I’ve found that when starting a project that uses Microsoft Azure as its cloud hosting platform, there are some initial questions that the team needs to answer. These include -
What does the overall solution architecture look like?
Which Azure services are required to support the solution?
How will these Azure services integrate with each other and external services?
Starting with a blank piece of paper can be overwhelming design-wise. There’s an ever-increasing selection of services in Azure, and selecting the most appropriate ones can be challenging.
For example, is Azure Service Bus or Event Grid the best fit for the solution’s messaging requirements? Or, is Data Factory a better choice over Databricks for data analytics? In addition, our solution needs to consider reliability, scalability, security and many other factors.
This is where design patterns or reference architectures can provide a helpful starting point and give us a head start on creating our solution architecture.
In this blog post, I’ll demonstrate how leveraging these can help kick-start your Azure project.
What is a Design Pattern?
A design pattern is a solution that can be used repeatedly to solve a common problem or requirement in cloud architecture and software development.
Design patterns also help avoid reinventing the wheel and address common challenges and trade-offs in cloud development. For example, handling data consistency, performance, security, fault tolerance, messaging, and more.
Here’s an example design pattern from Azure's Architecture Center -
The design pattern is helpful to us because it shows -
Potential use cases
Suggested Azure services
Dataflows between the services
Additional documentation and resources
Unless it’s a bespoke project, typically, it will follow a “it’s another one of those”. Or it’s a “bit of one of those” and “some of that”. Chances are that another organisation elsewhere has already implemented something similar.
There’ll always be intricacies and complexities specific to each organisation, but it’s likely to follow a common pattern.
This was one of the key reasons Microsoft started publishing reference Azure architectures and design patterns. They found that their support teams and product groups encountered similar scenarios and patterns across many of their Azure customers.
Where to find Design Patterns?
For Azure-specific design patterns, I've found these Microsoft resources the most valuable -
800+ example reference architectures
Often includes a downloadable Visio file
ARM templates (@Microsoft, please start providing these as Bicep templates!)
More generic application patterns such as Circuit Break and Ambassador
Whilst intended for Azure, they are also applicable to other cloud providers.
A good resource for helping identify the most suitable Azure services for a workload, i.e. choosing a container hosting option
Finally, I've also found the AWS Architecture Center a good reference. Please proceed cautiously, but many of their architectures and patterns are generic enough to be reused in Azure by swapping out the AWS services with the corresponding Azure ones.
Six Steps for Leveraging Design Patterns
The steps I follow for incorporating a design pattern in a design are -
Develop the high-level solution architecture first (abstract building blocks)
Identify suitable design pattern/s
Create a detailed architecture diagram customized to your project
Use the Well-Architected Framework to help inform and provide additional guidance on those chosen Azure services
As part of a Proof of Concept (PoC), deploy the architecture to Azure to validate the services integrate as required in your environment and meet the project requirements
Update the architecture diagram as needed based on learnings from the PoC
TIP: don’t fall into the trap of selecting your Azure services first based on personal preference or familiarity. Create the high-level solution architecture first, then select the most appropriate Azure services that support the architecture.
However, this needs to be balanced against your organisation’s experience with existing Azure services. For example, suppose your engineering teams are already highly experienced in Azure Kubernetes Service (AKS). In that case, even though Container Apps may be a good fit for the new project, deploying it may not be advisable as this introduces additional support, monitoring and upskilling requirements.
As mentioned in my previous post, Azure Architecture - Design Principles and Lessons Learned, once you’re out of the initial solution design phase and embarked on your project, refining the architecture as requirements become better known is common.
For example, if developing an application, the developers may find that Service Bus is a better fit than Event Hub due to a new feature requirement.
Remember that many readily available design patterns are intended to show how different Azure services can be used together to solve a specific scenario. In our overall architecture, we still need to consider other factors such as disaster recovery, high availability, cost, performance and scaling.
Example Scenario - Video Ingestion and Analysis
In this section, I'll work through an example scenario that customizes an existing design pattern to our specific requirements.
Our new Azure project is to build and deploy a solution that analyzes video files, looks for detected objects and flags them for review in a custom-built .NET application.
Searching the Azure Architecture Center finds the following Video ingestion and object detection on the edge reference architecture -
Okay, the design pattern isn't an exact fit for our requirements, but it's a good starting point. Changes I've made include -
Pre-recorded videos are ingested from Azure Files instead of from a mobile robot
Regular Azure SQL replaces Azure SQL Edge
Removed references to Azure Arc and Azure Stack Edge (no Edge services required)
Include an Azure Function that is automatically triggered when a new video analysis has been completed and added to the database
SendGrid (deployed as an Azure SaaS resource) added for email notifications
Azure Web Apps is included for hosting the .NET application
Entra (Azure AD) is included for integrated authentication to the Web App
Below is the new solution architecture -
I haven't considered disaster recovery, high availability or security at this stage. But it's enough to get started with a Proof of Concept deployment to validate the basic architecture.
Creating a Reusable Design Pattern
Imagine the scenario where our Azure project has been completed successfully. The design is complete, the environments are built, and the application is coded, deployed and accessed by our end users... congratulations!
We can now look at our project and consider whether any artefacts could be reused in the future. Specifically, does our solution solve a common challenge our organisation will likely face again?
For example, if the project developed and deployed a new customer-facing website, we would have needed to consider many factors, including Azure hosting and environments, security, authentication and deployment.
If our organisation is likely to embark on a similar project again, we can create a reusable design pattern from our very own architecture.
To create a reusable design pattern from a solution architecture, you may follow these tips:
Identify the common design challenges: Review the solution architecture and identify the common design challenges the system addresses. These challenges could be related to scalability, performance, security, or any other aspect of the system.
Abstract the solution: Once you have identified the common design challenges, abstract the solution. This means you should create a general solution that can be applied to different scenarios.
Document the pattern: Document the pattern clearly and concisely. This documentation should include the problem the pattern solves, the context in which it is applicable, and the steps required to implement it.
Validate the pattern: Validate the pattern by applying it to different scenarios. This will help you to identify any issues with the pattern and refine it further.
Publish the pattern: Once you have validated the pattern, publish it in a central repository. This will make it available to other developers who can use it to solve similar design challenges. Include references to relevant resources such as application libraries, infrastructure as code (IaC), diagrams, documentation, etc.
Conclusion
Design patterns and reference architectures can be used repeatedly to solve common problems or requirements in cloud architecture and software development.
At the time of writing, Microsoft has just refreshed the Well-Architected Framework, including the Azure Architecture Center. So, it is well worth a look for more detailed information and prescriptive guidance.
I hope you found this post useful. As always, feedback and comments are always welcome. How are you using Azure design patterns within your organisation and environments?