Microsoft Foundry: A Platform Architect’s Guide
- Andrew Kelleher
- 6 minutes ago
- 10 min read
I've recently completed another implementation of Microsoft Foundry - one of several I've worked on as organisations move from AI experimentation to something production-ready.
The questions are familiar by now; which models, which regions, how do we govern it? But one observation has held true on every engagement: standing Foundry up as a public-facing sandbox takes an afternoon. Deploying it privately, with the security, governance, and compliance posture an enterprise actually needs, is a non-trivial piece of platform design work.
There's a long list of things that have to be considered together - network isolation, private DNS, RBAC, governance boundaries, disaster recovery, cost management. Plus a set of adjacent services (Cosmos DB, Azure AI Search, Storage, Key Vault) that need the same treatment. Most Microsoft guidance covers these individually; very little walks through the end-to-end design decisions and the order in which to make them.
This post is a follow-on to my earlier Microsoft Foundry: A Mental Model post, which covered what Foundry is and how to think about it conceptually. This one is more hands-on and based on real-world delivery. I'll walk through the design decisions, trade-offs, and gotchas from these engagements, drawing mainly on the most recent design, and a framework for phasing those decisions across design, build and run.
What This Post Covers
This is a design guide from the perspective of a platform owner, AI or platform architect not a click-by-click how-to. I'll assume you're comfortable with core Azure concepts (VNETs, private endpoints, RBAC, Azure Policy), but I won't assume any prior Foundry experience.
Microsoft's own guidance is genuinely useful and comprehensive, but fragmented.
The rollout planning guide and the baseline Foundry landing zone reference architecture are good companion reading, and the foundry-samples repository provides deployable Bicep / Terraform for multiple secure topologies.
What those don't give you is the sequencing: which decisions you must make up front, which you can safely defer, and where the gaps between the documented happy path and a real enterprise estate appear.
One caveat before we start: Foundry is moving quickly. Everything below is accurate as of Q3 2026, but several limitations I've hit recently have already changed. I'll call those out as we go.
Foundry's Governance Model
The first thing to consider is where Foundry's governance boundaries sit. The Foundry resource (the account) is the primary boundary - networking, RBAC and policy controls are enforced at this level.
Within that boundary, projects act as a secondary logical segregation layer, separating teams, use cases and workloads while inheriting the security and compliance posture of the parent account. The Foundry architecture concepts page covers the resource model in more detail.
This shapes everything downstream. If your network posture, encryption settings and policies live at the account level, then the number of accounts you deploy becomes a key architectural decision.
In the most recent design, we landed on one Foundry instance per environment: DEV, UAT and PROD - each in its own Azure subscription, with projects providing the segregation between teams and use cases within each environment. This keeps the governance surface small (three accounts to secure, not thirty) while still giving teams their own space to work in.
How projects should be structured - per business area, per product, or per project - is a decision we deliberately didn't make at design time. The AI Engineers needed hands-on time with Foundry before they could commit to a structure, so we recorded it as an early build-phase decision. More on why that's fine later.
Who Creates Foundry Projects? Guardrails, Not Gates
Once you've settled the account/project model, the next question is the one that generates the most debate: who gets to create projects?
There are two obvious ends of the spectrum -
Approach | Pros | Cons |
Platform-managed (Terraform / Bicep) | Strong governance and consistency; enforced RBAC and security model; easier lifecycle management and auditability | Slower onboarding; request-driven; increased dependency on the platform team |
Self-service (AI Engineer-created projects) | Faster onboarding; enables rapid experimentation and PoCs; reduces platform team dependency | Risk of inconsistent configuration (naming, tagging, RBAC); potential project sprawl |
We adopted a hybrid: controlled self-service by AI engineers in DEV and UAT (with restricted RBAC roles and governance guardrails applied), and PROD projects provisioned exclusively via platform-managed IaC. This gives teams the agility to experiment where the blast radius is small, while keeping production consistent, auditable and compliant.
There are a couple of things worth highlighting -
Project provisioning and agent/model deployment are separate concerns, and can (perhaps should) be handled by different processes. A project in PROD might be created by the platform IaC while agents within it are deployed by the team's own ADO pipeline via AZ CLI commands
Guardrails beat gates. Azure Policy (allowed regions, required private endpoints, deny public network access, tag enforcement) does the enforcement work, so self-service doesn't have to mean uncontrolled.
Network Architecture (the Big One)
This is where the "public is easy, private is hard" observation tends to bite. A public Foundry endpoint works out of the box, which is ideal for early PoC experimentation.
A private one means wiring up all the infrastructure plumbing: disabling public access, private endpoints for Foundry and every associated service (Cosmos DB, AI Search, Storage, Key Vault, ACR), private DNS zones and egress control.
Microsoft's network isolation how-to is the main reference here, and the Private Networking and Inference in Microsoft Foundry post from the Foundry team is a good architectural companion.
For agent networking, there are two approaches: Bring-Your-Own (BYO) VNET, where you provision and manage a dedicated subnet for agent services, and Managed VNET, where Microsoft manages it for you.
As of Q3 2026, Microsoft's recommendation (confirmed in our discussions with them) is BYO VNET - it's currently the more feature-rich option, with many Managed VNET capabilities still in preview. The Agent Service virtual networks guide covers the setup.
That said, there are limitations worth highlighting whichever approach you take -
Teams / M365 publishing gets more involved. Agents can be published to Teams and M365 with public network access disabled, but there are additional set-up requirements e.g. fronting inbound traffic with Application Gateway + API Management. As Microsoft puts it: "You can publish your agent to Teams and M365 when your Foundry resource has public network access disabled. There are additional set-up requirements for this experience." (source)
Roughly half of agent tooling isn't yet supported with network isolation. Check the tool-by-tool support table before promising teams a particular capability - some tools work fully, some partially (Code Interpreter works, but without file upload), and some not at all.
There's no per-endpoint public/private control. Foundry can't selectively make one model endpoint public while keeping the rest private. If a single endpoint needs external exposure (we had exactly this - an integration needing to call a public model endpoint), the pattern is Application Gateway + APIM as an AI gateway in front of the private Foundry instance. I'd strongly recommend considering this pattern early - it also buys you token-based rate limiting and load balancing.
Limitations change under you. During our design, ACR had to be publicly accessible for hosted agents - we were preparing an Azure Policy exemption by our security team. By July, private ACR was supported and the exemption was unnecessary. Re-validate the limitations list before you finalise anything.
RBAC and Personas
Foundry makes a clear distinction between two planes -
Control plane - setting up and configuring resources, projects, networking, encryption and connections. This is platform/infrastructure engineer territory.
Data plane - running and using model inference, agent interactions, evaluation jobs and content safety calls. This is where AI engineers and developers live.
Microsoft provides built-in roles that map reasonably well onto real personas -
Built-in role | Persona in our design |
Foundry User | Developers and AI engineers using projects - the least-privilege starting point |
Foundry Project Manager | AI Platform Owner - manage projects, build within them, and conditionally assign Foundry User to others |
Foundry Account Owner | Platform team - full management of projects and resources |
Foundry Owner | Avoid by default - a highly privileged self-serve role designed for digital natives, not a governed enterprise |
Layer on the usual enterprise controls: separation of duties between the platform team, AI engineers and developesr. Use PIM / conditional access for privileged actions like changing networking or modifying data plane role assignments (ideally via IaC).
Models and Regions
Regional model availability is a real constraint if, for regulatory or security reasons, you need to deploy a model endpoint to a specific region. Not all models are available in all regions, and in UK South (and many others) the gap is noticeable.
Check the availability of the models you actually intend to use before committing to a region strategy. There are several different deployment approaches available -
Deployment type | What it means | Typical availability |
Regional | Model served from a specific Azure region (e.g. UK South) | Reduced availability (varies by region) |
Data zone | Model served from a data zone (e.g. EU / US / APAC) | Good availability |
Global | Model served from any region | Best availability |
There's some flexibility here: the core Foundry service and its BYO services can be deployed in one region with models deployed in another. But be aware that agents currently have limitations accessing models cross-region, so don't design around that split without validating it for your agent scenarios.
For the deployment approach itself, we followed Microsoft's recommendation and standardised on Data Zone Standard deployments - they provide the richest model capabilities and, per the resiliency guidance, the most recovery options. Provisioned Throughput (PTU) endpoints are the option to reach for where latency SLAs matter.
For UK-regulated organisations, there's a knock-on effect: UK West's limited AI capabilities effectively make this a single-region design. That feeds directly into the DR discussion below.
NFRs People Forget to Define
I continue to believe that non-functional requirements (NFRs) are the most undervalued artefact in platform design: they force the difficult conversations about availability, recovery, operability, cost and support that feature-led discussions tend to postpone. That's particularly important with Foundry, where the non-functional reality differs significantly from what platform teams may expect of a mature Azure service.
Availability. There is no SLA for Agent Service uptime. Straight from the HA and resiliency guidance:
"Agent Service has no availability or state durability Service Level Agreement (SLA). Standard mode offloads SLAs and data durability assurances to the underlying storage components." - Microsoft Learn
In other words, your availability story is the zone redundancy you configure on Cosmos DB, AI Search and Storage. Use Standard agent deployment mode (Basic offers almost no recovery options), enable Availability Zones on the data services, and set availability targets accordingly.
Disaster recovery. This is the one that surprised me most: Foundry cross-region DR is redeploy + reconstruct, not failover. The Agent Service DR guidance is refreshingly blunt about it:
"Agent Service doesn't provide built-in disaster recovery capabilities. It doesn't replicate state, create backups, or support point-in-time restore... The service doesn't have any supported method for active-active, multi-region replication." - Microsoft Learn
Critically, enabling native multi-region replication on Cosmos DB, AI Search, or Storage doesn't improve recovery outcomes; agent state can't be migrated or merged between projects or regions. As the same guidance puts it: "Warm standby environments start mostly empty. Recovery is reconstruction, not promotion of a hot replica."
Your real recovery mechanism is IaC plus pipeline-deployed agents - treat agent definitions as code, keep them in source control, and practice redeployment. Define RTO/RPO targets up front and classify workloads (PoC vs business-critical) so you know which agents justify that investment.
Observability. Native Log Analytics / App Insights covers DEV and UAT comfortably, including Foundry's built-in agent tracing and evaluation telemetry. The decision point is PROD: if your operations teams standardise on 3rd party enterprise tooling, integrating Foundry's services into that tooling becomes a service transition workstream in its own right - plan for it.
FinOps. AI platforms get expensive in surprising ways - vector indexing, re-indexing frequency, eval runs and logging volume are classic hotspots. Foundry now provides per-project cost attribution (in preview), which we adopted for chargeback rather than building something custom. Combine it with chargeback tags, budgets and quota ceilings per environment.
Data protection. Don't forget that in a BYO VNET agent architecture, your Cosmos DB is a persistent store of potentially sensitive prompts, responses and business content. Secure it appropriately: private endpoints, Entra ID authentication, managed identities and RBAC. The Azure security baseline for Foundry is a useful checklist here.
Environments and Release Management
Nothing revolutionary here, but it needs stating because manual portal-driven AI experimentation is the default behaviour you're designing against in production environments.
We kept the existing DEV/UAT/PROD subscription segregation, with Terraform deploying the core Foundry service and its associated Azure infrastructure across all three environments. Model and agent artefacts are promoted across environments via ADO pipelines (GitHub Actions works equally well) with stage approvals; manual changes within Foundry are reserved for ad hoc testing in lower environments. Developers can use the Foundry toolkit in VS Code to test locally before pushing changes to DEV.
The DR guidance above is why this matters. Microsoft's own resiliency guidance warns: "Avoid untracked changes made directly in the Foundry portal or Azure portal. Untracked production changes make recovery slower and error-prone."Â (source)
Your pipelines aren't just release management - they're your recovery plan.
A Framework: Phase Your Decisions
If there's one process lesson I'd pass on from these engagements, it's this: not every decision can - or should - be made at the high-level design (HLD) phase. In the most recent design, some decisions couldn't be made until the AI engineers had hands-on time with Foundry and fully understood how workloads and use cases could sit within its projects.
Rather than forcing premature answers, we recorded each decision against the phase that owns it -
HLDÂ - governance boundaries (accounts vs projects), target Azure landing zones, network posture (private-only, BYO VNET for agents), region strategy, environment strategy, DR approach (redeploy + reconstruct), support model.
LLDÂ - private endpoint and DNS specifics, RBAC role assignments, IaC module design, egress control.
Build - project structure, cost centre tagging, provisioning workflows - deferred deliberately so the AI engineers could explore Foundry and iterate on their preferred ways of working.
Run - model endpoint performance metrics, latency requirements, quota tuning, PTU decisions.
The important word is deliberately. Deferring a decision is fine; discovering at build time that nobody made it is not. Every deferred decision in our design log carried an owner and a target phase.
Lessons Learned
Having been through this a couple of times now, my key lessons have been -
Public is easy; private is a project. Budget for the infrastructure plumbing - VNETs, private endpoints, DNS, egress - not just the Foundry resource.
Validate limitations at design time, then re-validate at build. They change monthly - our ACR policy exemption became unnecessary mid-design.
Guardrails, not gates. Let AI Engineers self-serve in DEV/UAT with restricted roles and policy enforcement; keep PROD platform-managed via IaC and ADO pipelines
DR is redeploy + reconstruct. Invest in IaC and agent deployment pipelines, not multi-region data replication - it doesn’t improve recovery outcomes.
Defer decisions deliberately, not accidentally. Record which phase owns each open decision, and who's accountable for closing it.
Conclusion
Deploying Microsoft Foundry at a platform level is very achievable - but it's a platform design exercise, not a resource deployment. Get the governance boundaries, network posture and NFRs right up front, phase the decisions you can't yet make, and let IaC and pipelines carry both your release management and your recovery story. The framework above should adapt readily to your own organisation's deployment phases and gates.
I hope you found this post helpful. As always, feedback and questions are welcome.
You may also find the following links useful -