Microsoft Foundry: A Mental Model

Andrew Kelleher
5 days ago
6 min read

I've spent an increasing amount of time over the last few years working with Microsoft's AI tooling as it has evolved - from the original Azure AI Studio, through Azure AI Foundry, and most recently the new Microsoft Foundry experience announced at Ignite 2025.

Each iteration has added capabilities, but it's also meant that models, agents, data, tools, evaluation, deployment, and governance all needed to connect in ways that weren’t always obvious when you first open the portal.

With Microsoft Foundry, as with most new Microsoft services, I've found it really helpful to build a mental model of how the various components fit together.

In this blog post, I'll cover what Foundry is, the problem it's trying to solve, the core workflow it expects you to follow, and how to think about each of the building blocks.

The Short Version

Microsoft Foundry is a unified Azure platform for building, evaluating, deploying, and governing AI applications and agents. It brings models, agents, tools, data grounding, evaluation, and observability together under a single portal and a single Azure resource type - rather than leaving you to stitch together half a dozen separate services yourself.

Microsoft describes it as "the AI app and agent factory", which is more useful than it sounds. The factory framing is the mental model: raw materials (models, data, tools) go in one end, and governed, observable, production-ready AI applications come out the other.

One naming note before we go further - this is the new Foundry experience, formerly Azure AI Foundry (and before that, Azure AI Studio). If you've used hub-based projects in the older portal, those are now referred to as Foundry (classic). New investment is going into the new experience, so that's where I'd focus.

The Problem It Solves

Before the new Foundry experience, the hub-based Azure AI Studio / Azure AI Foundry (classic) approach typically meant stitching a solution together from a few separate building blocks -

A Hub + Project as the organisational containers
Model deployments (often via Azure OpenAI / Azure AI services endpoints) managed alongside, but not always feeling like part of one end-to-end workflow
Azure AI Search (or another retrieval layer) for grounding and RAG
Separate AI services for speech, vision, language, or document processing
Your own orchestration code for anything agent-shaped (prompt flow, SDK code, LangChain/Semantic Kernel, etc.)
Evaluation tooling that was often separate (prompt-flow runs, notebooks, scripts, or bespoke test harnesses)

Each piece worked, but the integration was yours to design, secure, and operate. Teams could experiment quickly or run responsibly in production, but doing both at once was hard work.

Foundry's answer is to bring these concerns into one platform surface. Models, agents, tools, and knowledge sit in a single project, with unified role-based access control (RBAC), networking, and policy under a single Azure resource provider, the Foundry account.

It’s worth calling out that in enterprise-scale Foundry deployment scenarios, where increased network security etc. is typically required, things do start to get a lot more complex with configuring BYO resources, VNET private endpoints etc. But the overall experience is still a lot more cohesive.

Image Courtesy Microsoft: (5) Getting to know the new Microsoft Foundry platform | LinkedIn

The Core Workflow

The platform makes much more sense once you realise it's built around a specific loop. In the following steps, we'll go through the workflow Foundry expects you to follow -

Create a Foundry resource - this is the top-level Azure resource (the “account”) that hosts Foundry projects and provides the shared control plane for identity/RBAC, networking, and governance across those projects.
Create a Foundry project - the project is your unit of organisation, access control, and billing context. These reside within the parent Foundry account
Choose your models from the catalog - foundation models, open-source options, and partner models, deployed as endpoints within your project
Add tools, data, or knowledge sources - connectors, APIs, and grounding data such as Azure AI Search indexes
Build agents or application workflows that combine the above
Evaluate and test behaviour using the built-in evaluation tooling
Deploy, then observe with tracing and monitoring, and iterate

That’s it — the whole Foundry experience hangs off this loop. If a capability or feature feels confusing, I ask myself where it sits within it.

How to Think About the Pieces

A couple of things are worth highlighting about each building block -

Agents. Agents are the centre of gravity in the new experience. An agent pairs a model with instructions, tools, and knowledge, and Foundry handles the hosting, threading, and lifecycle. This is the biggest conceptual shift from the older experience, which was primarily about managing model endpoints.

Models. The model catalog is broader than OpenAI’s GPT models - it includes open-source, partner models alongside frontier options from other labs such as Anthropic. Think of a model deployment as an endpoint that lives inside your project. If you're coming from a standalone Azure OpenAI resource, interestingly, you can upgrade it to a Foundry resource while preserving your endpoint and keys.

Tools and connectors. Tools are how agents act on the world - calling APIs, connecting to MCP servers, searching the web, or reaching into services like SharePoint and Fabric. What used to be Azure AI Services (speech, vision, language, documents) is now surfaced as Foundry Tools, plugged into the same platform.

Data, knowledge grounding, and Foundry IQ. Grounding is what stops your agent inventing answers. Knowledge sources provide agents with business context, and this is where retrieval-augmented generation (RAG) fits within the Foundry world. I also suspect that, over time, Foundry IQ will become a big thing for enterprises with many disparate data sources.

Evaluation and safety. Evaluation is built in rather than bolted on - you can run structured evaluations against datasets, compare model or prompt variants, and apply content safety controls. Where possible, run evaluations early; it's far cheaper than discovering issues in production.

Deployment and operations. Tracing, monitoring, and governance policies apply across the project. Because everything sits under one resource provider namespace, your platform team can reason about networking, identity, and policy in one place rather than six.

What Is Different for Teams

The unified surface changes the conversation for different roles -

Developers get a clearer build path - project, model, tools, agent, evaluate, deploy - with SDKs and a CLI that follow the same shape as the portal
Architects and Platform Engineers get a more coherent platform surface to design around - one resource type, unified RBAC and networking, and a governance story that doesn't require diagramming ten services
Leaders get a better way to discuss readiness - evaluation results, observability, and policy compliance are visible platform features rather than promises buried in a codebase

This isn't a strict segregation - in practice most of us wear more than one of these hats - but it helps explain why Microsoft keeps using the word "unified".

First Things to Try

If you want to become more familiar with Foundry and build your own mental model , I'd suggest the following -

Open the Foundry portal at ai.azure.com and create a Foundry resource and a small project (a non-production subscription is fine)
Deploy one model from the catalog and chat with it in the playground
Create a simple agent, give it instructions, and attach one tool or knowledge source
Run an evaluation against a handful of test prompts before you even think about production

Beware: it's tempting to jump straight to a multi-agent workflow with every connector enabled. Resist that. One project, one model, one agent, one evaluation - that's enough to understand how the pieces relate, and everything larger is a composition of the same loop.

Conclusion

This is a very high-level overview, but I hope you found this post useful. In my view, Microsoft Foundry is best understood not as a new product but as a consolidation - the point where Microsoft's AI building blocks stopped being a kit of parts and became a platform.

The mental model is the loop: project, models, tools and knowledge, agents, evaluation, deployment, observation.

Most importantly, Foundry is about reducing the friction between experimentation and production responsibility. You can move quickly and answer the governance questions when they come - which, in an enterprise, they always do.

You may also find the following links useful -