Python and Azure: A Perfect Match()?

I recently reflected on the popularity of Python and how it increasingly crops up in my day job.

Over the last few years, I’ve started writing more Python scripts for infrastructure automation and several personal projects. I’ve also worked with developer teams and AI engineers, for which Python was their primary language.

Is this just my perception, or is Python becoming more prevalent?

The latest TIOBE index appears to support this theory. Python’s usage has been steadily increasing since 2018, with a significant spike in 2024 -

image courtesy of tiobe.com

Significantly, this year, Python has now overtaken Java as the most used language on GitHub -

image courtesy of github.com

Microsoft even presented at PyCon US 24 this year.

One of the main reasons for Python’s popularity is its versatility. It’s used across a wide number of industries and use cases, e.g. -

• AI / Machine Learning

• Data analytics and processing

• General software development, e.g. API and backend services, frontend web services (Django, Flask)

• Automation and scripting

• Embedded Systems and IoT

• DevOps and automation

Microsoft Azure is where I spend most of my time, so as a fun exercise, I thought I’d investigate just how well-supported Python is across Azure’s multitude of services.

I’ve allocated each service to a particular job role. This isn’t a strict segregation; in reality, engineers and developers will often use various services to achieve a specific outcome.

Hopefully, this serves as a useful indication of the variety of services available.

Python Software Developers

1. Azure App Service

   • Web Apps: Host web applications built using Python frameworks like Django, Flask or FastAPI

   • API Apps: Create and deploy RESTful APIs using Python

2. Azure Functions

   • Serverless Functions: Write Python code to run as event-driven serverless functions.

3. Containers

   • Azure Kubernetes Service (AKS) - Containerize Python applications using Docker images and deploy them on Azure Kubernetes Service

   • Container Apps - same as AKS

   • Azure Container Instances - same as AKS

4. Azure Virtual Machines (VMs)

   • Python scripts and applications can be run within Azure VM’s

   • Many Azure Marketplace VM images already come with Python preinstalled

Data and AI Engineers

Given Python’s historically high usage within the data and machine learning communities, it’s no surprise that these job roles are well supported.

Okay, deep breath, here are the Data and AI-related Azure services that support Python -

1. Azure Batch

   • Run large-scale parallel and high-performance computing (HPC) applications efficiently in Python. Useful for batch processing jobs.

2. Azure Data Science Virtual Machines

   • Pre-configured VMs designed for data science workloads, including Python runtime

3. Azure Machine Learning

   • Develop and train machine learning models using Python. Supports Jupyter notebooks for interactive development and experimentation.

   • The SDK and many components of Azure ML utilize Python for scripting, model training, and deployment processes.

4. Azure Synapse Analytics

   • Run Python scripts to perform data processing and analytics tasks within Azure Synapse via native Synapse notebooks (PySpark) or the pyodbc library

5. Azure Databricks

   • Utilize Python code for big data analytics and machine learning, leveraging Apache Spark.

6. Azure IoT Edge

   • Deploy Python modules to IoT Edge devices for processing data locally before sending it to the cloud.

7. Azure Data Studio

   • Author Jupyter Notebooks within Azure Data Studio, enabling interactive data exploration and analysis using Python.

8. Azure AI Search

   • Many cognitive services provide Python SDKs for integrating AI and machine learning capabilities into applications.

Platform Engineer and DevOps

1. Azure Cloud Shell

   • Upload your Python scripts and run python from the command prompt

2. Azure CLI

   • Fun fact! - the Azure CLI is written in Python

   • “Built using Python, the Azure Command-Line Interface (CLI) allows you to manage Azure resources and services using command-line commands”

3. Azure SDK for Python

   • Provides comprehensive support for interacting with Azure services programmatically from Python scripts.

   • This can be an alternative to using Bicep / Terraform, especially if you prefer a programming approach over a declarative one. Whilst useful for some scenarios, you’d have to be committed to using this as your default option for deploying resources!

4. Azure Automation Runbooks

   • Automate the management of Azure resources using Python runbooks.

5. Integration with DevOps Pipelines

   • Python scripts can be part of CI/CD pipelines that deploy Azure resources using Bicep / Terraform. For instance, you can use Python to handle pre-deployment checks, validation, and coordination of the deployment process.

   • Azure DevOps Microsoft-hosted agents include Python

   
   

Summary

I hope you found this post interesting. Please let me know your comments below. Have you seen any other interesting use cases for running Python within Azure?

It's worth pointing out that from a developer's perspective, other popular languages, such as .NET, Java, etc., are also well supported within Azure. However, Python is certainly more prevalent, including data engineering, AI, and automation.

If you'd like to get started with Python on Azure, check out Microsoft's landing page for Python developers.

Also worth checking out is Microsoft's Python Pulse series of webcasts on YouTube.

Python is also cropping up in other areas of the Microsoft ecosystem. This neat video shows Python code running in Excel along with Microsoft 365 Copilot!