by: KarolPosted on:

Azure Managed DevOps Pool – Terraform

Azure Managed DevOps Pool is a new feature from Microsoft that provides a fully managed DevOps pool, eliminating the need for Azure Virtual Machine Scale Sets for DevOps agents. With this solution, you can create an agent pool within your virtual network. At a similar price point, you gain significantly more functionality. One of the most appealing aspects is the ability to leverage Microsoft-hosted runner images, so you don’t have to build a custom image with all the necessary tools. However, if you need a custom image to include additional software, check out my article on Azure DevOps self-hosted agents using Packer and VMSS for a step-by-step guide.

In the current scenario, Azure Managed DevOps Pool is the optimal choice for hosting agents within your Azure Virtual Network, providing a secure and fully managed environment for your CI/CD processes.

Status of the Azure Managed DevOps Pool deployment & test pipeline

Azure Managed DevOps Pool by Terraform

Prerequisites

Before you can fully leverage Azure Managed DevOps Pool in your Azure environment, you must complete a few essential setup steps. By taking care of these prerequisites – registering the correct resource providers, adjusting resource quotas, and assigning the necessary permissions – you’ll ensure a seamless configuration and be ready to take full advantage of Azure’s robust DevOps capabilities.

  1. Register Resource Providers
    • In your Azure subscription, go to Resource Providers in the left-hand menu.
    • Register the following providers:
      • Microsoft.DevOpsInfrastructure
      • Microsoft.DevCenter
  2. Adjust Resource Quotas
    • In the Azure portal, search for Quotas.
    • Select the Managed DevOps Pool tile.
    • Locate the Standard DADSv5 VM family; each agent in the pool uses 2 vCPUs.
    • Increase (or decrease) the quota to accommodate your project’s needs.
  3. Assign Permissions to Create Azure DevOps Agent Pools
    • In your Azure DevOps organization, navigate to Organization Settings.
    • Select Agent Pools, then click Security in the upper-right corner.
    • Locate your service principal and assign the Administrator role, then Save.
    • Repeat this process at both the Organization and Project levels.
    • Make sure your Azure DevOps organization and service principal are in the same Azure Entra ID (Azure AD) tenant.

By completing these steps, you’ll be prepared to configure and use Azure Managed DevOps Pool effectively, ensuring a secure, reliable, and high-performing environment for your CI/CD pipelines.

Azure Managed DevOps Pool Terraform: Complete Code

If you’re looking for a fully functional Terraform configuration to create an Azure DevOps Managed Pool, you can find the complete code in my GitHub repository. If you’re already familiar with configuring Terraform providers and managing Infrastructure as Code, feel free to skip the rest of this article and download the source code directly.

I hope you find it helpful – if you do, I’d greatly appreciate a comment or reaction on my LinkedIn. Your feedback not only helps improve the code but also fosters community collaboration and learning!

Azure Managed DevOps Pool Terraform configuration

In this solution, a main.auto.tfvars file is used to store variable values. Here’s an example of the values:

HCL
spn-client-id                  = "PRINCIPAL_CLIENT_ID"
spn-tenant-id                  = "TENANT_ID"
azure_devops_organization_name = "ADO_ORGANIZATION"
subscription-id                = "SUBSCRIPTION_ID"
ado_project_id                 = "ADO_PROJECT_ID"
pipeline_ids                   = ["PIPELINE_ID"]
maximum_concurrency            = 4

Obtaining Required IDs and Tokens

  1. Azure DevOps Project ID
    • You can retrieve this using a cURL command and your Personal Access Token (PAT) from Azure DevOps.
Bash
AUTH_HEADER=$(echo -n ":PAT_TOKEN" | base64)

curl -v -X GET \
-H "Content-Type: application/json" \
-H "Authorization: Basic $AUTH_HEADER" \
-H "Accept: application/json" \
"https://dev.azure.com/$ORGANIZATION/_apis/projects?api-version=7.1"
  1. Replace PAT_TOKEN with your Personal Access Token and $ORGANIZATION with your Azure DevOps organization name.
  2. Pipeline ID
    • You can find the Pipeline ID by looking at the URL of the pipeline in Azure DevOps.
  3. For example, if the URL is:
Bash
https://dev.azure.com/azure-way/basic-pipelines/_build?definitionId=1

The Pipeline ID is 1.

Applying Terraform with Client Secret

When running Terraform apply, you can pass in your service principal’s client secret like so:

HCL
terraform apply -var="spn-client-secret=SPN_CLIENT_SECRET"

Default Agent Settings

By default, the setup creates a maximum of 4 agents (maximum_concurrency = 4). Adjust this value to suit your organization’s needs for parallel builds.

Azure Managed DevOps Pool: Automated Testing

With both Terraform and Azure evolving rapidly, it’s important to make sure your solutions continue to function seamlessly amid ongoing updates. To achieve this, I created automated tests that verify the entire Azure Managed DevOps Pool setup. These tests do the following:

  1. Create a new Azure Managed DevOps Pool
  2. Add a new agent pool at both the project and organization levels
  3. Run a pipeline within Azure DevOps to confirm that everything is configured correctly
  4. The test checks the result of the ADO pipeline, if the status is successful then I assume that the agent pool is working.

Leveraging Multiple Images in Your Pipeline

One of the coolest features of Azure Managed DevOps Pool is the ability to specify different images on a per-job basis. In my automated test pipeline, each job requests a different image, such as ubuntu-20.04, ubuntu-22.04, windows-2019, or windows-2022. The pipeline you can find below:

YAML
trigger:
- main

jobs:
- job: Ubuntu_22_04
  displayName: 'Run on Ubuntu 22.04'
  pool: 
    name: azure-way-mdp # Name of Managed DevOps Pool
    demands:
    - ImageOverride -equals ubuntu-22.04
  steps:
  - script: |
      echo "Running on Ubuntu 22.04"
      uname -a
    displayName: 'Run Bash script on Ubuntu 22.04'

- job: Ubuntu_20_04
  displayName: 'Run on Ubuntu 20.04'
  pool: 
    name: azure-way-mdp # Name of Managed DevOps Pool
    demands:
    - ImageOverride -equals ubuntu-20.04
  steps:
  - script: |
      echo "Running on Ubuntu 20.04"
      uname -a
    displayName: 'Run Bash script on Ubuntu 20.04'

- job: Windows_2019
  displayName: 'Run on Windows 2019'
  pool: 
    name: azure-way-mdp # Name of Managed DevOps Pool
    demands:
    - ImageOverride -equals windows-2019
  steps:
  - powershell: |
      Write-Host "Running on Windows 2019"
      Get-ComputerInfo
    displayName: 'Run PowerShell script on Windows 2019'

- job: Windows_2022
  displayName: 'Run on Windows 2022'
  pool: 
    name: azure-way-mdp # Name of Managed DevOps Pool
    demands:
    - ImageOverride -equals windows-2022
  steps:
  - powershell: |
      Write-Host "Running on Windows 2022"
      Get-ComputerInfo
    displayName: 'Run PowerShell script on Windows 2022'

The repository is publicly available and can be accessed here.

Azure Managed DevOps Pool – Terraform setup

The first step in setting up your Azure Managed DevOps Pool with Terraform is to configure the necessary providers:

  1. Azure RM Provider – Enables you to manage Azure resources.
  2. Azure DevOps Provider – Allows you to configure Azure DevOps resources directly from Terraform.
HCL
provider "azuredevops" {
  org_service_url = local.azure_devops_organization_url

  client_id     = var.spn-client-id
  client_secret = var.spn-client-secret
  tenant_id     = var.spn-tenant-id
}

provider "azurerm" {
  features {}

  subscription_id = var.subscription-id
  client_id       = var.spn-client-id
  client_secret   = var.spn-client-secret
  tenant_id       = var.spn-tenant-id
}

In my setup, I’m using the same account for both Azure RM and Azure DevOps providers, but you’re free to use separate accounts if that suits your environment better.

The next step is to assign the required permissions to the built-in service principal responsible for configuring your networking. This principal requires at least the following permissions to function correctly:

  • Microsoft.Network/virtualNetworks/subnets/join/action
  • Microsoft.Network/virtualNetworks/subnets/serviceAssociationLinks/validate/action
  • Microsoft.Network/virtualNetworks/subnets/serviceAssociationLinks/write
  • Microsoft.Network/virtualNetworks/subnets/serviceAssociationLinks/delete
  • Reader

You obtain built-in service principal by using data:

HCL
data "azuread_service_principal" "this" {
  display_name = "DevOpsInfrastructure" 
}

For the Azure Managed DevOps Pool to function correctly within your virtual network, you must delegate the subnet to the appropriate resource type. Specifically, delegate your subnet to:

HCL
resource "azurerm_subnet" "this" {
  name                 = "managed-pool-subnet"
  resource_group_name  = azurerm_resource_group.this.name
  virtual_network_name = azurerm_virtual_network.this.name
  address_prefixes     = var.subnet_address_prefixes

  delegation {
    name = "delegation"

    service_delegation {
      name = "Microsoft.DevOpsInfrastructure/pools"
      actions = [
        "Microsoft.Network/virtualNetworks/subnets/join/action",
      ]
    }
  }
}

To properly configure your Azure Managed DevOps Pool, you must assign both the custom roles (for networking) and the Reader role to the pre-built service principal.

HCL
resource "azurerm_role_assignment" "reader" {
  principal_id         = data.azuread_service_principal.this.object_id
  role_definition_name = "Reader"
  scope                = azurerm_virtual_network.this.id
}

resource "azurerm_role_assignment" "subnet_join" {
  principal_id       = data.azuread_service_principal.this.object_id
  role_definition_id = azurerm_role_definition.this.role_definition_resource_id
  scope              = azurerm_virtual_network.this.id
}

To ensure your pipelines can use the newly created Azure DevOps Managed Pool, you need to perform two additional steps:

  1. Authorize the Agent Pool at the Project Level
    Grant your pipeline explicit permission to access the new pool.
  2. Create an Agent Queue
    In Azure DevOps, create an agent queue linked to the new pool so that your builds and releases can run seamlessly.
HCL
data "azuredevops_agent_queue" "this" {
  project_id = var.ado_project_id
  name       = module.managed_devops_pool.name
  depends_on = [module.managed_devops_pool]
}

data "azuredevops_build_definition" "example" {
  project_id = var.ado_project_id
  name = var.azure_devops_build_definition_name
}

resource "azuredevops_pipeline_authorization" "this" {
  for_each = toset(var.pipeline_ids)
  project_id  = var.ado_project_id
  resource_id = data.azuredevops_agent_queue.this.id
  type        = "queue"
  pipeline_id = each.key
}

Once you’ve registered all providers, delegated your subnet, and assigned the necessary roles to the service principal, you can proceed with the final step: creating your Azure Managed DevOps Pool

HCL
module "managed_devops_pool" {
  source  = "Azure/avm-res-devopsinfrastructure-pool/azurerm"
  version = "0.2.2"

  resource_group_name            = azurerm_resource_group.this.name
  location                       = azurerm_resource_group.this.location
  name                           = "azure-way-mdp"
  dev_center_project_resource_id = azurerm_dev_center_project.this.id
  subnet_id                      = azurerm_subnet.this.id
  maximum_concurrency            = var.maximum_concurrency
  organization_profile = {
    organizations = [{
      name     = var.azure_devops_organization_name
      projects = var.azure_devops_project_names
    }]
  }
  fabric_profile_images = [
    {
      well_known_image_name = "ubuntu-20.04/latest"
      aliases = [
        "ubuntu-20.04/latest",
        "ubuntu-20.04"
      ]
    },
    {
      well_known_image_name = "ubuntu-22.04/latest"
      aliases = [
        "ubuntu-22.04/latest",
        "ubuntu-22.04"
      ]
    },
    {
      well_known_image_name = "ubuntu-24.04/latest"
      aliases = [
        "ubuntu-24.04/latest",
        "ubuntu-24.04"
      ]
    },    
    {
      well_known_image_name = "windows-2019/latest"
      aliases = [
        "windows-2019/latest",
        "windows-2019"
      ]
    },
    {
      well_known_image_name = "windows-2022/latest"
      aliases = [
        "windows-2022/latest",
        "windows-2022"
      ]
    }
  ]
  enable_telemetry = var.enable_telemetry

  depends_on = [time_sleep.wait_1_minute]
}

Any images specified in the well_known_image_name property become immediately available to your pipelines. For a fully functional example, check out the complete code here.

Summary

Azure Managed DevOps Pool is Microsoft’s latest fully managed solution for hosting Azure DevOps build agents securely in your Azure Virtual Network. By using Terraform and leveraging Microsoft-hosted runner images (or custom images if needed), you can simplify your CI/CD pipelines without maintaining Azure Virtual Machine Scale Sets. To see the full Terraform setup, visit the linked GitHub repository – and don’t forget to follow me on LinkedIn and share your comments on my posts to help foster community collaboration and learning!

Leave a Reply

Your email address will not be published. Required fields are marked *