Create a single strategy multi-instance GPU node pool in Azure Kubernetes Service (AKS)

Certain NVIDIA GPUs can be divided into up to seven independent instances. Each instance has its own Streaming Multiprocessor (SM), which executes instructions in parallel, and GPU memory. For more information about GPU partitioning, see NVIDIA MIG.

Use single strategy multi-instance GPU (MIG) in your AKS cluster to run workloads on uniformly sized GPU partitions. Each workload requests nvidia.com/gpu, where one unit represents one partition rather than an entire physical GPU. AKS installs and configures the NVIDIA driver and device plugin for this path. For strategy selection and resource naming, see Multi-instance GPU strategies in AKS. To create different-sized partitions, see Configure mixed MIG strategy.

This article walks you through how to set single strategy with a MIG node pool by using a compatible VM size in an Azure Kubernetes Service (AKS) cluster.

Limitations

• Node pools with AzureContainerLinux and Windows OS SKUs aren't currently supported with this feature.
• You can't change the profile configuration after MIG node pool creation. Create a replacement pool to change the layout or stop using MIG. Don't try to disable MIG by updating the strategy to None.

Before you begin

• An Azure subscription. If you don't have one, create an Azure free account.
• Install Azure CLI version 2.85.0 or later and aks-preview version 22.0.0b15 or later. Run az --version to check your Azure CLI version.
• A supported MIG-capable GPU VM size available in your selected AKS region. Ensure that you have enough regional and GPU-family vCPU quota for the VM size and node count you plan to deploy.
• Ensure any existing self-managed NVIDIA device plugin or GPU operator doesn't also manage the new MIG node pool's nodes.

Choose a uniform GPU profile

Specify exactly one profile for Single so that AKS can create the uniform partition layout. The following table lists the profiles for the NVIDIA A100 80 GB GPU. Counts are per physical GPU, not per node. Standard_NC24ads_A100_v4 has one A100 80 GB GPU, so MIG3g gives two partitions on that VM size.

GPU memory per profile varies by NVIDIA GPU model. Don't apply this table's memory sizes or node counts to a different SKU without confirming its GPU configuration.

Set environment variables and prepare your Azure subscription

export SUBSCRIPTION_ID="<subscription-id>"
export RESOURCE_GROUP="<my-resource-group>"
export CLUSTER_NAME="<my-cluster-name>"
export SINGLE_POOL="migsingle"

Select the Azure subscription:

az account set --subscription "$SUBSCRIPTION_ID"

Connect to your AKS cluster:

az aks get-credentials \
  --resource-group "$RESOURCE_GROUP" \
  --name "$CLUSTER_NAME"

Register the preview features

Install or update the required extension:

az extension add --name aks-preview --upgrade --yes

Register the ManagedGPUExperiencePreview feature:

az feature register \
  --namespace Microsoft.ContainerService \
  --name ManagedGPUExperiencePreview

Register the MixedMIGStrategyPreview feature:

az feature register \
  --namespace Microsoft.ContainerService \
  --name MixedMIGStrategyPreview

Check each registration:

az feature show \
  --namespace Microsoft.ContainerService \
  --name ManagedGPUExperiencePreview \
  --query properties.state --output tsv

az feature show \
  --namespace Microsoft.ContainerService \
  --name MixedMIGStrategyPreview \
  --query properties.state --output tsv

After both return Registered, refresh the provider registration:

az provider register --namespace Microsoft.ContainerService

Enable single strategy MIG on a new node pool

az aks nodepool add \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --name "$SINGLE_POOL" \
  --node-vm-size Standard_NC24ads_A100_v4 \
  --enable-managed-gpu=true \
  --gpu-mig-strategy Single \
  --gpu-mig-profiles MIG3g \
  --node-count 1

az aks nodepool add \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --name "$SINGLE_POOL" \
  --node-vm-size Standard_NC24ads_A100_v4 \
  --os-sku AzureLinux \
  --enable-managed-gpu=true \
  --gpu-mig-strategy Single \
  --gpu-mig-profiles MIG3g \
  --node-count 1

Verify configuration and GPU partition capacity

Check the managed GPU settings, profile list, and AKS node image:

az aks nodepool show \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --name "$SINGLE_POOL" \
  --query "{state:provisioningState,image:nodeImageVersion,gpu:gpuProfile.nvidia}" \
  --output json

In the output, check the following values. The GPU values appear under the gpu key.

Wait for the pool's nodes to become Ready.

kubectl wait --for=condition=Ready node \
  -l agentpool="$SINGLE_POOL" --timeout=600s

Check all nodes in the node pool:

kubectl describe nodes -l agentpool="$SINGLE_POOL"

For one Standard_NC24ads_A100_v4 node with the MIG3g profile (one A100 80 GB GPU, which gives two partitions), the Allocatable section should include:

Allocatable:
  nvidia.com/gpu:  2

Each unit has a capacity of one 3g.40gb MIG partition.

Run a workload on one MIG partition

Create single-strategy-example.yaml with the following manifest. Update the selector if you changed the node pool name.

apiVersion: v1
kind: Pod
metadata:
  name: nvidia-single
spec:
  restartPolicy: Never
  nodeSelector:
    agentpool: migsingle
  containers:
    - name: gpu-test
      image: mcr.microsoft.com/azuredocs/samples-tf-mnist-demo:gpu
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: "1"

The selector matters because a full-GPU pool can also advertise nvidia.com/gpu. Kubernetes treats an omitted request as equal to the extended resource limit. If you set both, use equal whole-number values.

Deploy the pod:

kubectl apply -f single-strategy-example.yaml

Wait for successful completion:

kubectl wait --for=jsonpath='{.status.phase}'=Succeeded \
  pod/nvidia-single --timeout=300s

Read its device information:

kubectl logs nvidia-single

Confirm that nvidia-smi runs without errors and reports an assigned MIG device. For an explicit device-list check, change the sample command to ["nvidia-smi", "-L"], delete the completed pod, and apply the manifest again. Repeat the completion wait and log check. With Standard_NC24ads_A100_v4 and the MIG3g profile, the logs should list one assigned 3g.40gb MIG device.

Understand validation and existing feature compatibility

Compare ways to configure single strategy MIG

The --gpu-instance-profile feature remains available for configuration of your own NVIDIA GPU device plugin. The following options separate GPU layout configuration from device plugin ownership.

All three single-strategy paths expose partitions as nvidia.com/gpu when configured on an NVIDIA GPU node pool. The resource name alone doesn't identify the partition size or whether AKS manages the plugin.

Validate the MIG profiles configuration

• Supply exactly one profile value to --gpu-mig-profiles for Single, not repeated entries to specify a partition count.
• Don't also supply --gpu-instance-profile flag. You can't combine --gpu-instance-profile and --gpu-mig-profiles on the same MIG node pool.

The --gpu-instance-profile flag remains supported for the single strategy, and AKS doesn't automatically convert it to --gpu-mig-profiles. To verify a new node pool created with --gpu-mig-strategy and --gpu-mig-profiles, run az aks nodepool show and check gpuProfile.nvidia.migStrategy and gpuProfile.nvidia.migProfiles.

Migrate to use the managed MIG profiles

1. Inventory the current pool's VM size, gpuInstanceProfile, node image, and advertised partition capacity. Record workload selectors, affinity, taints, tolerations, and GPU requests. Check whether the existing NVIDIA device plugin or GPU operator serves other node pools.

2. Ensure that your self-managed NVIDIA device plugin and any GPU operator don't reconcile the replacement MIG pool's nodes. Adjust their node targeting by using the supported configuration for your installation.

3. Create a replacement MIG node pool with a distinct name. For example, to preserve a MIG3g layout on Standard_NC24ads_A100_v4, use the following command. Substitute the supported SKU and single profile that matches your migration plan. Don't pass --gpu-instance-profile.

   az aks nodepool add \
     --resource-group "$RESOURCE_GROUP" \
     --cluster-name "$CLUSTER_NAME" \
     --name migreplacement \
     --node-vm-size Standard_NC24ads_A100_v4 \
     --enable-managed-gpu=true \
     --gpu-mig-strategy Single \
     --gpu-mig-profiles MIG3g \
     --node-count 1
   

4. Run the configuration, readiness, capacity, and workload checks in this article against migreplacement. Update the sample selector to agentpool: migreplacement. Confirm the expected partition size and device access before moving application workloads.

5. Move workloads gradually by updating pool selectors or affinity and any required tolerations. Single-strategy workloads continue to request nvidia.com/gpu. Verify application behavior, throughput, latency, and memory usage on the replacement pool.

6. After verifying the replacement, drain or retire workloads on the old node pool and delete it. Keep the old pool available until your migration validation is complete when capacity permits. Remove the self-managed NVIDIA device plugin installation only after confirming that no remaining pools need it.

Troubleshoot and remove the node pool

Troubleshoot scheduling and partition advertisement

A pod that requests more nvidia.com/gpu resources than the eligible nodes can provide remains pending with an insufficient-resource scheduling event. Inspect its status and events:

kubectl describe pod nvidia-single

If nodes don't advertise partitions, inspect node conditions, managed settings, and the selected image and profile. Don't install a second plugin or patch AKS-managed components as a workaround.

The following errors can occur when you create the node pool:

Azure CLI might show BadRequest as the top-level code, so also read the validation message.

Delete the sample pod and MIG node pool

Delete the sample pod:

kubectl delete pod nvidia-single --ignore-not-found=true

Before deleting a node pool that runs other workloads, create replacement capacity, migrate the workloads, and verify application health. Then remove the node pool:

az aks nodepool delete \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --name "$SINGLE_POOL"

Verify that the MIG node pool is removed:

az aks nodepool list \
  --resource-group "$RESOURCE_GROUP" \
  --cluster-name "$CLUSTER_NAME" \
  --query "[?name=='$SINGLE_POOL'].name" --output json

The output should be []. Confirm that no Kubernetes nodes remain for the pool:

kubectl get nodes -l agentpool="$SINGLE_POOL"

Next steps

• Compare single and mixed MIG partitioning strategies in AKS.
• Configure mixed MIG strategy (preview).
• Learn more about managed GPU nodes.