Agent Not Reporting¶

1. Summary¶

An Azure Monitor Agent (AMA) extension appears installed on a VM or Arc-enabled server, but the machine stops sending heartbeats, performance counters, event logs, or syslog records to the Log Analytics workspace. This playbook applies when the problem is tied to AMA-collected data and you need to prove whether the failure is in extension health, DCR association, managed identity, guest runtime state, or outbound connectivity.

The key Microsoft Learn guidance is that a successful extension deployment does not prove the agent is healthy at runtime. AMA needs a valid data collection rule, access to identity and configuration endpoints, local service health, and network reachability to Azure Monitor. Use this playbook when Heartbeat is stale, when a newly onboarded VM never reports, or when only AMA-driven tables are missing while platform logs still arrive.

Typical incident window: 10-20 minutes from first missed heartbeat to human detection when stale-heartbeat alerting is in place. Time to resolution: 30 minutes to 2 hours depending on whether the break is DCR association, identity, guest runtime, or egress.

Use it when:

• Heartbeat is stale for one VM, a scale set set, or an Arc server fleet.
• The extension says Succeeded, but no new data reaches Perf, InsightsMetrics, or event tables.
• Data stopped after network changes, identity changes, or DCR rollout changes.
• Only AMA-collected tables are missing; other workspace tables still update.

flowchart TD
    A[AMA not reporting] --> B{Heartbeat stale?}
    B -->|Yes| C[Check extension and service health]
    C --> D{DCR association exists?}
    D -->|No| E[Create association]
    D -->|Yes| F{Identity and endpoint access healthy?}
    F -->|No| G[Fix identity or network]
    F -->|Yes| H[Inspect guest AMA logs]
    H --> I[Repair local runtime or config]

2. Common Misreadings¶

3. Competing Hypotheses¶

4. What to Check First¶

1. Confirm the VM identity state used by AMA-related calls

   az vm show \
       --resource-group $RG \
       --name $VM_NAME \
       --query "identity"
   

   Command	Purpose
   az vm show	Show properties of a virtual machine.
   --resource-group	Resource group that contains the resource.
   --name	Name of the resource.
   --query	JMESPath projection of the fields to return.

2. Confirm AMA extension deployment and publisher details

   az vm extension list \
       --resource-group $RG \
       --vm-name $VM_NAME \
       --query "[].{name:name,provisioningState:provisioningState,publisher:publisher}"
   

   Command	Purpose
   az vm extension list	List extensions installed on a virtual machine.
   --resource-group	Resource group that contains the resource.
   --vm-name	Name of the virtual machine.
   --query	JMESPath projection of the fields to return.

3. Query the workspace for current heartbeat state before changing the VM

   az monitor log-analytics query \
       --workspace $WORKSPACE_ID \
       --analytics-query "Heartbeat | where TimeGenerated > ago(1d) | summarize LastHeartbeat=max(TimeGenerated) by Computer, _ResourceId | order by LastHeartbeat asc" \
       --timespan "P1D"
   

   Command	Purpose
   az monitor log-analytics query	Run a KQL query against a Log Analytics workspace.
   --workspace	Log Analytics workspace ID for the query.
   --analytics-query	Kusto (KQL) query to execute.
   --timespan	Time range for the query.

4. List DCR associations on the affected resource

   az monitor data-collection rule association list \
       --resource $RESOURCE_ID \
       --output json
   

   Command	Purpose
   az monitor data-collection rule association list	List data collection rule associations.
   --resource	Target resource ID or name for the operation.
   --output	Output format for the result.

5. Inspect the DCR data flows and destination workspace

   az monitor data-collection rule show \
       --resource-group $RG \
       --name $DCR_NAME \
       --output json
   

   Command	Purpose
   az monitor data-collection rule show	Show a data collection rule.
   --resource-group	Resource group that contains the resource.
   --name	Name of the resource.
   --output	Output format for the result.

6. If Arc is involved, inspect the connected machine resource state

   az connectedmachine show \
       --resource-group $RG \
       --name $MACHINE_NAME \
       --query "{status:status,location:location,id:id}"
   

   Command	Purpose
   az connectedmachine show	Show a connected (Arc-enabled) machine.
   --resource-group	Resource group that contains the resource.
   --name	Name of the resource.
   --query	JMESPath projection of the fields to return.

5. Evidence to Collect¶

5.1 KQL Queries¶

// Last heartbeat per computer
Heartbeat
| where TimeGenerated > ago(3d)
| summarize LastHeartbeat=max(TimeGenerated) by Computer, OSType, _ResourceId
| order by LastHeartbeat asc
| take 30

// Compare AMA freshness against other workspace activity
union isfuzzy=true
    (Heartbeat | summarize LastSeen=max(TimeGenerated), Rows=count() by TableName="Heartbeat"),
    (AzureActivity | summarize LastSeen=max(TimeGenerated), Rows=count() by TableName="AzureActivity"),
    (Operation | summarize LastSeen=max(TimeGenerated), Rows=count() by TableName="Operation")
| extend MinutesSinceLastSeen = datetime_diff('minute', now(), LastSeen) * -1
| order by MinutesSinceLastSeen desc

// Missing or stale heartbeat concentration by resource group pattern
Heartbeat
| where TimeGenerated > ago(3d)
| summarize LastHeartbeat=max(TimeGenerated), Computers=dcount(Computer) by ResourceGroup=extract(@"resourceGroups/([^/]+)/", 1, _ResourceId)
| order by LastHeartbeat asc

// Estimate ingestion delay for recent heartbeats where data still flows
Heartbeat
| where TimeGenerated > ago(6h)
| extend DelayMinutes = datetime_diff('minute', ingestion_time(), TimeGenerated)
| summarize AvgDelay=avg(DelayMinutes), P95Delay=percentile(DelayMinutes, 95), MaxDelay=max(DelayMinutes) by OSType

5.2 CLI Investigation¶

# Check VM identity and provisioning basics
az vm show \
    --resource-group $RG \
    --name $VM_NAME \
    --output json

Sample output:

{
  "id": "/subscriptions/<subscription-id>/resourceGroups/rg-prod/providers/Microsoft.Compute/virtualMachines/vm-prod-02",
  "identity": {
    "type": "SystemAssigned",
    "principalId": "<object-id>",
    "tenantId": "<tenant-id>"
  },
  "location": "eastus",
  "name": "vm-prod-02"
}

Interpretation:

• Missing identity can block AMA token acquisition scenarios that depend on managed identity.
• Identity drift after rebuilds or automation changes is a common hidden cause.
• Capture this before spending time on guest-side remediation.

# Confirm AMA extension deployment on the VM
az vm extension list \
    --resource-group $RG \
    --vm-name $VM_NAME \
    --output table

Sample output:

Name                    Publisher                   ProvisioningState
----------------------  --------------------------  -----------------
AzureMonitorWindowsAgent Microsoft.Azure.Monitor    Succeeded

Interpretation:

• No AMA row means there is no agent to troubleshoot.
• Succeeded narrows the problem to runtime state, DCR, identity, or network.
• Extension name differs by OS, so compare against the expected Windows or Linux package.

# Check DCR association for the affected resource
az monitor data-collection rule association list \
    --resource $RESOURCE_ID \
    --output json

Sample output:

[
  {
    "dataCollectionRuleId": "/subscriptions/<subscription-id>/resourceGroups/rg-monitor/providers/Microsoft.Insights/dataCollectionRules/dcr-vm-baseline",
    "name": "ama-baseline",
    "provisioningState": "Succeeded"
  }
]

Interpretation:

• Empty result means the VM cannot receive AMA collection instructions.
• If the wrong DCR is associated, data may flow but not for the expected streams.
• Use the returned DCR ID in the next command to inspect data flows.

# Inspect DCR streams and destination workspace
az monitor data-collection rule show \
    --resource-group $RG \
    --name $DCR_NAME \
    --output json

Sample output:

{
  "dataFlows": [
    {
      "streams": [
        "Microsoft-Perf",
        "Microsoft-Event",
        "Microsoft-Syslog"
      ],
      "destinations": [
        "la-workspace"
      ]
    }
  ],
  "destinations": {
    "logAnalytics": [
      {
        "workspaceResourceId": "/subscriptions/<subscription-id>/resourceGroups/rg-monitor/providers/Microsoft.OperationalInsights/workspaces/law-prod"
      }
    ]
  }
}

Interpretation:

• If the missing stream is not listed, the agent may be healthy but collecting the wrong data set.
• Wrong workspace destination explains why a machine appears silent in the expected workspace.
• DCR content also helps prove whether this is one-machine drift or a fleet-wide configuration issue.

6. Validation and Disproof by Hypothesis¶

Hypothesis 1: No DCR association exists for the resource¶

Proves if: Section 5.2 CLI command 3 returns no association or a failed association.

Disproves if: A valid association exists and points to the intended DCR.

Test with: Section 5.2 CLI command 3.

Hypothesis 2: AMA extension or service runtime is unhealthy¶

Proves if: Section 5.1 Query 1 shows stale heartbeat and guest logs show service startup, plugin, or configuration errors.

Disproves if: Heartbeat is fresh or guest logs show healthy runtime.

Test with: Section 5.1 Query 1 plus Section 5.2 CLI command 2, then inspect guest logs under the Microsoft Learn-documented AMA log locations.

Hypothesis 3: Managed identity or token flow is broken¶

Proves if: Section 5.2 CLI command 1 shows missing or changed identity and AMA guest logs show authentication failures.

Disproves if: Identity is present and guest logs show successful configuration retrieval.

Test with: Section 5.2 CLI command 1 and guest-side AMA logs.

Hypothesis 4: Endpoint access to IMDS or Azure Monitor is blocked¶

Proves if: DCR and extension are present but the guest cannot reach IMDS or required Azure Monitor endpoints.

Disproves if: Endpoint checks succeed and another hypothesis fits better.

Test with: Section 5.2 CLI commands 3 and 4 for expected config, then validate network reachability from the guest host.

Hypothesis 5: DCR exists but does not contain the expected streams¶

Proves if: Heartbeat may exist but the missing data type is absent from dataFlows.streams.

Disproves if: The stream is present and properly routed.

Test with: Section 5.2 CLI command 4.

Hypothesis 6: Workspace-side issue is being blamed incorrectly¶

Proves if: Other machines still report to the same workspace and Section 5.1 Query 2 shows other workspace tables are current.

Disproves if: Many unrelated senders are stale too.

Test with: Section 5.1 Queries 2 and 3.

7. Likely Root Cause Patterns¶

Normal vs Abnormal Comparison¶

8. Immediate Mitigations¶

1. Recreate the missing DCR association.

   az monitor data-collection rule association create \
       --name ama-baseline \
       --resource $RESOURCE_ID \
       --rule-id $DCR_ID
   

   Command	Purpose
   az monitor data-collection rule association create	Associate a data collection rule with a resource.
   --name	Name of the resource.
   --resource	Target resource ID or name for the operation.
   --rule-id	Resource ID of the data collection rule to associate.

2. Reinstall or update AMA if the extension is missing or damaged.

   az vm extension set \
       --resource-group $RG \
       --vm-name $VM_NAME \
       --publisher Microsoft.Azure.Monitor \
       --name AzureMonitorLinuxAgent \
       --enable-auto-upgrade true
   

   Command	Purpose
   az vm extension set	Install or update a virtual machine extension.
   --resource-group	Resource group that contains the resource.
   --vm-name	Name of the virtual machine.
   --publisher	Publisher of the VM extension.
   --name	Name of the resource.
   --enable-auto-upgrade	Enables automatic extension upgrades.

3. Restart AMA after configuration repair.

   az vm run-command invoke \
       --resource-group $RG \
       --name $VM_NAME \
       --command-id RunShellScript \
       --scripts "sudo systemctl restart azuremonitoragent"
   

   Command	Purpose
   az vm run-command invoke	Run a command on a virtual machine.
   --resource-group	Resource group that contains the resource.
   --name	Name of the resource.
   --command-id	Run-command identifier to invoke.
   --scripts	Inline script content to run.

4. Restore system-assigned identity when it was removed.

   az vm identity assign \
       --resource-group $RG \
       --name $VM_NAME
   

   Command	Purpose
   az vm identity assign	Assign a managed identity to a virtual machine.
   --resource-group	Resource group that contains the resource.
   --name	Name of the resource.

5. If the issue is stream mismatch, attach the correct DCR or restore missing streams immediately.

   az monitor data-collection rule show \
       --resource-group $RG \
       --name $DCR_NAME \
       --query "dataFlows"
   

   Command	Purpose
   az monitor data-collection rule show	Show a data collection rule.
   --resource-group	Resource group that contains the resource.
   --name	Name of the resource.
   --query	JMESPath projection of the fields to return.

9. Prevention¶

Prevent AMA reporting failures by treating onboarding as a four-part contract: extension, identity, DCR association, and endpoint access. Missing any one of them causes silent collection gaps.

Audit DCR associations regularly for monitored machines.

az monitor data-collection rule association list \
    --resource $RESOURCE_ID \
    --output table

Keep identity configuration explicit in VM build templates and post-deployment validation.

az vm show \
    --resource-group $RG \
    --name $VM_NAME \
    --query "identity.type"

Use standard network validation for Azure Monitor and IMDS endpoints when firewalls or private routing change.

az monitor data-collection rule show \
    --resource-group $RG \
    --name $DCR_NAME \
    --query "destinations"

Create a stale-heartbeat alert so a failed agent path is detected quickly.

az monitor scheduled-query create \
    --name "ama-heartbeat-stale" \
    --resource-group "$RG" \
    --scopes "$WORKSPACE_ID" \
    --condition "count 'StaleHeartbeat' > 0" \
    --condition-query "StaleHeartbeat=Heartbeat | where TimeGenerated < ago(15m)" \
    --evaluation-frequency "5m" \
    --window-size "5m" \
    --severity 2 \
    --skip-query-validation true \
    --description "Trigger when AMA heartbeat data is older than fifteen minutes." \
    --output json

Finally, review guest log paths during runbook design. Microsoft Learn explicitly directs troubleshooters to local AMA logs because workspace queries alone cannot explain why a machine never started sending.

See Also¶

• No Data in Workspace
• Operations: Data Collection Rules
• Operations: Workspace Management
• KQL: Resource Health

Sources¶

• Microsoft Learn: Troubleshoot Azure Monitor Agent on Windows virtual machines
• Microsoft Learn: Troubleshoot Azure Monitor Agent on Linux virtual machines
• Microsoft Learn: Azure Monitor Agent overview
• Microsoft Learn: Data collection rules in Azure Monitor
• Microsoft Learn: Heartbeat table reference