Windows Slot-Swap First-Request Latency¶

1. Summary¶

Symptom¶

A Windows App Service slot swap succeeds, yet the first request to a business route is slow. Later requests to the same route are fast. The worker PID might be unchanged.

Do not collapse this symptom into “the app recycled.” Investigate three independent questions:

1. Did the worker process or instance change?
2. Did the effective runtime configuration change?
3. Was this particular business route, including its dependencies, exercised during warm-up?

Decision flow¶

flowchart TD
    A[First business request is slow after successful swap] --> B[Build one timestamped swap and request timeline]
    B --> C{PID, process start time, or instance changed?}
    C -->|Yes| D[Evaluate recycle or instance replacement evidence]
    C -->|No| E[Keep recycle unproven; evaluate route-local initialization]
    D --> F{Effective configuration or deployment changed?}
    E --> F
    F -->|Yes| G[Compare changed values with slow code and dependency path]
    F -->|No| H[Compare root warm-up with safe representative route]
    G --> H
    H --> I{Safe route was warmed and dependency state was equivalent?}
    I -->|No| J[Route or dependency cold-state hypothesis supported]
    I -->|Yes| K[Inspect request traces and competing causes]
    J --> L[Conclude only with discriminating evidence]
    K --> L

Scope and conclusion rule¶

This playbook applies to native Windows App Service workloads behind IIS. It does not transfer Windows web.config or application-initialization assumptions to Linux. A successful swap proves that the documented swap gate completed; it does not prove that every application route or downstream dependency is warm.

If PID/start time, instance identity, effective configuration, route coverage, and dependency timings cannot distinguish the hypotheses, report INCONCLUSIVE.

2. Common Misreadings¶

• “Slow first request means a recycle occurred.” An unchanged PID and start time weaken that claim; lazy route initialization can occur in an existing process.
• “Successful swap means every route is warm.” Microsoft Learn documents root-path fallback and configurable initialization URLs, not automatic execution of every business route.
• “Same PID proves nothing changed.” Effective settings, route-local caches, credentials, connections, or external dependency state can still differ.
• “A fast health endpoint represents checkout/reporting/search.” A shallow endpoint may not execute the slow route's middleware, query compilation, cache fill, or connection setup.
• “The nearest event caused the latency.” Temporal adjacency prioritizes investigation but does not establish causation.

3. Competing Hypotheses¶

H1 and H2 are different: a PID can repeat on a different instance, and an instance can host a newly started process. Record both dimensions.

4. What to Check First¶

1. Preserve the Activity Log event for Swap Web App Slots, including start/end time, status, correlation ID, source slot, and target slot.
2. Capture PID, process start time, and instance identity before the controlled swap and immediately before each probe after cutover. Use an authenticated diagnostic endpoint or Kudu Process Explorer; do not expose this data publicly.
3. Export effective app settings, connection-string names and slot-stickiness, runtime/bitness, deployed commit/package identity, and effective web.config warm-up entries for both slots. Redact values and secrets.
4. Identify the root/readiness path and one representative read-only, synthetic, or isolated business endpoint. Never replay a state-changing production POST, payment, order, message, or mutation merely to warm it.
5. Capture request and dependency timings for cold and repeated invocations. Use the same instance-routing conditions when your application safely supports that comparison.

Documented, observed, and not proven¶

What is not documented¶

5. Evidence to Collect¶

Before/after evidence worksheet¶

The instance marker must not expose the raw instance identifier to unauthenticated clients. Hash it, restrict the endpoint, or collect it through existing telemetry.

Activity Log and configuration collection¶

az monitor activity-log list --resource-group <resource-group> --start-time <incident-start-utc> --end-time <incident-end-utc> --query "[?contains(operationName.value, 'Microsoft.Web/sites/slots/slotsswap/action')]" --output json
az webapp config appsettings list --resource-group <resource-group> --name <app-name> --slot <slot-name> --output json
az webapp config show --resource-group <resource-group> --name <app-name> --slot <slot-name> --output json

Store exports in an access-controlled incident location. App-setting output can contain secrets; produce a redacted name/slot-setting/hash comparison before sharing.

Request comparison¶

Use a preproduction slot or approved isolated test identity:

1. Record PID/start time and instance marker.
2. Request / or the configured readiness route once.
3. Request a representative safe GET such as /diagnostics/read-model-warmup twice with unique correlation IDs.
4. Compare route elapsed time and dependency spans across attempt 1 and attempt 2.
5. Repeat only if the workload, endpoint, and rate are approved. Never convert a state-changing business operation into an ad hoc warm-up call.

KQL: request timing by instance and route¶

The query uses only columns in the current AppServiceHTTPLogs Learn reference. ComputerName is the server that generated the entry; it is useful instance evidence but is not a PID.

let incidentStart = datetime(<incident-start-utc>);
let incidentEnd = datetime(<incident-end-utc>);
AppServiceHTTPLogs
| where TimeGenerated between (incidentStart .. incidentEnd)
| where CsMethod == "GET"
| where CsUriStem in ("/", "/health", "/diagnostics/read-model-warmup")
| project TimeGenerated, ComputerName, CsMethod, CsUriStem, ScStatus, TimeTaken
| order by TimeGenerated asc

Sanitized synthetic example output (not observed):

This synthetic pattern shows stable server identity, fast root warm-up, and a slower first safe business-route probe. It supports investigating route-local or dependency cold state; it does not prove that no process recycle occurred because AppServiceHTTPLogs has no PID or process-start column.

KQL: platform-message timeline¶

AppServicePlatformLogs documents TimeGenerated, Host, OperationName, Level, Message, DeploymentId, and ActivityId. Message text varies, so use this as a candidate-event search, not proof that every recycle is represented.

let incidentStart = datetime(<incident-start-utc>);
let incidentEnd = datetime(<incident-end-utc>);
AppServicePlatformLogs
| where TimeGenerated between (incidentStart .. incidentEnd)
| where Message has_any ("swap", "restart", "recycle", "worker")
    or OperationName has_any ("swap", "restart", "recycle")
| project TimeGenerated, Host, OperationName, Level, Message, DeploymentId, ActivityId
| order by TimeGenerated asc

Windows IIS and dependency evidence¶

• Treat AppServiceHTTPLogs as the documented Azure Monitor request-log table for IIS-style request fields.
• If filesystem IIS logs are enabled, preserve the raw files and document their schema separately; do not relabel them as an AppServiceIISLogs table.
• Capture application traces with correlation ID, route phase, and dependency duration. Azure Monitor request logs do not contain downstream dependency spans.
• Capture PID/start time from Kudu Process Explorer, an authenticated diagnostic endpoint, or existing APM process metadata; neither KQL table above documents a PID column.

Reproduction/test record¶

6. Validation and Disproof by Hypothesis¶

[Observed] Record only evidence captured from the investigated environment.

[Inferred] A changed PID plus startup traces supports recycling, but still does not prove why the recycle occurred.

[Not Proven] Swap success, temporal proximity, and one slow request alone prove none of H1-H6.

7. Likely Root Cause Patterns¶

• Default / warm-up succeeds, but the first business route initializes a separate code path or cache.
• Process and instance remain stable while the first dependency connection, token, DNS lookup, or query compilation dominates latency.
• Slot-specific configuration differs and selects a cold dependency or feature path after cutover.
• Swap preparation or another event starts a new process, and application startup does not fully initialize the business route.
• Traffic reaches another instance whose application-local state is cold.

Do not label a pattern as the root cause until its discriminating evidence is present.

8. Immediate Mitigations¶

• If impact is severe and the previous slot is known good, follow the documented immediate reverse-swap procedure after preserving evidence.
• Add or repair only an approved, idempotent, read-only initialization route that exercises the required dependencies without changing business state.
• Use swap with preview to validate effective target-slot settings and safe endpoints before cutover.
• Temporarily reduce optional first-use work or move it to controlled startup/background initialization with explicit readiness criteria.
• Roll back a configuration change only when you have captured the before state and have a tested restoration path.

9. Prevention¶

• Publish a warm-up contract listing every configured path, expected status, dependency coverage, side-effect policy, and timeout budget.
• Include PID/start time, deployment ID, safe instance marker, route timing, and dependency spans in incident telemetry.
• Test root-only versus safe representative-route warm-up in a nonproduction slot.
• Keep all warm-up endpoints idempotent, authenticated or network-restricted where appropriate, and incapable of creating orders, payments, messages, or other business mutations.
• Alert separately on worker changes, instance changes, configuration drift, and first-route latency instead of using one “cold start” label.

See Also¶

• Windows Slot-Swap Binding Drift and Worker Recycle
• Deployment Slots Operations
• Windows IIS web.config Startup Failures
• Windows Kudu and Diagnostic Tools
• Linux Slot Swap Restart / Config Drift / Warm-up Race
• Slot Swap Failed During Warm-up

Sources¶

• Set up staging environments in Azure App Service
• IIS application initialization add element
• AppServiceHTTPLogs table reference
• AppServicePlatformLogs table reference
• Microsoft.Web/sites Azure Monitor table catalog