App Service Plan Scale-Out Timing: Instance Warmup, In-Flight Request Handling, and State Retention

Status: Published

Experiment completed with real data on 2026-05-04.

1. Question

When an App Service plan scales out from 1 to 2 instances via a manual or autoscale trigger, how long does it take before the new instance starts receiving traffic, and what happens to in-flight requests and instance-local state during the scale-out window?

2. Why this matters

Scale-out adds capacity but introduces a warmup period before the new instance is healthy and traffic-ready. During this window, all traffic continues on the original instance. Once the new instance becomes active, the load balancer begins distributing requests — potentially routing clients to an instance that has no warm cache, no in-memory session, and no pre-initialized connection pools. The timing of this transition is not configurable and is not visible in real-time metrics without polling. Support engineers need to understand the expected scale-out duration to distinguish between a platform issue and expected warmup behavior.

3. Customer symptom

"We scaled out but the new instance doesn't seem to be taking traffic" or "After scaling, some users started getting slower responses" or "The instance count shows 2 but all traffic is still on one instance."

4. Hypothesis

• H1: After triggering a scale-out from 1 to 2 instances on B1 plan, the new instance begins receiving traffic within 60–120 seconds.
• H2: The new instance is detectable by observing two distinct hostnames in HTTP responses (each instance returns its own container hostname).
• H3: During the scale-out window (before the new instance is ready), all traffic is served by the original instance — no requests are dropped.

5. Environment

Parameter	Value
Service	Azure App Service
SKU / Plan	B1
Region	Korea Central
Runtime	Python 3.11 / gunicorn
OS	Linux
App name	app-batch-1777849901
Date tested	2026-05-04

6. Variables

Experiment type: Performance / Platform behavior

Controlled:

• Manual scale: az appservice plan update --number-of-workers 2
• Polling: 5 requests every 5 seconds, detecting unique hostname count
• Measurement: milliseconds from scale command to first 2-hostname detection

Observed:

• Time from scale command to first traffic on new instance
• HTTP success rate during scale-out window
• Hostname values identifying each instance

7. Instrumentation

• /worker endpoint returning {"hostname": "<container-id>", "pid": <n>} — per-instance identity
• date +%s%3N millisecond timestamps around scale command and detection
• Poll loop: 5 requests per iteration, sort -u to count unique hostnames

8. Procedure

1. Confirm 1 instance (all requests return same hostname).
2. Record SCALE_START timestamp. Issue az appservice plan update --number-of-workers 2.
3. Every 5 seconds: send 5 requests, collect hostnames, count unique values.
4. Record SCALE_END when unique count reaches 2.
5. Scale back to 1 instance.

9. Expected signal

• Scale-out completes within 60–120 seconds for B1 plan.
• All requests return HTTP 200 during scale-out window.
• Once 2 hostnames appear, distribution is approximately 50/50 round-robin.

10. Results

```text t=0ms: command issued: az appservice plan update --number-of-workers 2

t=497ms: hostnames=[ce2f97f05a4f x5] unique=1 t=6096ms: hostnames=[ce2f97f05a4f x5] unique=1 t=11705ms: hostnames=[ce2f97f05a4f x5] unique=1 t=17172ms: hostnames=[ce2f97f05a4f x5] unique=1 t=22603ms: hostnames=[ce2f97f05a4f x5] unique=1 t=28110ms: hostnames=[ce2f97f05a4f x5] unique=1 t=31589ms: hostnames=[ce2f97f05a4f x5] unique=1 t=37338ms: hostnames=[ce2f97f05a4f x5] unique=1 t=43195ms: hostnames=[ce2f97f05a4f x5] unique=1 t=48842ms: hostnames=[ce2f97f05a4f x5] unique=1 t=54370ms: hostnames=[ce2f97f05a4f x5] unique=1 t=59866ms: hostnames=[ce2f97f05a4f x5] unique=1

t=63948ms: hostnames=[ce2f97f05a4f x3, 0ee27d0fe65f x2] unique=2 ✓ NEW INSTANCE DETECTED: hostname=0ee27d0fe65f at t=63,948ms

```text Scale-out duration: ~64 seconds (from command to first traffic on new instance)

All 65+ polling requests during the window returned HTTP 200. No errors during scale-out.

11. Interpretation

• Measured: B1 plan scale-out from 1 to 2 instances takes approximately 64 seconds before the new instance begins receiving traffic. H1 is confirmed (within the 60–120s estimate). Measured.
• Measured: The new instance is detectable via the hostname field in the response — each container has a unique hostname (ce2f97f05a4f vs 0ee27d0fe65f). H2 is confirmed.
• Measured: All requests during the 64-second scale-out window returned HTTP 200. No request failures during scale-out. H3 is confirmed.
• Observed: For the first 60 seconds after the scale command, 100% of traffic went to the original instance. The new instance appeared abruptly at t=63,948ms, immediately receiving ~40% of traffic (2 of 5 polling requests).
• Inferred: The 64-second window includes: VM allocation, container image pull (or cache hit), gunicorn startup, and App Service health check verification. The image was likely cached (containerapps-helloworld would not be in cache here — this is App Service with a custom Python image).

12. What this proves

• B1 plan scale-out takes approximately 64 seconds from command to traffic on the new instance. Measured.
• No request failures occur during the scale-out window — traffic continues on the original instance. Measured.
• New instances are immediately visible via the container hostname in the HTTP response. Observed.

13. What this does NOT prove

• Scale-out triggered by autoscale rules (CPU/memory threshold) was not tested — timing may differ.
• Scale-in behavior (removing an instance) was not measured. In-flight requests on the removed instance may fail.
• P1v3, P2v3, or Premium V3 plan scale-out timing was not tested. Larger SKUs with more resources may start faster.
• The warmup period for a cold instance (connection pool initialization, cache warming) was not measured separately from the instance startup time.

14. Support takeaway

When customers report that scale-out doesn't seem to be working:

1. Scale-out takes ~60 seconds for B1 plan. If checking within 30 seconds, the new instance may not be ready yet.
2. To verify scale-out success: poll the app's response for a hostname or instance identifier field. Two distinct values confirm 2 active instances.
3. In-flight request failures during scale-out are not expected — the platform keeps the original instance serving until the new one is healthy.
4. If requests are slower after scale-out, the new instance may be cold (connection pools not warmed, application caches empty). This is expected for the first few requests.

15. Reproduction notes

```bash PLAN_ID="/subscriptions/<sub>/resourceGroups//providers/Microsoft.Web/serverfarms/"</sub>

Scale out

SCALE_START=$(date +%s%3N) az appservice plan update --ids $PLAN_ID --number-of-workers 2

Poll for 2nd instance

while true; do HOSTS=$(for i in 1 2 3 4 5; do curl -s https://.azurewebsites.net/worker | python3 -c "import sys,json; print(json.load(sys.stdin)['hostname'][:12])" done) UNIQUE=$(echo "$HOSTS" | sort -u | wc -l) ELAPSED=$(( $(date +%s%3N) - SCALE_START )) echo "t=${ELAPSED}ms: unique=$UNIQUE" [ "$UNIQUE" -ge 2 ] && echo "Scale-out complete at ${ELAPSED}ms" && break sleep 5 done

Scale back

az appservice plan update --ids $PLAN_ID --number-of-workers 1

```text

16. Related guide / official docs

• Scale up an app in Azure App Service
• Autoscale in Azure App Service
• App Service instance warmup