Measuring Cold Start Impact on TTFB

This guide is part of Edge Runtime Fundamentals & Platform Constraints. It shows how to answer one question with data rather than intuition: of the milliseconds a user waits before the first byte arrives, how many are attributable to a cold isolate?

Almost every cold-start investigation goes wrong in the same way. Someone sees a p99 TTFB of 900 ms, assumes cold starts, adds a warming cron, and the p99 does not move — because the slow requests were slow origin fetches from a region with no cache, and the cold isolates were never the expensive part. The measurement below is cheap, needs no vendor tooling, and tells you within a day whether cold starts are worth engineering effort at all.

The problem

TTFB is a composite. It contains DNS resolution, TLS negotiation, the network leg to the nearest PoP, whatever the edge does, and — if the edge forwards — a full round trip to your origin, which may be thousands of kilometres away. Cold-start cost is one slice of that, and it is not usually the biggest one.

Worse, the slice is invisible from outside. A synthetic check from a monitoring service measures a warm isolate almost every time, because the check itself keeps the isolate alive. Real users hit cold isolates in exactly the situations you cannot reproduce: the first request into a PoP after a quiet period, the first request after a deploy, the first request from a region you have no synthetic probe in.

Root cause: the isolate lifecycle is invisible to the request

An edge runtime evaluates your module once per isolate, then reuses that isolate for as many requests as arrive before it is evicted. The evaluation cost — parsing the bundle, running top-level code, building any module-scoped caches — is paid by whichever unlucky request triggers the boot. Nothing in the Request object tells the handler whether it is that request.

Three consequences follow. First, the only entity that knows an isolate is new is module scope itself, because module scope runs exactly once per isolate; anything you want to know about isolate identity or age must be captured there. Second, cold and warm requests are not two points on one distribution, they are two distributions, so any single summary statistic mixing them is meaningless. Third, the cold rate is a property of your traffic pattern and the platform’s eviction policy, not of your code — which is why the useful alert is on the fraction of requests that are cold, not on latency.

Step 1 — Capture isolate identity and birth time in module scope

Module scope runs once per isolate. Two values captured there give you everything: an identifier that is stable for the isolate’s lifetime, and the timestamp at which evaluation finished.

// lib/isolate.ts — evaluated exactly once per isolate, on the cold request.
const BOOT_AT = Date.now();
const BOOT_ID = crypto.randomUUID();

let servedCount = 0;

export interface IsolateMarker {
  bootId: string;
  /** Milliseconds since this isolate finished evaluating its modules. */
  ageMs: number;
  /** 1 on the request that booted the isolate, incrementing thereafter. */
  requestIndex: number;
  /** True only for the first request an isolate ever serves. */
  cold: boolean;
}

export function markIsolate(): IsolateMarker {
  const requestIndex = ++servedCount;
  return {
    bootId: BOOT_ID,
    ageMs: Date.now() - BOOT_AT,
    requestIndex,
    cold: requestIndex === 1,
  };
}

crypto.randomUUID is Web Crypto, available in every edge runtime, so no Node import is involved. The counter is a plain module-level variable, which is safe here precisely because isolates are single-threaded per request context; there is no shared mutable state across isolates to race on.

Two subtleties are worth stating. Date.now() at the edge is often coarse — several runtimes clamp timer resolution to a millisecond or freeze the clock within a synchronous block as a side-channel defence — so treat ageMs as an ordering and bucketing signal, not as a microbenchmark. And requestIndex === 1 is the honest definition of cold: it is the request that paid for evaluation. A request arriving 5 ms later on the same isolate is warm even though the isolate is nearly new.

Step 2 — Attribute the segments so cold cost is separable

The marker tells you whether a request was cold. To know what that cost, you also need the parts of the handler you can time, and you need the browser or client to be able to see them. Server-Timing is the standard channel: it survives to the browser’s performance entries and to most log pipelines.

// middleware.ts
import { markIsolate } from "./lib/isolate";

export default async function middleware(request: Request): Promise<Response> {
  const handlerStart = Date.now();
  const iso = markIsolate();

  const originStart = Date.now();
  const upstream = await fetch(request);
  const originMs = Date.now() - originStart;

  const handlerMs = Date.now() - handlerStart;
  const headers = new Headers(upstream.headers);

  headers.set(
    "server-timing",
    [
      `iso;desc="${iso.cold ? "cold" : "warm"}"`,
      `isoage;dur=${iso.ageMs}`,
      `origin;dur=${originMs}`,
      `mw;dur=${handlerMs - originMs}`,
    ].join(", "),
  );
  headers.set("x-isolate-boot", iso.bootId);

  return new Response(upstream.body, {
    status: upstream.status,
    headers,
  });
}

The arithmetic is deliberately simple: mw is handler time minus origin time, which is the work your middleware did on the CPU. What it does not include is module evaluation, because by the time the handler runs, evaluation has already happened. That is the point of the boot marker — you cannot time module evaluation from inside the handler, so you infer its cost statistically by comparing the cold population against the warm one.

Do not try to compute a per-request “cold start duration” from inside the isolate. It is not observable there. The difference between the cold and warm distributions is the cost, and it is only meaningful in aggregate.

Step 3 — Emit one structured line per request

The distribution is computed from logs, so the log line must carry every dimension you will want to slice by. Emit one JSON object, one line, no nesting beyond a single level — nested objects make most log query languages awkward.

// lib/log.ts
export interface RequestLog {
  event: "edge.request";
  bootId: string;
  cold: boolean;
  isoAgeMs: number;
  reqIndex: number;
  originMs: number;
  mwMs: number;
  status: number;
  colo: string;
  route: string;
}

export function logRequest(entry: RequestLog): void {
  // One line, one object. Sampling belongs here, not at the collector.
  console.log(JSON.stringify(entry));
}

export function routeOf(url: string): string {
  const { pathname } = new URL(url);
  // Collapse identifiers so the route dimension has bounded cardinality.
  return pathname
    .replace(/\/[0-9a-f]{8,}\b/gi, "/:id")
    .replace(/\/\d+\b/g, "/:id")
    .slice(0, 64);
}

Route normalization is not cosmetic. If route carries raw paths, the cold population fragments across thousands of distinct values and you will never accumulate enough cold samples in any single bucket to compute a percentile. Bounded cardinality is what makes the analysis in Step 4 possible.

Sample warm requests if volume is a concern — but never sample cold ones. Cold requests are rare by construction, and dropping them at the same rate as warm requests destroys the very population you are trying to measure. A rule of “log every cold request, log one in fifty warm” keeps volume manageable while preserving the comparison.

Step 4 — Build the cold-versus-warm distribution

With cold as a boolean dimension, the analysis is two percentile computations over the same latency field. The shape you are looking for is two clearly separated distributions, not one distribution with a tail.

// analyze.ts — run over exported logs; the same maths your query language does.
interface Sample { cold: boolean; ttfbMs: number }

function quantile(sorted: number[], q: number): number {
  if (sorted.length === 0) return NaN;
  const pos = (sorted.length - 1) * q;
  const lo = Math.floor(pos);
  const hi = Math.ceil(pos);
  return lo === hi ? sorted[lo] : sorted[lo] + (sorted[hi] - sorted[lo]) * (pos - lo);
}

export function summarize(samples: Sample[]) {
  const cold = samples.filter((s) => s.cold).map((s) => s.ttfbMs).sort((a, b) => a - b);
  const warm = samples.filter((s) => !s.cold).map((s) => s.ttfbMs).sort((a, b) => a - b);
  const coldFraction = cold.length / Math.max(samples.length, 1);

  return {
    coldFraction,
    cold: { n: cold.length, p50: quantile(cold, 0.5), p95: quantile(cold, 0.95) },
    warm: { n: warm.length, p50: quantile(warm, 0.5), p95: quantile(warm, 0.95) },
    // The honest headline number: how much slower a cold request is at the median.
    bootCostP50: quantile(cold, 0.5) - quantile(warm, 0.5),
    // What cold starts contribute to the overall average, in milliseconds.
    blendedPenaltyMs: coldFraction * (quantile(cold, 0.5) - quantile(warm, 0.5)),
  };
}

bootCostP50 is the number to quote. Comparing medians rather than means is not pedantry: the mean of the cold population is dragged around by a handful of requests that also happened to miss cache or hit a slow origin, and with a small cold sample count a single outlier can move the mean by tens of milliseconds. The median of each population is stable at sample sizes you will realistically collect.

Percentiles matter more than averages here for a second, sharper reason. If two percent of requests are cold and cold requests cost 180 ms more, the average rises by 3.6 ms — invisible, and easy to dismiss. But those two percent are entirely concentrated in the tail, so p99 can move by the full 180 ms. Averages hide cold starts almost perfectly; tail percentiles expose them. Any dashboard built on means will tell you cold starts do not matter, right up until users complain.

Step 5 — Alert on the cold fraction, not on latency

Absolute latency is the wrong alert target for this. It is dominated by origin behaviour and network conditions you do not control, so it fires for reasons unrelated to isolates, and it stays quiet when the cold rate doubles but origin happens to be fast that hour.

The cold fraction is the right target because it is a direct, low-variance measure of the thing you can act on. It rises when traffic to a PoP thins out, when you deploy (every deploy invalidates every isolate), when your bundle grows enough to slow evaluation, or when a route’s matcher starts spawning isolates that serve one request each.

// alert.ts — evaluate against a rolling window of summarize() output.
interface Window { coldFraction: number; bootCostP50: number; samples: number }

export type Alert =
  | { fire: false }
  | { fire: true; reason: "cold_rate" | "boot_cost"; detail: string };

export function evaluate(w: Window, baselineColdFraction: number): Alert {
  // Never alert on a window too small to be meaningful.
  if (w.samples < 500) return { fire: false };

  // Deploys reset every isolate, so allow a generous multiple, not a fixed number.
  if (w.coldFraction > Math.max(baselineColdFraction * 3, 0.02)) {
    return {
      fire: true,
      reason: "cold_rate",
      detail: `cold fraction ${(w.coldFraction * 100).toFixed(2)}% vs baseline ${(baselineColdFraction * 100).toFixed(2)}%`,
    };
  }

  // A rising per-boot cost means the bundle or top-level work grew.
  if (w.bootCostP50 > 150) {
    return { fire: true, reason: "boot_cost", detail: `boot cost p50 ${w.bootCostP50.toFixed(0)} ms` };
  }

  return { fire: false };
}

Two thresholds, two different meanings. A cold-fraction alert says “traffic shape or isolate lifetime changed” and usually resolves itself or points at routing. A boot-cost alert says “module evaluation got more expensive”, which is nearly always a dependency someone added — the same failure mode that tree-shaking dependencies for edge middleware addresses. Suppress the cold-rate alert for a fixed window after each deploy, or it will fire on every release by design.

Matcher and platform configuration

Instrumenting assets pollutes the sample with requests that never had a meaningful origin leg, so scope the middleware before you scope the analysis.

export const config = {
  runtime: "edge",
  matcher: ["/((?!_next/static|_next/image|favicon.ico|healthz).*)"],
};

On Cloudflare, enable platform observability so the same JSON lines land in a queryable store without you running a collector:

name = "edge-middleware"
main = "src/middleware.ts"
compatibility_date = "2026-01-15"

[observability]
enabled = true
head_sampling_rate = 1.0

Keep head_sampling_rate at 1.0 while you are establishing a baseline. Sampling at the platform level drops cold and warm requests at the same rate, which is exactly the bias Step 3 warned about; do your sampling in code where you can exempt cold requests.

Provider behaviour and cold-start characteristics

The pattern across all three is the same: boot cost scales with how much work happens at module evaluation, not with how much your handler does. A bundle that builds a large lookup table at the top level pays for it on every cold request, forever.

Local versus production divergence

The first two rows make local measurement of cold-start cost actively misleading: locally, everything is cold and the origin is free, so the boot band looks like the entire request. Production is the opposite on both counts.

Validating with Vitest

Test the parts that are deterministic: the marker’s cold/warm classification, the percentile maths, and the alert thresholds. The isolate lifecycle itself is not testable, so isolate it behind a module you can re-import.

// isolate.test.ts
import { describe, expect, it, beforeEach, vi } from "vitest";
import { summarize } from "./analyze";
import { evaluate } from "./alert";

describe("markIsolate", () => {
  beforeEach(() => vi.resetModules());

  it("reports cold only for the first request of an isolate", async () => {
    const { markIsolate } = await import("./lib/isolate");
    expect(markIsolate()).toMatchObject({ cold: true, requestIndex: 1 });
    expect(markIsolate()).toMatchObject({ cold: false, requestIndex: 2 });
    expect(markIsolate()).toMatchObject({ cold: false, requestIndex: 3 });
  });

  it("issues a fresh boot id per isolate", async () => {
    const first = (await import("./lib/isolate")).markIsolate().bootId;
    vi.resetModules();
    const second = (await import("./lib/isolate")).markIsolate().bootId;
    expect(second).not.toBe(first);
  });
});

describe("summarize", () => {
  it("separates the two populations and reports the boot cost", () => {
    const samples = [
      ...Array.from({ length: 98 }, (_, i) => ({ cold: false, ttfbMs: 30 + (i % 5) })),
      ...Array.from({ length: 2 }, () => ({ cold: true, ttfbMs: 210 })),
    ];
    const s = summarize(samples);
    expect(s.coldFraction).toBeCloseTo(0.02, 4);
    expect(s.bootCostP50).toBeGreaterThan(170);
    expect(s.blendedPenaltyMs).toBeLessThan(5);
  });
});

describe("evaluate", () => {
  it("stays quiet on windows too small to be meaningful", () => {
    expect(evaluate({ coldFraction: 0.5, bootCostP50: 900, samples: 12 }, 0.005))
      .toEqual({ fire: false });
  });

  it("fires on a tripled cold fraction", () => {
    const a = evaluate({ coldFraction: 0.03, bootCostP50: 40, samples: 5000 }, 0.005);
    expect(a).toMatchObject({ fire: true, reason: "cold_rate" });
  });
});

vi.resetModules() is the trick that makes the marker testable at all: re-importing the module simulates a fresh isolate, because module scope is re-evaluated. The blendedPenaltyMs assertion encodes the central insight in a test — a 180 ms boot cost at a two percent cold rate is under 5 ms of average, which is why the average is the wrong metric.

Common pitfalls and resolutions

Cold fraction reads as zero — the marker is being created inside the handler instead of at module scope, so every request looks like request one, or none does. BOOT_AT and BOOT_ID must be top-level const declarations.

Cold p95 is noisier than warm p95 — you have too few cold samples. Stop sampling cold requests, widen the window, and compare medians rather than p95 until the count is in the hundreds.

Cold starts appear to cost seconds — you are measuring TTFB including the origin fetch and the cold requests happen to be the ones missing cache. Compare mw and origin from Server-Timing separately before blaming boot.

Every deploy pages the on-call engineer — the cold-rate alert has no deploy suppression. Every deploy invalidates every isolate; mute the rule for a fixed window after each release.

Warm requests show a large isoage but count as warm — that is correct behaviour. Age and coldness are different things; only the first request on an isolate paid for evaluation.

Synthetic monitoring never sees a cold start — the probe keeps the isolate alive. Cold measurement must come from real traffic logs, not from probes.

Production deployment checklist

• BOOT_AT and BOOT_ID are declared at module scope, never inside the handler
• Every response carries Server-Timing with separate origin and middleware durations
• One structured JSON log line per request, one level deep
• Route labels are normalized so cardinality stays bounded
• Cold requests are never sampled away; warm requests may be
• Percentiles are computed per population, and medians are compared, not means
• The primary alert targets cold fraction against a rolling baseline
• A second alert targets boot cost, catching bundle growth
• Cold-rate alerting is suppressed for a window after each deploy
• Alerts require a minimum sample count before firing

Frequently Asked Questions

Related

• Managing cold starts in serverless environments
• How to debug cold start latency on Vercel
• Structured logging for edge functions
• Tree-shaking dependencies for edge middleware
• Instrumenting edge middleware with OpenTelemetry