Reducing Memory Usage in Edge Middleware

This guide is part of Memory and CPU Limits Across Edge Providers. It shows how to keep an edge middleware function inside a 128 MB isolate heap that it does not own exclusively, by changing five specific habits carried over from Node.

The problem

Your middleware works. It has worked for weeks. Then traffic doubles on a Tuesday afternoon and a fraction of requests start returning 1102 Worker exceeded memory limit on Cloudflare, or a bare 500 with EDGE_FUNCTION_INVOCATION_FAILED on Vercel, or a Deno OutOfMemory trace in the Netlify logs. The failures are not evenly spread: some PoPs are fine, one region is failing constantly, and the request that fails is usually not the request that caused the problem.

That last detail is the whole story. Memory exhaustion at the edge is almost never one request allocating too much. It is one isolate accumulating too much across many requests, and then an ordinary request arriving to find no room left. Debugging the failing request tells you nothing, because the failing request is a victim.

Root cause: the heap holds your module graph, and every concurrent request shares it

An edge function does not get a process. It gets an isolate — a sandboxed heap inside a runtime that is hosting many other isolates on the same machine. On Cloudflare Workers and the Vercel Edge Runtime that heap is capped at roughly 128 MB. Netlify Edge Functions run on Deno Deploy with a more generous 512 MB, but the same structural facts apply.

Three of those facts explain nearly every out-of-memory incident.

The module graph is charged to your budget. Everything your bundle imports is parsed, compiled and instantiated into that same 128 MB. A 900 KB bundle is not 900 KB of heap; once parsed into functions, closures, string tables and any top-level data structures it constructs, it can easily be several megabytes. If you ship a country-to-locale table, a compiled route list, or a JSON blob imported at the top level, it is resident for the isolate’s entire life. Trimming what you import — see tree-shaking dependencies for edge middleware — is a memory optimisation, not just a cold-start one.

One isolate serves many concurrent requests. The runtime does not spin up a fresh isolate per request; it multiplexes. If ten requests are in flight and each buffers a 6 MB upload into an ArrayBuffer, that is 60 MB of transient heap on top of your module graph, and the eleventh request is the one that dies. Your per-request working set must be multiplied by peak concurrency before you compare it to the limit.

Nothing in module scope is ever collected. Module-scope state lives for the isolate’s lifetime, which can be minutes to hours on a busy PoP. A Map you populate on each request and never prune is not a cache; it is a leak with a friendly name. It grows monotonically until the isolate is evicted or killed, which is why the symptom appears hours after the deploy and only on high-traffic PoPs.

Step 1 — Account for the module graph before touching request code

Start by measuring what the isolate costs when it is doing nothing. Deploy a middleware that returns immediately, note the reported memory, then add your real imports back one at a time. On Cloudflare, wrangler dev --inspect exposes a Chrome DevTools endpoint where you can take a heap snapshot of a freshly booted isolate; the retained-size column tells you exactly which top-level structure is expensive.

The usual offenders are data, not code:

// middleware/tables.ts — BAD: 1.4 MB of JSON parsed at module scope,
// resident for the entire isolate lifetime, used by 2% of requests.
import geoTable from './geoip-cities.json';

export function cityFor(ip: string) {
  return geoTable[ip] ?? null;
}

// middleware/tables.ts — GOOD: keep only what every request needs in memory
// and move the rare, large lookup behind a KV read.
const COUNTRY_TO_LOCALE: Record<string, string> = {
  DE: 'de-DE', FR: 'fr-FR', JP: 'ja-JP', US: 'en-US',
};

export function localeFor(country: string): string {
  return COUNTRY_TO_LOCALE[country] ?? 'en-US';
}

export async function cityFor(ip: string, kv: KVNamespace): Promise<string | null> {
  return kv.get(`geo:${ip}`);
}

A 40-entry object costs a few kilobytes. A 1.4 MB JSON import costs several megabytes once parsed into V8 objects, permanently, on every isolate in every PoP. Reach for KV for anything large and sparsely accessed.

Step 2 — Stream bodies instead of buffering them

await request.text(), await request.json() and await request.arrayBuffer() all do the same thing to your heap: they hold the entire body in memory, and for text() and json() they hold it twice for a moment — once as bytes, once as the decoded result. On a 20 MB upload that is 40-60 MB of transient allocation on a 128 MB budget.

Middleware rarely needs the whole body. If you are inspecting a header, rewriting a URL, or checking a signature over a prefix, pass the body through untouched:

// middleware.ts — the body is never read into the heap; the runtime
// pipes it from client to origin as it arrives.
export default async function middleware(request: Request): Promise<Response> {
  const headers = new Headers(request.headers);
  headers.set('x-request-id', crypto.randomUUID());

  return fetch(
    new Request(request.url, {
      method: request.method,
      headers,
      body: request.body,      // ReadableStream, not bytes
      // @ts-expect-error duplex is required by the spec when body is a stream
      duplex: 'half',
    }),
  );
}

When you genuinely must look inside the body — scanning an upload for a forbidden token, say — do it chunk by chunk with a bounded window rather than materialising the whole thing:

// Scan a streaming body for a marker without ever holding more than
// one chunk plus a small overlap window in memory.
const OVERLAP = 64;

export async function bodyContains(
  body: ReadableStream<Uint8Array>,
  marker: string,
  maxBytes = 8 * 1024 * 1024,
): Promise<boolean> {
  const reader = body.getReader();
  const decoder = new TextDecoder();
  let carry = '';
  let seen = 0;

  try {
    while (true) {
      const { done, value } = await reader.read();
      if (done) return false;

      seen += value.byteLength;
      if (seen > maxBytes) return false;              // refuse, do not grow

      const window = carry + decoder.decode(value, { stream: true });
      if (window.includes(marker)) return true;

      // Keep only enough tail to catch a marker split across chunks.
      carry = window.slice(-OVERLAP);
    }
  } finally {
    reader.releaseLock();
    await body.cancel().catch(() => {});
  }
}

The maxBytes guard is not optional. Without it, an attacker chooses your memory usage by choosing their upload size. See using Web Streams instead of Node streams at the edge for the wider streaming model, and transforming HTML with HTMLRewriter at the edge for the response side.

Step 3 — Never materialise a large iterable

Array.from(...), spread syntax and [...headers] all allocate a dense array holding every element at once. On small collections this is invisible. On a large Headers object, a KV list result, or a decoded body it doubles your footprint for no benefit, because the very next line usually reduces it to a single value.

// BAD: allocates a full array of entries, then throws it away.
const total = Array.from(entries).filter((e) => e.active).length;

// GOOD: one counter, constant memory, one pass.
let total = 0;
for (const e of entries) if (e.active) total++;

The same applies to string building. Concatenating in a loop produces intermediate strings that survive until the next collection; joining a pre-sized array is usually better, but iterating and writing directly into a TransformStream is better still when the output is a response body. And avoid structuredClone on request-derived data — it deep-copies, doubling whatever you hand it.

One case deserves a specific warning. new Response(await res.text()) inside a transformation step buffers the entire upstream response before you emit a byte. If all you are doing is rewriting a header, construct the response from the existing stream:

const upstream = await fetch(request);
const headers = new Headers(upstream.headers);
headers.set('x-cache-tier', 'edge');

// upstream.body is passed through untouched — zero body copies.
return new Response(upstream.body, {
  status: upstream.status,
  statusText: upstream.statusText,
  headers,
});

Step 4 — Bound every module-scope cache

The unbounded Map is the classic edge memory leak, and it looks completely reasonable in review:

// The leak. Nothing here is wrong except that it never forgets anything.
const sessionCache = new Map<string, Session>();

export function remember(id: string, session: Session) {
  sessionCache.set(id, session);
}

Every distinct session id adds an entry. Entries are never removed. On a PoP handling a hundred thousand distinct users a day, in an isolate that lives for two hours, this reaches hundreds of megabytes. Because the isolate is recycled periodically, the leak appears to “fix itself” — until traffic grows enough that the isolate dies before it is recycled.

Bound it with a size cap and a TTL. A small insertion-ordered LRU is about fifteen lines and removes the entire failure mode:

// lib/bounded-cache.ts
interface Entry<V> { value: V; expiresAt: number }

export class BoundedCache<V> {
  private readonly map = new Map<string, Entry<V>>();

  constructor(
    private readonly maxEntries = 500,
    private readonly ttlMs = 60_000,
  ) {}

  get(key: string): V | undefined {
    const hit = this.map.get(key);
    if (!hit) return undefined;
    if (hit.expiresAt <= Date.now()) {
      this.map.delete(key);
      return undefined;
    }
    // Re-insert so Map iteration order reflects recency.
    this.map.delete(key);
    this.map.set(key, hit);
    return hit.value;
  }

  set(key: string, value: V): void {
    if (this.map.has(key)) this.map.delete(key);
    this.map.set(key, { value, expiresAt: Date.now() + this.ttlMs });

    while (this.map.size > this.maxEntries) {
      // Map keys iterate in insertion order, so the first is the coldest.
      const oldest = this.map.keys().next().value as string | undefined;
      if (oldest === undefined) break;
      this.map.delete(oldest);
    }
  }

  get size(): number {
    return this.map.size;
  }
}

export const sessionCache = new BoundedCache<Session>(500, 60_000);

Pick maxEntries from arithmetic, not intuition: measure a serialised entry, multiply by the cap, and confirm the product is a small single-digit percentage of the heap limit. Five hundred entries at 2 KB each is 1 MB — safe. Fifty thousand entries at 2 KB is 100 MB — fatal.

Step 5 — Release references before you await

Garbage collection cannot reclaim anything a live variable still points at. An await is exactly where the collector gets a chance to run, and exactly where a forgotten const keeps a large object pinned across a slow network call.

// BAD: `raw` and `parsed` stay reachable for the whole upstream fetch.
const raw = await request.text();          // 6 MB string
const parsed = JSON.parse(raw);            // 8 MB of objects
const userId = parsed.user.id;
const profile = await fetch(`${API}/users/${userId}`);   // 120 ms
return renderWith(profile, userId);

// GOOD: extract the scalar you need, then let the block end so both
// large values become unreachable before the slow await begins.
function extractUserId(payload: string): string {
  const parsed = JSON.parse(payload) as { user: { id: string } };
  return parsed.user.id;
}

const userId = extractUserId(await request.text());
const profile = await fetch(`${API}/users/${userId}`);   // heap already released
return renderWith(profile, userId);

Scoping the large values inside a helper is more reliable than assigning null, because it removes the binding entirely rather than hoping the engine treats the reassignment as a release point. The rule generalises: extract the small thing you need, then make sure the big thing cannot be named any more before you await on anything slow. The same discipline keeps CPU spikes down, which matters for avoiding CPU time limit errors in Cloudflare Workers.

Step 6 — Prove it with a synthetic load test

Memory bugs only show up under concurrency, so a single curl proves nothing. You need sustained parallel traffic with realistic key cardinality, because an unbounded cache keyed on session id only leaks when session ids are diverse.

// scripts/loadtest.ts — runs anywhere with fetch: no Node built-ins.
interface Result { status: number; ms: number }

async function hit(url: string, sessionId: string): Promise<Result> {
  const started = Date.now();
  const res = await fetch(url, {
    headers: { cookie: `sid=${sessionId}` },
  });
  await res.arrayBuffer();                     // drain, so we measure fully
  return { status: res.status, ms: Date.now() - started };
}

export async function run(url: string, concurrency = 50, total = 20_000) {
  const results: { ok: number; failed: number; p95: number } = { ok: 0, failed: 0, p95: 0 };
  const latencies: number[] = [];
  let issued = 0;

  async function worker() {
    while (issued < total) {
      const n = issued++;
      // High key cardinality is the point: a unique session per request
      // is what exposes an unbounded module-scope cache.
      const r = await hit(url, `sess_${n}`);
      if (r.status >= 500) results.failed++;
      else results.ok++;
      latencies.push(r.ms);
    }
  }

  await Promise.all(Array.from({ length: concurrency }, worker));
  latencies.sort((a, b) => a - b);
  results.p95 = latencies[Math.floor(latencies.length * 0.95)] ?? 0;
  return results;
}

Run it against a preview deployment, not production, and watch two signals together: the 5xx count and the p95. A leak shows up first as a slow p95 climb — the isolate is spending more time in garbage collection — and only later as errors. A run that ends with failed: 0 and a flat p95 across 20,000 requests at 50 concurrent is a real pass. Locally, wrangler dev --inspect plus a heap snapshot before and after the run will name the retaining object directly.

# Baseline snapshot, load, then compare in Chrome DevTools -> Memory.
npx wrangler dev --inspect --port 8787 &
npx tsx scripts/loadtest.ts http://127.0.0.1:8787/dashboard

Configuration and matcher scope

The cheapest memory optimisation is not running at all. Every static asset that reaches your middleware allocates a Request, a Headers map and whatever your handler builds before deciding it had nothing to do.

// middleware.ts
export const config = {
  runtime: 'edge',
  matcher: [
    // Skip framework assets, images and public files entirely.
    '/((?!_next/static|_next/image|favicon.ico|robots.txt|assets/).*)',
  ],
};

On Cloudflare the equivalent lever is the route pattern in wrangler.toml, and on Netlify it is the path and excludedPath fields of the config export.

# wrangler.toml — keep the Worker off asset paths entirely.
name = "edge-middleware"
main = "src/index.ts"
compatibility_date = "2026-01-15"

[[routes]]
pattern = "example.com/app/*"
zone_name = "example.com"

Local versus production divergence

The first and fourth rows explain why this class of bug reaches production so reliably: local development actively hides it. Hot reload discards module scope every few seconds, so the leak never has time to form.

Validating with Vitest

You cannot assert on heap bytes portably, but you can assert on the thing that causes the growth — cache size — and on the thing that causes the spike — buffering. Both are deterministic.

// bounded-cache.test.ts
import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
import { BoundedCache } from './lib/bounded-cache';
import { bodyContains } from './lib/scan';

describe('BoundedCache', () => {
  beforeEach(() => vi.useFakeTimers());
  afterEach(() => vi.useRealTimers());

  it('never exceeds maxEntries under unique-key traffic', () => {
    const cache = new BoundedCache<number>(500, 60_000);
    for (let i = 0; i < 50_000; i++) cache.set(`sess_${i}`, i);
    expect(cache.size).toBe(500);
  });

  it('evicts the least recently used entry first', () => {
    const cache = new BoundedCache<string>(2, 60_000);
    cache.set('a', 'A');
    cache.set('b', 'B');
    cache.get('a');                 // 'a' becomes most recent
    cache.set('c', 'C');            // evicts 'b'
    expect(cache.get('b')).toBeUndefined();
    expect(cache.get('a')).toBe('A');
  });

  it('drops entries once the TTL has elapsed', () => {
    const cache = new BoundedCache<string>(10, 60_000);
    cache.set('k', 'v');
    vi.advanceTimersByTime(60_001);
    expect(cache.get('k')).toBeUndefined();
    expect(cache.size).toBe(0);
  });
});

describe('bodyContains', () => {
  function streamOf(chunks: string[]): ReadableStream<Uint8Array> {
    const encoder = new TextEncoder();
    return new ReadableStream({
      start(controller) {
        for (const c of chunks) controller.enqueue(encoder.encode(c));
        controller.close();
      },
    });
  }

  it('finds a marker split across two chunks', async () => {
    const found = await bodyContains(streamOf(['aaaaSE', 'CRETbbbb']), 'SECRET');
    expect(found).toBe(true);
  });

  it('refuses bodies over the byte ceiling instead of growing', async () => {
    const big = 'x'.repeat(1024 * 1024);
    const chunks = Array.from({ length: 12 }, () => big);
    const found = await bodyContains(streamOf(chunks), 'SECRET', 8 * 1024 * 1024);
    expect(found).toBe(false);
  });
});

The first test is the important one. It encodes the invariant that no traffic pattern, however adversarial the key cardinality, can make the cache grow — which is precisely what the unbounded version could not promise. For the broader harness setup see unit testing edge middleware with Vitest.

Common pitfalls and resolutions

Worker exceeded memory limit only on your busiest PoP — module-scope growth. The busiest PoP keeps an isolate alive longest, so it hits the ceiling first. Audit every module-scope Map, Set and array for a bound.

Memory fine in staging, fatal in production — staging has low key cardinality. A leak keyed on user id needs many distinct users to manifest. Re-run the load test with unique keys per request.

p95 latency climbing over hours with no code change — garbage-collection pressure from a growing heap, not a slow upstream. Check retained size before blaming the origin.

Large uploads fail intermittently — a buffered body times its allocation with other in-flight requests. Stream the body and add an explicit byte ceiling.

Adding a dependency pushed you over — the module graph is charged to the same heap. Check what the dependency instantiates at import time, and prefer tree-shakeable imports.

structuredClone or spread on request data doubling usage — both deep-copy. Read the fields you need instead of cloning the container.

Production deployment checklist

• No large JSON or data table is imported at module scope
• Request bodies are streamed, never read with text()/json()/arrayBuffer() unless bounded
• Every body inspection has an explicit maximum byte ceiling
• Response bodies are passed through as streams when only headers change
• No Array.from or spread is applied to a collection whose size the client influences
• Every module-scope cache has a maxEntries cap and a TTL
• Cap arithmetic is documented: entry size times cap is under 5% of the heap limit
• Large intermediates are scoped inside helpers so they release before slow awaits
• A synthetic load test with unique keys runs at 50 concurrent against a preview deploy
• The matcher excludes static assets so middleware never runs on them
• An alert fires on the platform’s out-of-memory error code and on p95 drift

Frequently Asked Questions

Related

• Memory and CPU limits across edge providers
• Comparing memory limits on Netlify versus Vercel Edge
• Avoiding CPU time limit errors in Cloudflare Workers
• Optimizing bundle size for edge runtime deployment
• Using Web Streams instead of Node streams at the edge