Why architecture matters here
GC architecture matters because in most Java services GC is the biggest source of tail latency. A well-tuned service with a modern collector sees GC pauses under 10 milliseconds; a poorly tuned service sees 500-millisecond pauses. That is the difference between a service that meets p99 SLOs and one that doesn't. It is also the difference between a service where users notice nothing and one where they see occasional hangs.
Cost matters too. GC pressure translates directly into CPU consumption. A service with 30% of CPU spent on GC is a service that needs 40% more instances than a well-tuned equivalent. Reducing GC overhead by half can cut infrastructure cost meaningfully.
Reliability is the third driver. Full GC events on the old-style CMS collector could take tens of seconds — long enough to fail health checks and get the container killed. Modern collectors (ZGC, Shenandoah) all but eliminate the risk of long pauses, but require different tuning and behave differently under memory pressure. Choosing wisely and configuring correctly prevents the catastrophic pause that takes a service down.
The architecture: every piece explained
Walk the diagram top to bottom.
Application Threads. Your code. Every new object allocation happens in an application thread's Thread-Local Allocation Buffer (TLAB) — a pre-allocated chunk of the young generation reserved for that thread. Allocation is O(1) — bump a pointer. This is why Java allocation is "free" in tight loops.
JVM Heap. The managed memory region. Divided into generations (young, old) or regions (in G1, ZGC, Shenandoah). Sizing the heap is one of the two most impactful tuning decisions; too small causes constant GC, too large causes long pauses on non-concurrent collectors.
GC Threads. A pool of dedicated threads that perform garbage collection. Modern collectors run most GC work concurrently with application threads, pausing only for brief coordination phases ("stop-the-world" or STW).
Young Generation (Eden + Survivor). Where new objects live. Divided into Eden (where allocation happens) and Survivor spaces (where objects that survive a Minor GC are copied). The generational hypothesis says most objects die young; young generation collection is where GC does most of its work and is fast because most objects are unreachable.
Old Generation (Tenured). Objects that survive enough Minor GCs are promoted here. Old-gen collection is more expensive because live objects are the majority. Modern collectors run old-gen work concurrently to avoid long pauses.
G1 (Garbage-First). Default collector since Java 9. Divides the heap into ~2048 regions and prioritizes collecting regions with the most garbage. Runs mostly concurrent with short STW pauses (target 200ms; achievable 10-100ms with tuning). Good for heaps under ~30 GB and mixed workloads.
ZGC. Ultra-low-pause collector introduced in Java 15+. Uses colored pointers (metadata embedded in unused pointer bits) to run virtually all work concurrently. Pauses under 1ms regardless of heap size. Available on x86-64 and ARM64. Best for latency-critical services with large heaps.
Shenandoah. Similar goals to ZGC but different approach — concurrent evacuation via Brooks pointers. Pauses under 10ms typically. Available in OpenJDK builds. Choose based on distro and workload; both are strong low-pause choices.
Write barrier and card table. When an old-gen object gets a reference to a young-gen object, the GC needs to know so young GC doesn't miss the reference. A write barrier fires on every reference update; a card table tracks which regions have cross-gen references. This is invisible to the developer but has performance implications.
Escape analysis and TLAB. Optimizations that keep short-lived objects out of the heap entirely. Escape analysis can stack-allocate objects that don't escape a method. TLABs eliminate contention on the heap pointer for allocation. Both are essential to the "allocation is free" story.
Observability. Java Flight Recorder (JFR) captures detailed GC events with minimal overhead. GC logs (enabled with -Xlog:gc*) show every collection. JMX exposes GC metrics to Prometheus or Datadog. Alerting on GC pause time is standard operator practice.
End-to-end allocation and GC flow
Trace a request. An HTTP handler receives a POST. It parses the JSON into objects, calls a service, formats the response. Every object it creates goes into the current thread's TLAB in Eden.
Eden fills up after millions of allocations. The JVM triggers a Minor GC. All application threads pause briefly. GC threads walk the roots (thread stacks, class statics), find live objects in Eden, copy them into a Survivor space, and free Eden. Objects that survive several Minor GCs are promoted to Old Gen. The pause is a few milliseconds on modern hardware.
Application threads resume. Allocation continues. Weeks or months pass; the old gen slowly fills.
Eventually old gen crosses a threshold. G1 initiates a concurrent marking phase — GC threads walk the reachability graph while application threads run. When marking completes, G1 schedules mixed collections that reclaim garbage from both young gen and the most garbage-dense old-gen regions. Each mixed collection pauses briefly.
Under normal load, G1 keeps up. GC overhead is 2-5% of CPU. Pause times are under 50ms. Life is good.
Under abnormal load — a memory leak, a sudden burst of allocation, a large collection cached in memory — old gen fills faster than G1 can clean. G1 falls back to a Full GC that pauses the entire application for potentially seconds. This is the pause you never want to see. Modern collectors (ZGC, Shenandoah) avoid the Full GC fallback entirely.