Why architecture matters here
JIT surprises are common. Cold JVMs are slow; hot ones can hit a code cache limit and never recompile; a monomorphic call site turns polymorphic and deopts. Each has a specific fix rooted in understanding the pipeline.
The architecture matters because tuning happens at ops (code cache sizing, AOT precompile) and code (avoid megamorphic dispatch, allow escape-analysis-friendly patterns).
With the pipeline in mind, you can design services that reach steady state fast and stay there.
The architecture: every piece explained
The top strip is the compilation path. Class loader loads bytecode. Interpreter runs it warm-up while profiling. Hot methods escalate to C1 (client) — quick compilation with basic optimizations. Hotter ones escalate to C2 / Graal — aggressive optimizations, inlining, unroll, vectorize.
The middle row is the intelligence. Tiered compilation runs C1 and C2 concurrently, using profile data to decide when to recompile. Inline caches speed polymorphic dispatch by remembering recent target types. Escape analysis allows stack allocation and lock elision when objects don't escape. Deoptimization handles the case when an optimistic assumption breaks — a safepoint transitions execution back to the interpreter.
The lower rows are ops. Code cache holds compiled methods; when full, JIT stops compiling and performance drops. Diagnostics — JFR, JITWatch, PrintCompilation — show compilation events. Ops covers warmup strategies, code cache sizing, and AOT precompile options.
End-to-end flow
End-to-end: JVM starts. Interpreter runs the request path. After 10,000 invocations, C1 compiles a hot method; latency drops from 5 ms to 1 ms. After 100,000, C2 recompiles with aggressive inlining and escape analysis; latency drops to 200 μs. Warmup takes 2 minutes. Later, a call site that was monomorphic (always Impl1) sees Impl2 for the first time; deopts back to interpreter briefly then recompiles polymorphically. Ops watches PrintCompilation to spot patterns and sizes code cache to keep JIT active.