GraalVM Architecture in Depth

Why architecture matters here

GraalVM architecture matters because ahead-of-time compilation changes Java's economics for cold-start-sensitive workloads. Removing class loading, verification, and JIT warmup from process start cuts startup by an order of magnitude, and dropping the compiler, code cache, and class metadata out of the process leaves resident memory dominated by the live heap. The price is peak throughput, because an AOT compiler has no observed profiles to optimise against.

Cost is engineering — reflection config, framework compatibility. Frameworks like Micronaut and Quarkus greatly reduce this.

Reliability of AOT-compiled apps is high — no JIT warmup surprises.

The architecture: every piece explained

Walk the diagram top to bottom.

Java code. Standard bytecode; but must be AOT-compatible (limited reflection etc).

GraalVM Compiler. One compiler used in two distinct modes. As a JIT inside an ordinary JVM it replaces C2 and wins on abstraction-heavy code through better inlining and partial escape analysis. As the back end of Native Image it compiles ahead of time, with no runtime profiles to work from - a very different set of trade-offs. Conflating the two is the source of most contradictory claims about GraalVM performance.

Native Image. AOT compiles to standalone executable. No JVM at runtime.

Instant startup. Millisecond boot; no class loading, no JIT warmup.

Low memory. No compiler, code cache, profiling counters, or metadata for unloaded classes in the process, so footprint is dominated by the application's live heap rather than by runtime machinery.

Reflection config. AOT can't discover reflection at runtime; must be listed in config. Tracing agent helps generate.

Truffle framework. Language implementation framework — Python, Ruby, JS all run on Graal via Truffle.

Serverless fit. Lambda cold starts drop from seconds to milliseconds.

Peak throughput. Sometimes less than JIT — HotSpot has decades of optimizations.

Frameworks. Micronaut + Quarkus designed for AOT; Spring Boot 3 supports.

Three different products share one name

Almost every confused GraalVM conversation is two people using the word for two different artefacts. The name covers at least three things that are built from a shared compiler but have entirely different operational profiles.

The Graal compiler as a JIT. You run a completely ordinary JVM - class loading, bytecode interpreter, deoptimisation, the whole dynamic runtime - and only swap out the top-tier optimising compiler. Nothing about your application's deployment, packaging, or dynamic behaviour changes. You are still shipping a jar and still paying warmup.

Native Image. An ahead-of-time compiler that consumes your closed classpath and emits a standalone executable with no JVM at run time. This changes what your program is allowed to do. It is a different execution model, not a faster JVM.

Truffle and the polyglot runtimes. A framework for implementing other languages on top of the same compiler, which is why JavaScript, Python, and Ruby implementations carry the GraalVM name.

The practical cost of the conflation: someone reads that "GraalVM has slower peak throughput" - a Native Image property - and rejects the Graal JIT, which is a pure JIT-versus-JIT comparison with no closed-world constraint at all. Or someone reads that "GraalVM optimises better than C2" and expects that from a native binary, which is precisely where the optimiser has the least information. Keep the two apart and almost every claim about GraalVM becomes checkable.

The Graal JIT - a compiler written in Java

HotSpot's optimising compiler, C2, is a large body of C++ that has been tuned for decades. Graal is a re-implementation of that role in Java. It plugs into HotSpot through JVMCI, the JVM Compiler Interface: HotSpot hands a compiler a method's bytecode together with the profile it has accumulated - branch counts, receiver types seen at each call site, type-check outcomes - and the compiler hands back an installed machine-code blob plus the metadata needed to deoptimise back into the interpreter when a speculation fails. Everything C2 does through internal APIs, Graal does through that interface.

The interesting consequence is tooling, not language taste. A compiler written in Java can be unit-tested, profiled, and debugged with ordinary Java tools, and its IR - a sea-of-nodes graph, the same lineage as C2's - can be dumped and inspected with a graph viewer. That lowers the cost of adding an optimisation phase enough that Graal ships several C2 does not.

libgraal: why the compiler does not need its own warmup

A compiler written in Java running inside the JVM it is compiling for is a circular problem. Run Graal as ordinary Java code on the host JVM and it must itself be interpreted and then JIT-compiled before it produces good code, it allocates into the application's heap, and its garbage disturbs the application's GC. The answer is to compile Graal itself with Native Image into a shared library - libgraal - which is loaded as native code with its own isolated heap. The compiler starts at full speed, never triggers an application GC, and never appears in an application heap dump. This is a neat illustration of the two halves of the project feeding each other: the AOT compiler exists partly to make the JIT compiler practical.

Where the Graal JIT beats C2, and where it does not

Two differences account for most of the observed wins.

Inlining policy. C2's inlining is governed largely by bytecode-size thresholds applied per call. Graal's policy reasons about the callee's graph after the compiler has already simplified it, so a method that is large in bytecode but collapses to almost nothing once constants are folded is still a candidate. Deeply layered code - a stream pipeline, a functional wrapper four abstraction levels deep, a builder that expands into a handful of field writes - is exactly the shape where per-call size limits stop early and the abstraction never gets flattened.

Partial escape analysis. Classic escape analysis is all-or-nothing: an object either provably never escapes, in which case it can be replaced by its scalar fields, or it escapes somewhere and the allocation stands. As the escape analysis article notes, the mainline HotSpot optimisation is strongest when the object never escapes at all, and partial escapes are the hard case. Graal implements that hard case: it sinks the allocation down into only the branches where the object actually escapes, and on every other path the object is scalar-replaced and disappears. The pattern this rescues is extremely common - an object built on the hot path, consumed locally, and only stored or thrown on a rare error branch. Under all-paths analysis one rare branch defeats the whole optimisation; under partial escape analysis it only costs you that branch.

What C2 keeps: a long-tuned catalogue of intrinsics and loop optimisations, and a great deal of accumulated tuning against real workloads. Graal's advantage is largest on heavily abstracted, allocation-dense, object-oriented code, and smallest on tight numeric loops that were already being handled well. Availability is the practical catch - which JDK builds ship the Graal JIT, and whether a flag such as -XX:+UseJVMCICompiler is present at all, has changed across releases and across distributions. Check what your build actually offers rather than assuming.

Native Image and the closed-world assumption

Native Image is a different job. You give native-image a classpath or module path and an entry point; it computes the set of code that can ever execute, compiles all of it to machine code, links in a runtime, and emits a platform executable. At run time there is no bytecode, no class loading, no verifier, no interpreter, and no compiler.

All of that rests on one assumption: the closed world. The set of classes, methods, and fields that can ever be reached is fixed when the image is built. What that forbids follows directly:

• Loading a class that was not on the build classpath. There is no mechanism to compile it.
• Generating bytecode at run time - the classic proxy-and-instrumentation trick used by ORM and mocking libraries that spin classes with ASM, ByteBuddy, or cglib during startup.
• Custom class loaders that fabricate classes rather than reading known ones.
• Attaching a JVMTI agent to a running process.
• Class.forName on a name that is only known at run time. If the name is a compile-time constant the builder can follow it; if it comes from user input it cannot.

Lambdas and method references are fine - they are materialised during the build rather than at run time. It is worth being precise that the closed world is not an incidental limitation that a future release will lift. It is the enabling assumption. Dead-code stripping, a pre-populated heap, the absence of class metadata for classes that are never loaded, and the absence of any profiling or compilation infrastructure are all consequences of knowing the whole program. Take the assumption away and you have a JVM again.

Points-to analysis - what actually survives into the image

The builder does not simply mark called methods. It runs an iterative points-to (type-flow) analysis: starting from roots - the entry point, registered reflection targets, JNI entry points, class initialisers - it propagates the set of concrete types that can flow into every field, parameter, and return value, and iterates until it reaches a fixpoint.

That gives more than reachability. If the analysis proves only one concrete type can ever arrive at a virtual call site, the call is devirtualised statically - no inline cache, no guard, no deoptimisation path needed, because there is no possibility of a second type appearing later. This is one thing AOT does that a JIT structurally cannot: a JIT's monomorphic assumption is always a speculation guarded by a bailout, because a class implementing the interface might still be loaded.

Two operational consequences follow. First, registering something for reflection is not free in a local sense: it widens the type flow, which can drag whole subgraphs into the image and grow the binary noticeably. Second - and this is the failure mode that surprises teams - code the analysis cannot see is not stubbed out or lazily resolved. It is simply absent. The symptom is a ClassNotFoundException or NoSuchMethodException raised by a binary whose entire JVM test suite passed. When the binary is unexpectedly large, the builder's analysis report and call-tree output tell you which root pulled a subsystem in; that is usually a faster path than guessing.

Build-time initialisation and the image heap

The second big mechanism is that class initialisers can run during the build, inside the builder JVM, and the resulting object graph is serialised into the executable's data section. That graph is the image heap. At startup the process maps it in; for those classes there is no initialisation work at run time at all.

This, more than machine code, is where the startup win comes from. A framework's configuration parsing, dependency-injection wiring, ORM metadata, and routing tables can all already exist as live objects before main is entered. The work was done once on a build machine instead of on every process start in production.

The safety rules, and why they are semantic

You steer this with --initialize-at-build-time and --initialize-at-run-time, scoped to classes or packages. Anything that captures environment must be deferred to run time:

• Random seeds. A SecureRandom seeded at build time means every deployed copy of that binary shares the seed. This is a security bug, not a performance quirk.
• Current time, hostname, environment variables, system properties. A class initialiser that reads a system property bakes in the builder's value. The deployment's value is never consulted.
• File descriptors, sockets, channels, native library handles, Thread objects. These have no meaning after the builder exits.

The characteristic build failure is the builder refusing to write some object into the image heap - an error saying that instances of a given type are not allowed there. It means a build-time initialiser transitively constructed something environment-bound. The builder can print the reference chain that reached the offending object, which is how you find which initialiser to push to run time. Worth internalising: this is a real behavioural difference from the JVM, not a configuration chore. Two builds of the same source can differ because the build machines differed.

Reachability metadata: reflection, proxies, JNI, resources

Everything the static analysis cannot follow has to be declared. Historically these were separate JSON files placed on the image build path, and the concepts are stable even though the file layout has been consolidated across releases:

• Reflection. reflect-config.json names each class and, per class, the constructors, methods, and fields to retain, plus whether unsafe allocation is permitted. This is the same file the Scala reflection article describes from the Scala side, and the mechanism is identical regardless of source language.
• Dynamic proxies. Registered as interface sets, because a JDK proxy class is identified by its exact ordered interface list. Registering the interfaces individually does not help.
• Resources. Matched by regular expression over resource paths. A file sitting on the classpath is not in the binary unless a pattern matches it - the most common cause of a missing properties file or resource bundle at run time.
• JNI. Native code doing GetMethodID is looking up by name from outside the analysis entirely.
• Serialization. Deserialisation constructs objects without calling their declared constructors, which the analysis cannot model.

The tracing agent, and why this is a test-coverage problem

Hand-writing this metadata for a real dependency tree is not viable. The standard workflow is to run the application on a stock JVM under the tracing agent, -agentlib:native-image-agent=config-output-dir=..., exercise it, and let the agent record every reflective lookup, proxy creation, resource read, and JNI call that actually happened. Repeated runs can be merged into a single config directory.

The word "actually" is the whole point. The agent records executed paths only. An error-handling branch that reflects on a class and is never hit during the recording run produces metadata that omits it, and a binary that works in every test and fails the first time that branch is taken in production. Native image correctness therefore converts directly into integration-test coverage - a genuinely different quality bar than "the unit tests pass".

Two things reduce the burden. Libraries can ship their own metadata inside their jars under META-INF/native-image, and there is a community-maintained shared metadata repository for libraries that do not. And frameworks built for this - Quarkus, Micronaut, and Spring's AOT processing - attack it from the other end: they move dependency injection, ORM mapping, and configuration binding to build time via annotation processors or build plugins, so the run-time path is plain, statically analysable code, and whatever metadata remains is emitted automatically. "GraalVM works with this framework" usually means "this framework was designed to be statically analysable", not that the builder got cleverer.

The trade: startup and footprint versus peak throughput

The wins are structural. Startup skips class loading, verification, interpretation, and JIT compilation entirely, and a large part of the object graph arrives pre-built in the image heap, so process start is dominated by the operating system mapping the binary. This is an order-of-magnitude change against a JVM start, not a percentage. Footprint drops because there is no compiler in the process, no profiling counters, no code cache, and no class metadata for classes that were never loaded - resident memory ends up dominated by the live application heap itself rather than by runtime machinery.

The loss is peak throughput, and it has a specific cause worth stating precisely. A JIT compiles with observed profiles: which branch this program actually takes, which receiver type this call site actually sees, which types this cast actually encounters, which loops are actually hot. It inlines the observed target aggressively, guards the assumption with a cheap check, and deoptimises to the interpreter and recompiles if the assumption ever breaks. An AOT compiler has no observations. It has a sound but far less precise static type-flow result, and - critically - it has no interpreter and no compiler at run time, so it cannot speculate at all: there is nowhere to bail out to. The concrete effects are that genuinely polymorphic call sites stay polymorphic, cold branches get the same code quality as hot ones, and code layout is not driven by real branch frequencies.

PGO is a partial answer, not a fix

Profile-guided optimisation closes part of the gap. The shape is: build an instrumented image, run it under a representative workload, collect a profile, then rebuild consuming that profile. The second build gets real branch frequencies and call-site type distributions and can make inlining and layout decisions much closer to a JIT's.

Why it is only partial: the profile is a fixed artefact from a past run, not a runtime that keeps measuring. A workload whose hot path shifts with traffic mix, tenant, or season carries a stale profile until someone rebuilds. It also puts a representative load test inside the build pipeline, which is a real engineering commitment. There has additionally been work on inferring profiles statically when no instrumented run is available. Which of these mechanisms is present in which build has moved across editions and releases - Oracle GraalVM, the community builds, and downstream distributions have not always carried the same feature set - so check the documentation for the exact version you build with rather than treating any of it as universally available.

Substrate VM - the runtime inside the binary

A native image is not bare compiled code. It embeds Substrate VM: an allocator, a garbage collector, thread management, monitors, signal handling, exception dispatch, and a trimmed JDK. SVM is itself mostly Java compiled ahead of time, with a small amount of C.

Structurally it is missing the parts a closed world does not need: no class loading, no bytecode interpreter, no code cache, no deoptimisation-to-interpreter path. What it keeps is memory management, and the collector choice is a build option. A serial generational collector is the default in common builds and suits the target profile well - short-lived processes and small heaps, where a simple stop-the-world collector's pauses are irrelevant because the process may not collect at all. An epsilon-style non-collecting option exists for processes that exit before they exhaust the heap, trading all reclamation for zero GC cost. A G1-based option has been offered in some builds and editions. The reasoning about which to pick is the same trilemma as on HotSpot and is covered in the JVM GC architecture article; nothing about it is GraalVM-specific.

What is SVM-specific: heap parameters can be fixed at build time as well as overridden at run time, so a binary can ship with its own sensible maximum heap rather than inheriting a container-derived default; and the image heap portion is mapped from the executable rather than allocated and traced from scratch. SVM also supports isolates - multiple fully independent heaps within one process, each disposable in one operation. That is what makes it viable to embed a native image as a library inside a host process, including a non-Java one.

Build time and build memory are a real CI constraint

Underestimating this is the most common way a native-image adoption stalls. The build is not a compilation step; it is a whole-program static analysis followed by compiling every reachable method, including the JDK code you reached. It takes minutes rather than seconds and needs multiple gigabytes of builder heap. A builder out-of-memory failure on a CI runner with default memory limits is very often the first thing a team hits, and it is a resource problem, not a code problem.

There is also no incremental build. Change one line and you pay the entire cost again, which rules native image out as an inner development loop. The practical pipeline shape that works:

• Develop and run the full test suite on a normal JVM. That stays the fast loop.
• Run the native build as a separate, later CI stage, on a runner with explicitly sized memory.
• Treat the produced binary as a distinct artefact needing its own smoke and integration tests - your JVM tests did not execute it, and the failure modes above are precisely the ones JVM tests cannot catch.
• Budget one build per target platform and architecture. The output is a platform binary, so cross-targeting means more builds, usually in containers or on matching runners.

A quick-build mode exists that trades generated code quality for build time; it is useful for iterating on configuration problems, not for producing production binaries. It is also common in practice to build with a distribution that packages just the builder - Mandrel is the downstream build maintained in the Quarkus ecosystem for exactly this - rather than installing a full GraalVM. Editions and packaging here have shifted across releases, so treat the specific artefact names as version-dependent.

Debugging and observing a native binary

The JVM's introspection machinery is largely absent unless you ask for it, because including it costs binary size and startup.

Attach-based tooling does not work. jcmd, jstack, and JMX attach all depend on a JVM that is not there. Monitoring support - JFR, jvmstat, heap dumps - is enabled at build time through the monitoring flags, and JFR's event coverage in native images has historically lagged the JVM's. Decide before the build; you cannot turn it on for a running process.

Debugging is native debugging. Building with debug info lets you attach a platform debugger such as gdb and see Java source lines and variables, given the right source path configuration. It is not a Java debugger session; there is no JDWP agent. Symbols are frequently stripped for size, and stripping is what turns a production crash into an unreadable address dump - keep an unstripped copy of every shipped binary, indexed by build, or your crash reports are worthless.

Stack traces have caveats. Exceptions still carry traces, but inlining is aggressive and there is no deoptimisation path that can reconstruct an interpreter frame, so frames may be missing or attributed to an inlining parent. Line-number fidelity depends on the debug-info level chosen at build time.

Profiling changes shape. The JVM-attach profilers most teams rely on cannot attach. You fall back on OS-level sampling of the binary with perf and friends, or on an instrumented build. In practice the workable pattern is to do performance investigation on the JVM build of the same code, where the full toolchain works, and use the native build's own instrumentation only for questions specific to the binary.

Where it fits, and where it does not

The decision is not "is AOT better". It is which resource you are allowed to spend.

Strong fit. Short-lived CLI tools, where the process lifetime is shorter than a JVM's warmup - there is no throughput sacrifice because the JIT would never have reached peak anyway, so this is strictly better. Serverless functions, where cold start is user-visible latency and, on many platforms, billed time. High-density deployment, where per-instance resident memory determines how many replicas fit on a node and the footprint reduction converts directly into cost. Scale-to-zero services, sidecars, operators, and admission controllers. Embedding Java in a host process, where isolates and a shared library are the only workable packaging.

Poor fit. A long-running throughput-critical service: it runs for days, so warmup amortises to nothing, and you would be paying the peak-throughput gap continuously for a startup benefit that is measured once. Anything that must load plugins or user-supplied code at run time - closed world forbids it outright, and no amount of configuration works around it. A codebase leaning on dynamic bytecode generation in dependencies you do not control. A team without the CI headroom for the build.

The middle ground is often the right answer. Two JVM-preserving options attack startup without giving up the JIT. Class-Data Sharing removes class loading and verification cost from startup by mapping a pre-parsed archive, while keeping the full dynamic runtime and the JIT - a much smaller win than AOT, at close to zero engineering cost and zero behavioural risk. CRaC goes further by restoring a process image that was checkpointed after warmup, so JIT-compiled code and accumulated profiles survive into the restored process - keeping peak throughput while removing warmup, at the cost of a checkpoint/restore lifecycle and resources that must be released and reacquired around the checkpoint. Ongoing OpenJDK work continues to explore shifting work earlier in the application lifecycle while keeping the dynamic runtime. Order the options by what you can afford to give up: dynamism, peak throughput, engineering effort, or startup latency.

Truffle and polyglot - the third thing

Truffle is a framework for implementing programming languages as self-optimising abstract syntax tree interpreters. A language implementer writes an interpreter whose nodes specialise themselves as they observe actual values - a generic arithmetic node rewrites itself into an integer-specific node once it has only ever seen integers, with a guard that falls back if that stops being true. Graal then partially evaluates the interpreter against a specific program: it takes the interpreter code plus the specialised AST and compiles them together down to machine code for that one program. This is the first Futamura projection made practical, and its payoff is that writing an interpreter gets you a JIT compiler for free rather than as a separate multi-year project.

The practical consequences are integration ones. Languages implemented this way share one runtime, one heap, and one toolchain, and can pass objects between each other through an interoperability protocol rather than serialising across a foreign-function boundary. If you want to embed a scripting language inside a JVM process without running a second runtime, that is the argument. The caveats are that performance and ecosystem compatibility vary enormously by language, and largely on how much of that ecosystem depends on native extensions written against the reference implementation's C API. A Truffle language can itself be compiled into a native image, which is how these runtimes are distributed as standalone binaries. Which languages ship, at what maturity, and under which packaging has changed considerably across releases. Treat this as a separate product line that happens to share the compiler - it has almost nothing in common operationally with either running the Graal JIT or building a native image of your own service.

End-to-end Native Image build + run

Trace a build. Java app using Quarkus. Build with `mvn package -Pnative`. GraalVM Native Image starts.

Classpath analysis: which classes reachable? Only reachable ones compiled.

Reflection config consulted: which reflective accesses to support? Compile those hooks.

Native compilation: converts bytecode to machine code; embeds runtime (garbage collector, etc.).

Output: a standalone platform executable with the Substrate VM runtime linked in; no JVM needed to run it. Binary size tracks how much of the classpath the reachability analysis kept.

Deploy to a function platform. Cold start drops by roughly an order of magnitude, because the work a JVM does at start - class loading, verification, initialisation, interpretation - has already happened at build time. Warm invocations are comparable to, or somewhat slower than, a warmed-up JIT.

Alternative: Spring Boot 3. Similar but needs more reflection config initially. Tracing agent runs app under HotSpot with agent that logs reflection uses; config generated; native build with that config.

Peak throughput generally sits below a fully warmed JIT, and the gap is workload-dependent rather than a fixed percentage: it widens with polymorphism and with how much the JIT's speculation was buying you. Profile-guided optimisation narrows it by feeding a recorded profile back into a rebuild.