Layers of Logic

13.1

Where Everything Actually Lives

You have been told since Phase IV that locals live on the stack and objects live on the heap. Both halves are true and neither is the whole picture, and the two areas nobody mentions are where your classes and your string literals are.

Core20 min read4 exercises
01

Previously on

Section 4.2 used a conceptual frame-and-object diagram for an array and its reference. It also warned that this is a model of Java semantics, not a promise about physical addresses.

Since then you have seen the same model many times. Objects have storage, methods have local state, linked structures add node objects, and millions of wrappers cost memory. The JVM may move, inline, or eliminate some of that storage, so now we will separate the model from a measured implementation.

This section opens the box.

02

The problem

Here is a method with three things in it. Say where each one lives.

void process() {
    int count = 5;
    Student s = new Student("Aditya", 101);
    String name = "Aditya";
}

If you said stack, heap, and heap, you are mostly right and it is not enough to be useful.

Try harder questions.

count dies when the method ends. s also goes out of scope. Does the Student object die too? If it does, why does returning it work? If it does not, who deletes it, and when?

The literal "Aditya" appears in your source code. It exists before the method runs and after it ends. Which area is it in? Not the frame, because it survives. Not really the heap either, in the way the Student is.

And where is Student itself? Not the object. The class. Its bytecode, its method definitions, its static fields. Something has to hold those, and it is neither of the two areas you have been told about.

Then the question that decides the next phase. Two threads run this method at once. How many count variables exist? How many Student objects can each thread see?

Two areas cannot answer these. There are four.

03

The idea

A running JVM keeps memory in areas with different rules and different lifetimes.

AreaWhat is in it, and who can see it
Stackone per threadframes: locals, parameters, return addresses. Private to that thread
Heapone, sharedevery object ever created with new. Visible to all threads
Method areaone, sharedclass definitions, method bytecode, static fields, the string pool
PC registerone per threadwhich instruction this thread is on

The stack is per thread, and it is why locals are safe.

Every method invocation has a frame in the JVM execution model, as described in Section 5.1. The frame holds parameters, locals and return state. A JIT may inline a call physically while preserving the same observable behaviour.

No collector is involved. Popping a frame is one pointer moving, and that is the whole cleanup.

Two threads running the same method have two stacks and two frames. Neither can reach the other’s locals, which is a guarantee you get for nothing.

The heap is shared, and that is where objects go.

Student s = new Student("Aditya", 101);

Two things happen. The object is built on the heap. A reference to it goes in the frame.

Stack while process is running

process

int count5the number itself, right here
Student s@1a2ban address, not the object

Heap

Student@1a2b

String name"Aditya"
int rollNumber101
count dies with the frame. The address dies with the frame. The object does not.

When the method ends, the reference dies with the frame. The object does not. It stays on the heap until nothing can reach it any more, and then the garbage collector takes it at some later moment of its choosing.

That gap is why return s works at all. The object was never tied to the method that made it.

The method area holds the classes themselves.

Not the objects, the blueprints. The bytecode of every method, the field definitions, the static variables, the constant pool. Loaded once when the class is first used, and shared by every thread and every instance.

There is exactly one copy of Student’s toString bytecode no matter how many Student objects you make. Static fields live here too, which is why every instance sees the same one.

The string pool is in here, and it is where "Aditya" from the problem lives. That is Section 9.1 with an address attached.

The PC register is one number per thread.

Which instruction that thread is executing. It exists because threads get interrupted and have to be resumed exactly where they stopped. One per thread, and you never touch it.

04

Under the hood

Going deeper

Now the cost, measured. Ten million numbers, held two ways, on Java 21 with a fixed heap:

How they are heldHeap used
int[10_000_000]one object, values packed together40 MB
Integer[10_000_000]one array of references, plus 10M objects192 MB

Nearly five times, for the same ten million numbers.

The int[] is one object. A header, a length, then forty million bytes of values sitting next to each other.

The Integer[] is one array of ten million references, and then ten million separate objects for them to point at. Each Integer has a header of its own before it gets to the four bytes you wanted.

Every object carries a header. On a 64 bit JVM it is twelve bytes: a mark word holding the identity hash and lock state, and a class pointer saying which class this is. Objects are padded to a multiple of eight bytes, so the smallest possible object is sixteen.

An Integer is twelve bytes of header and four of number. Seventy five percent of it is not your data.

That single fact settles three arguments from earlier phases. It is why IntStream beat Stream<Integer> in Section 11.6. It is why IntUnaryOperator beat Function<Integer, Integer> in Section 11.2. And it is most of why ArrayList beat LinkedList at appending in Section 10.4, because a LinkedList node is a header plus three references for every single element.

Two ways to run out, and they mean different things.

StackOverflowError means one thread’s stack filled up. Almost always runaway recursion. The stack is around 512 KB to 1 MB per thread by default, and -Xss changes it.

OutOfMemoryError: Java heap space means the heap is full of objects that are still reachable. More memory postpones it. If they are reachable because of a bug, more memory postpones it by a few hours.

Each thread has its own stack, and that costs real memory. A thousand threads at 1 MB each is a gigabyte of stacks before your program holds any data. This is the number that makes thread pools necessary in Phase XIV, and the one virtual threads were built to change.

05

What it costs

Almost none of this is under your control, and that is mostly a good thing.

You cannot choose where an object goes. Every new allocates on the heap, and there is no keyword for putting a small object on the stack the way C lets you. The JVM sometimes does it for you when it can prove an object never escapes a method, and you cannot ask for it or rely on it.

Sizes are not portable either. Twelve byte headers, eight byte padding, the -128 to 127 cache: all of it can differ between JVM versions and vendors. Compressed references also switch off automatically above a 32 GB heap, which makes every reference in your program bigger. Code that depends on an exact object size will break.

The per thread stack cost is the one that shapes real designs. A thread is not free before it does anything, and that single fact is why you will spend most of Phase XIV pooling them rather than making them.

Boxing remains the easiest expensive mistake in the language. List<Integer> is the natural thing to write, it is five times the memory of the array you meant, and nothing about the code looks wrong.

06

Check yourself

Answer from memory before opening the explanation. Then compare your rule with the answer and revisit the example if they differ.

  1. `int x = 5;` inside a method, and `new Student()` inside the same method. Where does each one live, and when does each one die?

    Show the answer

    x is four bytes inside that method's stack frame. It dies at the closing brace, because the whole frame is discarded when the method returns. Nothing collects it and nothing needs to.

    The Student object is on the heap. The reference to it is in the frame and dies with the frame, but the object does not. It stays until nothing can reach it any more, and then the collector takes it whenever it next runs.

    That gap is the point. Return the reference and the object outlives the method that made it. Return an int and you return a copy of a number, which is Section 6.3 again.

  2. Two threads run the same method at the same time. What do they share and what do they not?

    Show the answer

    Each thread has its own stack. Two threads in the same method have two frames, two copies of every local variable, and neither can see the other's. Locals are private to a thread for free, without any effort on your part.

    They share the heap and the method area. Every object either one creates is visible to both, and so is every static field.

    That single split is the whole reason Phase XIV exists. A local is safe because nobody else can reach it. Anything on the heap is reachable by every thread at once, and that is where the trouble starts.

  3. Ten million ints in an `int[]` against ten million in an `Integer[]`. How much heap does each take, and where does the difference come from?

    Show the answer

    Measured on Java 21: 40 MB against 192 MB.

    The int[] is one object holding ten million four byte values, packed together. Forty megabytes and one header.

    The Integer[] is one array of ten million references, and ten million separate objects for those references to point at. Each Integer carries a header as well as its four bytes of number, which is most of the extra.

    This is the cost that made IntStream and IntUnaryOperator worth having in Phase XI, and it is why ArrayList beat LinkedList at appending in Phase X.

07

Exercises

Write each solution in VS Code, predict its result, and then run the checks. Reading an example does not replace implementing it.

4 exercises100 pointsabout 95 minutes

The Layers of Logic VS Code extension runs the checks for exercises marked checked. For a manual exercise, run the program and compare its behaviour with the stated requirements and sample output.
A

Measure the Boxing, In Megabytes

Real work·25 min·25 points

checkedex-13-1-a

Three phases have used this as an argument. Now put a number on it.

Do the measurement carefully or you will measure nothing at all. Hold a live reference while reading, and fix the heap size so the JVM is not resizing underneath you. Both of those mistakes silently give you zero, which looks like a result.

When you have the two numbers, work out the per element cost and account for every byte. Most of the gap is not the number you wanted. It is the header on each object that holds it.

What to write

These exact signatures. The checks call them by name, so a different name or a different parameter type will not build.

class Weigh {
    static long usedMb()
    static long primitiveMb(int n)
    static long boxedMb(int n)
    static int bytesPerPrimitive()
    static int bytesPerBoxed()
    static long predictedBoxedMb(int n)
    static boolean liveReferenceNeeded()
}

Both measuring methods hold a live reference in a static field while measuring, or the collector takes the array first and you measure zero. bytesPerBoxed counts the Integer object AND the reference in the array pointing at it.

What your program must do

  • Measure both arrays and record the two numbers in megabytes
  • Hold a live reference while measuring, and say why
  • Predict the boxed cost from the layout before measuring
  • Account for the gap byte by byte
Weigh.java
public class Weigh {

    static Object keep;   // hold a live reference while measuring

    static long usedMb() { return 0; }  // TODO

    static long primitiveMb(int n) { return 0; }  // TODO: an int[n], filled
    static long boxedMb(int n)     { return 0; }  // TODO: an Integer[n], filled

    // Work these out from the layout, do not measure them.
    static int bytesPerPrimitive() { return 0; }  // TODO
    static int bytesPerBoxed()     { return 0; }  // TODO: the object AND the reference to it
    static long predictedBoxedMb(int n) { return 0; }  // TODO

    static boolean liveReferenceNeeded() { return false; }  // TODO

    public static void main(String[] args) {
        // TODO: measure both at 10_000_000, with -Xms3g -Xmx3g
    }
}
Hint 1
Fix the heap with -Xms3g -Xmx3g. Without it the JVM grows and shrinks the heap underneath you and the readings mean nothing.
Hint 2
Expect around 40 MB and 192 MB. The int array is values packed together. The Integer array is references, plus ten million separate objects for them to point at.
Hint 3almost the answer
Each Integer is 16 bytes: twelve of header and four of number, already a multiple of eight. Add four more per element for the reference, which gives twenty against four.
What this is really testing

Whether "ten million Integer objects is expensive" is a phrase you repeat or a number you have measured. It has been an argument in three phases now.

B

Two Ways to Run Out

Real work·20 min·20 points

checkedex-13-1-b

Break both on purpose and notice how differently they behave.

One is a thread running out of frames, and it happens in milliseconds. The other is the shared heap filling with objects nothing has released, and it can take a while and slow down badly first.

Then try the flags the wrong way round. -Xmx will not save you from recursion and -Xss will not save you from a growing list, and understanding why is the point of having four areas rather than one.

What to write

These exact signatures. The checks call them by name, so a different name or a different parameter type will not build.

class RunOut {
    static void deeper(int depth)
    static int stackDepth()
    static String fillStack()
    static String fillHeap(int mbLimit)
    static String flagFor(String failure)
    static boolean sameArea()
}

deeper recurses and records the depth in a shared field. stackDepth catches the error and returns how far it got. fillStack and fillHeap return the simple name of whatever came out. flagFor takes a failure name and returns the JVM flag that moves that limit.

What your program must do

  • Fill the stack and record the depth reached
  • Fill the heap and read the different error
  • Say which flag moves which limit
  • Show that the wrong flag does nothing for the other failure
RunOut.java
import java.util.*;

public class RunOut {

    static int deepest = 0;

    static void deeper(int depth) { }  // TODO: recurse, recording the depth

    static int stackDepth() { return 0; }  // TODO: catch the error, return the depth

    static String fillStack() { return ""; }  // TODO: the simple name of what came out

    // Fill the HEAP instead, up to a limit in megabytes. Run with -Xmx128m.
    static String fillHeap(int mbLimit) { return ""; }  // TODO

    // Which JVM flag moves each limit?
    static String flagFor(String failure) { return ""; }  // TODO

    // Do the two failures come from the same memory area?
    static boolean sameArea() { return true; }  // TODO

    public static void main(String[] args) {
        // TODO: find your stack depth, then run again with -Xss256k and -Xss4m
        // TODO: fill the heap with -Xmx128m so it does not take all day
    }
}
Hint 1
-Xss is the stack size per thread and -Xmx is the maximum heap. Different areas, different flags, and neither one helps with the other's failure.
Hint 2
The depth is a fact about your stack size, not about Java. Change the flag and the number moves, which tells you what the limit was made of.
Hint 3almost the answer
A StackOverflowError is nearly always runaway recursion, and a bigger stack postpones it. An OutOfMemoryError means the heap is full of objects that are still reachable, and if a bug is holding them, more heap buys you a few hours.
What this is really testing

Whether StackOverflowError and OutOfMemoryError feel like the same thing. They come from different areas with different rules, and the fix for one is never the fix for the other.

C

Where Does the Literal Live

Real work·25 min·25 points

checkedex-13-1-c

Four things in one small file, and they live in three different places.

Predict every comparison before running. The Integer pair is the one worth thinking hardest about, because the answer changes at 128 and nothing in the code shows you the boundary.

Finish by naming the area for each of the four. The static and the instance field look almost identical in the source, and where they live is the entire difference between them.

What to write

These exact signatures. The checks call them by name, so a different name or a different parameter type will not build.

class WhichArea {
    static boolean twoLiteralsAreOneObject()
    static boolean literalEqualsNew()
    static boolean internMatchesLiteral()
    static boolean smallIntegersAreShared()
    static boolean largeIntegersAreShared()
    static int sharedCounterAfter(int objects)
    static int instanceFieldOfEach()
    static String areaOf(String what)
}

areaOf takes a description and returns "method area", "heap" or "stack". sharedCounterAfter makes that many objects, increasing the static counter and each object's own field, and returns the static one.

What your program must do

  • Predict all five identity comparisons before running
  • Show that a thousand objects share one static field
  • Name the area for each of the four kinds of thing
  • Say why the Integer answer changes between 100 and 200
WhichArea.java
public class WhichArea {

    static int counter = 0;      // which area?
    int instanceField = 0;       // which area?

    // PREDICT all five before running.
    static boolean twoLiteralsAreOneObject() { return false; }  // TODO
    static boolean literalEqualsNew()        { return false; }  // TODO
    static boolean internMatchesLiteral()    { return false; }  // TODO
    static boolean smallIntegersAreShared()  { return false; }  // TODO: valueOf(100)
    static boolean largeIntegersAreShared()  { return false; }  // TODO: valueOf(200)

    static int sharedCounterAfter(int objects) { return 0; }  // TODO
    static int instanceFieldOfEach()           { return -1; } // TODO

    // "method area", "heap" or "stack".
    static String areaOf(String what) { return ""; }  // TODO

    public static void main(String[] args) {
        // TODO: predict all five comparisons, then run them
        // TODO: name the area for the static, the instance field, the literal and the new String
    }
}
Hint 1
Literals are pooled, so two with the same characters are one object. new String deliberately makes a second, and intern asks the pool for the shared one.
Hint 2
Integer.valueOf caches -128 to 127 and builds a new object above that. It is a memory decision, and the == behaviour is a side effect of it.
Hint 3almost the answer
The static lives in the method area with the class, which is why every instance sees the same one. The instance field lives inside each object on the heap, which is why every instance has its own.
What this is really testing

Whether the method area is a real place to you. It holds your classes, your statics and your string literals, and each of those has behaviour you can observe from ordinary code.

D

A Thread Costs a Megabyte

Hard·25 min·30 points

checkedex-13-1-d

Start threads until the machine says no, and count.

The number itself matters less than what it is made of. Each thread reserves a stack whether or not it has anything to do, so a thousand sleeping threads have already cost you most of a gigabyte.

Shrink the stack with the flag and the count goes up, which tells you exactly what the limit was made of. Keep the number in mind: it is the reason the whole next phase is about reusing threads instead of creating them.

What to write

These exact signatures. The checks call them by name, so a different name or a different parameter type will not build.

class ThreadCost {
    static int maxPlatformThreads(int ceiling)
    static long stackBytesPerThread()
    static long megabytesFor(int threads)
    static int virtualThreadsIn(int count) throws Exception
    static boolean threadCostsBeforeItWorks()
}

maxPlatformThreads starts daemon threads that wait on a latch, stopping at the ceiling or when the machine refuses, and releases them all in a finally. Catch Throwable, since the failure is an OutOfMemoryError. virtualThreadsIn returns how many finished.

What your program must do

  • Start threads until it fails, and record how many you managed
  • Keep them alive while counting, and release them afterwards
  • Work out what a thousand idle threads cost in memory
  • Show virtual threads going far past that ceiling
ThreadCost.java
import java.util.*;
import java.util.concurrent.*;

public class ThreadCost {

    // Start threads until the machine refuses, holding them ALIVE while you count.
    // Make them daemons and release them in a finally, or the tests will hang.
    static int maxPlatformThreads(int ceiling) { return 0; }  // TODO

    static long stackBytesPerThread() { return 0; }  // TODO: the usual default
    static long megabytesFor(int threads) { return 0; }  // TODO

    // Virtual threads instead. Return how many finished.
    static int virtualThreadsIn(int count) throws Exception { return 0; }  // TODO

    // Is the memory reserved before the thread does any work?
    static boolean threadCostsBeforeItWorks() { return false; }  // TODO

    public static void main(String[] args) throws Exception {
        // TODO: find the platform ceiling, then try -Xss256k and compare
        // TODO: work out what a thousand idle threads cost before doing anything
    }
}
Hint 1
A thread that finishes gives its stack back, so hold them on a CountDownLatch and count down in a finally. Make them daemons or the JVM will not exit.
Hint 2
The failure is an OutOfMemoryError, which is an Error and not an Exception, so catch Throwable or you will not catch it at all.
Hint 3almost the answer
Around a megabyte per thread, reserved at start and not at first use. A thousand idle threads is most of a gigabyte, which is why the next phase pools them and why virtual threads exist.
What this is really testing

Whether you understand why thread pools exist before Phase XIV tells you. Each thread gets a stack, and a stack is real memory reserved before the thread does anything at all.

08

After the credits

One sentence in this section was doing a lot of unexplained work.

It stays on the heap until nothing can reach it any more, and then the garbage collector takes it at some later moment of its choosing.

Both halves need answering. What does “nothing can reach it” mean exactly, given that objects point at each other and two objects can point at each other while nothing points at either? And what decides “some later moment”?

Section 13.2 answers both. It also explains why the heap is split into a young part and an old part. That follows from a single observation about how long objects actually live. And it explains why Java’s default collector is judged on how long it stops your program rather than how fast it runs.

That last point is the connection to what comes next. A pause stops every thread, and Phase XIV is where having many threads starts to matter.

Threads you opened in this section

The memory areas will return in 13.2 - What Makes an Object Garbage