Layers of Logic

14.1

What a Thread Actually Is

Your single threaded program has six threads in it and always has. This section is about what each one owns, what they all share, and why that one split makes everything after it hard.

Core19 min read4 exercises
01

Previously on

Section 13.1 gave you the areas and one sentence that was pointing here:

Each thread has its own stack. All of them share one heap.

Section 11.6 gave you the failure:

IntStream.range(0, 100_000).parallel().forEach(list::add);   // 40787 of 100000

Sixty thousand values lost. No exception, and a different number every run. This is the phase that explains it.

02

The problem

Start with something that sounds beyond question. Your program is single threaded.

Ask it:

Thread.getAllStackTraces().keySet().forEach(t -> System.out.println(t.getName()));
Common-Cleaner (daemon)
Finalizer (daemon)
Notification Thread (daemon)
Reference Handler (daemon)
Signal Dispatcher (daemon)
main

Six. Your main is one of them, and the JVM was running the other five before your first line. One handles references for the garbage collector from Phase XIII. Another watches for signals from the operating system.

You have been writing concurrent programs the whole time. You have not had to think about it, because those threads never touch your data.

Now the reason you need more than one. Read a file, then call a network service, then write a report:

String data = readFile();        // 100 ms, the disk is working, the processor is idle
String extra = callService();    // 200 ms, the network is working, the processor is idle
writeReport(data, extra);        //  50 ms

Three hundred and fifty milliseconds, and for three hundred of them your processor did nothing at all. It was waiting. Meanwhile the machine has eight cores and seven of them were never used.

Two different reasons to want more threads, and they are not the same reason.

Waiting is one. While one thread waits for a disk, another can be doing useful work. This helps even on a single core, because waiting is not work.

Splitting is the other. Eight cores can do eight things at once, which is what parallelStream was doing in Section 11.6.

The obvious question is why not just start more processes. Because a process gets its own memory. Two processes cannot see each other’s objects at all. Passing data between them means writing it to a pipe, a socket or a file, and reading it back. Starting one is expensive, and sharing anything is work.

03

The idea

A process is a running program with its own memory. A thread is one path of execution inside a process, sharing that memory with the others.

ProcessThread
Memoryits own, isolatedshared with every other thread in the process
Starting oneexpensive: a new address spacecheaper, and still not free
Talking to anotherpipes, sockets, filesread the same object
One crashingthe others are fineusually takes the whole process down

Inside the JVM the split is exactly the areas from Section 13.1:

Per thread                    Shared by all threads
──────────                    ─────────────────────
Stack                         Heap
  frames                        every object
  local variables               every array
  parameters                  Method area
PC register                     static fields
                                class definitions

That is the whole of concurrency in one picture. Locals are private for free, because no other thread can reach a frame that is not its own. Everything on the heap is reachable by all of them at once.

Making one:

Thread t = new Thread(() -> {
    System.out.println("running on " + Thread.currentThread().getName());
});
t.start();

The lambda is a Runnable, which has one abstract method, so it is a functional interface from Section 11.1. Nothing new.

And now the mistake everyone makes once:

new Thread(() -> System.out.println(Thread.currentThread().getName())).run();
main

run() is an ordinary method call. It runs on the thread you called it from and finishes before the next line. No thread was created. The code compiles, prints something sensible, and is entirely sequential.

start() is the one that asks the operating system for a thread. Only start() does that.

Proof that stacks are separate:

Runnable show = () -> {
    int local = 0;
    for (int i = 0; i < 3; i++) local++;
    System.out.println(Thread.currentThread().getName() + " local=" + local);
};
new Thread(show, "worker-1").start();
new Thread(show, "worker-2").start();
worker-1 local=3
worker-2 local=3

One Runnable, one local in the source, two values in memory. Each thread has its own frame, so each has its own copy, and neither can affect the other.

Change local to a field on a shared object and everything about that changes. That is Section 14.4.

04

Under the hood

Going deeper

A thread is an operating system thread. new Thread(...).start() asks the OS for a real, schedulable thread. Java does not simulate them.

Which means the OS decides who runs and when. With eight cores and fifty threads, the scheduler gives each a slice of time and switches between them. You do not control the order, and it is different on every run.

What a context switch costs

  1. Save the current threadIts registers and program counter are written to memory so it can be resumed exactly where it stopped.
  2. Choose the next oneThe scheduler picks from the runnable threads using priorities and how long each has waited.
  3. Load itIts saved registers and program counter are restored.
  4. Pay for the cachesThe new thread wants different memory, so the processor caches are full of the wrong data. This is usually the largest part of the cost, and it does not show up in any timer.

More threads than cores means constant switching. Past a point, adding threads makes a program slower while every measurement still says the processor is busy.

Each thread reserves a stack when it starts, around 512 KB to 1 MB by default. Not when it does work: when it starts. A thousand idle threads is most of a gigabyte, before your program holds a single object.

That is the number the whole phase is built around. It is why Section 14.10 pools threads instead of making them, and it is exactly the problem virtual threads in Section 14.11 were designed to remove.

Daemon threads do not keep the JVM alive. The five threads listed at the start are all daemons. When every non daemon thread has finished, the JVM exits and takes the daemons with it wherever they had got to.

t.setDaemon(true);   // must be before start()

Your main is not a daemon, and neither is a thread you create unless you say so. That is why a program with a running worker thread does not exit when main returns.

Priorities are a hint, and mostly not even that. setPriority takes 1 to 10 and passes it to the operating system, which is free to ignore it. On some platforms it does nothing at all. Correctness that depends on one thread being scheduled before another is not correctness.

05

What it costs

Threads are not free at any point in their life. Starting one involves the operating system, and it reserves a stack before doing anything. Switching between them costs cache misses that never appear in a profiler as a line you can point at.

More of them stops helping and starts hurting, and the turn is not marked. Past the number of cores, extra threads add switching without adding work, and every measurement still shows the processor fully occupied.

The harder cost is that you have given up the order of events. Two threads produce output in whatever order the scheduler chose that time, and it will differ between runs on the same machine. Every debugging habit you have relies on being able to run something again and see the same thing.

Sharing memory is what makes threads useful and it is also the whole problem. Two threads reading and writing one field is where this phase’s failures come from, and Section 11.6 already showed you what it looks like: a number that is quietly wrong, with no error attached.

And a thread that fails fails alone. An uncaught exception ends that thread and prints its trace, and the rest of the program carries on with no idea anything happened. Nothing propagates back to whoever started it.

06

Check yourself

Answer from memory before opening the explanation. Then compare your rule with the answer and revisit the example if they differ.

  1. What does a thread have of its own, and what does it share?

    Show the answer

    Its own: a stack, and a program counter. That means every local variable, every parameter and every frame belongs to one thread and no other thread can see them.

    Shared: the heap and the method area. Every object, every array and every static field is visible to every thread at once.

    Those are the areas from Section 13.1, split by who can reach them. A local is safe without any effort on your part, because reaching it is not possible. Anything on the heap is reachable by all of them, and that is where every problem in this phase comes from.

  2. What is the difference between calling `run()` and calling `start()`?

    Show the answer

    start() asks the operating system for a new thread, which then calls run() on its own stack.

    run() is an ordinary method call. It runs on the thread you called it from, right now, and finishes before the next line. No thread is created at all.

    It compiles, it produces the right output, and it is completely sequential. Verified: calling run() directly on a new Thread prints main as the current thread name.

    This is the most common first mistake with threads, and nothing warns you. Your program works and gets none of the concurrency you wrote it for.

  3. Why is starting a thread expensive, when starting a process is more expensive still?

    Show the answer

    A process gets its own memory space. Starting one means the operating system building a new address space, loading the program, and setting up its own heap. Nothing is shared, so two processes talk through pipes, sockets or files.

    A thread lives inside a process and shares its memory. Starting one is much cheaper than a process, and it is not free: the operating system is involved, and the thread reserves a stack, usually around 512 KB to 1 MB, before it does anything.

    A thousand idle threads is most of a gigabyte of stacks. That number is why the rest of this phase pools threads rather than making them.

07

Exercises

Write each solution in VS Code, predict its result, and then run the checks. Reading an example does not replace implementing it.

4 exercises85 pointsabout 85 minutes

The Layers of Logic VS Code extension runs the checks for exercises marked checked. For a manual exercise, run the program and compare its behaviour with the stated requirements and sample output.
A

Count the Threads You Did Not Start

Warm up·15 min·15 points

ex-14-1-a

Guess the number before you run it. Most people say one.

The answer is six, and five of them were running before your first line. They are daemons, which means they will not keep the JVM alive when your work finishes, and they have been doing real jobs the entire time.

Look up two of them properly. One exists to service the reference types you met in Section 13.2, which means the garbage collector has had a helper thread in your program since the day you started this course.

What your program must do

  • Predict the thread count before running, then print every name
  • Say which are daemons and what that means for shutdown
  • Say which of them exists because of the garbage collector
  • Start one of your own and confirm it appears in the list
WhoIsRunning.java
public class WhoIsRunning {
    public static void main(String[] args) {
        // TODO: print the name of every thread currently alive.
        //       Predict how many there will be before you run it.
        // TODO: mark which ones are daemons
        // TODO: look up what Reference Handler and Common-Cleaner are for.
        //       One of them connects straight back to Phase XIII.
        // TODO: start a thread of your own and print the list again
    }
}
Hint 1
Thread.getAllStackTraces().keySet() gives you every live thread. t.isDaemon() tells you which kind each one is.
Hint 2
Expect around six: main, plus Reference Handler, Finalizer, Signal Dispatcher, Notification Thread and Common-Cleaner. All five of those are daemons.
Hint 3almost the answer
Reference Handler processes the weak and soft references from Section 13.2. Your program has been co-operating with the collector on a separate thread the whole time.
What this is really testing

Whether "single threaded" was ever true. Your smallest program has been running six threads, and knowing what they are makes the JVM feel less like a black box.

B

run() Is Not start()

Warm up·20 min·20 points

ex-14-1-b

Two characters of difference and a threefold difference in time.

Predict both numbers first. The run() version is sequential, so the sleeps add up. The start() version overlaps them, so it takes about as long as one.

The last part is the one worth repeating a few times. With real threads the output order changes between runs on the same machine with the same code. That is not a bug in your program, and getting used to it now will save you a lot of confusion in the next few sections.

What your program must do

  • Predict both timings before running, then measure
  • Print the thread name inside the job and compare the two versions
  • Say why the run() version takes three times as long
  • Run the start() version several times and describe the output order
RunOrStart.java
public class RunOrStart {
    static void job(String label) {
        System.out.println(label + " on " + Thread.currentThread().getName());
        try { Thread.sleep(300); } catch (Exception e) { }
        System.out.println(label + " done");
    }

    public static void main(String[] args) {
        // TODO: three threads, started with run(). Time the whole thing.
        // TODO: three threads, started with start(). Time it again.
        // TODO: predict both timings BEFORE running
        // TODO: print the thread name inside each. What differs?
        // TODO: with start(), is the output order the same every run?
    }
}
Hint 1
run() is an ordinary method call. It runs on the caller's thread, finishes, and only then does the next line begin. No thread is created.
Hint 2
Each job sleeps 300 ms. Three sequential calls take about 900 ms. Three real threads take about 300 ms, because they sleep at the same time.
Hint 3almost the answer
Run the start() version five times and note the order of the lines. It will not always match, and nothing in your code decides it. The scheduler does.
What this is really testing

Whether you can spot a program that looks concurrent and is not. This mistake compiles, prints the right answers, and gives you nothing.

C

Two Stacks, One Heap

Real work·25 min·25 points

ex-14-1-c

The same loop, twice, and only one version can go wrong.

Write down what you expect from each before running. The local version is safe and always gives the same answer, and you did not have to do anything to make that true.

Then run the shared version five times and write down all five totals. They will differ, they will be short, and no exception will appear anywhere. That is the failure you saw in Section 11.6 with the parallel stream, reproduced in six lines.

What your program must do

  • Run the local version on two threads and predict both values
  • Do the same with a shared static field and predict the total
  • Run the shared version five times and record all five totals
  • Explain both results in terms of which memory area the variable is in
Split.java
public class Split {
    static int sharedField = 0;

    public static void main(String[] args) throws Exception {
        Runnable withLocal = () -> {
            int local = 0;
            for (int i = 0; i < 100_000; i++) local++;
            System.out.println(Thread.currentThread().getName() + " local=" + local);
        };

        // TODO: run withLocal on two threads. Predict both numbers first.
        // TODO: now write the same loop using sharedField instead of a local.
        //       Predict the total before running. Run it five times.
        // TODO: explain both results using stack and heap
        // TODO: which of the two could you have got wrong without any error appearing?
    }
}
Hint 1
A local lives in a stack frame, and each thread has its own stack. Two threads running the same code have two separate copies, and neither can reach the other's.
Hint 2
A static field lives in the method area and there is one of it. Both threads read and write the same memory.
Hint 3almost the answer
Expect 100000 from each local, every time. Expect the shared total to be less than 200000, and to be a different number on each run. Nothing throws, and that is the point.
What this is really testing

Whether the split from Section 13.1 is a diagram or something you can demonstrate. A local is private for free. A field is not, and the difference is the whole phase.

D

Waiting Is Not Working

Real work·25 min·25 points

ex-14-1-d

Two reasons to want threads, and they behave nothing alike.

Sleeping threads do not compete for anything, so adding more of them keeps helping long past the number of cores. Computing threads compete for the same eight cores, and past that point you are paying for switching and getting nothing back.

The 64 thread run is the one that makes it obvious. Same number of threads, same machine, and the two graphs go in opposite directions. That single observation is how every thread pool in the rest of this phase gets sized.

What your program must do

  • Time the waiting work sequentially and on eight threads
  • Do the same for the computing work
  • Try 64 threads of each and record what happens to both
  • Say how you would size a pool for each kind of work, and why they differ
TwoReasons.java
public class TwoReasons {
    static void waiting()  { try { Thread.sleep(500); } catch (Exception e) { } }
    static long working()  { long x = 0; for (int i = 0; i < 200_000_000; i++) x += i; return x; }

    public static void main(String[] args) throws Exception {
        // TODO: run waiting() eight times, sequentially. Time it.
        // TODO: run it on eight threads. Time it. Predict first.
        // TODO: now do the same two experiments with working()
        // TODO: try 64 threads of each. Which one improves and which gets worse?
        // TODO: write one sentence on how you would size a pool for each kind of work
    }
}
Hint 1
Waiting is not work. Eight threads asleep finish in about the time one takes, because nothing is competing for a processor.
Hint 2
Computing is work. Eight threads on eight cores can go roughly eight times faster, and that is the whole ceiling.
Hint 3almost the answer
At 64 threads the waiting version still improves, and the computing version gets worse. That is why pools for network calls are sized much larger than the core count, and pools for computation are sized to the cores.
What this is really testing

Whether you can tell the two reasons for using threads apart. One helps on a single core and the other cannot, and mixing them up leads to sizing pools badly.

08

After the credits

You made a thread with a lambda, which works because Runnable has one method.

There is another way, older and still in every codebase:

class Worker extends Thread {
    public void run() { ... }
}

Section 14.2 compares them and explains why extending Thread is nearly always the wrong choice, using the inheritance rules from Phase VII. It also walks the six states a thread moves through between new and dead, and shows you how to observe them from a running program.

That state diagram is not decoration. BLOCKED, WAITING and TIMED_WAITING are three different reasons a thread is not running, and telling them apart is how you diagnose a program that has stopped without crashing.

Threads you opened in this section

Thread will return in 14.10 - Stop Making Threads