Layers of Logic

14.4

Three Ways Two Threads Break One Field

You expect 200,000 and get 123,267. Then 115,782. Then 111,996. Nothing throws, the code looks right, and there are three separate problems here rather than one.

Core22 min read4 exercises
01

Previously on

Every failure in this course so far has been repeatable. Run it again, see the same thing, fix it.

Section 11.6 broke that:

IntStream.range(0, 100_000).parallel().forEach(list::add);   // 40787 of 100000

Different number every run, no exception. Section 14.1 explained why it is possible: each thread has its own stack and they all share one heap. This is the section where that becomes precise.

02

The problem

Two threads, one counter, a hundred thousand increments each.

static int counter = 0;

// thread 1: for (int i = 0; i < 100_000; i++) counter++;
// thread 2: for (int i = 0; i < 100_000; i++) counter++;

Two hundred thousand. Five runs:

wanted 200000, got 123267
wanted 200000, got 115782
wanted 200000, got 111996
wanted 200000, got 114211
wanted 200000, got 107227

Five wrong answers, all different, no exception anywhere. About forty percent of the work vanished and the code has no bug you can point at.

Now something that fails differently. One thread spins until a flag is set. Another sets it.

static boolean flag = false;

// reader: while (!flag) { }        then print and finish
// writer: sleep 300ms, flag = true

Verified:

writer set plain = true
reader alive after 2s?  true   <- it never saw the write

The write happened. Two seconds later the reader was still spinning. It is not slow and it is not a wrong answer. That thread will never finish.

And a third one, which is the hardest to believe. These two lines have no relationship:

a = 1;
b = 2;

The compiler and the processor are both allowed to swap them. On one thread you can never tell, because nothing observes the order. On two threads you can, and code that looked at b and drew a conclusion about a has been lied to.

Three failures, three different causes. They get treated as one topic because they show up together, and the fixes are different.

03

The idea

Problem one is atomicity, and the bytecode gives it away.

counter++ looks like one thing. Ask the compiler:

0: getstatic     #7    // read c
3: iconst_1            // push 1
4: iadd                // add
5: putstatic     #7    // write c

Read, change, write. Three steps with two gaps in them, and a thread can be paused in either gap.

How two increments become one

  1. Thread A reads 5It now holds the value 5 in a register. The field still says 5.
  2. Thread B reads 5Before A got to write. B also holds 5. The field still says 5.
  3. Thread A writes 6Its work is done and correct, on its own terms.
  4. Thread B writes 6Also correct on its own terms, and it has just overwritten A. Two increments, one result.

That is a race condition: the answer depends on the order two threads happen to interleave, and nothing in your code decides that order.

Problem two is visibility, and it is not about speed.

A processor core has its own caches. A thread reading a field in a tight loop can keep reading a cached copy and never go back to main memory. The compiler is also allowed to notice that nothing inside the loop changes flag and read it once before the loop starts.

Both are legal, because Java only promises that a single thread sees its own actions in order. Between threads, nothing is promised unless you ask.

volatile is how you ask.

static volatile boolean flag = false;
writer set marked = true
reader2 stopped after 485899624 spins

The same loop, one keyword, and now it stops. A volatile write goes to main memory and every later volatile read of that field sees it.

volatile also fixes problem three. Reads and writes are not allowed to be moved across a volatile access, so anything written before a volatile write is visible to anything that reads it after.

So the three problems and their fixes:

ProblemWhat it needs
Atomicityread-change-write can interleavea lock, or an atomic class. Section 14.5 and 14.8
Visibilitya thread never sees another thread`s writevolatile, or a lock
Orderinginstructions get reorderedvolatile, or a lock

A lock fixes all three. volatile fixes two. That is the whole reason both exist.

04

Under the hood

Going deeper

Why reordering is allowed at all. Because it makes everything faster, and on one thread it is undetectable.

The compiler reorders to keep the processor busy. The processor reorders to avoid waiting for memory. Both follow one rule, called as-if-serial: the result must be what a single thread would have got. Nothing says anything about what another thread sees while it is happening.

That was a reasonable trade in 1995 and it is why the Java Memory Model had to be written. It is the specification saying exactly which writes are guaranteed visible to which reads, and it is built on one relationship called happens-before.

Happens-before is the only guarantee you actually have.

If action A happens-before action B, then everything A did is visible to B. The rules that create it:

  • Anything a thread does happens-before anything it does later, on that thread.
  • Unlocking a monitor happens-before any later locking of the same monitor.
  • A write to a volatile field happens-before any later read of that field.
  • Thread.start() happens-before anything the new thread does.
  • Anything a thread does happens-before another thread returning from join() on it.

Notice the shape. Every rule is about a pair of actions with something in common. Without one of these connections there is no guarantee at all, however obvious the code looks.

That is why start() and join() work without any extra thought. They are on that list.

What volatile compiles to. A memory barrier, which is an instruction telling the processor not to move loads and stores across this point and to make its writes visible.

It is not free. A volatile write is more expensive than a plain one, and a volatile field in a hot loop can be measurably slower. It is much cheaper than a lock, which is why it is the right tool for a flag.

This explains Section 11.6 exactly. ArrayList.add is:

elementData[size] = element;
size = size + 1;

Read size, write into that slot, write size + 1 back. Two threads reading the same size both write into the same slot, and one value is overwritten. Both then set size to the same number.

100,000 became 40,787. Not an error, not a corruption the JVM could detect. Every thread performed a correct sequence of steps, and the steps interleaved.

05

What it costs

The worst cost is that these bugs pass every test you write. Your machine has eight cores, your test has two threads and ten iterations, and it works every time. The failure needs a particular interleaving, and the odds of hitting it go up with load, cores and time. Production has all three.

Adding logging often makes them disappear. A print statement is slow and does its own synchronisation, so it changes the timing enough to hide the race, and you conclude the bug is elsewhere.

They are also not reproducible on demand. There is no input that triggers it. Running the same binary on the same machine gives a different answer, so the ordinary loop of reproduce, fix, verify does not close.

volatile looks like a solution and is a solution to two thirds of the problem. Used on a counter it makes the failure rarer without removing it, which converts a bug you might have caught into one you will not.

And the memory model is genuinely hard. Happens-before is a small set of rules with large consequences. Code that reads as plainly correct can be broken in ways that need a specification to explain. Almost nobody reasons about it directly, which is why the honest advice is to use the tools in the next few sections rather than build your own.

06

Check yourself

Answer from memory before opening the explanation. Then compare your rule with the answer and revisit the example if they differ.

  1. Why does `counter++` lose updates when two threads run it?

    Show the answer

    Because it is three instructions, not one. The bytecode for c++ is:

    getstatic read the current value, then iconst_1 and iadd to add one, then putstatic to write it back.

    Read, change, write. Two threads can both read 5, both work out 6, and both write 6. Two increments happened and the counter went up by one.

    Verified over five runs: 123267, 115782, 111996, 114211, 107227, where 200000 was wanted. Different every time, and nothing throws.

  2. A thread spins on `while (!flag)` and another thread sets `flag = true`. Why might the first one never stop?

    Show the answer

    Because nothing promised it would ever look again.

    Each processor core has its own caches, and a thread reading the same field in a tight loop can keep reading a cached copy. The compiler is also allowed to notice that the loop body never changes flag and hoist the read out of the loop entirely.

    Verified: with a plain boolean, the reader was still spinning two seconds after the write. With volatile, it stopped.

    This one is worse than lost updates, because it is not a wrong number. The thread never finishes at all, and it usually only happens on the machine you do not own.

  3. What does `volatile` fix, and what does it not?

    Show the answer

    Fixes: visibility and ordering. A write to a volatile field is seen by every thread that reads it afterwards, and the compiler and processor may not move other reads and writes across it.

    Does not fix: anything made of several steps. volatile int c; c++; is still read, change, write, and two threads can still interleave those three. Every value read is fresh and updates are still lost.

    So volatile is right for a flag that one thread writes and others read. It is wrong for a counter, and reaching for it there is one of the most common mistakes in concurrent Java.

07

Exercises

Write each solution in VS Code, predict its result, and then run the checks. Reading an example does not replace implementing it.

4 exercises100 pointsabout 95 minutes

The Layers of Logic VS Code extension runs the checks for exercises marked checked. For a manual exercise, run the program and compare its behaviour with the stated requirements and sample output.
A

Five Runs, Five Wrong Answers

Warm up·20 min·20 points

ex-14-4-a

Run it five times and write down all five numbers before doing anything else.

They will differ. That is the part worth sitting with, because every bug you have fixed until now gave the same answer twice.

The small version is the one to be careful about. A hundred iterations will usually come out right, and if that had been your test you would have shipped it. The window did not close, it just got harder to hit, which is exactly what makes these bugs reach production.

What your program must do

  • Run the two thread increment five times and write down all five totals
  • Try it with only 100 iterations each and describe what you see
  • Add a print inside the loop and say what happens to the bug
  • Disassemble counter++ and name the three instructions
FiveRuns.java
public class FiveRuns {
    static int counter = 0;

    public static void main(String[] args) throws Exception {
        // TODO: two threads, 100_000 increments each. Run the whole thing five times.
        //       Print all five totals.
        // TODO: try it with 100 increments each instead. What happens, and why is
        //       that more dangerous than the first result?
        // TODO: put a print inside the loop and run again. What changes?
        // TODO: look at the bytecode for counter++ with javap -c
    }
}
Hint 1
Expect totals well below 200000 and different every run. One set of five was 123267, 115782, 111996, 114211 and 107227.
Hint 2
At 100 iterations it will usually be correct. That is the dangerous case: a test that passes is not evidence, it is a smaller window.
Hint 3almost the answer
javap -c shows getstatic, iconst_1, iadd, putstatic. Read, change, write, with two gaps in the middle where another thread can get in.
What this is really testing

Whether a race condition is a phrase or something you have watched happen. The important part is not that it is wrong, it is that it is differently wrong every time.

B

The Write Nobody Saw

Real work·25 min·25 points

ex-14-4-b

Two threads, one boolean, and one of them waits forever.

There is no lost update here and no interleaving to reason about. A write happened, a reader kept reading, and it never saw it. Locking the write would fix it too, but for a different reason than you might expect.

The print at the end is the cruel part. Add logging to investigate and the bug goes away, which is how visibility problems waste entire afternoons.

What your program must do

  • Show the plain flag never being seen, with a timeout so the test ends
  • Show the volatile flag being seen
  • Add a print inside the spin loop and describe what happens to the bug
  • Explain why this problem is separate from lost updates
NeverSaw.java
public class NeverSaw {
    static boolean plain = false;
    static volatile boolean marked = false;

    public static void main(String[] args) throws Exception {
        // TODO: a thread spinning on while (!plain). Another sets plain = true
        //       after 300 ms. Wait two seconds. Is the reader still alive?
        // TODO: the same with the volatile field. Compare.
        // TODO: add a System.out.println inside the spin loop and try the plain
        //       version again. Does it stop now? Why is that infuriating?
        // TODO: say why this is NOT fixed by synchronizing the counter
    }
}
Hint 1
Make the reader a daemon thread, or your program will not exit when the reader never stops.
Hint 2
The print usually makes the bug disappear, because println synchronises internally and forces the thread to look at memory again. A bug that vanishes when you add logging is the signature of a visibility problem.
Hint 3almost the answer
Nothing here is a lost update. One thread writes once and another never sees it. Atomicity is about several steps interleaving, and this is about whether a write is visible at all.
What this is really testing

Whether visibility is a separate problem from atomicity. This one is not a wrong number, it is a thread that never finishes, and no amount of locking the counter would fix it.

C

What volatile Cannot Do

Real work·20 min·25 points

ex-14-4-c

The obvious fix, applied to the obvious place, and it does not work.

Run it five times. The numbers may look better than the plain version, and none of them will be 200000. Every read was fresh. Every write was visible. And updates were still lost, because visibility was never the problem with a counter.

That gap between “looks improved” and “is correct” is the whole reason this exercise exists. Reaching for volatile here is one of the most common mistakes in concurrent Java. It makes the failure rarer rather than removing it.

What your program must do

  • Run the volatile counter five times and record the totals
  • Compare against the plain version and say whether it improved
  • Fix it with synchronized and confirm the total is exact
  • State in one line each what volatile does and does not fix
NotEnough.java
public class NotEnough {
    static volatile int counter = 0;

    public static void main(String[] args) throws Exception {
        // TODO: two threads, 100_000 increments each, on the VOLATILE counter.
        //       Predict the total. Run it five times.
        // TODO: compare with the non-volatile version from the first exercise.
        //       Is it closer to correct? Is it correct?
        // TODO: now fix it properly with synchronized and run five more times
        // TODO: write one line each on what volatile fixes and what it does not
    }
}
Hint 1
volatile makes every read fresh and every write visible. It does not turn three instructions into one.
Hint 2
Both threads can read the same value from main memory, both add one, and both write the same result back. Every one of those reads and writes was perfectly visible.
Hint 3almost the answer
Expect the volatile version to still be wrong, and sometimes closer to correct than the plain one. That is worse, not better: a bug that is nearly right is harder to catch than one that fails every time.
What this is really testing

Whether volatile looks like a general fix. It solves two of the three problems completely and the third not at all, and using it on a counter makes the bug rarer instead of gone.

D

Rebuild the Parallel Stream Failure

Hard·30 min·30 points

ex-14-4-d

Section 11.6 showed you a number and promised an explanation. Go and get it.

Start by reproducing the stream version, then reproduce the identical failure with plain threads. Once you can do it without streams, it stops being something mysterious about parallel pipelines and becomes three lines inside ArrayList.

The three fixes at the end are where the real lesson is. All three are correct and they are not equivalent, and timing them tells you why collect exists rather than a thread safe list.

What your program must do

  • Reproduce the parallel stream failure and record five sizes
  • Reproduce the same failure with plain threads and no streams
  • Name the steps inside ArrayList.add that make it unsafe
  • Fix it three ways and compare the timings
Rebuilt.java
import java.util.*;
import java.util.stream.*;

public class Rebuilt {
    public static void main(String[] args) throws Exception {
        // TODO: reproduce the Section 11.6 failure with a parallel stream.
        //       Run it five times and record all five sizes.
        // TODO: now reproduce the SAME failure with plain threads and no streams
        // TODO: look at what ArrayList.add does and name the steps
        // TODO: fix it three ways: a synchronized list, a collect, and a
        //       per thread list merged at the end. Time all three.
    }
}
Hint 1
ArrayList.add writes the value into elementData[size] and then sets size = size + 1. Two threads reading the same size write into the same slot, and one value is overwritten.
Hint 2
Without streams it is the same shape: several threads calling list.add on one shared ArrayList in a loop.
Hint 3almost the answer
The three fixes are not equal. A synchronized list is correct and makes every thread queue at one lock. collect gives each thread its own partial result and combines at the end, which is why it is faster and why it is what the API was designed around.
What this is really testing

Whether you can connect Section 11.6 to this section. The 40,787 was not a stream bug, it was ArrayList.add being three steps, and you can reproduce it without streams at all.

08

After the credits

You have three problems and a keyword that fixes two of them.

The missing piece is a way to say one thread at a time through here, and Java has had it since version one:

synchronized (lock) {
    counter++;
}

Section 14.5 is that keyword. It fixes all three problems at once, and the interesting part is what the lock actually is: every object in Java has one built in, going back to the Object class in Section 8.3, and you have been carrying it around unused this whole time.

The section also answers the question that catches people out. synchronized on an instance method and synchronized on a static method lock two completely different things, and code that mixes them protects nothing at all.

Threads you opened in this section

Visibility will return in Phase XIV. Concurrency