Futures That Do Not Block, and Threads That Are Nearly Free

CompletableFuture is a Future you attach to instead of waiting on.

CompletableFuture.supplyAsync(() -> fetchUser(id))
    .thenApply(User::name)
    .thenAccept(System.out::println);

Nothing blocks. Each step runs when the one before it finishes.

Those are the interfaces from Section 11.2. thenApply is map, thenCompose is flatMap, and a chain of futures reads like a stream because it is built from the same four shapes.

The page, rewritten:

CompletableFuture<User> user = CompletableFuture.supplyAsync(() -> fetchUser(id));
CompletableFuture<Orders> orders = CompletableFuture.supplyAsync(() -> fetchOrders(id));

user.thenCompose(u -> CompletableFuture.supplyAsync(() -> fetchRecs(u)))
    .thenCombine(orders, (recs, ord) -> render(recs, ord))
    .exceptionally(e -> renderError(e))
    .thenAccept(this::send);

The user and orders calls start together. Recommendations start the moment the user arrives, not after the orders. No thread is parked at any point, and the failure path is part of the chain instead of a try around it.

ThreadLocal gives each thread its own copy.

private static final ThreadLocal<String> requestId = new ThreadLocal<>();

requestId.set(id);          // at the start of the request
...
requestId.get();            // anywhere, five layers down, on the same thread

One variable, one value per thread, no sharing and no locking. It is how logging frameworks carry a request id and how transactions are tracked. It works because a thread’s own storage is not shared, which is the split from Section 14.1.

The trap is a pool. Threads are reused, so a value left behind is still there for the next task, which is somebody else’s request. remove() in a finally is not optional.

ForkJoinPool is the one that has been running your parallel streams.

ForkJoin common parallelism: 7

Seven on an eight core machine, because the calling thread joins in. It is built for work that splits into smaller work, and its trick is work stealing: every thread has its own queue, and a thread that runs out takes from the back of somebody else’s.

That is why a parallelStream over an uneven workload still finishes reasonably. Threads that get easy pieces go and help.

And now the thing that changes the shape of everything above.

Answer from memory before opening the explanation. Then compare your rule with the answer and revisit the example if they differ.

1. What can `CompletableFuture` do that `Future` cannot?

   Show the answer

   Say what happens next, instead of waiting for the answer.

   Future.get() blocks. The only way to use the result is to stop a thread until it exists, which undoes much of the point of having moved the work elsewhere.

   CompletableFuture takes a callback. thenApply transforms the result when it arrives, thenCompose chains another asynchronous call, thenCombine waits for two and joins them, and exceptionally handles a failure. Nothing blocks anywhere.

   The methods are the ones from Phase XI. thenApply takes a Function and thenAccept takes a Consumer, so a chain of futures reads like a stream.

2. Why can a machine run a million virtual threads and only two thousand platform ones?

   Show the answer

   Because of the stack. A platform thread is an operating system thread and reserves around a megabyte before doing anything, from Section 13.1. Verified: this machine failed at 2,027 with an OutOfMemoryError, because pthread_create refused.

   A virtual thread is a Java object on the heap with a small stack that grows as needed. It is not an operating system thread at all. When it blocks, its stack is put aside and the platform thread underneath goes and runs another one.

   Verified: one million virtual threads, each sleeping 100 ms, finished in 2,953 ms on the same machine.

   So the thing that made threads expensive was never the thread. It was the stack the operating system insisted on reserving.

3. Virtual threads make thread pools unnecessary. Why?

   Show the answer

   Because a pool exists to reuse something expensive, and a virtual thread is not expensive.

   Pooling was the answer to two facts: a thread costs a megabyte, and a thread that has finished cannot be started again. Neither is true of a virtual thread, so making one per task is the right design again.

   Executors.newVirtualThreadPerTaskExecutor() is not a pool. It makes a new virtual thread for every task and throws it away afterwards.

   Pools still matter for work that computes rather than waits, because there you are limiting how many things run at once, and that limit is your cores. Virtual threads do not add cores.

Write each solution in VS Code, predict its result, and then run the checks. Reading an example does not replace implementing it.

4 exercises115 pointsabout 110 minutes

import java.util.concurrent.*;

public class Ceiling {
    public static void main(String[] args) throws Exception {
        // TODO: start platform threads in a loop, each sleeping, keeping them alive.
        //       Find the point where it fails. Catch the error and print the count.
        // TODO: read the failure message. What does it name as the cause?
        // TODO: now start a MILLION virtual threads, each sleeping 100 ms. Time it.
        // TODO: try -Xss256k on the platform version. Does the ceiling move?
        // TODO: say what the ceiling was actually made of
    }
}

import java.util.concurrent.*;

public class Pipeline {
    static String fetchUser(int id)      { sleep(200); return "user" + id; }
    static String fetchOrders(int id)    { sleep(300); return "orders" + id; }
    static String fetchRecs(String user) { sleep(200); return "recs for " + user; }
    static void sleep(int ms) { try { Thread.sleep(ms); } catch (Exception e) { } }

    public static void main(String[] args) throws Exception {
        // TODO: do it with Future and get(). Time it. Note where threads park.
        // TODO: do it with CompletableFuture: user and orders in parallel,
        //       recs chained off user, then combine. Time it.
        // TODO: add exceptionally and make fetchOrders throw. What happens?
        // TODO: which methods here are map, and which is flatMap?
    }
}

import java.util.concurrent.*;

public class LeftBehind {
    static final ThreadLocal<String> requestId = new ThreadLocal<>();

    public static void main(String[] args) throws Exception {
        // TODO: a pool of two. Submit ten "requests", each setting requestId
        //       at the start and NOT removing it.
        // TODO: have each task print the requestId it saw BEFORE setting its own
        // TODO: show a task seeing somebody else's id
        // TODO: fix it with remove() in a finally, and confirm
        // TODO: say what the same bug does to memory, and why Section 13.2 applies
    }
}

import java.util.concurrent.*;
import java.util.concurrent.locks.*;

public class Pinned {
    static final Object monitor = new Object();
    static final ReentrantLock lock = new ReentrantLock();

    public static void main(String[] args) throws Exception {
        // TODO: many virtual threads that each take `monitor` with synchronized
        //       and then sleep. Time the whole batch.
        // TODO: the same batch using the ReentrantLock instead. Time it.
        // TODO: run with -Djdk.tracePinnedThreads=full and read the output
        // TODO: now run a CPU-heavy task on a million virtual threads.
        //       Is it faster than eight platform threads? Why not?
    }
}