05 - Java Virtual Threads (Project Loom): Thread Platform vs Virtual Threads and Structured Concurrency

The problem: one thread per request, but threads are few

A server that dedicates one thread to each request is the simplest model to write, to read in a debugger and to analyze in a stack trace. Its limit is that every classic Java thread (platform thread) is tied to an operating system thread, and those are few: if each request spends most of its time waiting on a database, your throughput is set by the number of threads, not by the CPU. The historical answer was asynchronous programming (CompletableFuture, Reactor), which scales but shatters the code into callbacks.

JEP 444, which made virtual threads stable in Java 21, states as its goal to make applications written in the thread-per-request style scale "with near-optimal hardware utilization". A virtual thread is still a Thread, with the same stack trace and the same tools, but it is scheduled by the JVM: when it blocks on a blocking operation, it gives up the OS thread it was running on (the carrier) to another virtual thread. For a comparison with the other models, see the first article in the series.

All the listings were compiled and run with OpenJDK 21.0.13 (javac 21); the comparison on pinning was also run with OpenJDK 25.0.5. The timings are single measurements on one machine and change from one run to another.

Creating virtual threads: three ways, one real case

There are three entry points. Thread.ofVirtual() creates a single thread with a builder, useful for experiments. Executors.newVirtualThreadPerTaskExecutor() creates a new virtual thread for every task submitted, and it is the form to use in practice; unlike a pool, there is nothing to size or to reuse, because a virtual thread is cheap. The third, in the listing, is a million tasks.

import java.util.*;
import java.util.concurrent.*;

public class Api {
    public static void main(String[] args) throws Exception {
        // 1. A named virtual thread
        Thread vt = Thread.ofVirtual().name("my-virtual").start(() ->
            System.out.println("inside: virtual=" + Thread.currentThread().isVirtual()
                + ", name=" + Thread.currentThread().getName()));
        vt.join();

        // 2. An executor that creates one virtual thread per task
        try (ExecutorService exec = Executors.newVirtualThreadPerTaskExecutor()) {
            Future<String> f = exec.submit(() -> {
                Thread.sleep(100);              // suspends the virtual thread, not the carrier
                return "result";
            });
            System.out.println(f.get());
        }

        // 3. A million tasks, one virtual thread each
        long t0 = System.nanoTime();
        long somma = 0;
        try (ExecutorService exec = Executors.newVirtualThreadPerTaskExecutor()) {
            List<Future<Long>> futures = new ArrayList<>();
            for (int i = 0; i < 1_000_000; i++) {
                final long id = i;
                futures.add(exec.submit(() -> id * 2));
            }
            for (Future<Long> f : futures) somma += f.get();
        }
        System.out.println("sum = " + somma);
        System.out.println("time: " + (System.nanoTime() - t0) / 1_000_000 + " ms (varies from machine to machine)");
    }
}

// Output:
// inside: virtual=true, name=my-virtual
// result
// sum = 999999000000
// time: 569 ms (varies from machine to machine)

The first thread confirms isVirtual() == true. The second task sleeps 100 ms with Thread.sleep, which suspends only the virtual thread. The third runs a million tasks, one per virtual thread, and the sum 999,999,000,000 is exact (2 times the sum of 0..999,999). The try block with the executor matters: since Java 21 ExecutorService is AutoCloseable, and close() waits for all tasks to finish, so no virtual thread outlives the block. The time (569 ms here) is an order of magnitude; with an OS thread per task the same program would not be practical on a normal machine, but I did not try it.

Mount and unmount: what happens while you wait

A virtual thread is mounted on a carrier while it runs code, and unmounted when it blocks on an operation the JVM knows how to suspend (network I/O, sleep, java.util.concurrent locks). The scheduler is a ForkJoinPool whose parallelism, the JEP says, by default equals the number of available processors and is changed with the jdk.virtualThreadScheduler.parallelism property. To see the effect I reduce it to a single carrier: if the three tasks slept while keeping the carrier busy, it would take 300 ms.

import java.util.concurrent.*;

public class Montaggio {
    public static void main(String[] args) throws Exception {
        // With a single carrier (-Djdk.virtualThreadScheduler.parallelism=1) three virtual threads
        // sleeping together take 100 ms in total, not 300.
        long t0 = System.nanoTime();
        try (ExecutorService exec = Executors.newVirtualThreadPerTaskExecutor()) {
            for (int i = 1; i <= 3; i++) {
                final int n = i;
                exec.submit(() -> {
                    String prima = Thread.currentThread().toString();
                    try { Thread.sleep(100); } catch (InterruptedException e) { }
                    String dopo = Thread.currentThread().toString();
                    System.out.println("vt" + n + " before: " + prima.substring(prima.indexOf('/') + 1)
                        + " | after: " + dopo.substring(dopo.indexOf('/') + 1));
                });
            }
        }
        System.out.println("total: " + (System.nanoTime() - t0) / 1_000_000 + " ms");
    }
}

// Output:
// vt1 before: runnable@ForkJoinPool-1-worker-1 | after: runnable@ForkJoinPool-1-worker-1
// vt2 before: runnable@ForkJoinPool-1-worker-1 | after: runnable@ForkJoinPool-1-worker-1
// vt3 before: runnable@ForkJoinPool-1-worker-1 | after: runnable@ForkJoinPool-1-worker-1
// total: 156 ms

All three tasks run on the same carrier (ForkJoinPool-1-worker-1) but the total is 156 ms, not 300: while one sleeps, the carrier is free for the others. The listing prints the carrier's name before and after sleep; with a single carrier it cannot change, with several carriers a virtual thread may resume on one different from where it started. This is why you must never rely on the identity of the OS thread, nor use ThreadLocal as if it were tied to the carrier.

Pinning: when a virtual thread cannot unmount

JEP 444 lists two cases in which a virtual thread stays pinned to its carrier while it blocks: inside a synchronized block or method, and while running a native method. If at that moment it does I/O, the carrier and the OS thread stay blocked. The practical consequence is not an error, it is a loss of scalability: with few carriers, a few waits inside synchronized are enough to stop everything else. The listing measures it with a single carrier and four 200 ms tasks, each with its own lock (no contention: the only cost is the pinning).

import java.util.concurrent.*;
import java.util.concurrent.locks.ReentrantLock;

public class Pinning {
    static long misura(Runnable lavoro) throws Exception {
        long t0 = System.nanoTime();
        try (ExecutorService exec = Executors.newVirtualThreadPerTaskExecutor()) {
            for (int i = 0; i < 4; i++) exec.submit(lavoro);
        }
        return (System.nanoTime() - t0) / 1_000_000;
    }

    static void dormi() {
        try { Thread.sleep(200); } catch (InterruptedException e) { }
    }

    public static void main(String[] args) throws Exception {
        // Each task has ITS OWN lock: no contention, the only cost is the pinning.
        long conSynchronized = misura(() -> {
            Object lock = new Object();
            synchronized (lock) { dormi(); }
        });
        long conReentrantLock = misura(() -> {
            ReentrantLock lock = new ReentrantLock();
            lock.lock();
            try { dormi(); } finally { lock.unlock(); }
        });
        System.out.println("Java " + Runtime.version().feature()
            + " - 4 tasks of 200 ms on 1 carrier");
        System.out.println("  synchronized:    " + conSynchronized + " ms");
        System.out.println("  ReentrantLock:   " + conReentrantLock + " ms");
    }
}

// Output:
// Java 21 - 4 tasks of 200 ms on 1 carrier
//   synchronized:    831 ms
//   ReentrantLock:   200 ms
//
// Java 25 - 4 tasks of 200 ms on 1 carrier
//   synchronized:    213 ms
//   ReentrantLock:   200 ms

With Java 21 the synchronized takes 831 ms, that is, four times 200 ms in sequence plus a small overhead, while ReentrantLock takes 200 ms: the four tasks sleep together. With Java 25 the difference disappears (213 ms against 200 ms), because JEP 491, delivered in Java 24, makes virtual threads blocked in synchronized release the carrier. The old advice "replace synchronized with ReentrantLock" therefore holds only if you must stay on Java 21, and only for the blocks that enclose long blocking operations. It does not hold for native methods, which keep pinning.

To find pinning in Java 21, JEP 444 points to the -Djdk.tracePinnedThreads=short property, which prints the point where the thread blocks while pinned, and to a Java Flight Recorder event; JEP 491 explains that in the version without pinning the property is no longer needed and the jdk.VirtualThreadPinned event remains for the residual cases.

StructuredTaskScope: structured concurrency, still a preview

An important correction to what circulates: StructuredTaskScope is not stable in Java 25. In Java 21 it is a preview API (JEP 453); in Java 25 it is in its fifth preview (JEP 505), and in JEP 505 the API has changed: ShutdownOnFailure and ShutdownOnSuccess no longer appear, replaced by StructuredTaskScope.open() with a Joiner that decides when the scope ends. The idea stays the same: the child tasks (fork) live inside a try block and cannot outlive it. The listing uses the Java 21 API and must be compiled with --enable-preview; without it, javac answers "StructuredTaskScope is a preview API and is disabled by default".

import java.util.concurrent.*;
import java.util.concurrent.StructuredTaskScope.Subtask;

public class Scope21 {
    static String utente() throws Exception { Thread.sleep(50); return "mario"; }

    static String ordini() throws Exception {
        try { Thread.sleep(2000); return "3 orders"; }
        catch (InterruptedException e) { System.out.println("orders: interrupted"); throw e; }
    }

    static String impostazioni() throws Exception {
        Thread.sleep(100);
        throw new IllegalStateException("settings service unavailable");
    }

    public static void main(String[] args) throws Exception {
        long t0 = System.nanoTime();
        try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
            Subtask<String> u = scope.fork(Scope21::utente);
            Subtask<String> o = scope.fork(Scope21::ordini);
            Subtask<String> s = scope.fork(Scope21::impostazioni);
            scope.join().throwIfFailed();
            System.out.println(u.get() + ", " + o.get() + ", " + s.get());
        } catch (ExecutionException e) {
            System.out.println("failed: " + e.getClass().getSimpleName()
                + " with cause " + e.getCause());
        }
        System.out.println("time: " + (System.nanoTime() - t0) / 1_000_000 + " ms");

        // First valid answer: the others are cancelled
        try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
            scope.fork(() -> { Thread.sleep(300); return "primary"; });
            scope.fork(() -> { Thread.sleep(50);  return "cache"; });
            scope.fork(() -> { Thread.sleep(150); return "secondary"; });
            System.out.println("wins: " + scope.join().result());
        }
    }
}

// Output:
// orders: interrupted
// failed: ExecutionException with cause java.lang.IllegalStateException: settings service unavailable
// time: 143 ms
// wins: cache

In the first scope there are three tasks: utente (user) succeeds at once, ordini (orders) would wait two seconds, impostazioni (settings) fails after 100 ms. With ShutdownOnFailure the failure closes the scope: ordini receives an interruption ("orders: interrupted") and the block ends in 143 ms instead of 2000. Note the type of the exception: throwIfFailed throws an ExecutionException with the original error as its cause; it is not true that "the stack trace stays without wrapping", as you often read. The second scope, ShutdownOnSuccess, returns the first available result (the cache at 50 ms) and cancels the other two.

The Java 25 API, according to JEP 505, is the same idea with fewer classes: the common case becomes try (var scope = StructuredTaskScope.open()) { ...; scope.join(); }, in which join propagates the exception of the first failed task. I could not run it: the machine has only the Java 25 JRE, with no javac usable with --release 25. The fragment comes from the JEP, not from a run of mine.

Why not CompletableFuture: the comparison in a program

In code that uses CompletableFuture.allOf, a failure does not cancel the siblings. Even worse, allOf completes only when all of them have finished, even if one failed straight away. The listing starts a task that fails at 50 ms and one that would have finished at 300 ms.

import java.util.concurrent.*;

public class CfOrfano {
    public static void main(String[] args) throws Exception {
        CompletableFuture<String> utente = CompletableFuture.supplyAsync(() -> {
            sleep(50);
            throw new IllegalStateException("user not found");
        });
        CompletableFuture<String> ordini = CompletableFuture.supplyAsync(() -> {
            sleep(300);
            System.out.println("orders: finished anyway, for nobody");
            return "3 orders";
        });
        try {
            CompletableFuture.allOf(utente, ordini).get();
        } catch (ExecutionException e) {
            System.out.println("allOf: " + e.getCause());
        }
        sleep(500);   // give the orphan task time to show itself
    }

    static void sleep(long ms) { try { Thread.sleep(ms); } catch (InterruptedException e) { } }
}

// Output:
// orders: finished anyway, for nobody
// allOf: java.lang.IllegalStateException: user not found

The order of the lines is the proof: "orders: finished anyway, for nobody" appears before the error from allOf, because the composed future fires only when the work is done. Whoever awaits the result has waited 300 ms to learn of a failure that happened at 50 ms, and the orders work was done for nothing. With StructuredTaskScope the same situation took 143 ms and interrupted the superfluous task. That is the concrete advantage of structured concurrency: the lifetime of the child tasks is tied to the block in which they were born.

What it is really worth: 10,000 waits of 100 ms

The case where virtual threads shine is that of many short waits. The comparison in the listing is between a fixed pool of 200 OS threads and one virtual thread per task, on 10,000 tasks that sleep 100 ms. The expected figure for the pool is simple: 10,000 divided by 200 gives 50 rounds of 100 ms, that is, about 5000 ms.

import java.util.*;
import java.util.concurrent.*;

public class Bench {
    static long esegui(ExecutorService exec) throws Exception {
        long t0 = System.nanoTime();
        try (exec) {
            List<Future<?>> fs = new ArrayList<>();
            for (int i = 0; i < 10_000; i++) {
                fs.add(exec.submit(() -> { Thread.sleep(100); return null; }));
            }
            for (Future<?> f : fs) f.get();
        }
        return (System.nanoTime() - t0) / 1_000_000;
    }

    public static void main(String[] args) throws Exception {
        System.out.println("10,000 tasks with a 100 ms wait");
        System.out.println("  fixed pool of 200 platform threads: " + esegui(Executors.newFixedThreadPool(200)) + " ms");
        System.out.println("  virtual thread per task:            " + esegui(Executors.newVirtualThreadPerTaskExecutor()) + " ms");
    }
}

// Output:
// 10,000 tasks with a 100 ms wait
//   fixed pool of 200 platform threads: 5048 ms
//   virtual thread per task:            154 ms

The pool takes 5048 ms, consistent with the prediction; the virtual threads 154 ms, just over the 100 ms of a single wait. This is not a server benchmark: sleep uses neither CPU nor network, so the program shows only that waiting does not occupy an OS thread. For CPU-dominated loads virtual threads offer no advantage, because the number of carriers equals the number of processors.

Spring Boot: one line, and what stays the same

In Spring Boot 3.2 or later, with Java 21, the spring.threads.virtual.enabled=true property makes the web server and the automatically configured executors use virtual threads. The Spring Boot documentation specifies that the automatic AsyncTaskExecutor becomes a SimpleAsyncTaskExecutor with virtual threads. This block is a pattern to adapt, not a runnable program: it depends on a complete Spring project, which I did not start.

# application.properties (Spring Boot 3.2 or later, Java 21 or later)
spring.threads.virtual.enabled=true

# The database connection pool stays, and stays sized:
spring.datasource.hikari.maximum-pool-size=20

The second line is there to correct a wrong idea: the database connection pool does not become superfluous. One virtual thread per request potentially means thousands of requests asking for a connection; it is the pool, with its maximum, that protects the database. The limit moves from the number of threads to the number of connections, and rightly so. The same holds for any other limited resource: a virtual thread does not create capacity, it only makes it easier to use.

When virtual threads are not needed

• CPU-dominated loads (computation, compression): the number of cores remains the limit.
• Code that holds synchronized locks around I/O on Java 21, or that calls native code often: pinning cancels the advantage.
• Services that are already reactive and working: rewriting them for virtual threads is a cost with no urgency.

The list has a single criterion: virtual threads make waiting cheap, not computation. If your system waits little, has few threads or is already asynchronous, the gain is small. If instead it is a blocking service that holds thousands of requests waiting on a database or an API, it is the case they were born for.

What you can do now

You can create virtual threads and choose the right executor, you can predict what happens when one blocks, recognise the pinning of synchronized on Java 21 and know that since Java 24 it is gone. You can use StructuredTaskScope knowing it is a preview, and why it is better than CompletableFuture.allOf when a failure must interrupt the siblings.

The next article brings the same question to Rust: Future, Pin and Tokio. For structured cancellation in the other ecosystem, see asyncio.TaskGroup; for the defects of locks, race conditions and deadlocks. For another way of making concurrency cheap, with goroutines, see the article on goroutines and channels.

Timings and counts (569 ms, 156 ms, 831 ms against 200 ms, 143 ms, 5048 ms against 154 ms) come from single runs with OpenJDK 21.0.13; the pinning comparison was repeated with OpenJDK 25.0.5.