Virtual Threads ​

Virtual Threads are a lightweight thread implementation introduced in Java 19. In many other languages, they are known as coroutines, fibers, green threads, user-mode threads, etc.

Understanding Virtual Threads ​

Before understanding virtual threads, let's revisit the characteristics of traditional threads:

• Threads are resources created and scheduled by the operating system.
• Thread switching consumes a significant amount of CPU time.
• The number of threads that a system can schedule simultaneously is limited, typically in the hundreds to thousands range.

Therefore, threads are considered heavyweight resources. On the server side, user requests are typically handled by one thread per request. Since the number of user requests often far exceeds the number of threads that the operating system can schedule simultaneously, thread pools are usually used to minimize the cost of frequently creating and destroying threads.

For tasks that need to handle a large number of IO requests, using threads is inefficient because once performing read/write IO operations, the thread must enter a waiting state until the IO data is returned. Common IO operations include:

• Reading/writing files;
• Reading/writing networks, such as HTTP requests;
• Reading/writing databases, essentially implemented through JDBC network calls.

Let's take an example of a thread handling an HTTP request. It enters a waiting state when reading/writing networks or files:

Begin
────────
Blocking ──▶ Read HTTP Request
Wait...
Wait...
Wait...
────────
Running
────────
Blocking ──▶ Read Config File
Wait...
────────
Running
────────
Blocking ──▶ Read Database
Wait...
Wait...
Wait...
────────
Running
────────
Blocking ──▶ Send HTTP Response
Wait...
Wait...
────────
End

The actual CPU-executed code consumes very little time; most of the thread's time is spent waiting for IO. We refer to this type of task as IO-intensive.

Efficient Execution of IO-Intensive Tasks with Virtual Threads ​

To efficiently execute IO-intensive tasks, Java introduced virtual threads starting from Java 19. The interface of virtual threads is the same as that of regular threads, but their execution differs. Virtual threads are not scheduled by the operating system but by regular threads, meaning that hundreds or thousands of virtual threads can be scheduled by a single regular thread. At any moment, only one virtual thread can be executing, but once a virtual thread performs an IO operation and enters a waiting state, it is immediately "suspended," and the scheduler moves on to execute the next virtual thread. When the IO data returns, the suspended virtual thread is rescheduled to continue execution. Thus, multiple virtual threads can alternate running on a single regular thread:

Begin
───────────
V1 Running
V1 Blocking ──▶ Read HTTP Request
───────────
V2 Running
V2 Blocking ──▶ Read HTTP Request
───────────
V3 Running
V3 Blocking ──▶ Read HTTP Request
───────────
V1 Running
V1 Blocking ──▶ Read Config File
───────────
V2 Running
V2 Blocking ──▶ Read Database
───────────
V1 Running
V1 Blocking ──▶ Read Database
───────────
V3 Running
V3 Blocking ──▶ Read Database
───────────
V2 Running
V2 Blocking ──▶ Send HTTP Response
───────────
V1 Running
V1 Blocking ──▶ Send HTTP Response
───────────
V3 Running
V3 Blocking ──▶ Send HTTP Response
───────────
End

Synchronous-Looking Asynchronous Execution ​

If we look at the code of a single virtual thread in a method:

void register() {
    config = readConfigFile("./config.json"); // #1
    if (config.useFullName) {
        name = req.firstName + " " + req.lastName;
    }
    insertInto(db, name); // #2
    if (config.cache) {
        redis.set(key, name); // #3
    }
}

At the IO read/write points #1, #2, and #3, when executing these lines (entering the relevant JNI methods internally), the virtual thread automatically suspends and switches to other virtual threads. When the data returns, the suspended virtual thread is rescheduled and resumes execution. Therefore, the code appears to execute synchronously but is actually executing asynchronously.

Using Virtual Threads ​

The interface for virtual threads is identical to that of regular threads, with the only difference being that virtual threads can only be created through specific methods.

Method 1: Directly Creating and Running a Virtual Thread

// Pass a Runnable instance and start immediately:
Thread vt = Thread.startVirtualThread(() -> {
    System.out.println("Start virtual thread...");
    Thread.sleep(10);
    System.out.println("End virtual thread.");
});

Method 2: Creating a Virtual Thread Without Starting It Immediately

// Create a VirtualThread:
Thread vt = Thread.ofVirtual().unstarted(() -> {
    System.out.println("Start virtual thread...");
    Thread.sleep(1000);
    System.out.println("End virtual thread.");
});
// Start the VirtualThread:
vt.start();

Method 3: Creating Virtual Threads Using a ThreadFactory

// Create a ThreadFactory:
ThreadFactory tf = Thread.ofVirtual().factory();
// Create a VirtualThread:
Thread vt = tf.newThread(() -> {
    System.out.println("Start virtual thread...");
    Thread.sleep(1000);
    System.out.println("End virtual thread.");
});
// Start the VirtualThread:
vt.start();

Directly calling start() is actually scheduled by a ForkJoinPool thread. We can also create our own scheduler threads and run virtual threads:

// Create a scheduler:
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
// Create and schedule a large number of virtual threads:
ThreadFactory tf = Thread.ofVirtual().factory();
for (int i = 0; i < 100000; i++) {
    Thread vt = tf.newThread(() -> { /* ... */ });
    executor.submit(vt);
    // Alternatively, directly submit Runnable or Callable:
    executor.submit(() -> {
        System.out.println("Start virtual thread...");
        Thread.sleep(1000);
        System.out.println("End virtual thread.");
        return true;
    });
}

Since virtual threads are extremely lightweight resources, they should be created as needed and discarded after use, without pooling them.

Important Note: Virtual threads were officially released in Java 21. In Java 19/20, they are preview features and are disabled by default. To enable them, add the --enable-preview flag:

java --source 19 --enable-preview Main.java

Usage Limitations ​

Only code running as virtual threads will automatically suspend and switch to other virtual threads when performing IO operations. IO operations on regular threads will still block. For example, reading/writing files in the main() method will not trigger scheduling and automatic suspension.

Operations that can automatically trigger scheduling and switching include:

• File IO;
• Network IO;
• Waits triggered by the Concurrent library;
• Thread.sleep() operations.

This is because the JDK has modified the underlying operations to implement virtual threads, allowing application-level Java code to use virtual threads without modification. Languages that cannot automatically switch need to manually call await to implement asynchronous operations:

async function doWork() {
    await readFile();
    await sendNetworkData();
}

In virtual threads, bypassing JDK IO interfaces and directly performing file or network operations through JNI will not enable scheduling. Additionally, scheduling cannot occur within synchronized blocks.

Exercise ​

Use virtual threads to schedule 100,000 tasks and observe the time taken:

public class Main {
    public static void main(String[] args) {
        ExecutorService es = Executors.newVirtualThreadPerTaskExecutor();
        for (int i = 0; i < 100000; i++) {
            es.submit(() -> {
                Thread.sleep(1000);
                return 0;
            });
        }
        es.close();
    }
}

Then, change the ExecutorService to a thread pool model and compare the results.

Summary ​

• Virtual Threads, introduced in Java 19, are designed to handle IO-intensive tasks efficiently by allowing a large number of lightweight threads to be managed by a small number of regular threads.

• Execution Efficiency: Virtual threads automatically suspend and switch to other virtual threads when performing IO or blocking operations, preventing the main thread from waiting and maximizing thread execution efficiency.

• Interface Compatibility: Virtual threads use the same interfaces as regular threads, providing the significant advantage of enabling asynchronous execution of existing IO operations without modifying any code, thereby achieving greater throughput.

• Use Cases: Virtual threads are ideal for IO-intensive tasks. For CPU-intensive tasks, virtual threads should not be used; instead, performance should be enhanced by increasing CPU cores or utilizing distributed computing resources.