Project Loom

Revolution in concurrency
or obscure implementation detail?

Tomasz Nurkiewicz

nurkiewicz.com | @tnurkiewicz

Around IT in 256 seconds podcast

Java Champion, CTO at DevSkiller, trainer at Bottega

TL;DR:

aka. Agenda

1. Create millions of threads
2. Block and sleep() everywhere, like crazy
3. Does it make any difference to us, developers?

What is a thread?

...in Java

User threads

					
						Thread thread = new Thread(() -> System.out.println("Hello"));
						thread.start();
						thread.join();					
					
				

Kernel threads

The scheduler is the component of the kernel that selects which process to run next.

The scheduler [...] can be viewed as the code that divides the finite resource of processor time between the runnable processes on a system.

User threads ≠ Kernel threads

• User threads are created by the JVM
• Kernel thread are created and managed by kernel (duh!)

Many-to-One Model

aka. Green threads

Implementations of the many-to-one model (many user threads to one kernel thread) allow the application to create any number of threads that can execute concurrently. [...]

this multithreading model provides limited concurrency and does not exploit multiprocessors.

Many-to-Many Model

a user-level threads library provides sophisticated scheduling of user-level threads above kernel threads.

The kernel needs to manage only the threads that are currently active.

One-to-One Model

each user-level thread created by the application is known to the kernel [...]

The main problem with this model is that it places a restriction on you to be careful and frugal with threads, as each additional thread adds more "weight" to the process.

User threads = Kernel threads

jstack PID

						
							"Thread-0" #13 prio=5 os_prio=31 cpu=0.13ms elapsed=71.09s 
							tid=0x00007fd008917000 nid=0x6a03 
							waiting on condition  [0x000070000e2a6000]		
						
					

top -H

					
						-H
							[...] all individual threads will be displayed.
							Otherwise, top displays a summation of all threads in a process.
					
				

						
							PID    COMMAND      %CPU TIME     #TH    #WQ  #PORT MEM    PURG   CMPRS PGRP  PPID  STATE    BOOSTS          %CPU_ME %CPU_OTHRS UID  FAULTS     COW     MSGSENT     MSGRECV     SYSBSD      SYSMACH    CSW         PAGEINS
							75368  java         0.0  00:10.00 38     1    146   84M    0B     0B    75082 75082 sleeping *0[1]           0.00000 0.00000    502  47394      393     5635        2795        316143+     11062      158846+     8
							76787  java         0.0  00:20.31 43     1    158  130M    0B     0B    75082 75082 sleeping *0[1]           0.00000 0.00000    502  95759      400     12434       6179        324542      21096      168946      291     44984

						
					

Linux thread = Linux process

from the kernel point of view, only processes exist [...]

and a so-called thread is just a different kind of process

Why threads are expensive?

Thread.start() considered inefficient

• 1 MiB of memory (outside of heap, -Xss)
• Kernel thread

Project Loom

Goal:

JVM supports creating millions of threads

Virtual thread

Lightweight, cheap, user thread

Carrier thread

"Real" (kernel) thread running virtual threads

Continuation

A piece of code that can be paused, at almost no cost

Creating virtual threads

					
						Thread t = Thread.startVirtualThread(() ->
							System.out.println("Hello")
						);
					
				

					
						Thread t = Thread
							.ofVirtual()
							.name("MyThread")
							.unstarted(() -> task());		
					
				

Special ExecutorService

					
						ExecutorService es = Executors.newVirtualThreadPerTaskExecutor();
					
				

...and ThreadFactory

					
						ThreadFactory factory = Thread
							.ofVirtual()
							.factory();
					
				

Virtual thread

Continuation + scheduler

Continuations

sequential code that may suspend (itself) and resume (be resumed by a caller).

[...] may suspend or yield execution at some point [...] When [it] suspends, control is passed outside of the continuation, and when it is resumed, control returns to the last yield point, with the execution context up to the entry point intact

Pseudo-code

					
						void main() {
							c = continuation(foo)
							c.run() 
							...
							c.run() 
						}
 
						void foo() { 
							... 
							bar()
							... 
						}
						​
						void bar() {
							...
							yield() 
							buzz()
						}
					
				

Think:

• Coroutines / goroutines
• async / await
• Python generators
• Ruby fibers

Thread.sleep()

in the XXI century

					
						private static void sleepMillis(long millis) throws InterruptedException {
							if (currentThread() instanceof VirtualThread vthread) {
								vthread.sleepNanos(millis * 1000000);
							} else {
								sleep0(millis);
							}
						}
					
				

Simplified for clarity

VirtualThread.parkNanos()

					
						void parkNanos(long nanos) {
							Future<?> unparker = scheduleUnpark(nanos);
							try {
								continuation.yield();
							} finally {
								cancel(unparker);
							}
						}
					
				

Simplified for clarity

Hello, world!

					
						CountDownLatch latch = new CountDownLatch(COUNT);
						for (int i = 0; i < COUNT; i++) {
							Thread.startVirtualThread(() -> {
								try {
									Thread.sleep(1_000);
									latch.countDown();
								} catch (InterruptedException ignored) {}
							});
						}
						latch.await();
					
				

Carrier thread

					
						var carrierExecutor = Executors.newFixedThreadPool(1);

						ThreadFactory factory = Thread.ofVirtual()
							.name("Virtual-", 1)
							.scheduler(carrierExecutor)
							.factory();		

						var virtualExecutor = Executors.newThreadPerTaskExecutor(factory)
					
				

Structured concurrency

					
						try (ExecutorService exec = Executors.newVirtualThreadExecutor()) {
							//...
						}
					
				

when the flow of execution splits into multiple concurrent flows, they rejoin in the same code block

"Tasks, not Threads"

Run something after 8 hours

					
						Thread.startVirtualThread(() -> {
							TimeUnit.HOURS.sleep(8)
							runSomething();
						});
					
				

Handle each new player/connection/message

					
						var executor = Executors.newVirtualThreadExecutor();

						//...
						void onPlayer(player -> {
							executor.submit(() -> handle(player));
						});
					
				

Download 10K images

					
						try(var executor = Executors.newVirtualThreadExecutor()) {
							for(URL image: images) 
								executor.submit(() -> 
									downloadBlocking(url);
								);
						}
					
				

Hold on! Throttle 🦥

CountDownLatch

Semaphore

Problems and limitations

Deep stack

Preemption

Stack vs. heap memory

I don't need reactive anymore?

RestTemplate

What Loom addresses?

• 👍 Asynchronous programming
• ...
• 👎 Backpressure
• 👎 Change propagation
• 👎 Composability

Should I blindly switch to virtual threads?

Merged!

• github.com/openjdk/jdk/pull/8166
• Files changed: 1133
• +95870, -8270
• C++ / Java

More materials

• Virtual Threads: New Foundations for High-Scale Java Applications by Brian Goetz
• JEP draft: Virtual Threads (Preview)
• JEP draft: Structured Concurrency (Preview)
• Java’s Project Loom, Virtual Threads and Structured Concurrency with Ron Pressler
• Project Loom: Fibers and Continuations for the Java Virtual Machine
• Helidon Níma — Helidon on Virtual Threads
• AugustNagro/java-async-await

Presentations

• Cay Horstmann - Looming Changes in Concurrent Programming
• Heinz Kabutz - Prepare for what "Loom"s ahead
• Launching 10 millions virtual threads with Loom - JEP Café #12
• Adam Warski - Project Loom? Better Futures? What's next for JVM concurrent programming

Blog posts

• On Project Loom, the Reactive model and coroutines
• Structured concurrency: will Java Loom beat Kotlin’s coroutines?
• Going inside Java’s Project Loom and virtual threads
• Project Loom: virtual threads in Java
• Taking Project Loom for a spin
• Java Concurrency With Project Loom

Critical/skeptic

• 🎥 Project Loom - A Friend or Foe of Reactive? - Oleh Dokuka, VMware & Andrii Rodionov
• Will Project Loom obliterate Java Futures?
• Do Loom’s Claims Stack Up?
  1. Part 1: Millions of Threads?
  2. Part 2: Thread Pools?

Thank you!

Tomasz Nurkiewicz

nurkiewicz.com | @tnurkiewicz

Around IT in 256 seconds podcast

Slides: nurkiewicz.com/slides/loom