3400 words
17 minutes
ThreadLocal & ScopedValue
2023-04-11
Notes
ThreadLocal
/
ScopedValue
/
Concurrent

Introduction#

ThreadLocal is a solution for isolating variables between threads, also known as a thread-local variable table. In Java, each thread has a ThreadLocalMap variable under ThreadLocal, which stores the ThreadLocal objects defined in the thread. The keys of the ThreadLocalMap are weak references pointing to their respective ThreadLocal objects. However, every Java developer should note that if ThreadLocal variables are not removed in time using the remove() method, they can cause serious memory leak issues.

In the JDK 20 Early-Access Build 28 version, a new ScopedValue class was introduced as a redesign of the ThreadLocal class. ScopedValue is an incubating feature in JDK 20 and requires manual configuration to use. Its purpose is to serve as a substitute for ThreadLocal in certain scenarios. With ScopedValue, different code running on the same thread can share immutable values. ScopedValue primarily addresses some of the issues that may arise when using ThreadLocal with virtual threads.

This article introduces several real-world scenarios where ThreadLocal is used, followed by a detailed explanation of the new concurrency tools proposed in JDK 19 and the incubating ScopedValue class in JDK 20.

ThreadLocal#

Basic Concepts#

As briefly mentioned in the introduction, ThreadLocal exists not only for thread isolation but also to address object reuse issues. These concepts are often reflected in database connection pool frameworks. However, ThreadLocal can lead to memory leaks because ThreadLocalMap might not release the ThreadLocal object in a timely manner, even with the use of weak references (WeakReference). When a weakly referenced ThreadLocal is set to null but not promptly removed via the remove method, memory leaks can still occur.

Application Scenarios#

One common application of ThreadLocal is in scenarios requiring thread isolation, such as managing shopping cart information in Spring-based e-commerce projects.

  1. Thread Isolation in Spring Applications:

    @Service
public class ShoppingCartService {
    private ThreadLocal<ShoppingCart> cartHolder = new ThreadLocal<>();

    public ShoppingCart getCurrentCart() {
        ShoppingCart cart = cartHolder.get();
        if (cart == null) {
            cart = new ShoppingCart();
            cartHolder.set(cart);
        }
        return cart;
    }

    public void checkout() {
        // get current cart
        ShoppingCart cart = getCurrentCart();
        // Process checkout...
        cartHolder.remove(); // Prevent memory leaks
    }
}

// Class ShoppingCart
class ShoppingCart {
    private List<Product> products = new ArrayList<>();

    public void addProduct(Product product) {
        products.add(product);
    }

    public List<Product> getProducts() {
        return products;
    }
}

    In this code, ShoppingCartService is a Spring Bean managing shopping cart information. The ThreadLocal<ShoppingCart> ensures that each thread has its own shopping cart. After checkout, calling cartHolder.remove() clears the current thread’s shopping cart to prevent memory leaks. This approach ensures that in a multi-threaded environment, each thread’s cart is independent and unaffected by others.

  2. Using ThreadLocal in business logic is a common approach to handle thread-isolated data. Let’s consider a scenario where a series of interfaces require user authentication before performing operations on user data. This problem is straightforward: by integrating Spring Security with JWT, the Token passed from the frontend can be parsed to extract the username, which can then be validated. This process can be encapsulated in an aspect, and whenever the business logic needs access to this user, it can simply retrieve it from Spring Security. However, in some cases, the aspect might perform certain preprocessing on the user data, such as updating the timestamp of the user’s API access. Retrieving the user object from Spring Security again in such scenarios becomes inappropriate, as it may lead to discrepancies with the expected object. So, how can this be addressed? The solution is to use ThreadLocal to cache the user object, ensuring that this User instance remains singular throughout the entire HTTP session.

    @Aspect
@Component
public class UserConsistencyAspect {
	// Each UserVo enables thread isolation, starts to be created after entering the aspect, 
	// and is recycled by GC after being used up in the business logic
	private static final ThreadLocal<UserVo> userHolder = new ThreadLocal<>();
	
	@Pointcut("@annotation(org.nozomi.common.annotation.GetUser)")
	public void userAuthPoint() {}
	
	@Around("userAuthPoint()")
	public Object injectUserFromRequest(ProceedingJoinPoint joinPoint) throws Throwable {
		Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
		UserVo operator = (UserVo) authentication.getPrincipal();
		if (operator == null) {
			return Response.fail("User doesn't exist!");
		}
		userHolder.set(operator);
		return joinPoint.proceed();
	}
  
	/**  
	 * Get the UserVo object in the current thread. 
	 * These UserVo objects are isolated following the threads created by http.
	 *
	 * @return UserVo
	 */
	public static UserVo getUser() {
		return userHolder.get();
	}
}

    Use this aspect and the UserConsistencyAspect.getUser() method in business to get the User object in the http session.

  3. ThreadLocal is used to ensure the thread safety of Spring Bean when solving the thread insecurity problem of Spring Bean:

    @Service
public class ProductService {

    private final ThreadLocal<Session> sessionThreadLocal = new ThreadLocal<>();

    public Product getProductById(String id) {
        Session session = getSession();
        return session.get(Product.class, id);
    }

    public void updateProduct(Product product) {
        Session session = getSession();
        session.update(product);
    }

    private Session getSession() {
        Session session = sessionThreadLocal.get();
        if (session == null) {
            session = sessionFactory.openSession();
            sessionThreadLocal.set(session);
        }
        return session;
    }

    public void closeSession() {
        Session session = sessionThreadLocal.get();
        if (session != null) {
            session.close();
            sessionThreadLocal.remove();
        }
    }
}

    In many cases, developers use Spring to manage database sessions or transactions. However, such Beans are often not thread-safe, such as Hibernate’s SessionFactory or MyBatis’s SqlSessionFactory. The sessions produced by these factories are thread-unsafe. In e-commerce projects, a common scenario involves multiple database interactions during a single request. To ensure that the same database session (Session) is used throughout the request, the Session is typically stored in a ThreadLocal. This approach ensures that even within different methods of the same thread, the same Session can be accessed. In this example, each thread has its own Session instance stored in ThreadLocal. When different threads call the getSession() method, they retrieve the Session specific to that thread from ThreadLocal. However, in practice, the handling of such sessions is already managed internally by frameworks like MyBatis or Hibernate using ThreadLocal, so developers usually do not need to implement session isolation at the business logic level. This example primarily serves to explain how ThreadLocal works and is not a recommended approach for real-world development.

StructuredTaskScope#

Structured concurrency and virtual threads (Virtual Threads) are closely related. To understand ScopedValue, it is essential to first grasp these two concepts. Since JDK 5, the principle has always been that we should not interact directly with threads. The correct pattern is to submit tasks as Runnable or Callable to an ExecutorService or Executor and then operate on the returned Future. The Loom project has retained this model while introducing some excellent new features.

The first object to be introduced here is the Scope object, specifically of the type StructuredTaskScope.

This object can be seen as a virtual thread launcher. Tasks are submitted to it in the form of Callable, and a Future is returned. The Callable will execute within a virtual thread created by the Scope. This is quite similar to an Executor. However, there are significant differences between the two.

public static Weather readWeather() throws Exception {
	// try-with-resource
	try(var scope = new StructuredTaskScope<Weather>()) {
		Future<Weather> future = scope.fork(Weather::readWeatherFrom);
		scope.join();
		return future.resultNow();
	}
}

A StructuredTaskScope instance is AutoCloseable, allowing it to be used with the try-with-resources pattern. By using the fork() method, you can fork a task of type Callable. The fork() method returns a Future object. You can call the join() method to block the current thread until all tasks submitted (via fork) to the StructuredTaskScope are completed. Finally, you can use the resultNow() method of the Future to obtain and return the result. However, resultNow() will throw an exception if it is called before the Future is completed, so it should only be invoked after the join() method has been called.

ScopedValue#

Basic Concept#

More closely related to structured concurrency is the CompletableFuture introduced in JDK 8, which I will further elaborate on in a subsequent article. ScopedValue is a feature incubated in JDK 20 based on the principles of structured concurrency. It is not designed to replace ThreadLocal, but instead enables virtual threads in structured concurrency to have their own external variables.

Although ThreadLocal can also be used in structured concurrency, it inherently has several significant issues:

  1. ThreadLocal variables are mutable. Any code running in the current thread can modify the value of these variables, which can lead to difficult-to-debug bugs.
  2. The lifecycle of a ThreadLocal variable is long. Once a value is set for the current thread using the set method, it persists throughout the thread’s lifecycle unless explicitly removed using the remove method. However, most developers do not proactively call remove, which can result in memory leaks.
  3. ThreadLocal variables can be inherited. When a child thread inherits a ThreadLocal variable from its parent thread, it requires independently storing all ThreadLocal variables of the parent thread, leading to substantial memory overhead.

Virtual threads are characterized by their large numbers and short lifecycles, making memory leaks less likely. However, the memory overhead caused by thread inheritance becomes more pronounced. To address these challenges, ScopedValue was introduced. It retains the core characteristic of ThreadLocal—each thread having its own value—but differs by being immutable and having a defined scope, as suggested by the term “scoped” in its name.

Basic Usage#

ScopedValue objects are represented by the ScopedValue class in the jdk.incubator.concurrent package. The first step in using ScopedValue is to create a ScopedValue object, which is done via the static method newInstance. ScopedValue objects are typically declared as static final. Since ScopedValue is an incubating feature, its usage requires creating a module-info.java file in the same directory as the top-level package of your project to include it as a module：

module arena.app.module {  
	requires jdk.incubator.concurrent;  
}

Additionally, the preview feature --enable-preview needs to be enabled in the VM options. The next step is to specify the value and scope of the ScopedValue object, which is done through the static method where. The where method has three parameters:

  • The ScopedValue object
  • The value bound to the ScopedValue object
  • A Runnable or Callable object, representing the scope of the ScopedValue object

During the execution of the Runnable or Callable object, the code can use the get method of the ScopedValue object to access the value bound by the where method. This scope is dynamic, depending on the methods called by the Runnable or Callable object and the other methods they invoke. Once the Runnable or Callable object finishes executing, the ScopedValue object loses its binding, and the value can no longer be accessed through the get method. The value of the ScopedValue object is immutable within the current scope unless the where method is called again to bind a new value. This creates a nested scope, where the new value is only valid within the nested scope.

Using scope values has the following advantages:

  • Enhanced data security: Since scope values can only be accessed within the current scope, they help prevent data leakage or malicious modification.
  • Improved data efficiency: Because scope values are immutable and can be shared across threads, they reduce the overhead of data copying or synchronization.
  • Increased code clarity: Since scope values can only be accessed within the current scope, the need for parameter passing or global variables is reduced.

Java JEP 429 is a new feature currently incubating that provides a way to share immutable data within and between threads. As of the publication of this article, Java JEP 429 is still in the incubator phase and has not yet been officially included in the Java language specification.

public class Main {
    // Declares a static final instance of ScopedValue<String>
    // ScopedValue is a class that supports passing values within a specific scope (such as a task or thread)
    // Its usage is similar to ThreadLocal but is better suited for structured concurrency
    private static final ScopedValue<String> VALUE = ScopedValue.newInstance();
    
    public static void main(String[] args) throws Exception {
        System.out.println(Arrays.toString(stringScope()));
    }

    public static Object[] stringScope() throws Exception {
        return ScopedValue.where(VALUE, "value", () -> {
            // Uses try-with-resources to bind the scope of structured concurrency
            // This automatically manages the lifecycle of resources; this is a structured task scope
            // All subtasks created within this scope will be treated as part of the scope
            // If any task within the scope fails, all other tasks will be canceled
            try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
                // Uses scope.fork to create two parallel tasks
                // Each task retrieves the value of VALUE from the execution context and operates on it
                Future<String> user  = scope.fork(VALUE::get);
                Future<Integer> order = scope.fork(() -> VALUE.get().length());
                // The join() method waits for all tasks within the scope to complete
                // throwIfFailed() checks the results of all tasks, throwing an exception if any task fails
                scope.join().throwIfFailed();
                // After all tasks complete, use resultNow() to retrieve the result of each task and return them in an object array
                return new Object[]{user.resultNow(), order.resultNow()};
            }
        });
    }
}

This code shows how to use ScopedValue and structured concurrency to create and execute multiple parallel tasks, and safely pass and manipulate values ​​in the task context.

Source Code Analysis#

A value that is set once and is then available for reading for a bounded period of execution by a thread. A ScopedValue allows for safely and efficiently sharing data for a bounded period of execution without passing the data as method arguments. ScopedValue defines the where(ScopedValue, Object, Runnable) method to set the value of a ScopedValue for the bouned period of execution by a thread of the runnable’s run method. The unfolding execution of the methods executed by run defines a dynamic scope. The scoped value is bound while executing in the dynamic scope, it reverts to being unbound when the run method completes (normally or with an exception). Code executing in the dynamic scope uses the ScopedValue get method to read its value. Like a thread-local variable, a scoped value has multiple incarnations, one per thread. The particular incarnation that is used depends on which thread calls its methods.

Before starting the source code analysis of ScopedValue, let’s first look at the Java doc description: ScopedValue is an object that, once set, can be read by a thread for a limited period during execution. ScopedValue allows data to be safely and efficiently shared within a limited execution period without passing it as method parameters. ScopedValue defines the method where(ScopedValue, Object, Runnable), which sets the value of ScopedValue during the limited execution period of a thread executing the run method of a runnable. The method executed by run defines a dynamic scope. While executing in this dynamic scope, the scope value is bound, and when the run method completes (whether normally or exceptionally), it reverts to an unbound state. Code executed within the dynamic scope uses the get method of ScopedValue to read its value.

Similar to thread-local variables, scoped values have multiple instances, one per thread. The instance used depends on which thread calls its method. A typical use case of ScopedValue is to declare it in final and static fields. The accessibility of the field determines which components can bind or read its value. There are three internal classes in ScopedValue: Snapshot, Carrier, and Cache, each of which plays a crucial role within ScopedValue.

Snapshot#

An immutable map from ScopedValue to values. Unless otherwise specified, passing a null argument to a constructor or method in this class will cause a NullPointerException to be thrown.

The Snapshot is an immutable mapping from ScopedValue to its value. Unless explicitly stated otherwise, passing a null parameter to the constructor or methods of this class will result in a NullPointerException. The main purpose of this class is to create an immutable mapping of the ScopedValue instance so that at runtime, regardless of how other code modifies the original ScopedValue, the value in the Snapshot remains unchanged. It provides a safe way to share values in a multi-threaded environment.

Carrier#

A mapping of scoped values, as keys, to values. A Carrier is used to accumlate mappings so that an operation (a Runnable or Callable) can be executed with all scoped values in the mapping bound to values. The following example runs an operation with k1 bound (or rebound) to v1, and k2 bound (or rebound) to v2. ScopedValue.where(k1, v1).where(k2, v2).run(() -> ... ); A Carrier is immutable and thread-safe. The where method returns a new Carrier object, it does not mutate an existing mapping. Unless otherwise specified, passing a null argument to a method in this class will cause a NullPointerException to be thrown.

The Carrier class is used to accumulate mappings, allowing an operation (such as a Runnable or Callable) to execute where all scoped values in the mapping are bound to a value. Carrier is immutable and thread-safe. The where method returns a new Carrier object, which does not alter the existing mapping. This serves as a tool to create and maintain the mapping relationship between ScopedValue instances and their corresponding values, so these mappings can be applied together when performing the operation.

Cache#

A small fixed-size key-value cache. When a scoped value’s get() method is invoked, we record the result of the lookup in this per-thread cache for fast access in future.

The Cache is a small, fixed-size key-value cache. When calling the get() method of a scoped value, the result of the lookup is recorded in this thread-local cache for fast access in the future. The primary function of this class is to optimize performance. By caching the results of get() method calls, it avoids repeated lookups when accessing the same ScopedValue multiple times. The cache is updated only when the value of the ScopedValue changes.

where()#

The where() method is the core method and entry point of the ScopedValue class, and it accepts three parameters. When the operation completes (either normally or with an exception), the ScopedValue will revert to an unbound state in the current thread or return to the previous value it had when bound earlier.

Scoped values are intended to be used in a structured way. If op has already created a StructuredTaskScope but has not closed it, then exiting op will close each StructuredTaskScope created within the dynamic scope. This may require blocking until all child threads have completed their tasks. The closure is performed in the reverse order of their creation.

  1. Using ScopedValue.where(key, value, op); is equivalent to using ScopedValue.where(key, value).call(op);.
    public static <T, R> R where(ScopedValue<T> key,
							 T value,
							 Callable<? extends R> op) throws Exception {
	return where(key, value).call(op);
}
  2. This method delegates the first two parameters to the Carrier.of(key, value); method.
    /*  
 * Returns a new collection consisting of a single binding
 */  
static <T> Carrier of(ScopedValue<T> key, T value) {  
	return where(key, value, null);  
}
/**  
 * Adds a binding to the map, returning a new Carrier instance.
 */  
private static final <T> Carrier where(ScopedValue<T> key, T value,  
									   Carrier prev) {  
	return new Carrier(key, value, prev);  
}
  3. In the Carrier class, the where method returns a new Carrier object, which follows the chain-of-responsibility design pattern.

call()#

The where method primarily constructs the Carrier object, and then these will be delegated to the call method of subsequent Carrier objects to execute the Callable. The call method chain involves many details related to handling Snapshot and Cache, which may change in future Java versions, and thus will not be elaborated here.

Summary#

Both ThreadLocal and ScopedValue play crucial roles in Java concurrent programming, each suitable for different scenarios. Developers need to choose based on specific requirements.

ThreadLocal is mainly used in traditional concurrent programming. In Java, each thread has its own stack, where the local variables needed by the thread are stored. ThreadLocal provides a unique mechanism that allows each thread to have its own independent data, inaccessible to other threads. This mechanism is especially useful in scenarios requiring thread state isolation or data separation between threads, such as database connections or session management.

However, although ThreadLocal can achieve thread-level data isolation, it cannot solve more complex concurrency issues, such as controlling concurrency in asynchronous tasks or sharing data between asynchronous tasks. This is where a new tool, ScopedValue, comes into play.

ScopedValue is a new feature introduced in Java to support structured concurrency programming. Structured concurrency allows developers to manage the lifecycle of concurrent programs by defining their structure. ScopedValue provides a way to share a value within a defined execution scope, which is also referred to as a “scope.”

In structured concurrency, ScopedValue is mainly used for data sharing between concurrent tasks. Compared to ThreadLocal, ScopedValue offers better control over data sharing between tasks and more effective management of task lifecycles. For example, a thread can place a value into ScopedValue, and all child threads started by that thread can access this value. This helps avoid passing large numbers of parameters in asynchronous tasks, simplifying concurrent programming.

Reference#

[1] Java API Documentation. (2023). Scoped Value - JDK Incubator Documentation. Retrieved June 16, 2023, from https://download.java.net/java/early_access/loom/docs/api/jdk.incubator.concurrent/jdk/incubator/concurrent/ScopedValue.html

[2] Inside. (2023). From threadlocal to scopedvalue with loom - jep café. Retrieved June 16, 2023, from https://inside.java/2023/01/10/jepcafe16/

[3] John Rose. (2023). JEP 429: Scoped Values. Retrieved June 16, 2023, from https://openjdk.org/jeps/429

ThreadLocal & ScopedValue
https://biu.kim/posts/notes/threadlocal_scopedvalue/
Author
Moritz Arena
Published at
2023-04-11
Why need to encapsulate Logger again?
Structured Concurrency
苏ICP备2024141418号-1 © 2024 Moritz Arena. All Rights Reserved.
Powered by Astro & Fuwari