Single transaction across multiple threads solution

Question

As I understand it, all transactions are Thread-bound (i.e. with the context stored in ThreadLocal). For example if:

1. I start a transaction in a transactional parent method
2. Make database insert #1 in an asynchronous call
3. Make database insert #2 in another asynchronous call

Then that will yield two different transactions (one for each insert) even though they shared the same "transactional" parent.

For example, let's say I perform two inserts (and using a very simple sample, i.e. not using an executor or completable future for brevity, etc.):

@Transactional
public void addInTransactionWithAnnotation() {
    addNewRow();
    addNewRow();
}

Will perform both inserts, as desired, as part of the same transaction.

However, if I wanted to parallelize those inserts for performance:

@Transactional
public void addInTransactionWithAnnotation() {
    new Thread(this::addNewRow).start();
    new Thread(this::addNewRow).start();
}

Then each one of those spawned threads will not participate in the transaction at all because transactions are Thread-bound.

Key Question: Is there a way to safely propagate the transaction to the child threads?

The only solutions I've thought of to solve this problem:

1. Use JTA or some XA manager, which by definition should be able to do this. However, I ideally don't want to use XA for my solution because of it's overhead
2. Pipe all of the transactional work I want performed (in the above example, the addNewRow() function) to a single thread, and do all of the prior work in the multithreaded fashion.
3. Figuring out some way to leverage InheritableThreadLocal on the Transaction status and propagate it to the child threads. I'm not sure how to do this.

Are there any more solutions possible? Even if it's tastes a little bit of like a workaround (like my solutions above)?

Answer 1

The JTA API has several methods that operate implicitly on the current Thread's Transaction, but it doesn't prevent you moving or copying a Transaction between Threads, or performing certain operations on a Transaction that's not bound to the current (or any other) Thread. This causes no end of headaches, but it's not the worst part...

For raw JDBC, you don't have a JTA Transaction at all. You have a JDBC Connection, which has its own ideas about transaction context. In which case, the transaction is Connection bound, not thread bound. Pass the Connection around and the tx goes with it. But Connections aren't necessarily threadsafe and are probably a performance bottleneck anyhow, so sharing one between multiple concurrent threads doesn't really help you. You likely need multiple Connections that think they are in the same Transaction, which means you need XA, since that's how the db identifies such cases. At which point you're back to JTA, but now with a JCA in the picture to handle the Connection management properly. In short, you've reinvented the JavaEE application server.

For frameworks that layer on JDBC e.g. ORMs like Hibernate, you have an additional complication: their abstractions are not necessarily threadsafe. So you can't have a Session that is bound to multiple Threads concurrently. But you can have multiple concurrent Sessions that each participate in the same XA transaction.

As usual it boils down to Amdahl's law. If the speedup you get from using multiple Connections per tx to allow for multiple concurrent Threads to share the db I/O work is large relative to what you get from batching, then the overhead of XA is worthwhile. If the speedup is in local computation and the db I/O is a minor concern, then a single Thread that handles the JDBC Connection and offloads non-IO computation work to a Thread pool is the way to go.

Answer 2

First, a clarification: if you want to speed up several inserts of the same kind, as your example suggests, you will probably get the best performance by issuing the inserts in the same thread and using some type of batch inserting. Depending on your DBMS there are several techniques available, look at:

• Efficient way to do batch INSERTS with JDBC
• What's the fastest way to do a bulk insert into Postgres?

As for your actual question, I would personally try to pipe all the work to a worker thread. It is the simplest option as you don't need to mess with either ThreadLocals or transaction enlistment/delistment. Furthermore, once you have your units of work in the same thread, if you are smart you might be able to apply the batching techniques above for better performance.

Lastly, piping work to worker threads does not mean that you must have a single worker thread, you could have a pool of workers and achieve some parallelism if it is really beneficial to your application. Think in terms of producers/consumers.