Using Spring Batch Without Transaction

Question

I have a Spring Batch job that calls another job in its first step / tasklet. Naturally, this does not work with how jobs handle transactions on default.

In my application, I don't really need transactions all that much, and the steps do not have a database connection anyway, so I'd rather just remove the transactions completely. For that, for each step of both jobs I did the following:

  public Step stepX() {
    return this.stepBuilderFactory
        .get("stepX")
        .tasklet(new StepXTasklet())
        .transactionAttribute(transactionAttribute())
        .build();
  }

  private TransactionAttribute transactionAttribute() {
    DefaultTransactionAttribute attribute = new DefaultTransactionAttribute();
    attribute.setPropagationBehavior(Propagation.SUPPORTS.value());
    return attribute;
  }

Still the application will throw this exception:

org.springframework.dao.OptimisticLockingFailureException: Attempt to update step execution id=1 with wrong version (2), where current version is 3
    at org.springframework.batch.core.repository.dao.MapStepExecutionDao.updateStepExecution(MapStepExecutionDao.java:106)
    at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:196)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:353)
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:99)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy150.update(Unknown Source)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)

Which is how Spring Batch presents its internal TransactionSuspensionNotSupportedException for some reason.

So something is still using transactions, I'm just not sure what, and how to remove them. Maybe the JobExecutionListener? Maybe the chunks?

I'm leaning towards the chunks since after the second chunk (counted by the number of lines read) and before the third I get the output:

Commit failed while step execution data was already updated. Reverting to old version.

In case it makes a difference, the parent job is a "flow" job, the child job is a "reader-transformer-writer".

Which parts of the Spring Batch jobs framework uses transactions and how do I make sure these stop using them?

Spring Batch writes data about the job and step executions to the database. That's why it needs transactions. — Simon Martinelli, Mar 17 '21 at 13:34
If you don't want this behavoir checkout https://stackoverflow.com/questions/25077549/spring-batch-without-persisting-metadata-to-database — Simon Martinelli, Mar 17 '21 at 13:34
@SimonMartinelli But these transactions should not overlap in a "job inside a job" world, should they? They are only between the steps, so there should not be any `TransactionSuspensionNotSupportedException`, because no open transaction has to be suspended for another to run. — Stefan S., Mar 17 '21 at 13:40
Why do you think that this OptimisticLockingFailureException is a TransactionSuspensionNotSupportedException? — Simon Martinelli, Mar 17 '21 at 14:26
@SimonMartinelli Because I debugged it. The parent job starts the child job, and while creating a step it will try to open a transaction, which is not possible with the standard `TransactionManager`. The behavior is generally weird, because afterwards it restarts the parent job, which shouldn't happen either. And it's possibly not the step that gets the transaction, but the chunk? Because for the step I disabled transaction. — Stefan S., Mar 18 '21 at 07:24
I'm keen to debug your use case if you provide a minimal complete example that reproduces the issue. Is your step multi-threaded? From your stack trace, I see usage of `MapStepExecutionDao`. Are you using the Map based job repository? If yes, please note that it has been deprecated for removal, see https://github.com/spring-projects/spring-batch/issues/3780. I recommend using the Jdbc based job repository implementation with an in-memory db. Please try this and let me know if it fixes the issue. — Mahmoud Ben Hassine, Mar 18 '21 at 08:50
Any reason to run a separate job in a tasklet step? A tasklet step is not designed to launch a job in it, there is [JobStep](https://docs.spring.io/spring-batch/docs/4.3.x/api/org/springframework/batch/core/step/job/JobStep.html) which is specifically designed for that use case. — Mahmoud Ben Hassine, Mar 18 '21 at 08:59
@MahmoudBenHassine In the future we want to create micro-services from these two jobs, that's why we don't want to link them too tightly. Sadly, now they are in the same Spring application and get in the way of each other. I strongly think [this question](https://stackoverflow.com/questions/66690338/transactionattribute-not-working-for-simple-step) is the root cause of the problem state above, and it has a minimal broken example. — Stefan S., Mar 18 '21 at 11:40
@MahmoudBenHassine And we are using the default repository, so the map based on. I think JDBC won't work for us, since we are using MongoDB, and there is no implementation for that. — Stefan S., Mar 18 '21 at 11:46

score 0 · Accepted Answer · answered Mar 23 '21 at 09:25

The problem is with the default job repository. It seems its transaction handling is buggy. To fix this, replace this with the JDBC job repository with an in-memory database. Just add this class to the Spring context:

@Configuration
@EnableBatchProcessing
public class InMemoryBatchContextConfigurer extends DefaultBatchConfigurer {

  @Override
  protected JobRepository createJobRepository() throws Exception {
    JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
    factory.setDatabaseType(DatabaseType.H2.getProductName());
    factory.setDataSource(dataSource());
    factory.setTransactionManager(getTransactionManager());
    return factory.getObject();
  }

  public DataSource dataSource() {
    EmbeddedDatabaseBuilder embeddedDatabaseBuilder = new EmbeddedDatabaseBuilder();
    return embeddedDatabaseBuilder
        .addScript("classpath:org/springframework/batch/core/schema-drop-h2.sql")
        .addScript("classpath:org/springframework/batch/core/schema-h2.sql")
        .setType(EmbeddedDatabaseType.H2).build();
  }
}

Using Spring Batch Without Transaction

1 Answers1

Linked