Spring Batch Sporadically getting java.lang.IllegalStateException: Existing transaction detected in JobRepository

Question

My project executes a spring batch (version 2.1.9) based task from Quartz scheduler (version 2,2,0).
We are working in a multi-tenant environment and every tenant is maintaining is own job Repository on a tenant dedicated schema. The data source implantation is tenant sensitive and pool connections per tenant. The transaction manager used is org.springframework.jdbc.datasource.DataSourceTransactionManager

When we execute the scheduler the tasks are executed successfully. Once in a while I am getting the following exception from spring batch. Please note that other tasks share this scheduler as well. Some of them have other PlatformTransactionManagers assigned to them.

2014-08-14 17:10:00,082 INFO [tenant_id] [org.quartz.core.JobRunShell] - <Job ROOT_GROUP. dataGenerator threw a JobExecutionException: >
org.quartz.JobExecutionException: java.lang.IllegalStateException: Existing transaction detected in JobRepository.
Please fix this and try again (e.g. remove @Transactional annotations from client).
[See nested exception: java.lang.IllegalStateException: Existing transaction detected in JobRepository.
Please fix this and try again (e.g. remove @Transactional annotations from client).] 

I am familiar with this Exception when using @Transactional (see my answer post here).
But as my Job is actually running correctly from the scheduler and as a standalone app this is not a configuration issue as this happens sporadically. I guess that at some stage the transaction is delayed in a way that Spring Batch thinks is sharing the transaction. Or Maybe other tasks does not clear correctly the TransactionSynchronizationManager before returning the thread back to the Quartz Thread Pool

<bean id="jobTransactionManager"
            class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
            <property name="dataSource" ref="jobDataSource" />
</bean> 
<bean
    class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
    <property name="jobRegistry" ref="jobRegistry" />
</bean>

<bean id="jobRepository" class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean">
    <property name="dataSource" ref="jobDataSource"/>
    <property name="databaseType" value="SQLSERVER"/>
    <property name="transactionManager" ref="jobTransactionManager"/>
</bean>

Any idea how to trace down this issue?
Please find the full stack trace below:

2014-08-14 11:50:00,157 INFO [tenant_id] [org.quartz.core.JobRunShell] - <Job ROOT_GROUP.dataGenerator threw a JobExecutionException: >
org.quartz.JobExecutionException: java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove @Transactional annotations from client). [See nested exception: java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove @Transactional annotations from client).]
    at com.my.csd.scheduler.BatchJobWrapper.executeInternal(BatchJobWrapper.java:53)
    at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:111)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:560)
Caused by: java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove @Transactional annotations from client).
    at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:164)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
    at com.sun.proxy.$Proxy256.createJobExecution(Unknown Source)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:111)
    at org.springframework.batch.core.launch.support.SimpleJobOperator.startNextInstance(SimpleJobOperator.java:345)
    at org.springframework.batch.core.launch.support.SimpleJobOperator$$FastClassBySpringCGLIB$$44ee6049.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:640)
    at org.springframework.batch.core.launch.support.SimpleJobOperator$$EnhancerBySpringCGLIB$$540a28da.startNextInstance(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201)
    at com.sun.proxy.$Proxy257.startNextInstance(Unknown Source)
    at com.my.csd.schedule.tasks.SpringBatchTask.process(SpringBatchTask.java:32)
    at com.my.csd.schedule.tasks.AbstractScheduledTask.process(AbstractScheduledTask.java:64)
    at com.my.csd.scheduler.BatchJobWrapper.process(BatchJobWrapper.java:60)
    at com.my.csd.scheduler.BatchJobWrapper.executeInternal(BatchJobWrapper.java:49)
    ... 3 more