We Keep Coding

Spring Boot scheduled Runnable tasks continue executing even after server is shut down with an actuator

I'm currently developing a Spring-based web platform which makes use of several scheduled processes that access a central database. I wanted to introduce actuators for shutdown and restart. However, I am experiencing an issue: even though the shutdown request is accepted with a 200 response, the application context begins shutting down:
2022-10-10 17:37:25.235 INFO 9067 --- [ Thread-3] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2022-10-10 17:37:25.240 INFO 9067 --- [ Thread-3] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
And after that, I repeatedly get the following exception:
org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is java.lang.IllegalStateException: EntityManagerFactory is closed
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:467) ~[spring-orm-5.3.16.jar:5.3.16]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400) ~[spring-tx-5.3.16.jar:5.3.16]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373) ~[spring-tx-5.3.16.jar:5.3.16]
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:595) ~[spring-tx-5.3.16.jar:5.3.16]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:382) ~[spring-tx-5.3.16.jar:5.3.16]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) ~[spring-tx-5.3.16.jar:5.3.16]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.16.jar:5.3.16]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) ~[spring-aop-5.3.16.jar:5.3.16]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) ~[spring-aop-5.3.16.jar:5.3.16]
at usi.si.seart.service.TaskService$TaskServiceImpl$$EnhancerBySpringCGLIB$$4bf83a1d.getNext(<generated>) ~[classes/:na]
at usi.si.seart.scheduling.TaskRunner.run(TaskRunner.java:68) ~[classes/:na]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.3.16.jar:5.3.16]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Caused by: java.lang.IllegalStateException: EntityManagerFactory is closed
at org.hibernate.internal.SessionFactoryImpl.validateNotClosed(SessionFactoryImpl.java:547) ~[hibernate-core-5.6.5.Final.jar:5.6.5.Final]
at org.hibernate.internal.SessionFactoryImpl.createEntityManager(SessionFactoryImpl.java:636) ~[hibernate-core-5.6.5.Final.jar:5.6.5.Final]
at org.hibernate.internal.SessionFactoryImpl.createEntityManager(SessionFactoryImpl.java:158) ~[hibernate-core-5.6.5.Final.jar:5.6.5.Final]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.createNativeEntityManager(AbstractEntityManagerFactoryBean.java:585) ~[spring-orm-5.3.16.jar:5.3.16]
at jdk.internal.reflect.GeneratedMethodAccessor112.invoke(Unknown Source) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.invokeProxyMethod(AbstractEntityManagerFactoryBean.java:487) ~[spring-orm-5.3.16.jar:5.3.16]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean$ManagedEntityManagerFactoryInvocationHandler.invoke(AbstractEntityManagerFactoryBean.java:734) ~[spring-orm-5.3.16.jar:5.3.16]
at com.sun.proxy.$Proxy175.createNativeEntityManager(Unknown Source) ~[na:na]
at org.springframework.orm.jpa.JpaTransactionManager.createEntityManagerForTransaction(JpaTransactionManager.java:485) ~[spring-orm-5.3.16.jar:5.3.16]
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:410) ~[spring-orm-5.3.16.jar:5.3.16]
... 17 common frames omitted
So in spite of the fact that the server is reported as "shutting down", the scheduled processes still continue executing. My scheduler definition within the configuration class is as follows:
#Bean
public ThreadPoolTaskScheduler taskScheduler() {
ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
threadPoolTaskScheduler.setClock(Clock.systemUTC());
threadPoolTaskScheduler.setPoolSize(3);
threadPoolTaskScheduler.setThreadNamePrefix("PlatformScheduler");
threadPoolTaskScheduler.setExecuteExistingDelayedTasksAfterShutdownPolicy(false);
threadPoolTaskScheduler.setRemoveOnCancelPolicy(true);
threadPoolTaskScheduler.initialize();
threadPoolTaskScheduler.schedule(getRepoMaintainer(), new CronTrigger("0 0 0 * * SUN"));
threadPoolTaskScheduler.schedule(getTaskCleaner(), new CronTrigger("0 */15 * * * *"));
threadPoolTaskScheduler.scheduleWithFixedDelay(getTaskRunner(), 500);
return threadPoolTaskScheduler;
}
Where the getRepoMaintainer, getTaskCleaner and getTaskRunner are all methods that produce custom Runnable instances: RepoMaintainer, TaskCleaner and TaskRunner respectively.

Unfortunately, none of the fixes suggested by the other users worked. In the end, I managed to solve my problem by virtue of a custom ThreadPoolTaskScheduler implementation:
ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler() {
private final Set<ScheduledFuture<?>> futures = new HashSet<>();
#Override
public ScheduledFuture<?> scheduleWithFixedDelay(Runnable task, long delay) {
ScheduledFuture<?> future = super.scheduleWithFixedDelay(task, delay);
futures.add(future);
return future;
}
#Override
public ScheduledFuture<?> schedule(Runnable task, Trigger trigger) {
ScheduledFuture<?> future = super.schedule(task, trigger);
futures.add(future);
return future;
}
#Override
public void shutdown() {
futures.forEach(future -> future.cancel(true));
super.shutdown();
}
};
This way I ensure that all scheduled tasks are cancelled pre-shutdown no matter what.

As you have specified that your custom destroyMethod is "shutdown" have you defined that method with the required destruction statements?
Something like this,
public void shutdown() {
threadPoolTaskScheduler.shutdown();
}
Edit-
Most probably the error here is, even though the shutdown has been initiated the tasks are still running but the container is shutting down. This releases the resources required for the task to execute and the error is thrown.
What you can do here is perform a graceful shutdown where you prevent the shutdown of the container until the tasks have finished executing.
Your issue is similar to this one and you can try out his solution.
Or, you can configure a termination period like so, setAwaitTerminationSeconds(*seconds*) Check more here. But then you might face this issue.

Spring Boot with Kafka Streams throws ClassCastException only when set configuration in application.properties

I have a problem with Spring boot with Kafka Streams.
There are two ways to set configuration Kafka Streams.
First way is set configuration with #Bean like this.
#Bean(name = KafkaStreamsDefaultConfiguration.DEFAULT_STREAMS_CONFIG_BEAN_NAME)
KafkaStreamsConfiguration kStreamsConfig() {
Map<String, Object> props = new HashMap<>();
props.put(APPLICATION_ID_CONFIG, "kafka-streams");
props.put(BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
return new KafkaStreamsConfiguration(props);
}
When I using this way and deploy application on k8s, it works well and doesn't throw any Exception.
But, In second way using application.properties like this
// application.properties
spring.kafka.streams.bootstrap-servers=localhost:9092
spring.kafka.streams.application-id=kafka-streams
When I deploy application with this way on k8s, the application works well too but throw ClassCastException!!
20220728 18:08:17 [XTRA-KAFKA-PRPDUCER11 main] class java.util.ArrayList cannot be cast to class java.lang.String (java.util.ArrayList and java.lang.String are in module java.base of loader 'bootstrap') java.lang.ClassCastException: class java.util.ArrayList cannot be cast to class java.lang.String (java.util.ArrayList and java.lang.String are in module java.base of loader 'bootstrap')
java.lang.ClassCastException
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:666)
org.apache.kafka.streams.processor.internals.DefaultKafkaClientSupplier.getRestoreConsumer(DefaultKafkaClientSupplier.java:49)
org.apache.kafka.streams.processor.internals.StreamThread.create(StreamThread.java:343)
org.apache.kafka.streams.KafkaStreams.createAndAddStreamThread(KafkaStreams.java:956)
org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:948)
org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:845)
org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:751)
org.springframework.kafka.config.StreamsBuilderFactoryBean.start(StreamsBuilderFactoryBean.java:349)
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
java.base/java.lang.Iterable.forEach(Iterable.java:75)
org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935)
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586)
org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147)
org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
com.example.springkafka.SpringKafkaApplication.main(SpringKafkaApplication.java:10)
java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
java.base/java.lang.reflect.Method.invoke(Method.java:577)
org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
(That bootstrap-servers is an example. In my application, it isn't localhost:9092)
Additionally, In local environment using IntelliJ, there's no Exception in both ways.
I wonder why this problem is caused. and how to solve this problem..

Finally, I decided to ignore this Exception.
Because, When the exception thrown, application does not shut down or it works well.
Furthermore, In local environment, exception does not be thrown.
I think there's some problem in my k8s.
Thanks for all comments.

Spring data hibernate upsert transaction error #TransactionalEventListener and #Transactinal

I have error with transaction handling in spring data.
Spring boot version with correlating spring data - 2.3.7.
Java - 11
I have a method in my service class(annotated as a Service)
#Transactional
public void incrementSomething(SomeData data) {
myRepository.someIncrement(data);
}
My repository(I wanted to have upsert operation):
#Repository
public interface MyRepository extends PagingAndSortingRepository<SomeData, Long> {
#Transactional
#Modifying
#Query(value = "INSERT INTO my_table (.....) VALUES (.....) " +
"ON CONFLICT ON CONSTRAINT some_contraint DO UPDATE " +
"set something = my_table.something + 1", nativeQuery = true)
public void someIncrement(#Param("someparam") String data);
However I have a strange exception occurring(I have enabled TRACE transaction in springframework).
Completing transaction for [org.springframework.data.jpa.repository.support.SimpleJpaRepository.someIncrement] after exception: javax.persistence.TransactionRequiredException: Executing an update/delete query
I have no idea why is it happening, because I can see the logs that transaction started, and then completing transaction is happening with error. I have tried to look for similar errors on StackOverflow or similar pages, however none of them works.
Do You know why does this error occur? And how to fix it?
PS:
My Transactional method is from:
import org.springframework.transaction.annotation.Transactional;
Stacktrace:
org.springframework.dao.InvalidDataAccessApiUsageException: Executing an update/delete query; nested exception is javax.persistence.TransactionRequiredException: Executing an update/delete query
at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:403)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:257)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:531)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:154)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:149)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy222.incrementCounter(Unknown Source)
...
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
...
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:305)
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:190)
at org.springframework.transaction.event.ApplicationListenerMethodTransactionalAdapter$TransactionSynchronizationEventAdapter.processEvent(ApplicationListenerMethodTransactionalAdapter.java:129)
at org.springframework.transaction.event.ApplicationListenerMethodTransactionalAdapter$TransactionSynchronizationEventAdapter.afterCompletion(ApplicationListenerMethodTransactionalAdapter.java:118)
at org.springframework.transaction.support.TransactionSynchronizationUtils.invokeAfterCompletion(TransactionSynchronizationUtils.java:171)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.invokeAfterCompletion(AbstractPlatformTransactionManager.java:989)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.triggerAfterCompletion(AbstractPlatformTransactionManager.java:964)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:785)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:633)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:386)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
...
at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
...
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.persistence.TransactionRequiredException: Executing an update/delete query
at org.hibernate.internal.AbstractSharedSessionContract.checkTransactionNeededForUpdateOperation(AbstractSharedSessionContract.java:413)
at org.hibernate.query.internal.AbstractProducedQuery.executeUpdate(AbstractProducedQuery.java:1668)
at org.springframework.data.jpa.repository.query.JpaQueryExecution$ModifyingExecution.doExecute(JpaQueryExecution.java:238)
at org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:88)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:154)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:142)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor$QueryMethodInvoker.invoke(QueryExecutorMethodInterceptor.java:195)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:152)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:130)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:367)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
... 56 common frames omitted
UPDATE 1:
I have added tests with test containers and the same postgres version. I have verified that running the class which invokes this method works just as expected. The problem starts with the class before -> which sends an event and the event handler(annotated as #TransactionalEventListener) receives it after the commit and then the same path is being invoked and exception occurs.
So currently for me it looks like #TransactionalEventListener doesn't work well with #Transactional.
UPDATE 2:
It works only when #Transaction(propagation = REQUIRES_NEW) is set up(as stated in the comments. However I would like to understand why is that. Normal #Transaction works the same as #Transaction(propagation = REQUIRES_NEW) when no transaction is running and it is directly this case. I am still unable to tell why we need propagation requires new and no simple transactional.

I think you need to create a new transaction (#Transactional(propagation = REQUIRES_NEW) when invoking such services from within a transaction listener.

Conflicting Spring schedules

I've two scheduled tasks that run at different time intervals. The first task is scheduled to run every 5 seconds and the second task is scheduled to run every 10 minutes.
#EnableScheduling
public class ScheduledTask {
#Autowired
private taskService taskService;
#Scheduled(every 5 second)
public void scheduleTaskA() {
taskService.taskA()
}
#Scheduled(every 10 minute)
public void scheduleTaskB() {
taskService.taskB()
}
}
public class TaskServiceImpl implements TaskService {
#PersistenceContext
private EntityManager entityManager;
void taskA(){
StoredProcedureQuery query = entityManager.createStoredProcedureQuery("callStoreProcedure1");
if(query.execute())
query.getSingleResult();
}
void taskB(){
StoredProcedureQuery query = entityManager.createStoredProcedureQuery("callStoreProcedure2");
if(query.execute())
query.getSingleResult();
}
}
Every time the second task is running it throws java.lang.IllegalStateException: Session/EntityManager is closed. Looks like the first task is closing the entityManager. How can I avoid this without changing #PersistenceContext annotation?
Full stack trace
java.lang.IllegalStateException: Session/EntityManager is closed
at org.hibernate.internal.AbstractSharedSessionContract.checkOpen(AbstractSharedSessionContract.java:357) ~[hibernate-core-5.3.6.Final.jar:5.3.6.Final]
at org.hibernate.engine.spi.SharedSessionContractImplementor.checkOpen(SharedSessionContractImplementor.java:138) ~[hibernate-core-5.3.6.Final.jar:5.3.6.Final]
at org.hibernate.query.internal.AbstractProducedQuery.getMaxResults(AbstractProducedQuery.java:892) ~[hibernate-core-5.3.6.Final.jar:5.3.6.Final]
at org.hibernate.procedure.internal.ProcedureCallImpl.getResultList(ProcedureCallImpl.java:716) ~[hibernate-core-5.3.6.Final.jar:5.3.6.Final]
at org.hibernate.procedure.internal.ProcedureCallImpl.getSingleResult(ProcedureCallImpl.java:744) ~[hibernate-core-5.3.6.Final.jar:5.3.6.Final]
at sun.reflect.GeneratedMethodAccessor150.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.orm.jpa.SharedEntityManagerCreator$DeferredQueryInvocationHandler.invoke(SharedEntityManagerCreator.java:374) ~[spring-orm-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at com.sun.proxy.$Proxy149.getSingleResult(Unknown Source) ~[na:na]
at com.test.service.TaskServiceImpl.taskA(TaskServiceImpl.java:602) ~[classes/:0.0.1-SNAPSHOT]
at com.test.service.TaskServiceImpl....(TaskServiceImpl.java:112) ~[classes/:0.0.1-SNAPSHOT]
at com.test.service.TaskServiceImpl....(TaskServiceImpl.java:163) ~[classes/:0.0.1-SNAPSHOT]
at com.test.schedule.ScheduledTask.scheduledTaskA(ScheduledTask.java:45) ~[classes/:0.0.1-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) [spring-context-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_171]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_171]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_171]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]

I'm pretty sure you need to run the StoredProcedureQuery(s) in a transactional context. Please try adding #Transactional to each of your methods.
#Transactional
void taskA(){
StoredProcedureQuery query = entityManager.createStoredProcedureQuery("callStoreProcedure1");
if(query.execute())
query.getSingleResult();
}
#Transactional
void taskB(){
StoredProcedureQuery query = entityManager.createStoredProcedureQuery("callStoreProcedure2");
if(query.execute())
query.getSingleResult();
}
You might also want to set the propagation to new to insure these each run within their own separate transaction #Tansacational(propagation = Propagation.REQUIRES_NEW) .

You are getting the error because you are trying to share same entity manager. Which is short living object.
You can apply following two solutions :
1) Inject EntityManagerFactory and inside of EntityManager, in method use factory to create EntityManager.
#PersistenceUnit(unitName= "em")
private EntityManagerFactory emf;
2) Create two different services let each service have there own entity manger instance.
(Not sure it will work second time when method will called by scheduler. Test it and let me know.)

Apache camel Connection pool timeout with restlet even after configuring component options

I have a camel Java DSL route which invokes a restlet endpoint. And the route works without any issues when I hit the same manually. But, when I try to pump a larger number of requests through code I'm getting "Timeout waiting for connection from pool"
And the following is stackt-race of the same:
2016-01-29 14:09:38.650 WARN 20256 --- [pool-3-thread-2] org.restlet : An error occurred during the communication with the remote HTTP server.org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
at org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:412)
at org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRoute.java:298)
at org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:238)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:423)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.restlet.ext.httpclient.internal.HttpMethodCall.sendRequest(HttpMethodCall.java:339)
at org.restlet.engine.adapter.ClientAdapter.commit(ClientAdapter.java:105)
at org.restlet.engine.adapter.HttpClientHelper.handle(HttpClientHelper.java:119)
at org.restlet.Client.handle(Client.java:153)
at org.restlet.Restlet.handle(Restlet.java:275)
at org.apache.camel.component.restlet.RestletProducer.process(RestletProducer.java:79)
at org.apache.camel.component.restlet.RestletProducer.process(RestletProducer.java:98)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:141)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:668)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:596)
at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:237)
at org.apache.camel.processor.Splitter.process(Splitter.java:104)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:62)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:141)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
at org.apache.camel.processor.MulticastProcessor.doProcessParallel(MulticastProcessor.java:814)
at org.apache.camel.processor.MulticastProcessor.access$200(MulticastProcessor.java:84)
at org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:314)
at org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:299)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
And as per the doc I did configure the restlet component with the following code but I'm still seeing the same issue:
#Bean
public RestletComponent restlet()
{
RestletComponent restlet = new RestletComponent();
restlet.setMaxThreads(100);
restlet.setThreadMaxIdleTimeMs(10000);
restlet.setMaxQueued(20);
return restlet;
}
Note: The route did accepted 10 requests at a time then I started getting errors. And from configuration I could see the maxThreads by default is 10. Which means the that I did through Bean is not picking up properly.

Actually in my code I have defined restlet bean after route configure i.e as below
public class RoutesBuilder extends FatJarRouter {
....
#Override
public void configure() throws JAXBException {
......
}
#Bean(name={"restlet"})
public RestletComponent restlet()
{
.......
}
}
And now I have changed the order of defining the code as i.e first I have defined restlet and the route configure as below. With that in place I saw the restlet configuration is been picked up and I no more see the connection pool issue.
public class RoutesBuilder extends FatJarRouter {
....
#Bean(name={"restlet"})
public RestletComponent restlet()
{
.......
}
#Override
public void configure() throws JAXBException {
......
}
}

Check these things:
Make sure you have enabled <context:annotation-config/> in the Spring App Context XML file.
Use <context:component-scan base-package="<your package>" /> in the Spring XML.
Make sure you have annotated the containing that #Bean with #Configuration.
Add the name attribute to the #Bean: #Bean(name={"restlet"}).
Hope that helps.

Take a look at this example. Its Spring-boot APP. I created a separate config class and used spring-boot configuration annotation, so on startup spring-boot does all the configuration needed
Here is a working solution
https://github.com/RakeshBhat/rb-springbootcamelrestlet