I'm using FtpStreamingMessageSource in combination with poller with following config (#InboundChannelAdapter(channel = "ftpChannel", poller = #Poller("pollerMetadata"))):
#Bean
public PollerMetadata pollerMetadata(PlatformTransactionManager transactionManager) {
PeriodicTrigger trigger = new PeriodicTrigger(TimeUnit.SECONDS.toMillis(30));
trigger.setFixedRate(true);
MatchAlwaysTransactionAttributeSource source = new MatchAlwaysTransactionAttributeSource();
source.setTransactionAttribute(new DefaultTransactionAttribute());
TransactionInterceptor interceptor = new TransactionInterceptor(transactionManager, source);
PollerMetadata metadata = new PollerMetadata();
metadata.setTrigger(trigger);
metadata.setTransactionSynchronizationFactory(synchronizationFactory());
metadata.setAdviceChain(Collections.singletonList(interceptor));
return metadata;
}
It was working OK, until today I had a DB problem and an exception The last packet sent successfully to the server was 30,079 milliseconds ago. and (ERROR): LoggingHandler org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction; nested exception is java.sql.SQLException: Connection is
closed.
Somehow, poller stopped working after this, even though HikariCP managed to recover from exception after a while. Seems like a thread that was doing polling job was terminated.
Any idea how to make thread recover and continue with processing? After I restarted application, everything was back to normal.
UPDATE
this is the last exception I got before poller stopped working
2019-03-06 14:34:45 (ERROR): LoggingHandler org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction; nested exception is java.sql.SQLException: Connection is
closed
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:290)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:853)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:830)
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:503)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:285)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy71.call(Unknown Source)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:353)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:344)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Connection is closed
at com.zaxxer.hikari.pool.ProxyConnection$ClosedConnection.lambda$getClosedConnection$0(ProxyConnection.java:489)
at com.sun.proxy.$Proxy67.rollback(Unknown Source)
at com.zaxxer.hikari.pool.ProxyConnection.rollback(ProxyConnection.java:370)
at com.zaxxer.hikari.pool.HikariProxyConnection.rollback(HikariProxyConnection.java)
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:287)
... 22 more
There is no "termination" of a polling thread; polling uses a TaskScheduler and when the poll completes (whether an exception occurred or not) the thread is returned to the pool, ready for the next poll.
It's probably too late now (if you restarted your app) but if it happens again take a thread dump; most likely the poller thread is "stuck" somewhere in user (or DB) code.
Related
Using Spring Integration SFTP feature, and Inbound Channel Adapter, remote files are downloaded for processing.
While I use following start and stop routine to start/stop inbound channel adapter.
public void startInboundAdapter() throws ApplicationException {
SourcePollingChannelAdapter streamInboundAdapter = appContext.getBean("streamInboundAdapter", SourcePollingChannelAdapter.class);
streamInboundAdapter.start();
}
public void stopInboundAdapter() throws ApplicationException {
SourcePollingChannelAdapter streamInboundAdapter = appContext.getBean("streamInboundAdapter", SourcePollingChannelAdapter.class);
streamInboundAdapter.stop();
}
But I observe an exception after stop inbound adapter is called after all files processing is done. I am stuck here and looking out for a solution, please suggest.
Stacktrace
2020-08-15 15:44:24.298 INFO 9976 --- [ scheduling-1] o.s.i.e.SourcePollingChannelAdapter : stopped bean 'streamInboundAdapter'
2020-08-15 15:44:24.298 ERROR 9976 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessageHandlingException: error occurred in message handler [bean 'aggregatorFactoryBean'; defined in: 'class path resource [com/config/SFTPConfig.class]'; from source: 'org.springframework.core.type.classreading.SimpleMethodMetadata#40dd3977']; nested exception is org.springframework.messaging.MessagingException: Interrupted while obtaining lock; nested exception is java.lang.InterruptedException, failedMessage=GenericMessage [payload=sftp-inbound\upload_file.csv.bburpt.xls, headers={file_remoteHostPort=sftp.bloomberg.com:22, file_name=upload_file.csv.bburpt.xls, file_remoteDirectory=//report, file_originalFile=sftp-inbound\upload_file.csv.bburpt.xls, id=615015fe-b403-d247-c86e-f026a6ab3ad2, file_relativePath=upload_file.csv.bburpt.xls, file_remoteFile=upload_file.csv.bburpt.xls, timestamp=1597520532363}]
at org.springframework.integration.support.utils.IntegrationUtils.wrapInHandlingExceptionIfNecessary(IntegrationUtils.java:191)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:187)
at org.springframework.integration.handler.ReplyProducingMessageHandlerWrapper.handleRequestMessage(ReplyProducingMessageHandlerWrapper.java:49)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:127)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:170)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:403)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.handleMessage(SourcePollingChannelAdapter.java:234)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.messageReceived(AbstractPollingEndpoint.java:396)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:380)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.pollForMessage(AbstractPollingEndpoint.java:328)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$null$1(AbstractPollingEndpoint.java:275)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.lambda$execute$0(ErrorHandlingTaskExecutor.java:57)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:55)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.lambda$createPoller$2(AbstractPollingEndpoint.java:272)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.springframework.messaging.MessagingException: Interrupted while obtaining lock; nested exception is java.lang.InterruptedException
at org.springframework.integration.store.SimpleMessageStore.completeGroup(SimpleMessageStore.java:426)
at org.springframework.integration.aggregator.AggregatingMessageHandler.afterRelease(AggregatingMessageHandler.java:80)
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.processMessageForGroup(AbstractCorrelatingMessageHandler.java:503)
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.handleMessageInternal(AbstractCorrelatingMe ssageHandler.java:471)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:170)
... 31 more
Caused by: java.lang.InterruptedException
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1261)
at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:317)
at org.springframework.integration.store.SimpleMessageStore.completeGroup(SimpleMessageStore.java:412)
... 35 more
Stop is immediate and if you have an in-process task that is interruptible, it will be interrupted.
Please add a new feature suggestion issue on GitHub.
We should either implement Pausable allowing you to wait until it's paused before stopping, or add an "orderly" option for stop.
I have partitioned reader and writer. There are some queries getting transaction timedout while getting resultset back from cursor. But I couldn't able to get catch the exception. I need to catch the exception and update the status back to track those reports.
Below is the exception from logs:
com.ibm.ws.batch.JobLogger CWWKY0024W: The current chunk was rolled back to the previous checkpoint for step genGAHJSONReport in job instance 553 and job execution 553. Step metrics = [(READ_SKIP_COUNT,0), (PROCESS_SKIP_COUNT,0), (WRITE_SKIP_COUNT,0), (FILTER_COUNT,0), (COMMIT_COUNT,1), (READ_COUNT,5000), (WRITE_COUNT,5000), (ROLLBACK_COUNT,0)]
com.ibm.ws.batch.JobLogger CWWKY0030I: An exception occurred while running the step genGAHJSONReport.
com.ibm.jbatch.container.exception.BatchContainerRuntimeException: Failure in Read-Process-Write Loop
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:704)
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:795)
at com.ibm.jbatch.container.controller.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:295)
at com.ibm.jbatch.container.controller.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:118)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeCoreTransitionLoop(WorkUnitThreadControllerImpl.java:96)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeWorkUnit(WorkUnitThreadControllerImpl.java:178)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl$AbstractControllerHelper.runExecutionOnThread(WorkUnitThreadControllerImpl.java:503)
at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.runExecutionOnThread(WorkUnitThreadControllerImpl.java:92)
at com.ibm.jbatch.container.util.BatchWorkUnit.run(BatchWorkUnit.java:113)
at com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79)
at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:232)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.ibm.jbatch.container.exception.TransactionManagementException: javax.transaction.RollbackException
at com.ibm.jbatch.container.transaction.impl.JTAUserTransactionAdapter.commit(JTAUserTransactionAdapter.java:108)
at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:656)
... 15 more
Caused by: javax.transaction.RollbackException
at com.ibm.tx.jta.impl.TransactionImpl.stage3CommitProcessing(TransactionImpl.java:978)
at com.ibm.tx.jta.impl.TransactionImpl.processCommit(TransactionImpl.java:778)
at com.ibm.tx.jta.impl.TransactionImpl.commit(TransactionImpl.java:711)
at com.ibm.tx.jta.impl.TranManagerImpl.commit(TranManagerImpl.java:165)
at com.ibm.tx.jta.impl.TranManagerSet.commit(TranManagerSet.java:113)
at com.ibm.tx.jta.impl.UserTransactionImpl.commit(UserTransactionImpl.java:162)
at com.ibm.tx.jta.embeddable.impl.EmbeddableUserTransactionImpl.commit(EmbeddableUserTransactionImpl.java:101)
at com.ibm.ws.transaction.services.UserTransactionService.commit(UserTransactionService.java:72)
at com.ibm.jbatch.container.transaction.impl.JTAUserTransactionAdapter.commit(JTAUserTransactionAdapter.java:101)
We use redis in spring boot project. After running a period of time, the redis operation MAY throw a broken pipe error, but sometimes it will succeed. Restarting the service will resolve this problem, but it's not a good idea.
I can't tell the reason why it happens. It seems that some redis connections in the pool are unusable, but aren't closed and evicted from the pool.
my questions are:
possible reason causing the broken pipe error?
if there isn't redis operation for a long period, will the idle connection in the pool become unusable?
will the connection be closed and evicted from pool when broken pipe error happen?
pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
application.yml:
spring:
redis:
database: 0
host: ${REDIS_HOST:127.0.0.1}
password: ${REDIS_PASSWORD:password}
port: ${REDIS_PORT:6379}
timeout: ${REDIS_TIMEOUT:1000}
pool:
max-active: ${REDIS_MAX_ACTIVE:100}
max-wait: ${REDIS_MAX_WAIT:500}
max-idle: ${REDIS_MAX_IDLE:20}
min-idle: ${REDIS_MIN_IDLE:5}
error message:
org.springframework.data.redis.RedisConnectionFailureException: java.net.SocketException: Broken pipe (Write failed); nested exception is redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Broken pipe (Write failed)
at org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert(JedisExceptionConverter.java:67) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert(JedisExceptionConverter.java:41) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:37) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:37) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.connection.jedis.JedisConnection.convertJedisAccessException(JedisConnection.java:212) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.connection.jedis.JedisConnection.hSet(JedisConnection.java:2810) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.core.DefaultHashOperations$9.doInRedis(DefaultHashOperations.java:173) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:204) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:166) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:88) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
at org.springframework.data.redis.core.DefaultHashOperations.put(DefaultHashOperations.java:170) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Broken pipe (Write failed)
at redis.clients.jedis.Connection.flush(Connection.java:291) ~[jedis-2.8.2.jar!/:na]
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:220) ~[jedis-2.8.2.jar!/:na]
at redis.clients.jedis.BinaryJedis.hset(BinaryJedis.java:749) ~[jedis-2.8.2.jar!/:na]
at org.springframework.data.redis.connection.jedis.JedisConnection.hSet(JedisConnection.java:2808) ~[spring-data-redis-1.7.6.RELEASE.jar!/:na]
... 115 common frames omitted
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_111]
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109) ~[na:1.8.0_111]
at java.net.SocketOutputStream.write(SocketOutputStream.java:153) ~[na:1.8.0_111]
at redis.clients.util.RedisOutputStream.flushBuffer(RedisOutputStream.java:52) ~[jedis-2.8.2.jar!/:na]
at redis.clients.util.RedisOutputStream.flush(RedisOutputStream.java:216) ~[jedis-2.8.2.jar!/:na]
at redis.clients.jedis.Connection.flush(Connection.java:288) ~[jedis-2.8.2.jar!/:na]
... 118 common frames omitted
Answer my question:
why the broken pipe error happen?
TransactionSynchronizationManager will hold the RedisConnection in thread, and wont close it or return it to pool, see RedisTemplate.java and RedisConnectionUtils.java. After restarting the redis server, operation on the holded RedisConnection in thread will throw broken pipe error.
how to resolve it?
Adding try/catch for all redis operation, if the error happens , unbinding it from thread, and could get a new connection from pool and execute redis operation again.
private static final ExceptionTranslationStrategy EXCEPTION_TRANSLATION =
new FallbackExceptionTranslationStrategy(JedisConverters.exceptionConverter());
public Object req(RedisRequest req) {
try {
return req.request();
} catch (Exception ex) {
if (ex instanceof NullPointerException) {
throw ex;
}
DataAccessException exception = EXCEPTION_TRANSLATION.translate(ex);
if (exception instanceof RedisConnectionFailureException) {
RedisConnectionUtils.unbindConnection(factory);
/** retry again */
return req.request();
} else {
throw ex;
}
}
}
This can happen for n number of reasons, one of them could be when you use a long-living connection (e. g. connect to Redis on application start and then use the connection over and over).
Some of the things to do are:
Reconnect if the connection is broken (needs some try/catch magic to prevent that errors are propagated to your application logic)
or a better was is to use
TestOnBorrow - Sends a PING request when you ask for the resource.
TestOnReturn - Sends a PING when you return a resource to the pool.
TestWhileIdle - Sends periodic PINGS from idle resources in the pool.
Connect at the moment you need the connection and disconnect afterwards
Regarding
if there isn't redis operation for a long period, will the idle connection in the pool become unusable?
maxidle means at any given time the system allows 'maxIdle' many number of connections to be idle the rest will be constantly checked, closed and returned to the pool.
I dont know a reason when an idle connection be unusable. Anyways this can be got rid of by using ways mentioned above.
I'm the new to redis, I start the server about this tutorial. And it work. Then I use write the code using java to connect redis, then it's ok, like this:
Jedis jedis = new Jedis("localhost");
System.out.println("Connection to server sucessfully");
//store data in redis list
jedis.lpush("tutorial-list", "Redis");
jedis.lpush("tutorial-list", "Mongodb");
jedis.lpush("tutorial-list", "Mysql");
But, when I use multithread to push the redis, it will throw the exception "read time out":
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601) at
org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException:
java.net.SocketTimeoutException: Read timed out at
redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:201)
at
redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
at redis.clients.jedis.Protocol.process(Protocol.java:141) at
redis.clients.jedis.Protocol.read(Protocol.java:205) at
redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)
at
redis.clients.jedis.Connection.getBinaryMultiBulkReply(Connection.java:233)
at redis.clients.jedis.Jedis.keys(Jedis.java:185) at
org.v11.redis_mongo_task.UpdateApp.jobDetail(UpdateApp.java:23) at
org.v11.redis_mongo_task.UpdateApp.main(UpdateApp.java:42) ... 5 more
Caused by: java.net.SocketTimeoutException: Read timed out at
java.net.SocketInputStream.socketRead0(Native Method) at
java.net.SocketInputStream.read(SocketInputStream.java:150) at
java.net.SocketInputStream.read(SocketInputStream.java:121) at
java.net.SocketInputStream.read(SocketInputStream.java:107) at
redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195)
... 13 more
What happened for redis? why it can work in single thread?
According to this answer, a single Jedis instance is not threadsafe. You will have to use JedisPool for multithreading. You can read here on how use it and here to set the max connections and what will happen if those connections are all occupied.
I'm posting links since two of them are SO answers an they should get the credit and one is from github official repo, so if anything gets updated it should be reflected here too.
I am getting UnexpectedRollbackException. Here is the complete stack trace:
org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe due to a timeout); nested exception is javax.transaction.RollbackException
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1031)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:635)
at com.cmates.userIcon.service.IconUpdaterServiceImpl$$EnhancerByCGLIB$$78838aa7.persist(<generated>)
at com.cmates.userIcon.service.ScheduledIconUpdaterServiceImpl.doScheduledTask(ScheduledIconUpdaterServiceImpl.java:125)
at com.cmates.profile.services.IconSyncSingletonImpl.process(IconSyncSingletonImpl.java:121)
at com.cmates.profile.services.IconSyncJob.executeInternal(IconSyncJob.java:25)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: javax.transaction.RollbackException
at org.objectweb.jotm.TransactionImpl.commit(TransactionImpl.java:245)
at org.objectweb.jotm.Current.commit(Current.java:488)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
... 13 more
[18 Apr 2011 00:54:00,590] ERROR ErrorLogger - Job (DEFAULT.iconSyncJob threw an exception.
This Exception suddenly started showing in my logs. I didn't made any changes in my code.
I guess this might be due to a timeout?
I'm seeing a mention of a quartz job in the stacktrace. Looks like something has setup a periodic job and the AOP transaction management has picked up on it.
It looks like your transaction lasts than transaction timeout limit. You should increase transaction timeout limit..
But I am not sure about that, you should post more stack trace to understand the reason of rollback.