Spring Integration lock busy, another session is active - java

I'm connecting to a POP3 inbox (it doesn't support IMAP, unfortunately) using Spring Integration. Here's my config:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/mail http://www.springframework.org/schema/integration/mail/spring-integration-mail.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-mail="http://www.springframework.org/schema/integration/mail"
xmlns:util="http://www.springframework.org/schema/util">
<int:channel id="mailInputChannel" />
<int:channel id="channel1" />
<int:channel id="channel2" />
<int:channel id="discardChannel" />
<!-- replace 'userid and 'password' with the real values -->
<int-mail:inbound-channel-adapter id="pop3ShouldDeleteTrue"
store-uri="pop3://userid:password#host.example.com/Inbox"
channel="mailInputChannel"
should-delete-messages="true"
auto-startup="true"
java-mail-properties="javaMailProperties">
<!-- Will poll every 1 seconds -->
<int:poller max-messages-per-poll="1000" fixed-delay="1000"/>
</int-mail:inbound-channel-adapter>
<int:recipient-list-router id="incomingMailRouter" input-channel="mailInputChannel">
<int:recipient channel="channel1" selector-expression="payload.subject matches '^.*channel1.*$'"/>
<int:recipient channel="channel2" selector-expression="payload.subject matches '^.*channel2.*$'"/>
</int:recipient-list-router>
<util:properties id="javaMailProperties">
<prop key="mail.pop3.socketFactory.fallback">false</prop>
<prop key="mail.debug">false</prop>
</util:properties>
</beans>
Basically I have an inbound-channel-adapter that reads in the messages and a simple recipient-list-router that directs it accordingly.
When I start up the application, it runs fine. However, after it processes the first email, it sometimes hangs. The debug output is:
19260 [task-scheduler-1] DEBUG org.springframework.integration.mail.Pop3MailReceiver - connecting to store [pop3://username:password#mailhost.example.com/Inbox]
DEBUG POP3: mail.pop3.apop.enable: false
DEBUG POP3: mail.pop3.disablecapa: false
DEBUG POP3: connecting to host "mailhost.example.com", port 110, isSSL false
+OK Qpopper (version 4.0.5) at cobweb starting.
CAPA
+OK Capability list follows
TOP
USER
LOGIN-DELAY 0
EXPIRE 0
UIDL
RESP-CODES
AUTH-RESP-CODE
X-MANGLE
X-MACRO
X-LOCALTIME Wed, 10 Aug 2016 11:50:57 -0700
IMPLEMENTATION Qpopper-version-4.0.5
.
DEBUG POP3: authentication command trace suppressed
DEBUG POP3: authentication command failed
QUIT
+OK Pop server at cobweb signing off.
30529 [task-scheduler-1] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory - Returning cached instance of singleton bean 'errorChannel'
30531 [task-scheduler-1] DEBUG org.springframework.integration.channel.PublishSubscribeChannel - preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11), headers={id=43817279-7717-f4d7-971a-3c283e2dbec3, timestamp=1470849586452}]
30531 [task-scheduler-1] DEBUG org.springframework.integration.handler.LoggingHandler - _org.springframework.integration.errorLogger.handler received message: ErrorMessage [payload=org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11), headers={id=43817279-7717-f4d7-971a-3c283e2dbec3, timestamp=1470849586452}]
30534 [task-scheduler-1] ERROR org.springframework.integration.handler.LoggingHandler - org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11)
at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:131)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:209)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:245)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:58)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:190)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:186)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:353)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:344)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11)
at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:207)
at javax.mail.Service.connect(Service.java:295)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at org.springframework.integration.mail.AbstractMailReceiver.connectStoreIfNecessary(AbstractMailReceiver.java:286)
at org.springframework.integration.mail.AbstractMailReceiver.openFolder(AbstractMailReceiver.java:297)
at org.springframework.integration.mail.AbstractMailReceiver.receive(AbstractMailReceiver.java:319)
at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:112)
... 19 more
It stays in this loop for a (seemingly) random amount of time, sometimes up to 2 minutes, spitting out these exceptions every time it tries to connect. However, sometimes it connects just file right after processing a message. Processing a message is the only thing that causes it to fail.
Is Spring trying to hold on to that connection to that connection too long? Can I tell it to drop the connection right after it fetches the email?


Turns out it was a server-side issue. After switching to a production-grade mail server that supported IMAP, the issue was resolved.

Related

Mule Flow fails on first try but works after with the LDAP connector

I have a strange behavior with my Mule flow. It is the only one that does this and I have no idea why. I use the LDAP connector. When I try the next day the first try to the listener fails and I get the error that follows:
ERROR 2015-05-29 08:34:44,115 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Registered authentication is set to org.mule.module.spring.security.filters.http.HttpBasicAuthenticationFilter but there was no security context on the session. Authentication denied on endpoint none. Message payload is of type: NullPayload
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Registered authentication is set to org.mule.module.spring.security.filters.http.HttpBasicAuthenticationFilter but there was no security context on the session. Authentication denied on endpoint none. Message payload is of type: NullPayload (org.mule.api.security.UnauthorisedException)
org.mule.transport.http.filters.HttpBasicAuthenticationFilter:156 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/security/UnauthorisedException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.api.security.UnauthorisedException: Registered authentication is set to org.mule.module.spring.security.filters.http.HttpBasicAuthenticationFilter but there was no security context on the session. Authentication denied on endpoint none. Message payload is of type: NullPayload
at org.mule.transport.http.filters.HttpBasicAuthenticationFilter.authenticateInbound(HttpBasicAuthenticationFilter.java:156)
at org.mule.security.AbstractEndpointSecurityFilter.authenticate(AbstractEndpointSecurityFilter.java:54)
at org.mule.security.AbstractAuthenticationFilter.doFilter(AbstractAuthenticationFilter.java:52)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
WARN 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Search failed.
INFO 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Connection dude#host.host#ldap://host.host.host:389/ closed.
ERROR 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.retry.notifiers.ConnectNotifier: Failed to connect/reconnect: Work Descriptor. Root Exception was: Connection reset. Type: class java.net.SocketException
ERROR 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Failed to invoke search. Message payload is of type: NullPayload
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Connection reset (java.net.SocketException)
java.net.SocketInputStream:-1 (null)
2. javax.naming.CommunicationException: Connection reset [Root exception is java.net.SocketException: Connection reset]; remaining name 'OU=Musee,DC=mus, DC=civ'; resolved object com.sun.jndi.ldap.LdapCtx#2ba8e4bc (javax.naming.CommunicationException)
com.sun.jndi.ldap.LdapCtx:-1 (http://java.sun.com/j2ee/sdk_1.3/techdocs/api/javax/naming/CommunicationException.html)
3. Connection reset (org.mule.module.ldap.api.CommunicationException)
sun.reflect.NativeConstructorAccessorImpl:-2 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/module/ldap/api/CommunicationException.html)
4. Failed to invoke search. Message payload is of type: NullPayload (org.mule.api.MessagingException)
org.mule.devkit.processor.DevkitBasedMessageProcessor:128 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
When I call it the second time it always works and I don't know for how much time but it continues working even from another browser were the auth session isn't there it works on the first try after that
Here is the flow :
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:smtp="http://www.mulesoft.org/schema/mule/smtp" xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:wmq="http://www.mulesoft.org/schema/mule/ee/wmq" xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.6.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:spring-security="http://www.mulesoft.org/schema/mule/spring-security"
xmlns:security="http://www.springframework.org/schema/security"
xmlns:ldap="http://www.mulesoft.org/schema/mule/ldap"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/ldap http://www.mulesoft.org/schema/mule/ldap/current/mule-ldap.xsd
http://www.mulesoft.org/schema/mule/ee/wmq http://www.mulesoft.org/schema/mule/ee/wmq/current/mule-wmq-ee.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.mulesoft.org/schema/mule/smtp http://www.mulesoft.org/schema/mule/smtp/current/mule-smtp.xsd
http://www.mulesoft.org/schema/mule/spring-security http://www.mulesoft.org/schema/mule/spring-security/current/mule-spring-security.xsd
http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">
<ldap:config name="ldapConf" url="ldap://host.host.host:389/" authDn="dude#host.host" authPassword="password" doc:name="LDAP"/>
<http:listener-config name="HTTP_Listener_Configuration" host="host" port="8084" doc:name="HTTP Listener Configuration"/>
<spring:beans>
<security:authentication-manager alias="secAuthSample">
<security:authentication-provider>
<security:user-service id="authService">
<security:user name="ldap" password="ldap" authorities="ROLE_ADMIN"/>
</security:user-service>
</security:authentication-provider>
</security:authentication-manager>
</spring:beans>
<spring-security:security-manager>
<spring-security:delegate-security-provider name="memory-provider" delegate-ref="secAuthSample"/>
</spring-security:security-manager>
<flow name="activedirectoryGetUsers">
<http:listener config-ref="HTTP_Listener_Configuration" path="/ldap" doc:name="HTTP">
<http:response-builder>
<http:header headerName="content-type" value="text/plain"/>
<http:header headerName="Content-Disposition" value="attachment;filename=0544#[function:datestamp:yyyyMMdd].csv"/>
</http:response-builder>
</http:listener>
<spring-security:http-security-filter realm="" securityProviders="memory-provider"/>
<ldap:search config-ref="ldapConf" doc:name="LDAP" baseDn="OU=Musee,DC=mus, DC=civ" filter="(&(objectClass=user)(Company=*dasdasd)(!(extensionAttribute11=asdasd))(!(extensionAttribute12=asdasd)))" scope="SUB_TREE" pageSize="200000000" maxResults="20000000">
<ldap:attributes>
<ldap:attribute>dn</ldap:attribute>
<ldap:attribute>GivenName</ldap:attribute>
<ldap:attribute>sn</ldap:attribute>
<ldap:attribute>mail</ldap:attribute>
<ldap:attribute>telephonenumber</ldap:attribute>
<ldap:attribute>l</ldap:attribute>
<ldap:attribute>st</ldap:attribute>
<ldap:attribute>postalCode</ldap:attribute>
<ldap:attribute>streetAddress</ldap:attribute>
</ldap:attributes>
</ldap:search>
<collection-splitter doc:name="Split Result Set"/>
<ldap:ldap-entry-to-map doc:name="to LDIF"/>
<collection-aggregator failOnTimeout="false" doc:name="Aggregate entries"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy" file="adtocsv.groovy"></scripting:script>
</scripting:component>
<message-properties-transformer
overwrite="true" doc:name="Message Properties">
<add-message-property key="Content-Type"
value="text/plain;charset=ISO-8859-1;" />
<add-message-property key="MULE_ENCODING" value="ISO-8859-1" />
</message-properties-transformer>
</flow>
</mule>

Mule ESB LDAP connector able to bind but fails at lookup

So I am trying to create a Mule flow to authenticate against my LDAP. I want to bind sending a JSON in through PHP then check if it binds. It then returns the value of the binded user so I can store that in my session variable.
Sending the JSON works well and I can authenticate to the server as I have tried with wrong password and it doesn't work. It binds successfully but cannot do the lookup after that. I hope someone can help.
Here's my code :
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:ldap="http://www.mulesoft.org/schema/mule/ldap" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.6.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/ldap http://www.mulesoft.org/schema/mule/ldap/current/mule-ldap.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd">
<http:listener-config name="HTTP_Listener_Configuration" host="localhost" port="8080" doc:name="HTTP Listener Configuration"/>
<ldap:config name="ldapConf" url="ldap://address.address.address:389/" doc:name="LDAP">
<ldap:connection-pooling-profile evictionCheckIntervalMillis="1" minEvictionMillis="1" exhaustedAction="WHEN_EXHAUSTED_WAIT"/>
</ldap:config>
<flow name="loginldapFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/login" doc:name="HTTP">
<http:response-builder>
<http:header headerName="content-type" value="text/plain"/>
</http:response-builder>
</http:listener>
<json:json-to-object-transformer doc:name="JSON to Object" returnClass="java.util.Map"/>
<ldap:bind config-ref="ldapConf" authDn="#[payload.username]#address.addresss" authPassword="#[payload.password]" authentication="simple" doc:name="LDAP"/>
</flow>
</mule>
Here's the error :
INFO 2015-05-07 14:01:05,452 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Binded to ldap://address.address.address:389/ with simple authentication as Mule1#address.address
INFO 2015-05-07 14:01:05,452 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Already binded to ldap://address.address.address:389/ with simple authentication as Mule1#address.address. Closing connection first.
INFO 2015-05-07 14:01:05,452 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Connection Mule1#address.address#ldap://address.address.address:389/ closed.
INFO 2015-05-07 14:01:05,467 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Re-binding to ldap://address.address.address:389/ with simple authentication as Mule1#address.address
INFO 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Binded to ldap://address.address.address:389/ with simple authentication as Mule1#address.address
INFO 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.adapters.LDAPConnectorConnectionIdentifierAdapter: Bind was successful for user: Mule1#address.address
WARN 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Lookup failed.
WARN 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.adapters.LDAPConnectorConnectionIdentifierAdapter: Cannot retrieve entry for dn: Mule1#address.address
org.mule.module.ldap.api.InvalidAttributeException: Mule1#address.address: [LDAP: error code 34 - 0000208F: NameErr: DSID-03100225, problem 2006 (BAD_NAME), data 8350, best match of:
'Mule1#address.address'
Caused by: javax.naming.InvalidNameException: Mule1#address.address: [LDAP: error code 34 - 0000208F: NameErr: DSID-03100225, problem 2006 (BAD_NAME), data 8350, best match of:
'Mule1#address.address'
I've looked at what error 34 is but I cannot add DN to bind in Mule.
I have also tried to remove the connection pooling profile and other configurations with the same result.
Thanks

The error is that the DN is invalid. Look at the answers at this previous question: javax.naming.InvalidNameException: [LDAP: error code 34 - invalid DN].
You could try using a tool to explore the LDAP structure like http://jxplorer.org/ to understand what the server is expecting.

File Polling and Filtering Issues with SFTP Spring Integration

I am trying to pull remote files through SFTP Spring Integration and dump into my local drive, and kick-off a Spring Batch Job (Step-1 to read, process & write), followed by Step-2 where a Tasklet archives the local file downloaded to another location locally and deletes the downloaded file.
Assuming the files being downloaded is "data.service", "data.maintenance" and "data.transaction". I am only processing "data.service" but all 3 are archived and deleted.
I want the above steps to happen at a specific interval assuming that a new set of files will be overwritten at the remote location.
Bean def
<bean id="acceptAllFileListFilter" class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/>
<bean id="acceptOnceFileListFilter" class="org.springframework.integration.file.filters.AcceptOnceFileListFilter"/>
conf
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="${inbound.remote.directory}"
auto-create-local-directory="true"
delete-remote-files="false"
preserve-timestamp="true"
filename-pattern="*.*"
local-filter="acceptAllFileListFilter" >
<int:poller max-messages-per-poll="-1" fixed-rate="60000" />
</int-sftp:inbound-channel-adapter>
What is happening now is...
The above triggers this sequence and runs in an unending loop, without fetching the new files from remote. Details as below:
I start with an empty local folder, and remote server has all 3 files
[Start the program]
Spring Integration program downloads all 3 files to local location
Spring Batch Job Step-1 gets triggered Tasklet (Step-2) archives the
local files Tasklet (Step-2) deletes the local files Spring
Integration program downloads all 3 files to local location (why???)
program keeps running Step-1 to Step-5 in an unending loop
New file dropped on the remote server, same (1) to (5) no job triggered
Tried replacing as below, but the program just runs once, i.e., from (1) to (5) and nothing happens
local-filter="acceptOnceFileListFilter" >
Expected
Spring Integration program downloads all 3 files to local location
Spring Batch Job Step-1 gets triggered
Tasklet (Step-2) archives the local files
Tasklet (Step-2) deletes the local files
Spring Integration program waits for new files to be dropped in the remote location (polls every 60000 ms)
New file dropped on the remote server
Start with (1) through (6)
Per suggestion on a different post I have a unique job parameter as below:
#Transformer
public JobLaunchRequest toRequest(Message<File> message) {
JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
jobParametersBuilder.addString(fileParameterName, message.getPayload().getAbsolutePath());
jobParametersBuilder.addLong("currentTime", new Long(System.currentTimeMillis()));
return new JobLaunchRequest(job, jobParametersBuilder.toJobParameters());
}
Tried with different combinations of filter, none served my purpose. Any sample would be of great help, struggling with this since last week.
Here's my change per the suggestions:
Adding a composite filter as below, but what would be the argument that will be passed to SftpPersistentAcceptOnceFileListFilter? I could not see any samples in the internet.
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
<list>
<bean class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter"/>
<bean class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
</bean>
</list>
</constructor-arg>
</bean>
Including in the adapter as below:
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="${inbound.remote.directory}"
auto-create-local-directory="true"
delete-remote-files="false"
filter="compositeFilter">
<int:poller max-messages-per-poll="-1" fixed-rate="600000" />
</int-sftp:inbound-channel-adapter>
Update - Changes after adding persistent filter:
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="file:${inbound.remote.directory}"
auto-create-local-directory="false"
delete-remote-files="false"
filter="compositeFilter"
>
<int:poller max-messages-per-poll="-1" fixed-rate="60000" />
</int-sftp:inbound-channel-adapter>
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
<list>
<bean class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter">
<constructor-arg name="store" ref="metadataStore"/>
<constructor-arg value="*.*"/>
</bean>
<bean class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
<constructor-arg name="store" ref="metadataStore"/>
<constructor-arg value="*.*"/>
</bean>
</list>
</constructor-arg>
</bean>
<bean id="redisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<property name="port" value="7379"/>
</bean>
<bean name="metadataStore" class="org.springframework.integration.redis.metadata.RedisMetadataStore">
<constructor-arg name="connectionFactory" ref="redisConnectionFactory"/>
</bean>
This gives me the exception as below. I tried the suggestions in the Spring forum, but I don't have a connection issues.
2014-07-17 22:27:14,139 [task-scheduler-1] INFO com.jcraft.jsch -
Authentication succeeded (keyboard-interactive). 2014-07-17
22:27:14,192 [task-scheduler-1] INFO com.jcraft.jsch - Disconnecting
from localhost port 22 2014-07-17 22:27:14,195 [task-scheduler-1]
DEBUG
org.springframework.beans.factory.support.DefaultListableBeanFactory -
Returning cached instance of singleton bean 'errorChannel' 2014-07-17
22:27:14,197 [task-scheduler-1] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
preSend on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}] 2014-07-17 22:27:14,197 [task-scheduler-1]
DEBUG org.springframework.integration.handler.LoggingHandler - (inner
bean)#22 received message: [Payload MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}] 2014-07-17 22:27:14,199 [task-scheduler-1]
ERROR org.springframework.integration.handler.LoggingHandler -
org.springframework.messaging.MessagingException: Problem occurred
while synchronizing remote to local directory at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:193)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:187)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:52)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:143)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:278)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:272)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695) Caused by:
org.springframework.messaging.MessagingException: Failed to execute on
session at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:311)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 21 more Caused by: java.lang.ClassCastException:
com.jcraft.jsch.ChannelSftp$LsEntry cannot be cast to java.io.File at
org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter.fileName(FileSystemPersistentAcceptOnceFileListFilter.java:28)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.buildKey(AbstractPersistentAcceptOnceFileListFilter.java:88)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.accept(AbstractPersistentAcceptOnceFileListFilter.java:48)
at
org.springframework.integration.file.filters.AbstractFileListFilter.filterFiles(AbstractFileListFilter.java:40)
at
org.springframework.integration.file.filters.CompositeFileListFilter.filterFiles(CompositeFileListFilter.java:97)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.filterFiles(AbstractInboundFileSynchronizer.java:157)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:173)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
... 22 more
2014-07-17 22:27:14,199 [task-scheduler-1] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
postSend (sent=true) on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}]
Update post removing FileSystemPersistentAcceptOnceFileListFilter filter leaving the 'SftpPersistentAcceptOnceFileListFilter'
2014-07-18 09:29:21,942 [task-scheduler-3] INFO com.jcraft.jsch -
Authentications that can continue:
publickey,keyboard-interactive,password 2014-07-18 09:29:21,942
[task-scheduler-3] INFO com.jcraft.jsch - Next authentication method:
publickey 2014-07-18 09:29:21,942 [task-scheduler-3] INFO
com.jcraft.jsch - Authentications that can continue:
keyboard-interactive,password 2014-07-18 09:29:21,942
[task-scheduler-3] INFO com.jcraft.jsch - Next authentication method:
keyboard-interactive 2014-07-18 09:29:22,022 [task-scheduler-3] INFO
com.jcraft.jsch - Authentication succeeded (keyboard-interactive).
2014-07-18 09:29:22,067 [task-scheduler-3] DEBUG
org.springframework.data.redis.core.RedisConnectionUtils - Opening
RedisConnection 2014-07-18 09:29:22,068 [task-scheduler-3] INFO
com.jcraft.jsch - Disconnecting from localhost port 22 2014-07-18
09:29:22,068 [task-scheduler-3] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
preSend on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}] 2014-07-18 09:29:22,068 [Connect thread
localhost session] INFO com.jcraft.jsch - Caught an exception,
leaving main loop due to Socket closed 2014-07-18 09:29:22,068
[task-scheduler-3] DEBUG
org.springframework.integration.handler.LoggingHandler - (inner
bean)#22 received message: [Payload MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}] 2014-07-18 09:29:22,069 [task-scheduler-3]
ERROR org.springframework.integration.handler.LoggingHandler -
org.springframework.messaging.MessagingException: Problem occurred
while synchronizing remote to local directory at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:193)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:187)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:52)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:143)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:278)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:272)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695) Caused by:
org.springframework.messaging.MessagingException: Failed to execute on
session at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:311)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 21 more Caused by:
org.springframework.data.redis.RedisConnectionFailureException: Cannot
get Jedis connection; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:97)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:143)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:41)
at
org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:128)
at
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:91)
at
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:78)
at
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:177)
at
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:152)
at
org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:84)
at
org.springframework.data.redis.core.DefaultHashOperations.putIfAbsent(DefaultHashOperations.java:179)
at
org.springframework.data.redis.core.DefaultBoundHashOperations.putIfAbsent(DefaultBoundHashOperations.java:91)
at
org.springframework.data.redis.support.collections.RedisProperties.putIfAbsent(RedisProperties.java:228)
at
org.springframework.integration.redis.metadata.RedisMetadataStore.putIfAbsent(RedisMetadataStore.java:139)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.accept(AbstractPersistentAcceptOnceFileListFilter.java:51)
at
org.springframework.integration.file.filters.AbstractFileListFilter.filterFiles(AbstractFileListFilter.java:40)
at
org.springframework.integration.file.filters.CompositeFileListFilter.filterFiles(CompositeFileListFilter.java:97)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.filterFiles(AbstractInboundFileSynchronizer.java:157)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:173)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
... 22 more Caused by:
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool at
redis.clients.util.Pool.getResource(Pool.java:42) at
redis.clients.jedis.JedisPool.getResource(JedisPool.java:84) at
redis.clients.jedis.JedisPool.getResource(JedisPool.java:10) at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:90)
... 41 more Caused by:
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused at
redis.clients.jedis.Connection.connect(Connection.java:150) at
redis.clients.jedis.BinaryClient.connect(BinaryClient.java:71) at
redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1783) at
redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:65) at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:819)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:429)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:360)
at redis.clients.util.Pool.getResource(Pool.java:40) ... 44 more
Caused by: java.net.ConnectException: Connection refused at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:382) at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:241)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:228) at
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:431) at
java.net.Socket.connect(Socket.java:527) at
redis.clients.jedis.Connection.connect(Connection.java:144) ... 51
more
2014-07-18 09:29:22,069 [task-scheduler-3] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
postSend (sent=true) on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}]

Gist Here
Hopefully self-explanatory; it just prints the file name on the console, then deletes it.

It could be the case that if the new files dropped on the server has same name(which are downloaded in the first iteration) then spring integration will not attempt to download the files again. "acceptOnceFileListFilter" seems like something that would restrict sftp integration component to download the same file again.
Update: Here is some sample code which polls a remote directory every 10 secs without introducing any filters(I am not sure why you need them, as you want to download the files regardless). Given that you dont want to change the file names, the only thing that you will have to make sure is that your downloaded local needs to deleted when the next file is dropped on the remote server. I verified it and worked fine for me.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-ftp="http://www.springframework.org/schema/integration/ftp"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/ftp http://www.springframework.org/schema/integration/ftp/spring-integration-ftp.xsd">
<int:channel id="incomingChannel"/>
<bean id="ftpClientFactory" class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="host"/>
<property name="username" value="username"/>
<property name="password" value="password"/>
</bean>
<!-- poll every 10 secs without worrying about file names-->
<int-ftp:inbound-channel-adapter id="triggerFtpInBound"
channel="incomingChannel"
auto-create-local-directory="true"
local-directory="C:\home\temp\ftp"
remote-directory="/export/home/rbaljinder/ftp-test"
filename-pattern="*.*"
session-factory="ftpClientFactory">
<int:poller cron="1/10 * * * * *" max-messages-per-poll="1"/>
</int-ftp:inbound-channel-adapter>
UPDATE: seems working now.Added local-filter="acceptAllFileListFilter".
<int-ftp:inbound-channel-adapter id="triggerFtpInBound"
channel="incomingChannel"
auto-create-local-directory="true"
local-directory="C:\home\temp\ftp"
remote-directory="/export/home/cwk2/ftp-test"
filename-pattern="*.*"
session-factory="ftpClientFactory"
local-filter="acceptAllFileListFilter">
<int:poller cron="1/10 * * * * *" max-messages-per-poll="1"/>
</int-ftp:inbound-channel-adapter>
<bean id="acceptAllFileListFilter"
class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/>
<int:service-activator id="jobServiceActivator"
input-channel="incomingChannel"
ref="triggerJobLauncher"
method="launch"/>
#Component("triggerJobLauncher")
public static class TriggerJobLauncher {
#Autowired
JobLauncher jobLauncher;
public void launch(File file) throws Exception {
System.out.println("test:" + file);
}
}

Spring RmiExporter service fails after successful startup

I'm experiencing a situation where an RMI Service configured using Spring's RmiServiceExporter is starting up just fine, and is usable for a while - then after some unknown amount of time, the service is unavailable even though the java process for that Spring context is still running.
At the moment the work around is to reboot the java process, but this is hardly acceptable in a production environment. I cannot figure out, or even begin to guess, why this might be happening, or what to look for that might be going wrong. No S/O or Google search has been useful, because all I find are examples of RmiServiceExporter failing to start at all, not starting ok and then failing later on. Any clue at all would be useful.
Output of lsof | head -1;lsof | grep 1197, before the service fails:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 6882 ubuntu 176u IPv6 54677985 0t0 TCP *:1197 (LISTEN)
Service-side Spring config:
<bean class="org.springframework.remoting.rmi.RmiServiceExporter">
<property name="serviceName" value="MyRmiService" />
<property name="service" ref="myService" />
<property name="serviceInterface" value="package.MyService" />
<property name="registryPort" value="${my.rmi.port}" />
</bean>
Client-side Spring config:
<bean id="remoteService" class="org.springframework.remoting.rmi.RmiProxyFactoryBean">
<property name="serviceUrl" value="rmi://${my.rmi.host}:${my.rmi.port}/MyRmiService"/>
<property name="serviceInterface" value="package.MyService"/>
<property name="lookupStubOnStartup" value="false"/>
<property name="refreshStubOnConnectFailure" value="true"/>
</bean>
Client-side stacktrace, after the service fails (lsof reports nothing):
Exception in thread "main" org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed; nested exception is java.rmi.ConnectException: Connection refused to host: rmiservice.host; nested exception is:
java.net.ConnectException: Connection refused
at org.springframework.remoting.rmi.RmiClientInterceptor.lookupStub(RmiClientInterceptor.java:215)
at org.springframework.remoting.rmi.RmiClientInterceptor.getStub(RmiClientInterceptor.java:237)
at org.springframework.remoting.rmi.RmiClientInterceptor.invoke(RmiClientInterceptor.java:257)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy61.serviceMethod(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:191)
at package.Client.main(Client.java:47)
Caused by: java.rmi.ConnectException: Connection refused to host: rmiservice.host; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at java.rmi.Naming.lookup(Naming.java:84)
at org.springframework.remoting.rmi.RmiClientInterceptor.lookupStub(RmiClientInterceptor.java:200)
... 11 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:189)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
... 17 more

Which web server you are using ?
If it is Apache Tomcat, then add to CATALINA_OPTS variable
-Djava.rmi.server.hostname=10.0.34.11
in catalina.sh file and then export CATALINA_OPTS variable.

QPID - Spring CachingConnectionFactory - Reconnect

Spring Configuration
<bean id="jmsQueueConnectionFactory" class="org.apache.qpid.client.AMQConnectionFactory">
<constructor-arg index="0"
value="amqp://guest:guest#localhost/test?brokerlist='tcp://localhost:5672'" />
</bean>
<bean id="cachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="jmsQueueConnectionFactory" />
<property name="sessionCacheSize" value="1" />
<property name="reconnectOnException" value="true" />
</bean>
<bean id="myDestination" class="org.apache.qpid.client.AMQAnyDestination">
<constructor-arg index="0" value="ADDR:myqueue; {create: always}" />
</bean>
<bean id="myServiceBean" class="com.test.MyService" />
<bean id="myContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="cachingConnectionFactory" />
<property name="exceptionListener" ref="cachingConnectionFactory" />
<property name="messageListener" ref="myServiceBean" />
<property name="concurrentConsumers" value="1" />
<property name="autoStartup" value="true" />
<property name="destination" ref="myDestination" />
<property name="recoveryInterval" value="10000" />
</bean>
MyService.java
public class MyService implements MessageListener {
public void onMessage(Message msg) {
log.info("----On Message called :"+msg+", :"+msg.getClass().getName());
}
}
When I restart QPID
Without reconnectOnException=true,I keep getting this exception ,but not reconneting
3203 [myContainer-1] DEBUG org.springframework.jms.connection.CachingConnectionFactory - Creating cached JMS Session for mode 1: org.apache.qpid.client.AMQSession_0_10#1d03a4e
3312 [myContainer-1] DEBUG org.springframework.jms.connection.CachingConnectionFactory - Creating cached JMS MessageConsumer for destination ['myqueue'/None; {
'create': 'always'
}]: org.apache.qpid.client.BasicMessageConsumer_0_10#8a2023
99109 [myContainer-1] WARN org.springframework.jms.listener.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination ''myqueue'/None; {
'create': 'always'
}' - trying to recover. Cause: timed out waiting for session to become open (state=DETACHED)
org.apache.qpid.transport.SessionException: timed out waiting for session to become open (state=DETACHED)
at org.apache.qpid.transport.Session.invoke(Session.java:630)
at org.apache.qpid.transport.Session.invoke(Session.java:559)
at org.apache.qpid.transport.SessionInvoker.executionSync(SessionInvoker.java:84)
at org.apache.qpid.transport.Session.sync(Session.java:782)
at org.apache.qpid.transport.Session.sync(Session.java:770)
at org.apache.qpid.client.BasicMessageConsumer_0_10.getMessageFromQueue(BasicMessageConsumer_0_10.java:423)
at org.apache.qpid.client.BasicMessageConsumer.receive(BasicMessageConsumer.java:407)
at org.springframework.jms.connection.CachedMessageConsumer.receive(CachedMessageConsumer.java:74)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:429)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
at java.lang.Thread.run(Thread.java:619)
99109 [myContainer-1] INFO org.springframework.jms.listener.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
..
281125 [myContainer-3] INFO org.springframework.jms.listener.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
With reconnectOnException=true ,Its getting connected and disconnected
13015 [IoReceiver - localhost/127.0.0.1:5672] WARN org.springframework.jms.connection.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection
javax.jms.JMSException: connection aborted
at org.apache.qpid.client.AMQConnectionDelegate_0_10.closed(AMQConnectionDelegate_0_10.java:303)
at org.apache.qpid.transport.Connection.closed(Connection.java:568)
at org.apache.qpid.transport.network.Assembler.closed(Assembler.java:110)
at org.apache.qpid.transport.network.InputHandler.closed(InputHandler.java:202)
at org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:150)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.qpid.transport.ConnectionException: connection aborted
at org.apache.qpid.transport.Connection.closed(Connection.java:541)
... 4 more
13031 [IoReceiver - localhost/127.0.0.1:5672] DEBUG org.springframework.jms.connection.CachingConnectionFactory - Closing shared JMS Connection: AMQConnection:
Host: localhost
Port: 5672
Virtual Host: test
Client ID: localhost
Active session count: 1
73031 [IoReceiver - localhost/127.0.0.1:5672] DEBUG org.springframework.jms.connection.CachingConnectionFactory - Could not close shared JMS Connection
org.apache.qpid.client.JMSAMQException: timed out waiting for session to become open (state=DETACHED)
at org.apache.qpid.client.AMQConnection.stop(AMQConnection.java:824)
at org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:422)
at org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:321)
at org.springframework.jms.connection.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:197)
at org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:302)
at org.springframework.jms.connection.ChainedExceptionListener.onException(ChainedExceptionListener.java:57)
at org.apache.qpid.client.AMQConnectionDelegate_0_10.closed(AMQConnectionDelegate_0_10.java:306)
...
at org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:150)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.qpid.AMQException: timed out waiting for session to become open (state=DETACHED) [error code 541: internal error]
at org.apache.qpid.client.AMQSession_0_10.setCurrentException(AMQSession_0_10.java:1050)
at org.apache.qpid.client.AMQSession_0_10.sync(AMQSession_0_10.java:1030)
at org.apache.qpid.client.AMQSession_0_10.sendSuspendChannel(AMQSession_0_10.java:857)
at org.apache.qpid.client.AMQSession.suspendChannel(AMQSession.java:3006)
at org.apache.qpid.client.AMQSession.stop(AMQSession.java:2341)
at org.apache.qpid.client.AMQConnection.stop(AMQConnection.java:820)
... 11 more
104875 [myContainer-1] INFO org.springframework.jms.connection.CachingConnectionFactory - Established shared JMS Connection: AMQConnection:
Host: localhost
Port: 5672
Virtual Host: test
Client ID: localhost
Active session count: 0
104875 [myContainer-1] INFO org.springframework.jms.listener.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
104875 [myContainer-2] DEBUG org.springframework.jms.connection.CachingConnectionFactory - Creating cached JMS Session for mode 1: org.apache.qpid.client.AMQSession_0_10#191e4c
104937 [IoReceiver - localhost/127.0.0.1:5672] WARN org.springframework.jms.connection.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection
javax.jms.JMSException: 404
at org.apache.qpid.client.AMQConnection.exceptionReceived(AMQConnection.java:1230)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.qpid.AMQException: ch=0 id=0 ExecutionException(errorCode=NOT_FOUND, commandId=0, description=Queue: myqueue not found) [error code 404: not found]
at org.apache.qpid.client.AMQSession_0_10.setCurrentException(AMQSession_0_10.java:1050)
... 29 more
104937 [IoReceiver - localhost/127.0.0.1:5672] DEBUG org.springframework.jms.connection.CachingConnectionFactory - Closing shared JMS Connection: AMQConnection:
Host: localhost
Port: 5672

Have you tried omitting the exceptionListener setting on the DMLC? I don't recall ever needing to specify that, and it seems like your connection is being invalidated after connection recovery has been initiated.
Also, assuming you're not using JBoss 4, you might try using the DMLC's built-in caching mechanism (via the cacheLevel setting) instead of using a caching connection factory for your consumers; you may even get better performance, since DMLC can cache sessions and consumers as well as connections.

Categories