Mule ESB LDAP connector able to bind but fails at lookup - java

So I am trying to create a Mule flow to authenticate against my LDAP. I want to bind sending a JSON in through PHP then check if it binds. It then returns the value of the binded user so I can store that in my session variable.
Sending the JSON works well and I can authenticate to the server as I have tried with wrong password and it doesn't work. It binds successfully but cannot do the lookup after that. I hope someone can help.
Here's my code :
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:ldap="http://www.mulesoft.org/schema/mule/ldap" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.6.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/ldap http://www.mulesoft.org/schema/mule/ldap/current/mule-ldap.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd">
<http:listener-config name="HTTP_Listener_Configuration" host="localhost" port="8080" doc:name="HTTP Listener Configuration"/>
<ldap:config name="ldapConf" url="ldap://address.address.address:389/" doc:name="LDAP">
<ldap:connection-pooling-profile evictionCheckIntervalMillis="1" minEvictionMillis="1" exhaustedAction="WHEN_EXHAUSTED_WAIT"/>
</ldap:config>
<flow name="loginldapFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/login" doc:name="HTTP">
<http:response-builder>
<http:header headerName="content-type" value="text/plain"/>
</http:response-builder>
</http:listener>
<json:json-to-object-transformer doc:name="JSON to Object" returnClass="java.util.Map"/>
<ldap:bind config-ref="ldapConf" authDn="#[payload.username]#address.addresss" authPassword="#[payload.password]" authentication="simple" doc:name="LDAP"/>
</flow>
</mule>
Here's the error :
INFO 2015-05-07 14:01:05,452 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Binded to ldap://address.address.address:389/ with simple authentication as Mule1#address.address
INFO 2015-05-07 14:01:05,452 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Already binded to ldap://address.address.address:389/ with simple authentication as Mule1#address.address. Closing connection first.
INFO 2015-05-07 14:01:05,452 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Connection Mule1#address.address#ldap://address.address.address:389/ closed.
INFO 2015-05-07 14:01:05,467 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Re-binding to ldap://address.address.address:389/ with simple authentication as Mule1#address.address
INFO 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Binded to ldap://address.address.address:389/ with simple authentication as Mule1#address.address
INFO 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.adapters.LDAPConnectorConnectionIdentifierAdapter: Bind was successful for user: Mule1#address.address
WARN 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Lookup failed.
WARN 2015-05-07 14:01:05,468 [[loginldap].HTTP_Listener_Configuration.worker.01] org.mule.module.ldap.adapters.LDAPConnectorConnectionIdentifierAdapter: Cannot retrieve entry for dn: Mule1#address.address
org.mule.module.ldap.api.InvalidAttributeException: Mule1#address.address: [LDAP: error code 34 - 0000208F: NameErr: DSID-03100225, problem 2006 (BAD_NAME), data 8350, best match of:
'Mule1#address.address'
Caused by: javax.naming.InvalidNameException: Mule1#address.address: [LDAP: error code 34 - 0000208F: NameErr: DSID-03100225, problem 2006 (BAD_NAME), data 8350, best match of:
'Mule1#address.address'
I've looked at what error 34 is but I cannot add DN to bind in Mule.
I have also tried to remove the connection pooling profile and other configurations with the same result.
Thanks

The error is that the DN is invalid. Look at the answers at this previous question: javax.naming.InvalidNameException: [LDAP: error code 34 - invalid DN].
You could try using a tool to explore the LDAP structure like http://jxplorer.org/ to understand what the server is expecting.

Related

How to resolve error with Spring process looking for EJB JNDI endpoint?

I have an Apache Camel Spring process that connects to a JBoss 4.2.3 app server to get EJB remote proxies.
The following is the XML snippet of the configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jee="http://www.springframework.org/schema/jee"
xmlns:camel="http://camel.apache.org/schema/spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/jee
http://www.springframework.org/schema/jee/spring-jee.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd"
default-lazy-init="true">
<!-- ================================ -->
<!-- EJB Components -->
<!-- ================================ -->
<jee:remote-slsb id="brokingReferenceService"
jndi-name="BrokingReferenceServiceBean/remote"
business-interface="com.company.command.ejb.BrokingReferenceService"/>
<jee:remote-slsb id="strategyReferenceService"
jndi-name="StrategyReferenceServiceBean/remote"
business-interface="com.company.command.ejb.StrategyReferenceService"/>
<jee:remote-slsb id="instantMessageOrderService"
jndi-name="InstantMessageOrderServiceBean/remote"
business-interface="com.company.command.ejb.InstantMessageOrderService"/>
...
</beans>
I have a jndi.properties file in the classpath, which contains the following:
java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
java.naming.provider.url=http://10.10.0.240:1100
But I am receiving the following error when the process attempts to lookup the JNDI endpoint:
2017-01-03 09:15:07.057 DEBUG (org.springframework.jndi.JndiTemplate) Looking up JNDI object with name [java:comp/env/BrokingReferenceServiceBean/remote]
2017-01-03 09:15:09.322 DEBUG (org.jnp.interfaces.NamingContext) Failed to connect to http:1099
javax.naming.CommunicationException: Failed to connect to server http:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server http:1099 [Root exception is java.net.UnknownHostException: http]]
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:274)
at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1533)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:634)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:627)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at org.springframework.jndi.JndiTemplate$1.doInContext(JndiTemplate.java:154)
at org.springframework.jndi.JndiTemplate.execute(JndiTemplate.java:87)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:152)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:178)
at org.springframework.jndi.JndiLocatorSupport.lookup(JndiLocatorSupport.java:95)
at org.springframework.jndi.JndiObjectLocator.lookup(JndiObjectLocator.java:105)
at org.springframework.ejb.access.AbstractRemoteSlsbInvokerInterceptor.lookup(AbstractRemoteSlsbInvokerInterceptor.java:100)
at org.springframework.ejb.access.AbstractSlsbInvokerInterceptor.refreshHome(AbstractSlsbInvokerInterceptor.java:122)
at org.springframework.ejb.access.SimpleRemoteSlsbInvokerInterceptor.refreshHome(SimpleRemoteSlsbInvokerInterceptor.java:163)
at org.springframework.ejb.access.AbstractSlsbInvokerInterceptor.afterPropertiesSet(AbstractSlsbInvokerInterceptor.java:109)
...
Caused by: javax.naming.ServiceUnavailableException: Failed to connect to server http:1099 [Root exception is java.net.U
nknownHostException: http]
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:248)
... 48 more
Caused by: java.net.UnknownHostException: http
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at java.net.InetAddress.getByName(InetAddress.java:1076)
at org.jnp.interfaces.TimedSocketFactory.createSocket(TimedSocketFactory.java:76)
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:244)
... 48 more
Does anyone have an idea why it is not picking up the URL properly? I have defined the URL in the properties file, but it is parsing the URL as "http:1099".
Thanks for your help.
Try those things:
java.naming.provider.url=jnp://10.10.0.240:1100
java.naming.provider.url=10.10.0.240:1100
See the documentation here

Spring Integration lock busy, another session is active

I'm connecting to a POP3 inbox (it doesn't support IMAP, unfortunately) using Spring Integration. Here's my config:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/mail http://www.springframework.org/schema/integration/mail/spring-integration-mail.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-mail="http://www.springframework.org/schema/integration/mail"
xmlns:util="http://www.springframework.org/schema/util">
<int:channel id="mailInputChannel" />
<int:channel id="channel1" />
<int:channel id="channel2" />
<int:channel id="discardChannel" />
<!-- replace 'userid and 'password' with the real values -->
<int-mail:inbound-channel-adapter id="pop3ShouldDeleteTrue"
store-uri="pop3://userid:password#host.example.com/Inbox"
channel="mailInputChannel"
should-delete-messages="true"
auto-startup="true"
java-mail-properties="javaMailProperties">
<!-- Will poll every 1 seconds -->
<int:poller max-messages-per-poll="1000" fixed-delay="1000"/>
</int-mail:inbound-channel-adapter>
<int:recipient-list-router id="incomingMailRouter" input-channel="mailInputChannel">
<int:recipient channel="channel1" selector-expression="payload.subject matches '^.*channel1.*$'"/>
<int:recipient channel="channel2" selector-expression="payload.subject matches '^.*channel2.*$'"/>
</int:recipient-list-router>
<util:properties id="javaMailProperties">
<prop key="mail.pop3.socketFactory.fallback">false</prop>
<prop key="mail.debug">false</prop>
</util:properties>
</beans>
Basically I have an inbound-channel-adapter that reads in the messages and a simple recipient-list-router that directs it accordingly.
When I start up the application, it runs fine. However, after it processes the first email, it sometimes hangs. The debug output is:
19260 [task-scheduler-1] DEBUG org.springframework.integration.mail.Pop3MailReceiver - connecting to store [pop3://username:password#mailhost.example.com/Inbox]
DEBUG POP3: mail.pop3.apop.enable: false
DEBUG POP3: mail.pop3.disablecapa: false
DEBUG POP3: connecting to host "mailhost.example.com", port 110, isSSL false
+OK Qpopper (version 4.0.5) at cobweb starting.
CAPA
+OK Capability list follows
TOP
USER
LOGIN-DELAY 0
EXPIRE 0
UIDL
RESP-CODES
AUTH-RESP-CODE
X-MANGLE
X-MACRO
X-LOCALTIME Wed, 10 Aug 2016 11:50:57 -0700
IMPLEMENTATION Qpopper-version-4.0.5
.
DEBUG POP3: authentication command trace suppressed
DEBUG POP3: authentication command failed
QUIT
+OK Pop server at cobweb signing off.
30529 [task-scheduler-1] DEBUG org.springframework.beans.factory.support.DefaultListableBeanFactory - Returning cached instance of singleton bean 'errorChannel'
30531 [task-scheduler-1] DEBUG org.springframework.integration.channel.PublishSubscribeChannel - preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11), headers={id=43817279-7717-f4d7-971a-3c283e2dbec3, timestamp=1470849586452}]
30531 [task-scheduler-1] DEBUG org.springframework.integration.handler.LoggingHandler - _org.springframework.integration.errorLogger.handler received message: ErrorMessage [payload=org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11), headers={id=43817279-7717-f4d7-971a-3c283e2dbec3, timestamp=1470849586452}]
30534 [task-scheduler-1] ERROR org.springframework.integration.handler.LoggingHandler - org.springframework.messaging.MessagingException: failure occurred while polling for mail; nested exception is javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11)
at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:131)
at org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:209)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:245)
at org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:58)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:190)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:186)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:353)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:51)
at org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:344)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.mail.AuthenticationFailedException: [IN-USE] /var/spool/maildrop/.program_name.pop lock busy! Is another session active? (11)
at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:207)
at javax.mail.Service.connect(Service.java:295)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at org.springframework.integration.mail.AbstractMailReceiver.connectStoreIfNecessary(AbstractMailReceiver.java:286)
at org.springframework.integration.mail.AbstractMailReceiver.openFolder(AbstractMailReceiver.java:297)
at org.springframework.integration.mail.AbstractMailReceiver.receive(AbstractMailReceiver.java:319)
at org.springframework.integration.mail.MailReceivingMessageSource.receive(MailReceivingMessageSource.java:112)
... 19 more
It stays in this loop for a (seemingly) random amount of time, sometimes up to 2 minutes, spitting out these exceptions every time it tries to connect. However, sometimes it connects just file right after processing a message. Processing a message is the only thing that causes it to fail.
Is Spring trying to hold on to that connection to that connection too long? Can I tell it to drop the connection right after it fetches the email?
Turns out it was a server-side issue. After switching to a production-grade mail server that supported IMAP, the issue was resolved.

Mule Flow fails on first try but works after with the LDAP connector

I have a strange behavior with my Mule flow. It is the only one that does this and I have no idea why. I use the LDAP connector. When I try the next day the first try to the listener fails and I get the error that follows:
ERROR 2015-05-29 08:34:44,115 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Registered authentication is set to org.mule.module.spring.security.filters.http.HttpBasicAuthenticationFilter but there was no security context on the session. Authentication denied on endpoint none. Message payload is of type: NullPayload
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Registered authentication is set to org.mule.module.spring.security.filters.http.HttpBasicAuthenticationFilter but there was no security context on the session. Authentication denied on endpoint none. Message payload is of type: NullPayload (org.mule.api.security.UnauthorisedException)
org.mule.transport.http.filters.HttpBasicAuthenticationFilter:156 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/security/UnauthorisedException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.api.security.UnauthorisedException: Registered authentication is set to org.mule.module.spring.security.filters.http.HttpBasicAuthenticationFilter but there was no security context on the session. Authentication denied on endpoint none. Message payload is of type: NullPayload
at org.mule.transport.http.filters.HttpBasicAuthenticationFilter.authenticateInbound(HttpBasicAuthenticationFilter.java:156)
at org.mule.security.AbstractEndpointSecurityFilter.authenticate(AbstractEndpointSecurityFilter.java:54)
at org.mule.security.AbstractAuthenticationFilter.doFilter(AbstractAuthenticationFilter.java:52)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
WARN 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Search failed.
INFO 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.module.ldap.api.jndi.LDAPJNDIConnection: Connection dude#host.host#ldap://host.host.host:389/ closed.
ERROR 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.retry.notifiers.ConnectNotifier: Failed to connect/reconnect: Work Descriptor. Root Exception was: Connection reset. Type: class java.net.SocketException
ERROR 2015-05-29 08:34:50,111 [[reptel].HTTP_Listener_Configuration.worker.04] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Failed to invoke search. Message payload is of type: NullPayload
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. Connection reset (java.net.SocketException)
java.net.SocketInputStream:-1 (null)
2. javax.naming.CommunicationException: Connection reset [Root exception is java.net.SocketException: Connection reset]; remaining name 'OU=Musee,DC=mus, DC=civ'; resolved object com.sun.jndi.ldap.LdapCtx#2ba8e4bc (javax.naming.CommunicationException)
com.sun.jndi.ldap.LdapCtx:-1 (http://java.sun.com/j2ee/sdk_1.3/techdocs/api/javax/naming/CommunicationException.html)
3. Connection reset (org.mule.module.ldap.api.CommunicationException)
sun.reflect.NativeConstructorAccessorImpl:-2 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/module/ldap/api/CommunicationException.html)
4. Failed to invoke search. Message payload is of type: NullPayload (org.mule.api.MessagingException)
org.mule.devkit.processor.DevkitBasedMessageProcessor:128 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
When I call it the second time it always works and I don't know for how much time but it continues working even from another browser were the auth session isn't there it works on the first try after that
Here is the flow :
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:smtp="http://www.mulesoft.org/schema/mule/smtp" xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:wmq="http://www.mulesoft.org/schema/mule/ee/wmq" xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.6.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:spring-security="http://www.mulesoft.org/schema/mule/spring-security"
xmlns:security="http://www.springframework.org/schema/security"
xmlns:ldap="http://www.mulesoft.org/schema/mule/ldap"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/ldap http://www.mulesoft.org/schema/mule/ldap/current/mule-ldap.xsd
http://www.mulesoft.org/schema/mule/ee/wmq http://www.mulesoft.org/schema/mule/ee/wmq/current/mule-wmq-ee.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.mulesoft.org/schema/mule/smtp http://www.mulesoft.org/schema/mule/smtp/current/mule-smtp.xsd
http://www.mulesoft.org/schema/mule/spring-security http://www.mulesoft.org/schema/mule/spring-security/current/mule-spring-security.xsd
http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.xsd">
<ldap:config name="ldapConf" url="ldap://host.host.host:389/" authDn="dude#host.host" authPassword="password" doc:name="LDAP"/>
<http:listener-config name="HTTP_Listener_Configuration" host="host" port="8084" doc:name="HTTP Listener Configuration"/>
<spring:beans>
<security:authentication-manager alias="secAuthSample">
<security:authentication-provider>
<security:user-service id="authService">
<security:user name="ldap" password="ldap" authorities="ROLE_ADMIN"/>
</security:user-service>
</security:authentication-provider>
</security:authentication-manager>
</spring:beans>
<spring-security:security-manager>
<spring-security:delegate-security-provider name="memory-provider" delegate-ref="secAuthSample"/>
</spring-security:security-manager>
<flow name="activedirectoryGetUsers">
<http:listener config-ref="HTTP_Listener_Configuration" path="/ldap" doc:name="HTTP">
<http:response-builder>
<http:header headerName="content-type" value="text/plain"/>
<http:header headerName="Content-Disposition" value="attachment;filename=0544#[function:datestamp:yyyyMMdd].csv"/>
</http:response-builder>
</http:listener>
<spring-security:http-security-filter realm="" securityProviders="memory-provider"/>
<ldap:search config-ref="ldapConf" doc:name="LDAP" baseDn="OU=Musee,DC=mus, DC=civ" filter="(&(objectClass=user)(Company=*dasdasd)(!(extensionAttribute11=asdasd))(!(extensionAttribute12=asdasd)))" scope="SUB_TREE" pageSize="200000000" maxResults="20000000">
<ldap:attributes>
<ldap:attribute>dn</ldap:attribute>
<ldap:attribute>GivenName</ldap:attribute>
<ldap:attribute>sn</ldap:attribute>
<ldap:attribute>mail</ldap:attribute>
<ldap:attribute>telephonenumber</ldap:attribute>
<ldap:attribute>l</ldap:attribute>
<ldap:attribute>st</ldap:attribute>
<ldap:attribute>postalCode</ldap:attribute>
<ldap:attribute>streetAddress</ldap:attribute>
</ldap:attributes>
</ldap:search>
<collection-splitter doc:name="Split Result Set"/>
<ldap:ldap-entry-to-map doc:name="to LDIF"/>
<collection-aggregator failOnTimeout="false" doc:name="Aggregate entries"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy" file="adtocsv.groovy"></scripting:script>
</scripting:component>
<message-properties-transformer
overwrite="true" doc:name="Message Properties">
<add-message-property key="Content-Type"
value="text/plain;charset=ISO-8859-1;" />
<add-message-property key="MULE_ENCODING" value="ISO-8859-1" />
</message-properties-transformer>
</flow>
</mule>

File Polling and Filtering Issues with SFTP Spring Integration

I am trying to pull remote files through SFTP Spring Integration and dump into my local drive, and kick-off a Spring Batch Job (Step-1 to read, process & write), followed by Step-2 where a Tasklet archives the local file downloaded to another location locally and deletes the downloaded file.
Assuming the files being downloaded is "data.service", "data.maintenance" and "data.transaction". I am only processing "data.service" but all 3 are archived and deleted.
I want the above steps to happen at a specific interval assuming that a new set of files will be overwritten at the remote location.
Bean def
<bean id="acceptAllFileListFilter" class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/>
<bean id="acceptOnceFileListFilter" class="org.springframework.integration.file.filters.AcceptOnceFileListFilter"/>
conf
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="${inbound.remote.directory}"
auto-create-local-directory="true"
delete-remote-files="false"
preserve-timestamp="true"
filename-pattern="*.*"
local-filter="acceptAllFileListFilter" >
<int:poller max-messages-per-poll="-1" fixed-rate="60000" />
</int-sftp:inbound-channel-adapter>
What is happening now is...
The above triggers this sequence and runs in an unending loop, without fetching the new files from remote. Details as below:
I start with an empty local folder, and remote server has all 3 files
[Start the program]
Spring Integration program downloads all 3 files to local location
Spring Batch Job Step-1 gets triggered Tasklet (Step-2) archives the
local files Tasklet (Step-2) deletes the local files Spring
Integration program downloads all 3 files to local location (why???)
program keeps running Step-1 to Step-5 in an unending loop
New file dropped on the remote server, same (1) to (5) no job triggered
Tried replacing as below, but the program just runs once, i.e., from (1) to (5) and nothing happens
local-filter="acceptOnceFileListFilter" >
Expected
Spring Integration program downloads all 3 files to local location
Spring Batch Job Step-1 gets triggered
Tasklet (Step-2) archives the local files
Tasklet (Step-2) deletes the local files
Spring Integration program waits for new files to be dropped in the remote location (polls every 60000 ms)
New file dropped on the remote server
Start with (1) through (6)
Per suggestion on a different post I have a unique job parameter as below:
#Transformer
public JobLaunchRequest toRequest(Message<File> message) {
JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
jobParametersBuilder.addString(fileParameterName, message.getPayload().getAbsolutePath());
jobParametersBuilder.addLong("currentTime", new Long(System.currentTimeMillis()));
return new JobLaunchRequest(job, jobParametersBuilder.toJobParameters());
}
Tried with different combinations of filter, none served my purpose. Any sample would be of great help, struggling with this since last week.
Here's my change per the suggestions:
Adding a composite filter as below, but what would be the argument that will be passed to SftpPersistentAcceptOnceFileListFilter? I could not see any samples in the internet.
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
<list>
<bean class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter"/>
<bean class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
</bean>
</list>
</constructor-arg>
</bean>
Including in the adapter as below:
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="${inbound.remote.directory}"
auto-create-local-directory="true"
delete-remote-files="false"
filter="compositeFilter">
<int:poller max-messages-per-poll="-1" fixed-rate="600000" />
</int-sftp:inbound-channel-adapter>
Update - Changes after adding persistent filter:
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="file:${inbound.remote.directory}"
auto-create-local-directory="false"
delete-remote-files="false"
filter="compositeFilter"
>
<int:poller max-messages-per-poll="-1" fixed-rate="60000" />
</int-sftp:inbound-channel-adapter>
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
<list>
<bean class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter">
<constructor-arg name="store" ref="metadataStore"/>
<constructor-arg value="*.*"/>
</bean>
<bean class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
<constructor-arg name="store" ref="metadataStore"/>
<constructor-arg value="*.*"/>
</bean>
</list>
</constructor-arg>
</bean>
<bean id="redisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<property name="port" value="7379"/>
</bean>
<bean name="metadataStore" class="org.springframework.integration.redis.metadata.RedisMetadataStore">
<constructor-arg name="connectionFactory" ref="redisConnectionFactory"/>
</bean>
This gives me the exception as below. I tried the suggestions in the Spring forum, but I don't have a connection issues.
2014-07-17 22:27:14,139 [task-scheduler-1] INFO com.jcraft.jsch -
Authentication succeeded (keyboard-interactive). 2014-07-17
22:27:14,192 [task-scheduler-1] INFO com.jcraft.jsch - Disconnecting
from localhost port 22 2014-07-17 22:27:14,195 [task-scheduler-1]
DEBUG
org.springframework.beans.factory.support.DefaultListableBeanFactory -
Returning cached instance of singleton bean 'errorChannel' 2014-07-17
22:27:14,197 [task-scheduler-1] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
preSend on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}] 2014-07-17 22:27:14,197 [task-scheduler-1]
DEBUG org.springframework.integration.handler.LoggingHandler - (inner
bean)#22 received message: [Payload MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}] 2014-07-17 22:27:14,199 [task-scheduler-1]
ERROR org.springframework.integration.handler.LoggingHandler -
org.springframework.messaging.MessagingException: Problem occurred
while synchronizing remote to local directory at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:193)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:187)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:52)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:143)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:278)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:272)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695) Caused by:
org.springframework.messaging.MessagingException: Failed to execute on
session at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:311)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 21 more Caused by: java.lang.ClassCastException:
com.jcraft.jsch.ChannelSftp$LsEntry cannot be cast to java.io.File at
org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter.fileName(FileSystemPersistentAcceptOnceFileListFilter.java:28)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.buildKey(AbstractPersistentAcceptOnceFileListFilter.java:88)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.accept(AbstractPersistentAcceptOnceFileListFilter.java:48)
at
org.springframework.integration.file.filters.AbstractFileListFilter.filterFiles(AbstractFileListFilter.java:40)
at
org.springframework.integration.file.filters.CompositeFileListFilter.filterFiles(CompositeFileListFilter.java:97)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.filterFiles(AbstractInboundFileSynchronizer.java:157)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:173)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
... 22 more
2014-07-17 22:27:14,199 [task-scheduler-1] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
postSend (sent=true) on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}]
Update post removing FileSystemPersistentAcceptOnceFileListFilter filter leaving the 'SftpPersistentAcceptOnceFileListFilter'
2014-07-18 09:29:21,942 [task-scheduler-3] INFO com.jcraft.jsch -
Authentications that can continue:
publickey,keyboard-interactive,password 2014-07-18 09:29:21,942
[task-scheduler-3] INFO com.jcraft.jsch - Next authentication method:
publickey 2014-07-18 09:29:21,942 [task-scheduler-3] INFO
com.jcraft.jsch - Authentications that can continue:
keyboard-interactive,password 2014-07-18 09:29:21,942
[task-scheduler-3] INFO com.jcraft.jsch - Next authentication method:
keyboard-interactive 2014-07-18 09:29:22,022 [task-scheduler-3] INFO
com.jcraft.jsch - Authentication succeeded (keyboard-interactive).
2014-07-18 09:29:22,067 [task-scheduler-3] DEBUG
org.springframework.data.redis.core.RedisConnectionUtils - Opening
RedisConnection 2014-07-18 09:29:22,068 [task-scheduler-3] INFO
com.jcraft.jsch - Disconnecting from localhost port 22 2014-07-18
09:29:22,068 [task-scheduler-3] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
preSend on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}] 2014-07-18 09:29:22,068 [Connect thread
localhost session] INFO com.jcraft.jsch - Caught an exception,
leaving main loop due to Socket closed 2014-07-18 09:29:22,068
[task-scheduler-3] DEBUG
org.springframework.integration.handler.LoggingHandler - (inner
bean)#22 received message: [Payload MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}] 2014-07-18 09:29:22,069 [task-scheduler-3]
ERROR org.springframework.integration.handler.LoggingHandler -
org.springframework.messaging.MessagingException: Problem occurred
while synchronizing remote to local directory at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:193)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:187)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:52)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:143)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:278)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:272)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695) Caused by:
org.springframework.messaging.MessagingException: Failed to execute on
session at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:311)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 21 more Caused by:
org.springframework.data.redis.RedisConnectionFailureException: Cannot
get Jedis connection; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:97)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:143)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:41)
at
org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:128)
at
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:91)
at
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:78)
at
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:177)
at
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:152)
at
org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:84)
at
org.springframework.data.redis.core.DefaultHashOperations.putIfAbsent(DefaultHashOperations.java:179)
at
org.springframework.data.redis.core.DefaultBoundHashOperations.putIfAbsent(DefaultBoundHashOperations.java:91)
at
org.springframework.data.redis.support.collections.RedisProperties.putIfAbsent(RedisProperties.java:228)
at
org.springframework.integration.redis.metadata.RedisMetadataStore.putIfAbsent(RedisMetadataStore.java:139)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.accept(AbstractPersistentAcceptOnceFileListFilter.java:51)
at
org.springframework.integration.file.filters.AbstractFileListFilter.filterFiles(AbstractFileListFilter.java:40)
at
org.springframework.integration.file.filters.CompositeFileListFilter.filterFiles(CompositeFileListFilter.java:97)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.filterFiles(AbstractInboundFileSynchronizer.java:157)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:173)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
... 22 more Caused by:
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool at
redis.clients.util.Pool.getResource(Pool.java:42) at
redis.clients.jedis.JedisPool.getResource(JedisPool.java:84) at
redis.clients.jedis.JedisPool.getResource(JedisPool.java:10) at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:90)
... 41 more Caused by:
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused at
redis.clients.jedis.Connection.connect(Connection.java:150) at
redis.clients.jedis.BinaryClient.connect(BinaryClient.java:71) at
redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1783) at
redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:65) at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:819)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:429)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:360)
at redis.clients.util.Pool.getResource(Pool.java:40) ... 44 more
Caused by: java.net.ConnectException: Connection refused at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:382) at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:241)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:228) at
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:431) at
java.net.Socket.connect(Socket.java:527) at
redis.clients.jedis.Connection.connect(Connection.java:144) ... 51
more
2014-07-18 09:29:22,069 [task-scheduler-3] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
postSend (sent=true) on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}]
Gist Here
Hopefully self-explanatory; it just prints the file name on the console, then deletes it.
It could be the case that if the new files dropped on the server has same name(which are downloaded in the first iteration) then spring integration will not attempt to download the files again. "acceptOnceFileListFilter" seems like something that would restrict sftp integration component to download the same file again.
Update: Here is some sample code which polls a remote directory every 10 secs without introducing any filters(I am not sure why you need them, as you want to download the files regardless). Given that you dont want to change the file names, the only thing that you will have to make sure is that your downloaded local needs to deleted when the next file is dropped on the remote server. I verified it and worked fine for me.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-ftp="http://www.springframework.org/schema/integration/ftp"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/ftp http://www.springframework.org/schema/integration/ftp/spring-integration-ftp.xsd">
<int:channel id="incomingChannel"/>
<bean id="ftpClientFactory" class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="host"/>
<property name="username" value="username"/>
<property name="password" value="password"/>
</bean>
<!-- poll every 10 secs without worrying about file names-->
<int-ftp:inbound-channel-adapter id="triggerFtpInBound"
channel="incomingChannel"
auto-create-local-directory="true"
local-directory="C:\home\temp\ftp"
remote-directory="/export/home/rbaljinder/ftp-test"
filename-pattern="*.*"
session-factory="ftpClientFactory">
<int:poller cron="1/10 * * * * *" max-messages-per-poll="1"/>
</int-ftp:inbound-channel-adapter>
UPDATE: seems working now.Added local-filter="acceptAllFileListFilter".
<int-ftp:inbound-channel-adapter id="triggerFtpInBound"
channel="incomingChannel"
auto-create-local-directory="true"
local-directory="C:\home\temp\ftp"
remote-directory="/export/home/cwk2/ftp-test"
filename-pattern="*.*"
session-factory="ftpClientFactory"
local-filter="acceptAllFileListFilter">
<int:poller cron="1/10 * * * * *" max-messages-per-poll="1"/>
</int-ftp:inbound-channel-adapter>
<bean id="acceptAllFileListFilter"
class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/>
<int:service-activator id="jobServiceActivator"
input-channel="incomingChannel"
ref="triggerJobLauncher"
method="launch"/>
#Component("triggerJobLauncher")
public static class TriggerJobLauncher {
#Autowired
JobLauncher jobLauncher;
public void launch(File file) throws Exception {
System.out.println("test:" + file);
}
}

java.io.EOFException at weblogic

Hello I'm having hard time trying to configure weblogic for stateless bean (Spring). I'm new on that field.
I'm having 2-layer application. Server (weblogic + DB + java app) and client written in swing. When calling this line:
getCtx().lookup(SomeStatelessRemoteHome.class.getName()), SomeStatelessRemoteHome.class);
I got this error:
Caused by: javax.naming.CommunicationException [Root exception is weblogic.rjvm.PeerGoneException: ; nested exception is:
java.io.EOFException]
at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:74)
at weblogic.jndi.internal.WLContextImpl.translateException(WLContextImpl.java:472)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:428)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:412)
at javax.naming.InitialContext.lookup(Unknown Source)
at pl.com...SomeStatelessRemoteLbkProxy.<init>(SomeStatelessRemoteLbkProxy.java:49)
... 46 more
Caused by: weblogic.rjvm.PeerGoneException: ; nested exception is:
java.io.EOFException
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:237)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:348)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259)
at weblogic.jndi.internal.ServerNamingNode_1036_WLStub.lookup(Unknown Source)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:424)
... 49 more
Caused by: java.io.EOFException
at weblogic.rjvm.t3.MuxableSocketT3.endOfStream(MuxableSocketT3.java:345)
at weblogic.socket.SocketMuxer.deliverExceptionAndCleanup(SocketMuxer.java:827)
at weblogic.socket.SocketMuxer.deliverEndOfStream(SocketMuxer.java:761)
at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:958)
at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:889)
at weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:339)
at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
at weblogic.work.ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
EJB is auto-deployed on server. I've run weblogic with debug properties and there is nothing in log on weblogic server. Only on client it shows this stacktrace.
After ocurring of this exception weblogic lose all connection with client.
My config.xml on weblogic is:
<?xml version='1.0' encoding='UTF-8'?>
<domain xmlns="http://xmlns.oracle.com/weblogic/domain" xmlns:sec="http://xmlns.oracle.com/weblogic/security" xmlns:wls="http://xmlns.oracle.com/weblogic/security/wls" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/domain http://xmlns.oracle.com/weblogic/1.0/domain.xsd http://xmlns.oracle.com/weblogic/security http://xmlns.oracle.com/weblogic/1.0/security.xsd http://xmlns.oracle.com/weblogic/security/wls http://xmlns.oracle.com/weblogic/security/wls/1.0/wls.xsd http://xmlns.oracle.com/weblogic/security/extension http://xmlns.oracle.com/weblogic/1.0/security.xsd">
<name>mydomain</name>
<domain-version>10.3.2.0</domain-version>
<security-configuration>
<name>mydomain</name>
<realm>
<sec:authentication-provider xmlns:ext="http://xmlns.oracle.com/weblogic/security/extension" xsi:type="ext:custom-authenticatorType">
<sec:name>CustomAuthenticator</sec:name>
<sec:control-flag>SUFFICIENT</sec:control-flag>
<ext:host>66.66.66.66</ext:host>
<ext:port>389</ext:port>
<ext:principal>some_principals_censored</ext:principal>
<ext:results-time-limit>2000</ext:results-time-limit>
<ext:connect-timeout>2990</ext:connect-timeout>
<ext:connection-retry-limit>100</ext:connection-retry-limit>
<ext:credential-encrypted>some_credentialss_censored</ext:credential-encrypted>
</sec:authentication-provider>
<sec:authentication-provider xsi:type="wls:default-authenticatorType"></sec:authentication-provider>
<sec:authentication-provider xsi:type="wls:default-identity-asserterType">
<sec:active-type>AuthenticatedUser</sec:active-type>
</sec:authentication-provider>
<sec:role-mapper xsi:type="wls:default-role-mapperType"></sec:role-mapper>
<sec:authorizer xsi:type="wls:default-authorizerType"></sec:authorizer>
<sec:adjudicator xsi:type="wls:default-adjudicatorType"></sec:adjudicator>
<sec:credential-mapper xsi:type="wls:default-credential-mapperType"></sec:credential-mapper>
<sec:cert-path-provider xsi:type="wls:web-logic-cert-path-providerType"></sec:cert-path-provider>
<sec:cert-path-builder>WebLogicCertPathProvider</sec:cert-path-builder>
<sec:user-lockout-manager></sec:user-lockout-manager>
<sec:deploy-role-ignored>false</sec:deploy-role-ignored>
<sec:deploy-policy-ignored>false</sec:deploy-policy-ignored>
<sec:security-dd-model>Advanced</sec:security-dd-model>
<sec:combined-role-mapping-enabled>false</sec:combined-role-mapping-enabled>
<sec:name>myrealm</sec:name>
</realm>
<default-realm>myrealm</default-realm>
<credential-encrypted>some_credetials</credential-encrypted>
<web-app-files-case-insensitive>os</web-app-files-case-insensitive>
<compatibility-connection-filters-enabled>true</compatibility-connection-filters-enabled>
<node-manager-username>weblogic</node-manager-username>
<node-manager-password-encrypted>encrypted_pass</node-manager-password-encrypted>
<enforce-strict-url-pattern>false</enforce-strict-url-pattern>
</security-configuration>
<security>
<name>mydomain</name>
<realm>wl_default_realm</realm>
<password-policy>wl_default_password_policy</password-policy>
</security>
<jta>
<name>mydomain</name>
<timeout-seconds>60000</timeout-seconds>
</jta>
<server>
<name>myserver</name>
<jms-thread-pool-size>0</jms-thread-pool-size>
<max-message-size>80000000</max-message-size>
<complete-message-timeout>480</complete-message-timeout>
<ssl>
<name>myserver</name>
<enabled>false</enabled>
<identity-and-trust-locations>KeyStores</identity-and-trust-locations>
</ssl>
<log>
<stdout-severity>Warning</stdout-severity>
</log>
<execute-queue>
<name>weblogic.kernel.Default</name>
<thread-count>100</thread-count>
</execute-queue>
<messaging-bridge-thread-pool-size>100</messaging-bridge-thread-pool-size>
<listen-port>7001</listen-port>
<listen-address>127.0.0.1</listen-address>
<xml-registry xsi:nil="true"></xml-registry>
<xml-entity-cache xsi:nil="true"></xml-entity-cache>
<extra-ejbc-options>-Xmx1g</extra-ejbc-options>
<tunneling-enabled>false</tunneling-enabled>
<staging-mode>stage</staging-mode>
<server-diagnostic-config>
<wldf-diagnostic-volume>High</wldf-diagnostic-volume>
</server-diagnostic-config>
<transaction-log-jdbc-store>
<enabled>false</enabled>
</transaction-log-jdbc-store>
<data-source>
<rmi-jdbc-security xsi:nil="true"></rmi-jdbc-security>
</data-source>
</server>
<embedded-ldap>
<name>mydomain</name>
<credential-encrypted>some_credentials</credential-encrypted>
</embedded-ldap>
<configuration-version>10.3.2.0</configuration-version>
<app-deployment>
<name>_appsdir_appear</name>
<module-type>ear</module-type>
<source-path>autodeploy\app.ear</source-path>
<sub-deployment>
....Some Ejb libs
</sub-deployment>
<deployment-order>100</deployment-order>
<security-dd-model>Advanced</security-dd-model>
<staging-mode>stage</staging-mode>
<cache-in-app-directory>false</cache-in-app-directory>
</app-deployment>
<ws-reliable-delivery-policy>
<name>RMDefaultPolicy</name>
<jms-server>WSStoreForwardInternalJMSServermyserver</jms-server>
<default-retry-count>10</default-retry-count>
<default-time-to-live>60000</default-time-to-live>
</ws-reliable-delivery-policy>
<file-realm>
<name>wl_default_file_realm</name>
</file-realm>
<realm>
<name>wl_default_realm</name>
<file-realm>wl_default_file_realm</file-realm>
</realm>
<password-policy>
<name>wl_default_password_policy</name>
</password-policy>
<jms-server>
<name>WSStoreForwardInternalJMSServermyserver</name>
<target>myserver</target>
<persistent-store>FileStore</persistent-store>
<allows-persistent-downgrade>true</allows-persistent-downgrade>
<hosting-temporary-destinations>false</hosting-temporary-destinations>
</jms-server>
<jms-server>
<name>MyJMS Server</name>
<target>myserver</target>
<store-enabled>false</store-enabled>
<allows-persistent-downgrade>true</allows-persistent-downgrade>
<hosting-temporary-destinations>false</hosting-temporary-destinations>
</jms-server>
<web-app-container>
<relogin-enabled>true</relogin-enabled>
<allow-all-roles>true</allow-all-roles>
<filter-dispatched-requests-enabled>true</filter-dispatched-requests-enabled>
<rtexprvalue-jsp-param-name>true</rtexprvalue-jsp-param-name>
<jsp-compiler-backwards-compatible>true</jsp-compiler-backwards-compatible>
</web-app-container>
<startup-class>
<name>StartupSerwerCustom</name>
<target>myserver</target>
<deployment-order>1</deployment-order>
<class-name>pl.com...</class-name>
<load-before-app-deployments>true</load-before-app-deployments>
</startup-class>
<startup-class>
...other startupclasses
</startup-class>
<file-store>
<name>FileStore</name>
<directory>rmfilestore</directory>
<synchronous-write-policy>Direct-Write</synchronous-write-policy>
<target>myserver</target>
</file-store>
<jms-interop-module>
<name>interop-jms</name>
<sub-deployment>
<name>CustomConnectionFactory</name>
<target>myserver</target>
</sub-deployment>
<sub-deployment>
<name>WSStoreForwardInternalJMSServermyserver</name>
<target>WSStoreForwardInternalJMSServermyserver</target>
</sub-deployment>
<sub-deployment>
<name>MyJMS Server</name>
<target>MyJMS Server</target>
</sub-deployment>
<descriptor-file-name>jms/interop-jms.xml</descriptor-file-name>
</jms-interop-module>
<admin-server-name>myserver</admin-server-name>
<jdbc-system-resource>
<name>DS_Server</name>
<target>myserver</target>
<descriptor-file-name>jdbc/DS_Server-jdbc.xml</descriptor-file-name>
</jdbc-system-resource>
</domain>
I've censored it a little :).
I would appreciate for some help with it.
FIXED:
I've added debug options on Weblogic:
-Dweblogic.kernel.debug=true
-Dweblogic.debug.DebugConnection=true
-Dweblogic.StdoutDebugEnabled=true
-Dweblogic.log.LogSeverity=Debug
-Dweblogic.log.LoggerSeverity=Debug
One of jar file had wrong path. So it thrown "ClassNotFoundException". Debuging allowed me to find it... Simple mistakes are most annoying ones ;).

Categories