I have a Wildfly messaging live/backup pair and a single Wildfly instance as application server (all version 26.0.1). I'm using Wildfly with the integrated ActiveMQ server as remote messaging brokers (not plain ActiveMQ servers directly).
For testing I use
the helloworld-mdb quickstart example (https://github.com/wildfly/quickstart/tree/main/helloworld-mdb) deployed on the Wildfly application server with MDBs receiving messages and a servlet for sending messages using an injected JmsContext and injected Queue/Topic (I renamed the queue/topic and removed the #JMSDestinationDefinitions in order to define the destinations in the messaging-activemq subysystem configuration)
and two simple JMS client programs: one for sending messages to a queue, and one for receiving messages from a queue (they just go the default route: obtain an InitialContext, look up a destination and a connection factory, and create a connection and a session)
My setup almost works. The client programs and MDBs reconnect automatically on messaging broker failover/failback. However, the servlet from the quickstart example blocks when sending messages after the live/backup pair fails over, and no messages are sent. It fails after around one minute with the following stack trace:
12:15:53,496 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /helloworld-mdb/HelloWorldMDBServletClient: javax.jms.JMSRuntimeException: AMQ219014: Timed out after waiting 30,000 ms for response when sending packet 71
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.JmsExceptionUtils.convertToRuntimeException(JmsExceptionUtils.java:88)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.ActiveMQJMSProducer.send(ActiveMQJMSProducer.java:100)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.ActiveMQJMSProducer.send(ActiveMQJMSProducer.java:124)
at deployment.helloworld-mdb.war//org.jboss.as.quickstarts.servlet.HelloWorldMDBServletClient.doGet(HelloWorldMDBServletClient.java:95)
at javax.servlet.api#2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:503)
at javax.servlet.api#2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:590)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at io.opentracing.contrib.opentracing-jaxrs2//io.opentracing.contrib.jaxrs2.server.SpanFinishingFilter.doFilter(SpanFinishingFilter.java:52)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.security.elytron-web.undertow-server#1.10.1.Final//org.wildfly.elytron.web.undertow.server.ElytronRunAsHandler.lambda$handleRequest$1(ElytronRunAsHandler.java:68)
at org.wildfly.security.elytron-base#1.18.3.Final//org.wildfly.security.auth.server.FlexibleIdentityAssociation.runAsFunctionEx(FlexibleIdentityAssociation.java:103)
at org.wildfly.security.elytron-base#1.18.3.Final//org.wildfly.security.auth.server.Scoped.runAsFunctionEx(Scoped.java:161)
at org.wildfly.security.elytron-base#1.18.3.Final//org.wildfly.security.auth.server.Scoped.runAs(Scoped.java:73)
at org.wildfly.security.elytron-web.undertow-server#1.10.1.Final//org.wildfly.elytron.web.undertow.server.ElytronRunAsHandler.handleRequest(ElytronRunAsHandler.java:67)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.core#2.2.14.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.core#2.2.14.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.core#2.2.14.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at org.wildfly.security.elytron-web.undertow-server-servlet#1.10.1.Final//org.wildfly.elytron.web.undertow.server.servlet.CleanUpHandler.handleRequest(CleanUpHandler.java:38)
at io.undertow.core#2.2.14.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow#26.0.1.Final//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.core#2.2.14.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow#26.0.1.Final//org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
at io.undertow.core#2.2.14.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:275)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:79)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:134)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:131)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow#26.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at org.wildfly.extension.undertow#26.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at org.wildfly.extension.undertow#26.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at org.wildfly.extension.undertow#26.0.1.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1544)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:255)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:79)
at io.undertow.servlet#2.2.14.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:100)
at io.undertow.core#2.2.14.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
at io.undertow.core#2.2.14.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:852)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads#2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at org.jboss.xnio#3.8.5.Final//org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: javax.jms.JMSException: AMQ219014: Timed out after waiting 30,000 ms for response when sending packet 71
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:539)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:443)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendFullMessage(ActiveMQSessionContext.java:552)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.sendRegularMessage(ClientProducerImpl.java:296)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:268)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:143)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:125)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx(ActiveMQMessageProducer.java:483)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:220)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:207)
at org.apache.activemq.artemis.ra#2.19.0//org.apache.activemq.artemis.ra.ActiveMQRAMessageProducer.send(ActiveMQRAMessageProducer.java:137)
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.jms.client.ActiveMQJMSProducer.send(ActiveMQJMSProducer.java:97)
... 53 more
Caused by: ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ219014: Timed out after waiting 30,000 ms for response when sending packet 71]
... 65 more
I followed 7.8.12. Connect a pooled-connection-factory to a Remote Artemis Server, but I'm using http-connectors instead of remote-connectors because of using Wildfly as remote brokers.
All configurations are based on standalone-full-ha.xml with just the following modifications.
Messaging live Wildfly server "messaging-node-1":
changed cluster password
added replication-master
changed cluster connection name
added two destinations
...
<subsystem xmlns="urn:jboss:domain:messaging-activemq:13.0">
<server name="default">
<security elytron-domain="ApplicationDomain"/>
<cluster password="${jboss.messaging.cluster.password:**changed-password**}"/>
<statistics enabled="${wildfly.messaging-activemq.statistics-enabled:${wildfly.statistics-enabled:false}}"/>
<replication-master cluster-name="messaging-cluster" group-name="activemq-group"/><!-- added -->
<security-setting name="#">
<role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
</security-setting>
<address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10" redistribution-delay="1000"/>
<http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/>
<http-connector name="http-connector-throughput" socket-binding="http" endpoint="http-acceptor-throughput">
<param name="batch-delay" value="50"/>
</http-connector>
<in-vm-connector name="in-vm" server-id="0">
<param name="buffer-pooling" value="false"/>
</in-vm-connector>
<http-acceptor name="http-acceptor" http-listener="default"/>
<http-acceptor name="http-acceptor-throughput" http-listener="default">
<param name="batch-delay" value="50"/>
<param name="direct-deliver" value="false"/>
</http-acceptor>
<in-vm-acceptor name="in-vm" server-id="0">
<param name="buffer-pooling" value="false"/>
</in-vm-acceptor>
<jgroups-broadcast-group name="bg-group1" jgroups-cluster="activemq-cluster" connectors="http-connector"/>
<jgroups-discovery-group name="dg-group1" jgroups-cluster="activemq-cluster"/>
<cluster-connection name="messaging-cluster" address="jms" connector-name="http-connector" discovery-group="dg-group1"/>
<jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
<jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
<jms-queue name="TestQueue" entries="java:jboss/exported/queue/TestQueue" durable="true"/>
<jms-topic name="TestTopic" entries="java:jboss/exported/topic/TestTopic"/>
<connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
<connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector" ha="true" block-on-acknowledge="true" reconnect-attempts="-1"/>
<pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" transaction="xa"/>
</server>
</subsystem>
...
Messaging backup Wildfly server "messaging-node-2"
Same as above except replication-master replaced by replication-slave
...
<subsystem xmlns="urn:jboss:domain:messaging-activemq:13.0">
<server name="default">
...
<replication-slave cluster-name="messaging-cluster" group-name="activemq-group"/>
...
</server>
</subsystem>
...
Application Wildfly server "application-node-1"
Replaced integrated ActiveMQ server definition with a remote broker and added external destinations. Notice the use of the default name and JNDI entries for the connection factory (for the EE and EJB subsystems):
...
<subsystem xmlns="urn:jboss:domain:messaging-activemq:13.0">
<http-connector name="node-1-http-connector" socket-binding="messaging-node-1" endpoint="http-acceptor"/>
<http-connector name="node-2-http-connector" socket-binding="messaging-node-2" endpoint="http-acceptor"/>
<pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="node-1-http-connector node-2-http-connector" user="JmsTest" password="JmsTest"/>
<external-jms-queue name="TestQueue" entries="java:/queue/TestQueue"/>
<external-jms-topic name="TestTopic" entries="java:/topic/TestTopic"/>
</subsystem>
...
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
...
<outbound-socket-binding name="messaging-node-1">
<remote-destination host="..." port="8180"/>
</outbound-socket-binding>
<outbound-socket-binding name="messaging-node-2">
<remote-destination host="..." port="8180"/>
</outbound-socket-binding>
</socket-binding-group>
...
The servlet:
#WebServlet("/HelloWorldMDBServletClient")
public class HelloWorldMDBServletClient extends HttpServlet {
private static final long serialVersionUID = -8314035702649252239L;
private static final int MSG_COUNT = 5;
#Inject
private JMSContext context;
#Resource(lookup = "java:/queue/TestQueue")
private Queue queue;
#Resource(lookup = "java:/topic/TestTopic")
private Topic topic;
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
resp.setContentType("text/html");
PrintWriter out = resp.getWriter();
out.write("<h1>Quickstart: Example demonstrates the use of <strong>JMS 2.0</strong> and <strong>EJB 3.2 Message-Driven Bean</strong> in JBoss EAP.</h1>");
try {
boolean useTopic = req.getParameterMap().keySet().contains("topic");
final Destination destination = useTopic ? topic : queue;
out.write("<p>Sending messages to <em>" + destination + "</em></p>");
out.write("<h2>The following messages will be sent to the destination:</h2>");
for (int i = 0; i < MSG_COUNT; i++) {
String text = "This is message " + (i + 1);
**context.createProducer().send(destination, text); // fails**
out.write("Message (" + i + "): " + text + "</br>");
}
out.write("<p><i>Go to your JBoss EAP server console or server log to see the result of messages processing.</i></p>");
} finally {
if (out != null) {
out.close();
}
}
}
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
doGet(req, resp);
}
}
Procedure: start the live/backup pair and the application server, deploy helloworld-mdb.war to the application server and load the servlet page in a web browser. For failover, shut down the live messaging server. Reload the servlet page in the web browser. Result: the web browser keeps waiting for around one minute and then receives what the servlet has printed so far ("...The following messages will be sent to the destination:" <EOF>).
Sending and receiving messages using the JMS client programs works after failover. MDBs keep receiving messages after failover. Just application server message sending fails.
I have unsuccessfully
tried changing the timeout configuration values of the cluster connection
moved message sending to a stateless session bean (in order to test transactions)
replaced the JmsContext with a connection factory to no avail; createSession() blocks, then fails with "...Caused by: ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ219014: Timed out after waiting 30,000 ms for response when sending packet 63]"
...
#Resource(mappedName = "java:/JmsXA")
private ConnectionFactory connectionFactory;
#Resource(mappedName = "java:/queue/TestQueue")
private Queue queue;
...
Connection connection = connectionFactory.createConnection();
Session session = connection.createSession();
Questions:
Are the messaging and application server configurations correct for my setup?
What could be the reason that sending messages using the injected JmsContext and Queue fails after failover?
I would appreciate any help.
Update 1
Just FYI: The application server notices when the live messaging server goes down:
17:11:40,908 WARN [org.apache.activemq.artemis.core.client] (Thread-2 (ActiveMQ-client-global-threads)) AMQ212037: Connection failure to 172.*.*.*/172.*.*.*:8180 has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED]
...repeated around 30 times and an Exception:
17:11:40,904 WARN [org.jboss.activemq.artemis.wildfly.integration.recovery] (Thread-2 (ActiveMQ-client-global-threads)) being disconnected for server shutdown: ActiveMQDisconnectedException[errorType=DISCONNECTED message=AMQ219015: The connection was disconnected because of server shutdown]
at org.apache.activemq.artemis#2.19.0//org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run(ClientSessionFactoryImpl.java:1089)
at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Related
Writing a simple JMS client I am getting:
09:00:59.771 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider
09:00:59.774 [main] INFO org.wildfly.naming - WildFly Naming version 1.0.13.Final
09:00:59.905 [main] INFO org.wildfly.security - ELY00001: WildFly Elytron version 1.1.0.Final
09:00:59.963 [main] INFO org.xnio - XNIO version 3.5.1.Final
09:00:59.974 [main] INFO org.xnio.nio - XNIO NIO Implementation Version 3.5.1.Final
09:01:00.524 [XNIO-1 I/O-1] DEBUG org.xnio.nio - Started channel thread 'XNIO-1 I/O-1', selector sun.nio.ch.WindowsSelectorImpl#497899f6
09:01:00.525 [XNIO-1 Accept] DEBUG org.xnio.nio - Started channel thread 'XNIO-1 Accept', selector sun.nio.ch.WindowsSelectorImpl#25c2b866
09:01:00.532 [main] INFO org.jboss.remoting - JBoss Remoting version 5.0.0.Final
09:01:01.549 [main] DEBUG org.apache.commons.beanutils.converters.BooleanConverter - Setting default value: false
...
09:01:02.015 [main] DEBUG org.apache.commons.beanutils.converters.StringConverter - Converting 'Integer' value '65536' to type 'String'
Got ConnectionFactory
Attempting to acquire destination "jms/queue/ExternalSignalQueue"
javax.naming.NameNotFoundException: jms/queue/ExternalSignalQueue -- service jboss.naming.context.java.jboss.exported.jms.queue.ExternalSignalQueue
at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:106)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:207)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:193)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:189)
at org.wildfly.naming.client.remote.RemoteServerTransport.handleLookup(RemoteServerTransport.java:200)
at org.wildfly.naming.client.remote.RemoteServerTransport$1.handleMessage(RemoteServerTransport.java:120)
at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430)
at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:991)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
with the corresponding Java code
String destinationString = "java:jms/queue/ExternalSignalQueue";
ConnectionFactory connectionFactory = (ConnectionFactory) namingContext.lookup(connectionFactoryString);
System.out.println("Got ConnectionFactory");
System.out.println("Attempting to acquire destination \"" + destinationString + "\"");
Destination destination = (Destination) namingContext.lookup(destinationString);
for the following Destination (per the management console):
ExternalSignalQueue
Queue Address: jms.queue.ExternalSignalQueue
JNDI Names: java:/jms/queue/ExternalSignalQueue,java:jboss/exported/jms/queue/ExternalSignalQueue
What is the queue name, as it should be specified in Java (and why)? I have also tried using java:jboss/exported/jms/queue/ExternalSignalQueue and queue/ExternalSignalQueue as the queue name - with the same error but different queue "names" seeming to be checked.
This is the XML for the subsystem:
<subsystem xmlns="urn:jboss:domain:messaging-activemq:8.0">
<server name="default">
<statistics enabled="${wildfly.messaging-activemq.statistics-enabled:${wildfly.statistics-enabled:false}}"/>
<security-setting name="#">
<role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
<role name="admin" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
</security-setting>
<address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10"/>
<http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/>
<http-connector name="http-connector-throughput" socket-binding="http" endpoint="http-acceptor-throughput">
<param name="batch-delay" value="50"/>
</http-connector>
<in-vm-connector name="in-vm" server-id="0">
<param name="buffer-pooling" value="false"/>
</in-vm-connector>
<http-acceptor name="http-acceptor" http-listener="default"/>
<http-acceptor name="http-acceptor-throughput" http-listener="default">
<param name="batch-delay" value="50"/>
<param name="direct-deliver" value="false"/>
</http-acceptor>
<in-vm-acceptor name="in-vm" server-id="0">
<param name="buffer-pooling" value="false"/>
</in-vm-acceptor>
<jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
<jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
<jms-queue name="ExternalSignalQueue" entries="java:/jms/queue/ExternalSignalQueue,java:jboss/exported/jms/queue/ExternalSignalQueue"/>
<connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
<connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector"/>
<pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" transaction="xa"/>
</server>
</subsystem>
I believe you should be using jms/queue/ExternalSignalQueue rather than java:jms/queue/ExternalSignalQueue. This is because jms/queue/ExternalSignalQueue is what appears after java:jboss/exported/ in the JNDI binding (which is what is made available to remote JNDI clients).
I am load testing an application deployed on Wildfly 20 using JMeter. The observation is that under load of 300 requests/sec and more Wildfly server doesn't respond to few requests and those requests time out. On analyzing server logs (both system and application) and thread dumps nothing related to this is found and memory utilization is also normal.
On tuning some parameters such as Open File Descriptors, Wildfly IO threads/Task Max Threads, Max Connections and Apache configuration [details below] the number of requests getting timed out have significantly reduced but still a lot of requests time out.
Using Wildfly Runtime for monitoring during load testing one thing drew my attention that even on setting the Undertow's Http Max Connections parameter to a value lets say 1000 the Connection Count value (circled in screenshot below) never exceeds 256. On playing with this parameter a bit I saw that Http Max Connections parameter sets to any value less than 256 but never more. And it looks like once this limit is reached no new request is served as gets timed out until some connections are freed.
Is there is some other configuration (Apache or OS) which puts this limit or any other Wildfly configuration responsible for this? Am I missing anything?
[Used Wildfly Management Console for all configurations]
[from standalone.xml]
<subsystem xmlns="urn:jboss:domain:io:3.0">
<worker name="default" io-threads="300" task-max-threads="300"/>
<buffer-pool name="default"/>
</subsystem>
...
<subsystem xmlns="urn:jboss:domain:undertow:11.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other" statistics-enabled="true">
<buffer-cache name="default"/>
<server name="default-server">
<http-listener name="default" max-connections="500" socket-binding="http" max-cookies="100" record-request-start-time="true" redirect-socket="https" enable-http2="true"/>
<https-listener name="https" socket-binding="https" max-cookies="100" record-request-start-time="true" security-realm="ApplicationRealm" enable-http2="true"/>
<host name="default-host">
<location name="/" handler="welcome-content"/>
<filter-ref name="limit-connections"/>
<http-invoker security-realm="ApplicationRealm"/>
</host>
</server>
<servlet-container name="default">
<jsp-config/>
<websockets/>
</servlet-container>
<handlers>
<file name="welcome-content" path="${jboss.home.dir}/welcome-content"/>
</handlers>
<filters>
<request-limit name="limit-connections" max-concurrent-requests="10000" queue-size="10000"/>
</filters>
</subsystem>
[Apache configuration]
<IfModule mpm_worker_module>
ServerLimit 40
StartServers 2
MaxClients 1000
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
[Wildfly Runtime monitoring console during testing]
Maybe your server is not tuned to cater the maximum number of processes? Add the following to your /etc/security/limits.conf file:
jboss soft nproc 8000
jboss hard nproc 12000
jboss soft nofile 9999
jboss hard nofile 9999
I'm currently migrating our code from Jboss7 to Wildfly10.
The Server itself starts up totaly fine.
When trying to connect our client with the working new wildfly10 server for ejb-remote calls it just won't work.
The only thing I get to work with is the following error:
org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector setupEJBReceivers
WARN: Could not register a EJB receiver for connection to remote-ip:8080
java.lang.RuntimeException: Operation failed with status WAITING at
org.jboss.ejb.client.remoting.IoFutureHelper.get(IoFutureHelper.java:94)
at
org.jboss.ejb.client.remoting.ConnectionPool.getConnection(ConnectionPool.java:80)
at
org.jboss.ejb.client.remoting.RemotingConnectionManager.getConnection(RemotingConnectionManager.java:51)
at
org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:161)
at
org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.getCurrent(ConfigBasedEJBClientContextSelector.java:118)
at
org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.getCurrent(ConfigBasedEJBClientContextSelector.java:47)
at
org.jboss.ejb.client.EJBClientContext.getCurrent(EJBClientContext.java:281)
at
org.jboss.ejb.client.EJBClientContext.requireCurrent(EJBClientContext.java:291)
at
org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:178)
at
org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
at com.sun.proxy.$Proxy2.connect(Unknown Source) at
de.cinovo.rcp.test.RemoteEJBClient.invokeStatelessBean(RemoteEJBClient.java:39)
at de.cinovo.rcp.test.RemoteEJBClient.main(RemoteEJBClient.java:25)
Exception in thread "main" java.lang.IllegalStateException:
EJBCLIENT000025: No EJB receiver available for handling
[appName:de.cinovo.tcc.server-ear,
moduleName:de-cinovo-tcc-server-ejb-6.0-SNAPSHOT, distinctName:]
combination for invocation context
org.jboss.ejb.client.EJBClientInvocationContext#180542f at
org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:798)
at
org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:128)
at
org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:186)
at
org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:255)
at
org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:200)
at
org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183)
at
org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
at com.sun.proxy.$Proxy2.connect(Unknown Source) at
de.cinovo.rcp.test.RemoteEJBClient.invokeStatelessBean(RemoteEJBClient.java:39)
at de.cinovo.rcp.test.RemoteEJBClient.main(RemoteEJBClient.java:25)
There is no error, warning, info or anything showing up in the server log, while trying to connect.
There is some action on the port via tcp while watching during a call attempt.
The realy funny part is:
If I use the same wildfly setup on my local machine, the exact same connection method works, but only while using localhost as the ip address within the jboss-ejb-client.properties.
As soon as I change the ip into 127.0.0.1 or my current ip address, it will fail with the same error as above.
Relevant information:
Wildfly will respond to a telnet on port 8080
Wildfly is the only service listening on 8080
My /etc/hosts is correctly configured
Changing the port doesn't fix the problem
Wildfly Version 10.1.0.Final
Relevant parts from my standalone.xml
<subsystem xmlns="urn:jboss:domain:remoting:3.0">
<endpoint/>
<http-connector name="http-remoting-connector" connector-ref="default" security-realm="ApplicationRealm"/>
</subsystem>
[...]
<subsystem xmlns="urn:jboss:domain:undertow:3.1">
<buffer-cache name="default"/>
<server name="default-server">
<http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true"/>
[...]
</subsystem>
[...]
<interfaces>
<interface name="public">
<any-address/>
</interface>
</interfaces>
[...]
<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
<socket-binding name="http" interface="public" port="${jboss.http.port:8080}"/>
<socket-binding name="https" port="${jboss.https.port:8443}"/>
[...]
</socket-binding-group>
My jboss-ejb-client.properties
endpoint.name=client-endpoint
remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED=false
remote.connections=default
remote.connection.default.host=<host-ip>
remote.connection.default.port=8080
remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.connection.default.username=<usernmae>
remote.connection.default.password=<pswd>
Client-Code
final Hashtable jndiProperties = new Hashtable();
jndiProperties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
final Context context = new InitialContext(jndiProperties);
[...]
return context.lookup("ejb:" + appName + "/" + moduleName + "/" + distinctName + "/" + beanName + "!" + viewClassName);
EJB-Client-Maven-Dependency :
<dependency>
<groupId>org.wildfly</groupId>
<artifactId>wildfly-ejb-client-bom</artifactId>
<version>10.1.0.Final</version>
<type>pom</type>
</dependency>
Anyone out there who had the same problem and knows what I'm doing wrong?
Looks like there is a missing definition in the standalone.xml in the socket.binding-group:
<outbound-socket-binding name="remote-ejb">
<local-destination socket-binding-ref="http"/>
</outbound-socket-binding>
So for everybody interested, here is the solution to my problem:
based on the comment by Steve C and the help of a friend, we figured out that the problem is not server based.
It appears that there are some antivirus programs, which do something with your HTTP messages, as soon as the HTTP-upgrade negotiations with the Wildfly/server are done. They seem to manipulate the sent/received packages, which leads to the problem in the client, as it is no longer able to understand the answers.
Therefore it never gets to react, since the package appears to have been lost - hence the IoFuture exception and EJB receiver not found.
Long story short: removing the antivirus program from our systems (in our case Bitdefender) leads to everything working as intended...
I am migrating an old application which is running on Jboss 6.1.0 currently to wildfly 10.1.0.
I have created all queue from old application to the default configuration of standalone-full.xml But while server start up i am getting issue "cannot be cast to javax.resource.spi.endpoint.MessageEndpoint".
I have also added code snippet which is throwing this error.
Any idea what i am doing wrong ?
standalone-full.xml
...
...
<subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0">
<server name="default">
<security-setting name="#">
<role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/>
</security-setting>
<address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10"/>
<address-setting name="jms.queue.InboundNotificationResponseQueue" dead-letter-address="jms.queue.bblDeadLetterQueue" expiry-address="jms.queue.bblExpiryQueue"/>
<address-setting name="jms.queue.EventDispatchQueue" dead-letter-address="jms.queue.bblDeadLetterQueue" expiry-address="jms.queue.bblExpiryQueue"/>
<address-setting name="jms.queue.OutboundEmailQueue" dead-letter-address="jms.queue.bblDeadLetterQueue" expiry-address="jms.queue.bblExpiryQueue"/>
<http-connector name="http-connector" socket-binding="http" endpoint="http-acceptor"/>
<http-connector name="http-connector-throughput" socket-binding="http" endpoint="http-acceptor-throughput">
<param name="batch-delay" value="50"/>
</http-connector>
<in-vm-connector name="in-vm" server-id="0"/>
<http-acceptor name="http-acceptor" http-listener="default"/>
<http-acceptor name="http-acceptor-throughput" http-listener="default">
<param name="batch-delay" value="50"/>
<param name="direct-deliver" value="false"/>
</http-acceptor>
<in-vm-acceptor name="in-vm" server-id="0"/>
<jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
<jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
<jms-queue name="bblDeadLetterQueue" entries="java:/bbl/bblDeadLetterQueue"/>
<jms-queue name="OutboundEmailQueue" entries="java:/bbl/OutboundEmailQueue"/>
<jms-queue name="InboundNotificationResponseQueue" entries="java:/bbl/InboundNotificationResponseQueue"/>
<jms-queue name="EventDispatchQueue" entries="java:/bbl/EventDispatchQueue"/>
<jms-queue name="bblExpiryQueue" entries="java:/bbl/bblExpiryQueue"/>
<connection-factory name="InVmConnectionFactory" entries="java:/ConnectionFactory" connectors="in-vm"/>
<connection-factory name="RemoteConnectionFactory" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector"/>
<pooled-connection-factory name="activemq-ra" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm" transaction="xa"/>
</server>
</subsystem>
...
...
Code : EmailDispatchMDB.java
package com.mcg.bbl.email.dispatcher;
import java.util.Arrays;
import java.util.List;
import javax.annotation.Resource;
import javax.ejb.ActivationConfigProperty;
import javax.ejb.EJB;
import javax.ejb.MessageDriven;
import javax.ejb.MessageDrivenContext;
import javax.ejb.TransactionAttribute;
import javax.ejb.TransactionAttributeType;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageListener;
import javax.jms.ObjectMessage;
import org.apache.commons.lang.time.StopWatch;
import org.apache.log4j.Logger;
#MessageDriven(name = "EmailDispatchMDB", activationConfig = {
#ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"),
#ActivationConfigProperty(propertyName="destination", propertyValue="java:/bbl/OutboundEmailQueue")
})
public class EmailDispatchMDB implements MessageListener {
private static final Logger LOG = Logger.getLogger(EmailDispatchMDB.class);
#Resource
private MessageDrivenContext context;
#EJB(mappedName="bbl-server/NotificationService/remote")
private NotificationService notificationService;
public EmailDispatchMDB(){
}
public void finalize() throws Throwable {
}
#TransactionAttribute(TransactionAttributeType.REQUIRED)
public void onMessage( Message message ) {
/* Custom Code */
}
}
Log Statements :
17:09:21,506 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 64) AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=false,journalDirectory=D:\projects\MCG\Software\wildfly-10.1.0.Final\standalone\data\activemq\journal,bindingsDirectory=D:\projects\MCG\Software\wildfly-10.1.0.Final\standalone\data\activemq\bindings,largeMessagesDirectory=D:\projects\MCG\Software\wildfly-10.1.0.Final\standalone\data\activemq\largemessages,pagingDirectory=D:\projects\MCG\Software\wildfly-10.1.0.Final\standalone\data\activemq\paging)...17:09:28,604 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 64) AMQ221001: Apache ActiveMQ Artemis Message Broker version 1.1.0.wildfly-017 [nodeID=a3b04b10-d656-11e6-8bd3-c391bfc23687]...17:09:28,888 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 72) WFLYMSGAMQ0002: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
17:09:28,898 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 73) AMQ221003: trying to deploy queue jms.queue.OutboundEmailQueue...17:09:49,857 INFO [org.jboss.as.ejb3] (MSC service thread 1-5) WFLYEJB0042: Started message driven bean 'EmailDispatchMDB' with 'activemq-ra.rar' resource adapter...
17:09:56,420 WARN [org.apache.activemq.artemis.ra] (default-threads - 5) AMQ152005: Failure in broker activation org.apache.activemq.artemis.ra.inflow.ActiveMQActivationSpec(ra=org.apache.activemq.artemis.ra.ActiveMQResourceAdapter#78712571 destination=java:/bbl/OutboundEmailQueue destinationType=javax.jms.Queue ack=Auto-acknowledge durable=false clientID=null user=null maxSession=15): java.lang.ClassCastException: com.MCG.bbl.email.dispatcher.EmailDispatchMDB$$$endpoint2 cannot be cast to javax.resource.spi.endpoint.MessageEndpoint
at org.jboss.as.ejb3.inflow.JBossMessageEndpointFactory.createEndpoint(JBossMessageEndpointFactory.java:75)
at org.jboss.as.ejb3.inflow.JBossMessageEndpointFactory.createEndpoint(JBossMessageEndpointFactory.java:63)
at org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.setup(ActiveMQMessageHandler.java:187)
at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation.setup(ActiveMQActivation.java:318)
at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation$SetupActivation.run(ActiveMQActivation.java:719)
at org.jboss.jca.core.workmanager.WorkWrapper.run(WorkWrapper.java:223)
at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)
at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:808)
at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)
at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:828)
at java.lang.Thread.run(Thread.java:744)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
...
17:09:58,488 INFO [org.apache.activemq.artemis.ra] (default-threads - 5) AMQ151001: Attempting to reconnect org.apache.activemq.artemis.ra.inflow.ActiveMQActivationSpec(ra=org.apache.activemq.artemis.ra.ActiveMQResourceAdapter#78712571 destination=java:/bbl/OutboundEmailQueue destinationType=javax.jms.Queue ack=Auto-acknowledge durable=false clientID=null user=null maxSession=15) ...17:09:58,693 ERROR [org.apache.activemq.artemis.ra] (default-threads - 5) AMQ154003: Unable to reconnect org.apache.activemq.artemis.ra.inflow.ActiveMQActivationSpec(ra=org.apache.activemq.artemis.ra.ActiveMQResourceAdapter#78712571 destination=java:/bbl/OutboundEmailQueue destinationType=javax.jms.Queue ack=Auto-acknowledge durable=false clientID=null user=null maxSession=15): java.lang.ClassCastException: com.mcg.bbl.email.dispatcher.EmailDispatchMDB$$$endpoint2 cannot be cast to javax.resource.spi.endpoint.MessageEndpoint
at org.jboss.as.ejb3.inflow.JBossMessageEndpointFactory.createEndpoint(JBossMessageEndpointFactory.java:75)
at org.jboss.as.ejb3.inflow.JBossMessageEndpointFactory.createEndpoint(JBossMessageEndpointFactory.java:63)
at org.apache.activemq.artemis.ra.inflow.ActiveMQMessageHandler.setup(ActiveMQMessageHandler.java:187)
at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation.setup(ActiveMQActivation.java:318)
at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation.reconnect(ActiveMQActivation.java:678)
at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation$SetupActivation.run(ActiveMQActivation.java:722)
at org.jboss.jca.core.workmanager.WorkWrapper.run(WorkWrapper.java:223)
at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)
at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:808)
at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)
at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:828)
at java.lang.Thread.run(Thread.java:744)
at org.jboss.threads.JBossThread.run(JBossThread.java:320)
This issue is resolved, it was jar related issue. Since I was upgrading from jboss 6 to wildfly 10, somehow the project was referring jar "jboss-connector-api_1.5_spec-1.0.0.Final.jar" which is specific to jboss 6. I corrected this dependency and configured correct module "javax.resource.api" which has latest jar "jboss-connector-api_1.7_spec-1.0.0.Final.jar", and error got resolved.
so conclusion is project was referring older version of jar which needed to be corrected.
Hope this helps!!
I am trying to pull remote files through SFTP Spring Integration and dump into my local drive, and kick-off a Spring Batch Job (Step-1 to read, process & write), followed by Step-2 where a Tasklet archives the local file downloaded to another location locally and deletes the downloaded file.
Assuming the files being downloaded is "data.service", "data.maintenance" and "data.transaction". I am only processing "data.service" but all 3 are archived and deleted.
I want the above steps to happen at a specific interval assuming that a new set of files will be overwritten at the remote location.
Bean def
<bean id="acceptAllFileListFilter" class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/>
<bean id="acceptOnceFileListFilter" class="org.springframework.integration.file.filters.AcceptOnceFileListFilter"/>
conf
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="${inbound.remote.directory}"
auto-create-local-directory="true"
delete-remote-files="false"
preserve-timestamp="true"
filename-pattern="*.*"
local-filter="acceptAllFileListFilter" >
<int:poller max-messages-per-poll="-1" fixed-rate="60000" />
</int-sftp:inbound-channel-adapter>
What is happening now is...
The above triggers this sequence and runs in an unending loop, without fetching the new files from remote. Details as below:
I start with an empty local folder, and remote server has all 3 files
[Start the program]
Spring Integration program downloads all 3 files to local location
Spring Batch Job Step-1 gets triggered Tasklet (Step-2) archives the
local files Tasklet (Step-2) deletes the local files Spring
Integration program downloads all 3 files to local location (why???)
program keeps running Step-1 to Step-5 in an unending loop
New file dropped on the remote server, same (1) to (5) no job triggered
Tried replacing as below, but the program just runs once, i.e., from (1) to (5) and nothing happens
local-filter="acceptOnceFileListFilter" >
Expected
Spring Integration program downloads all 3 files to local location
Spring Batch Job Step-1 gets triggered
Tasklet (Step-2) archives the local files
Tasklet (Step-2) deletes the local files
Spring Integration program waits for new files to be dropped in the remote location (polls every 60000 ms)
New file dropped on the remote server
Start with (1) through (6)
Per suggestion on a different post I have a unique job parameter as below:
#Transformer
public JobLaunchRequest toRequest(Message<File> message) {
JobParametersBuilder jobParametersBuilder = new JobParametersBuilder();
jobParametersBuilder.addString(fileParameterName, message.getPayload().getAbsolutePath());
jobParametersBuilder.addLong("currentTime", new Long(System.currentTimeMillis()));
return new JobLaunchRequest(job, jobParametersBuilder.toJobParameters());
}
Tried with different combinations of filter, none served my purpose. Any sample would be of great help, struggling with this since last week.
Here's my change per the suggestions:
Adding a composite filter as below, but what would be the argument that will be passed to SftpPersistentAcceptOnceFileListFilter? I could not see any samples in the internet.
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
<list>
<bean class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter"/>
<bean class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
</bean>
</list>
</constructor-arg>
</bean>
Including in the adapter as below:
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="${inbound.remote.directory}"
auto-create-local-directory="true"
delete-remote-files="false"
filter="compositeFilter">
<int:poller max-messages-per-poll="-1" fixed-rate="600000" />
</int-sftp:inbound-channel-adapter>
Update - Changes after adding persistent filter:
<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
channel="inboundFileChannel"
session-factory="sftpSessionFactory"
local-directory="file:${inbound.local.directory}"
remote-directory="file:${inbound.remote.directory}"
auto-create-local-directory="false"
delete-remote-files="false"
filter="compositeFilter"
>
<int:poller max-messages-per-poll="-1" fixed-rate="60000" />
</int-sftp:inbound-channel-adapter>
<bean id="compositeFilter" class="org.springframework.integration.file.filters.CompositeFileListFilter">
<constructor-arg>
<list>
<bean class="org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter">
<constructor-arg name="store" ref="metadataStore"/>
<constructor-arg value="*.*"/>
</bean>
<bean class="org.springframework.integration.sftp.filters.SftpPersistentAcceptOnceFileListFilter">
<constructor-arg name="store" ref="metadataStore"/>
<constructor-arg value="*.*"/>
</bean>
</list>
</constructor-arg>
</bean>
<bean id="redisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<property name="port" value="7379"/>
</bean>
<bean name="metadataStore" class="org.springframework.integration.redis.metadata.RedisMetadataStore">
<constructor-arg name="connectionFactory" ref="redisConnectionFactory"/>
</bean>
This gives me the exception as below. I tried the suggestions in the Spring forum, but I don't have a connection issues.
2014-07-17 22:27:14,139 [task-scheduler-1] INFO com.jcraft.jsch -
Authentication succeeded (keyboard-interactive). 2014-07-17
22:27:14,192 [task-scheduler-1] INFO com.jcraft.jsch - Disconnecting
from localhost port 22 2014-07-17 22:27:14,195 [task-scheduler-1]
DEBUG
org.springframework.beans.factory.support.DefaultListableBeanFactory -
Returning cached instance of singleton bean 'errorChannel' 2014-07-17
22:27:14,197 [task-scheduler-1] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
preSend on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}] 2014-07-17 22:27:14,197 [task-scheduler-1]
DEBUG org.springframework.integration.handler.LoggingHandler - (inner
bean)#22 received message: [Payload MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}] 2014-07-17 22:27:14,199 [task-scheduler-1]
ERROR org.springframework.integration.handler.LoggingHandler -
org.springframework.messaging.MessagingException: Problem occurred
while synchronizing remote to local directory at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:193)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:187)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:52)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:143)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:278)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:272)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695) Caused by:
org.springframework.messaging.MessagingException: Failed to execute on
session at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:311)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 21 more Caused by: java.lang.ClassCastException:
com.jcraft.jsch.ChannelSftp$LsEntry cannot be cast to java.io.File at
org.springframework.integration.file.filters.FileSystemPersistentAcceptOnceFileListFilter.fileName(FileSystemPersistentAcceptOnceFileListFilter.java:28)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.buildKey(AbstractPersistentAcceptOnceFileListFilter.java:88)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.accept(AbstractPersistentAcceptOnceFileListFilter.java:48)
at
org.springframework.integration.file.filters.AbstractFileListFilter.filterFiles(AbstractFileListFilter.java:40)
at
org.springframework.integration.file.filters.CompositeFileListFilter.filterFiles(CompositeFileListFilter.java:97)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.filterFiles(AbstractInboundFileSynchronizer.java:157)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:173)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
... 22 more
2014-07-17 22:27:14,199 [task-scheduler-1] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
postSend (sent=true) on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=84cccecf-20ec-f67b-3f74-a938bf9abf3d,
timestamp=1405661234197}]
Update post removing FileSystemPersistentAcceptOnceFileListFilter filter leaving the 'SftpPersistentAcceptOnceFileListFilter'
2014-07-18 09:29:21,942 [task-scheduler-3] INFO com.jcraft.jsch -
Authentications that can continue:
publickey,keyboard-interactive,password 2014-07-18 09:29:21,942
[task-scheduler-3] INFO com.jcraft.jsch - Next authentication method:
publickey 2014-07-18 09:29:21,942 [task-scheduler-3] INFO
com.jcraft.jsch - Authentications that can continue:
keyboard-interactive,password 2014-07-18 09:29:21,942
[task-scheduler-3] INFO com.jcraft.jsch - Next authentication method:
keyboard-interactive 2014-07-18 09:29:22,022 [task-scheduler-3] INFO
com.jcraft.jsch - Authentication succeeded (keyboard-interactive).
2014-07-18 09:29:22,067 [task-scheduler-3] DEBUG
org.springframework.data.redis.core.RedisConnectionUtils - Opening
RedisConnection 2014-07-18 09:29:22,068 [task-scheduler-3] INFO
com.jcraft.jsch - Disconnecting from localhost port 22 2014-07-18
09:29:22,068 [task-scheduler-3] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
preSend on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}] 2014-07-18 09:29:22,068 [Connect thread
localhost session] INFO com.jcraft.jsch - Caught an exception,
leaving main loop due to Socket closed 2014-07-18 09:29:22,068
[task-scheduler-3] DEBUG
org.springframework.integration.handler.LoggingHandler - (inner
bean)#22 received message: [Payload MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}] 2014-07-18 09:29:22,069 [task-scheduler-3]
ERROR org.springframework.integration.handler.LoggingHandler -
org.springframework.messaging.MessagingException: Problem occurred
while synchronizing remote to local directory at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:193)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizingMessageSource.receive(AbstractInboundFileSynchronizingMessageSource.java:167)
at
org.springframework.integration.endpoint.SourcePollingChannelAdapter.receiveMessage(SourcePollingChannelAdapter.java:124)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.doPoll(AbstractPollingEndpoint.java:187)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint.access$000(AbstractPollingEndpoint.java:52)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:146)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$1.call(AbstractPollingEndpoint.java:143)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller$1.run(AbstractPollingEndpoint.java:278)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:52)
at
org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
at
org.springframework.integration.util.ErrorHandlingTaskExecutor.execute(ErrorHandlingTaskExecutor.java:49)
at
org.springframework.integration.endpoint.AbstractPollingEndpoint$Poller.run(AbstractPollingEndpoint.java:272)
at
org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at
org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695) Caused by:
org.springframework.messaging.MessagingException: Failed to execute on
session at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:311)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.synchronizeToLocalDirectory(AbstractInboundFileSynchronizer.java:167)
... 21 more Caused by:
org.springframework.data.redis.RedisConnectionFailureException: Cannot
get Jedis connection; nested exception is
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:97)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:143)
at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:41)
at
org.springframework.data.redis.core.RedisConnectionUtils.doGetConnection(RedisConnectionUtils.java:128)
at
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:91)
at
org.springframework.data.redis.core.RedisConnectionUtils.getConnection(RedisConnectionUtils.java:78)
at
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:177)
at
org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:152)
at
org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:84)
at
org.springframework.data.redis.core.DefaultHashOperations.putIfAbsent(DefaultHashOperations.java:179)
at
org.springframework.data.redis.core.DefaultBoundHashOperations.putIfAbsent(DefaultBoundHashOperations.java:91)
at
org.springframework.data.redis.support.collections.RedisProperties.putIfAbsent(RedisProperties.java:228)
at
org.springframework.integration.redis.metadata.RedisMetadataStore.putIfAbsent(RedisMetadataStore.java:139)
at
org.springframework.integration.file.filters.AbstractPersistentAcceptOnceFileListFilter.accept(AbstractPersistentAcceptOnceFileListFilter.java:51)
at
org.springframework.integration.file.filters.AbstractFileListFilter.filterFiles(AbstractFileListFilter.java:40)
at
org.springframework.integration.file.filters.CompositeFileListFilter.filterFiles(CompositeFileListFilter.java:97)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer.filterFiles(AbstractInboundFileSynchronizer.java:157)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:173)
at
org.springframework.integration.file.remote.synchronizer.AbstractInboundFileSynchronizer$1.doInSession(AbstractInboundFileSynchronizer.java:167)
at
org.springframework.integration.file.remote.RemoteFileTemplate.execute(RemoteFileTemplate.java:302)
... 22 more Caused by:
redis.clients.jedis.exceptions.JedisConnectionException: Could not get
a resource from the pool at
redis.clients.util.Pool.getResource(Pool.java:42) at
redis.clients.jedis.JedisPool.getResource(JedisPool.java:84) at
redis.clients.jedis.JedisPool.getResource(JedisPool.java:10) at
org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:90)
... 41 more Caused by:
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused at
redis.clients.jedis.Connection.connect(Connection.java:150) at
redis.clients.jedis.BinaryClient.connect(BinaryClient.java:71) at
redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1783) at
redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:65) at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:819)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:429)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:360)
at redis.clients.util.Pool.getResource(Pool.java:40) ... 44 more
Caused by: java.net.ConnectException: Connection refused at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:382) at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:241)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:228) at
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:431) at
java.net.Socket.connect(Socket.java:527) at
redis.clients.jedis.Connection.connect(Connection.java:144) ... 51
more
2014-07-18 09:29:22,069 [task-scheduler-3] DEBUG
org.springframework.integration.channel.PublishSubscribeChannel -
postSend (sent=true) on channel 'errorChannel', message: [Payload
MessagingException
content=org.springframework.messaging.MessagingException: Problem
occurred while synchronizing remote to local
directory][Headers={id=55304e38-6f82-8350-e763-0f5bcb507788,
timestamp=1405700962068}]
Gist Here
Hopefully self-explanatory; it just prints the file name on the console, then deletes it.
It could be the case that if the new files dropped on the server has same name(which are downloaded in the first iteration) then spring integration will not attempt to download the files again. "acceptOnceFileListFilter" seems like something that would restrict sftp integration component to download the same file again.
Update: Here is some sample code which polls a remote directory every 10 secs without introducing any filters(I am not sure why you need them, as you want to download the files regardless). Given that you dont want to change the file names, the only thing that you will have to make sure is that your downloaded local needs to deleted when the next file is dropped on the remote server. I verified it and worked fine for me.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:int="http://www.springframework.org/schema/integration"
xmlns:int-ftp="http://www.springframework.org/schema/integration/ftp"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/ftp http://www.springframework.org/schema/integration/ftp/spring-integration-ftp.xsd">
<int:channel id="incomingChannel"/>
<bean id="ftpClientFactory" class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host" value="host"/>
<property name="username" value="username"/>
<property name="password" value="password"/>
</bean>
<!-- poll every 10 secs without worrying about file names-->
<int-ftp:inbound-channel-adapter id="triggerFtpInBound"
channel="incomingChannel"
auto-create-local-directory="true"
local-directory="C:\home\temp\ftp"
remote-directory="/export/home/rbaljinder/ftp-test"
filename-pattern="*.*"
session-factory="ftpClientFactory">
<int:poller cron="1/10 * * * * *" max-messages-per-poll="1"/>
</int-ftp:inbound-channel-adapter>
UPDATE: seems working now.Added local-filter="acceptAllFileListFilter".
<int-ftp:inbound-channel-adapter id="triggerFtpInBound"
channel="incomingChannel"
auto-create-local-directory="true"
local-directory="C:\home\temp\ftp"
remote-directory="/export/home/cwk2/ftp-test"
filename-pattern="*.*"
session-factory="ftpClientFactory"
local-filter="acceptAllFileListFilter">
<int:poller cron="1/10 * * * * *" max-messages-per-poll="1"/>
</int-ftp:inbound-channel-adapter>
<bean id="acceptAllFileListFilter"
class="org.springframework.integration.file.filters.AcceptAllFileListFilter"/>
<int:service-activator id="jobServiceActivator"
input-channel="incomingChannel"
ref="triggerJobLauncher"
method="launch"/>
#Component("triggerJobLauncher")
public static class TriggerJobLauncher {
#Autowired
JobLauncher jobLauncher;
public void launch(File file) throws Exception {
System.out.println("test:" + file);
}
}