We Keep Coding

ActiveMQ Artemis: javax.jms.ConnectionFactory does not receive cluster topology in case of failover

On my Windows machine I set up a local test cluster of 2 brokers (version 2.19.0): 1 master, 1 slave. The ha-policy is replication and cluster communicating goes over JGroups.
master broker.xml:
<configuration xmlns="urn:activemq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xi="http://www.w3.org/2001/XInclude"
xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
<core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:activemq:core ">
<name>broker1</name>
<persistence-enabled>true</persistence-enabled>
<connectors>
<!-- Connector used to be announced through cluster connections and notifications -->
<connector name="netty-connector">tcp://127.0.0.2:61616</connector>
</connectors>
<graceful-shutdown-enabled>true</graceful-shutdown-enabled>
<acceptors>
<!-- Acceptor for every supported protocol -->
<acceptor name="artemis">tcp://127.0.0.2:61616</acceptor>
<!-- STOMP Acceptor. -->
<acceptor name="stomp">tcp://127.0.0.2:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true;anycastPrefix=/queue/;multicastPrefix=/topic/</acceptor>
</acceptors>
<cluster-user>user</cluster-user>
<cluster-password>pw</cluster-password>
<ha-policy>
<replication>
<master>
<check-for-live-server>true</check-for-live-server>
</master>
</replication>
</ha-policy>
<broadcast-groups>
<broadcast-group name="bg-group1">
<jgroups-file>test-jgroups-jdbc_ping.xml</jgroups-file>
<jgroups-channel>active_broadcast_channel</jgroups-channel>
<connector-ref>netty-connector</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="dg-group1">
<jgroups-file>test-jgroups-jdbc_ping.xml</jgroups-file>
<jgroups-channel>active_broadcast_channel</jgroups-channel>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="my-cluster">
<connector-ref>netty-connector</connector-ref>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>ON_DEMAND</message-load-balancing>
<max-hops>1</max-hops>
<discovery-group-ref discovery-group-name="dg-group1"/>
</cluster-connection>
</cluster-connections>
[...]
</core>
</configuration>
slave broker.xml:
<configuration xmlns="urn:activemq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xi="http://www.w3.org/2001/XInclude"
xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
<core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:activemq:core ">
<name>broker2</name>
<persistence-enabled>true</persistence-enabled>
<connectors>
<!-- Connector used to be announced through cluster connections and notifications -->
<connector name="netty-connector">tcp://127.0.0.3:61616</connector>
<connector name="master1-netty-connector">tcp://127.0.0.2:61616</connector>
</connectors>
<graceful-shutdown-enabled>true</graceful-shutdown-enabled>
<acceptors>
<!-- Acceptor for every supported protocol -->
<acceptor name="artemis">tcp://127.0.0.3:61616</acceptor>
<!-- STOMP Acceptor. -->
<acceptor name="stomp">tcp://127.0.0.3:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true;anycastPrefix=/queue/;multicastPrefix=/topic/</acceptor>
</acceptors>
<cluster-user>user</cluster-user>
<cluster-password>pw</cluster-password>
<!-- failover config -->
<ha-policy>
<replication>
<slave>
<allow-failback>true</allow-failback>
</slave>
</replication>
</ha-policy>
<broadcast-groups>
<broadcast-group name="bg-group1">
<jgroups-file>test-jgroups-jdbc_ping.xml</jgroups-file>
<jgroups-channel>active_broadcast_channel</jgroups-channel>
<connector-ref>netty-connector</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="dg-group1">
<jgroups-file>test-jgroups-jdbc_ping.xml</jgroups-file>
<jgroups-channel>active_broadcast_channel</jgroups-channel>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<cluster-connections>
<cluster-connection name="my-cluster">
<connector-ref>netty-connector</connector-ref>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>ON_DEMAND</message-load-balancing>
<max-hops>1</max-hops>
<discovery-group-ref discovery-group-name="dg-group1"/>
</cluster-connection>
</cluster-connections>
[...]
</core>
</configuration>
I left out the default config lines, so that it won't be too long. For reproducing the example, just replace the upper parts in broker.xml.
I am running also a spring boot application (2.3.8), which owns a javax.jms.ConnectionFactory and some consumers/producers which are connected to the Artemis server cluster.
For the initial connection, the application connects to master broker (one host).
import javax.jms.ConnectionFactory;
import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
[...]
#Bean
public ConnectionFactory filterConnectionFactory(ArtemisConfig artemisConfig) {
return new ActiveMQConnectionFactory(
"tcp://127.0.0.2:61616?ha=true&blockOnDurableSend=false",
artemisConfig.getUserName(), artemisConfig.getPassword());
}
My expectation in case of failover:
master broker crashes.
backup broker becomes live.
ConnectionFactory receives updated cluster topology and connects to slave broker. (host of slave is received via topology update)
Reality:
By now it does not. So if failover occurs, the connections can not be recovered by the application.
I guess the broker config itself should be ok, because the master is replicating all addresses to the slave in case of failover.
Also failback is working.
Is my setup wrong in any point?
And is there a way to log receiving of topology?
test-jgroups-jdbc_ping.xml:
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:org:jgroups"
xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups.xsd">
<TCP recv_buf_size="${tcp.recv_buf_size:5M}"
send_buf_size="${tcp.send_buf_size:5M}"
max_bundle_size="64K"
max_bundle_timeout="30"
sock_conn_timeout="300"
timer_type="new3"
timer.min_threads="4"
timer.max_threads="10"
timer.keep_alive_time="3000"
timer.queue_max_size="500"
thread_pool.enabled="true"
thread_pool.min_threads="2"
thread_pool.max_threads="8"
thread_pool.keep_alive_time="5000"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10000"
thread_pool.rejection_policy="discard"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="1"
oob_thread_pool.max_threads="8"
oob_thread_pool.keep_alive_time="5000"
oob_thread_pool.queue_enabled="false"
oob_thread_pool.queue_max_size="100"
oob_thread_pool.rejection_policy="discard"/>
<!-- <TRACE/> -->
<!-- <SSL_KEY_EXCHANGE
keystore_name="./activemq.example.keystore"
keystore_password="activemqexample"
/> -->
<JDBC_PING connection_url="jdbc:postgresql://127.0.0.1:5432/test" connection_username="test" connection_password="test" connection_driver="org.postgresql.Driver" initialize_sql="CREATE TABLE IF NOT EXISTS JGROUPSPING (own_addr varchar(200),bind_addr varchar(200),created timestamp DEFAULT CURRENT_TIMESTAMP,cluster_name varchar(200),ping_data BYTEA,constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name))"/>
<MERGE3 min_interval="10000"
max_interval="30000"/>
<FD_SOCK/>
<FD timeout="3000" max_tries="3" />
<VERIFY_SUSPECT timeout="1500" />
<BARRIER />
<pbcast.NAKACK2 use_mcast_xmit="false"
discard_delivered_msgs="true"/>
<UNICAST3 />
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
max_bytes="4M"/>
<pbcast.GMS print_local_addr="true" join_timeout="2000"
view_bundling="true"/>
<MFC max_credits="2M"
min_threshold="0.4"/>
<FRAG2 frag_size="60K" />
<!--RSVP resend_interval="2000" timeout="10000"/-->
<pbcast.STATE_TRANSFER/>
</config>
master logs:
2021-11-10 16:58:53,209 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server
2021-11-10 16:58:53,330 INFO [org.apache.activemq.artemis.core.server] AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=data/journal,bindingsDirectory=data/bindings,largeMessagesDirectory=data/large-messages,pagingDirectory=data/paging)
2021-11-10 16:59:15,465 INFO [org.apache.activemq.artemis.core.server] AMQ221013: Using NIO Journal
2021-11-10 16:59:15,528 INFO [org.apache.activemq.artemis.core.server] AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size (-Xmx). being defined as 536.870.912
2021-11-10 16:59:20,988 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE
2021-11-10 16:59:20,990 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP
2021-11-10 16:59:20,990 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
2021-11-10 16:59:20,991 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT
2021-11-10 16:59:20,992 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE
2021-11-10 16:59:20,992 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP
2021-11-10 16:59:22,317 INFO [org.apache.activemq.artemis.core.server] AMQ221080: Deploying address DLQ supporting [ANYCAST]
2021-11-10 16:59:22,328 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying ANYCAST queue DLQ on address DLQ
2021-11-10 16:59:22,393 INFO [org.apache.activemq.artemis.core.server] AMQ221080: Deploying address ExpiryQueue supporting [ANYCAST]
2021-11-10 16:59:22,394 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying ANYCAST queue ExpiryQueue on address ExpiryQueue
2021-11-10 16:59:24,953 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started NIO Acceptor at 127.0.0.2:61616 for protocols [CORE,MQTT,AMQP,HORNETQ,STOMP,OPENWIRE]
2021-11-10 16:59:24,972 INFO [org.apache.activemq.artemis.core.server] AMQ221020: Started NIO Acceptor at 127.0.0.2:61613 for protocols [STOMP]
2021-11-10 16:59:24,973 INFO [org.apache.activemq.artemis.core.server] AMQ221007: Server is now live
2021-11-10 16:59:24,973 INFO [org.apache.activemq.artemis.core.server] AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.19.0 [broker1, nodeID=1a4624d9-423f-11ec-b875-40167e37963a]
2021-11-10 16:59:25,776 INFO [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized activemq-branding plugin
2021-11-10 16:59:26,092 INFO [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized artemis-plugin plugin
2021-11-10 16:59:26,267 INFO [org.apache.activemq.artemis.core.server] AMQ221025: Replication: sending NIOSequentialFile D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\master1\data\journal\activemq-data-2.amq (size=10.485.760) to replica.
2021-11-10 16:59:27,597 INFO [org.apache.activemq.artemis.core.server] AMQ221025: Replication: sending NIOSequentialFile D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\master1\data\bindings\activemq-bindings-3.bindings (size=1.048.576) to replica.
2021-11-10 16:59:27,628 INFO [org.apache.activemq.artemis.core.server] AMQ221025: Replication: sending NIOSequentialFile D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\master1\data\bindings\activemq-bindings-2.bindings (size=1.048.576) to replica.
2021-11-10 16:59:28,338 INFO [io.hawt.HawtioContextListener] Initialising hawtio services
2021-11-10 16:59:28,355 INFO [io.hawt.system.ConfigManager] Configuration will be discovered via system properties
2021-11-10 16:59:28,358 INFO [io.hawt.jmx.JmxTreeWatcher] Welcome to Hawtio 2.14.0
2021-11-10 16:59:28,365 INFO [io.hawt.web.auth.AuthenticationConfiguration] Starting hawtio authentication filter, JAAS realm: "activemq" authorized role(s): "amq" role principal classes: "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
2021-11-10 16:59:28,376 INFO [io.hawt.web.auth.LoginRedirectFilter] Hawtio loginRedirectFilter is using 1800 sec. HttpSession timeout
2021-11-10 16:59:28,394 INFO [io.hawt.web.proxy.ProxyServlet] Proxy servlet is disabled
2021-11-10 16:59:28,403 INFO [io.hawt.web.servlets.JolokiaConfiguredAgentServlet] Jolokia overridden property: [key=policyLocation, value=file:/D:/apache-artemis-2.19.0-bin/apache-artemis-2.19.0/bin/master1/etc/\jolokia-access.xml]
2021-11-10 16:59:29,401 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://127.0.0.2:8261
2021-11-10 16:59:29,401 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://127.0.0.2:8261/console/jolokia
2021-11-10 16:59:29,403 INFO [org.apache.activemq.artemis] AMQ241004: Artemis Console available at http://127.0.0.2:8261/console
slave logs:
2021-11-10 16:58:55,465 INFO [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting ActiveMQ Artemis Server
2021-11-10 16:58:55,597 INFO [org.apache.activemq.artemis.core.server] AMQ221000: backup Message Broker is starting with configuration Broker Configuration (clustered=true,journalDirectory=data/journal,bindingsDirectory=data/bindings,largeMessagesDirectory=data/large-messages,pagingDirectory=data/paging)
2021-11-10 16:58:55,625 INFO [org.apache.activemq.artemis.core.server] AMQ222162: Moving data directory D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\backup1\data\journal to D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\backup1\data\journal\oldreplica.1
2021-11-10 16:58:55,631 INFO [org.apache.activemq.artemis.core.server] AMQ221055: There were too many old replicated folders upon startup, removing D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\backup1\data\paging\oldreplica.22
2021-11-10 16:58:55,721 INFO [org.apache.activemq.artemis.core.server] AMQ222162: Moving data directory D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\backup1\data\paging to D:\apache-artemis-2.19.0-bin\apache-artemis-2.19.0\bin\backup1\data\paging\oldreplica.24
2021-11-10 16:58:55,847 INFO [org.apache.activemq.artemis.core.server] AMQ221013: Using NIO Journal
2021-11-10 16:58:55,955 INFO [org.apache.activemq.artemis.core.server] AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size (-Xmx). being defined as 536.870.912
2021-11-10 16:58:56,419 INFO [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized activemq-branding plugin
2021-11-10 16:58:56,683 INFO [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized artemis-plugin plugin
2021-11-10 16:58:58,188 INFO [io.hawt.HawtioContextListener] Initialising hawtio services
2021-11-10 16:58:58,210 INFO [io.hawt.system.ConfigManager] Configuration will be discovered via system properties
2021-11-10 16:58:58,214 INFO [io.hawt.jmx.JmxTreeWatcher] Welcome to Hawtio 2.14.0
2021-11-10 16:58:58,223 INFO [io.hawt.web.auth.AuthenticationConfiguration] Starting hawtio authentication filter, JAAS realm: "activemq" authorized role(s): "amq" role principal classes: "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
2021-11-10 16:58:58,234 INFO [io.hawt.web.auth.LoginRedirectFilter] Hawtio loginRedirectFilter is using 1800 sec. HttpSession timeout
2021-11-10 16:58:58,258 INFO [io.hawt.web.proxy.ProxyServlet] Proxy servlet is disabled
2021-11-10 16:58:58,273 INFO [io.hawt.web.servlets.JolokiaConfiguredAgentServlet] Jolokia overridden property: [key=policyLocation, value=file:/D:/apache-artemis-2.19.0-bin/apache-artemis-2.19.0/bin/backup1/etc/\jolokia-access.xml]
2021-11-10 16:58:59,913 INFO [org.apache.activemq.artemis] AMQ241001: HTTP Server started at http://127.0.0.4:8261
2021-11-10 16:58:59,913 INFO [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia REST API available at http://127.0.0.4:8261/console/jolokia
2021-11-10 16:58:59,915 INFO [org.apache.activemq.artemis] AMQ241004: Artemis Console available at http://127.0.0.4:8261/console
2021-11-10 16:59:06,946 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE
2021-11-10 16:59:06,947 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP
2021-11-10 16:59:06,948 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
2021-11-10 16:59:06,948 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol support for: MQTT
2021-11-10 16:59:06,949 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding protocol support for: OPENWIRE
2021-11-10 16:59:06,949 INFO [org.apache.activemq.artemis.core.server] AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP
2021-11-10 16:59:20,433 WARNING [org.jgroups.protocols.TCP] JGRP000032: D114336-63392: no physical address for 53866421-fef5-9b4b-fdcb-ed1196697da8, dropping message
2021-11-10 16:59:25,161 INFO [org.apache.activemq.artemis.core.server] AMQ221109: Apache ActiveMQ Artemis Backup Server version 2.19.0 [null] started, waiting live to fail before it gets active
2021-11-10 16:59:28,533 INFO [org.apache.activemq.artemis.core.server] AMQ221024: Backup server ActiveMQServerImpl::name=broker2 is synchronized with live server, nodeID=1a4624d9-423f-11ec-b875-40167e37963a.
2021-11-10 16:59:28,679 INFO [org.apache.activemq.artemis.core.server] AMQ221031: backup announced
To be clear, I am using JGroups, as this should be used later in an k8s environment and I wanted to test it. I first tried the standard UDP configuration, but also with that, I did not receive topology. So my mistake must be somewhere else.

I believe the problem is that you need reconnectAttempts > 0 in your URL (e.g. reconnectAttempts=25). The default value for reconnectAttempts is 0.
Also, if you didn't receive a topology update when you connected you'd see an ActiveMQConnectionTimedOutException with a message like this:
Timed out waiting to receive cluster topology.
I also recommend that you specify the backup's host & port in your URL along with the primary's host & port. That way, if the primary is down when the application starts it will still be able to connect to the backup. Since you only have the primary's host & port then anytime the primary is down and the application starts or even if the application just re-creates the ActiveMQConnectionFactory then it will fail. Here's a simple example:
#Bean
public ConnectionFactory filterConnectionFactory(ArtemisConfig artemisConfig) {
return new ActiveMQConnectionFactory(
"(tcp://127.0.0.2:61616,tcp://127.0.0.3:61616)?ha=true&reconnectAttempts=25&blockOnDurableSend=false",
artemisConfig.getUserName(), artemisConfig.getPassword());
}

Unable to create oracle data source in Jboss EAP 7.0 server

I need to create an oracle data source in JBOSS EAP 7.0 server
I deployed ojdbc6.jar from the JBOSS management CLI(Command Line Interface) using the below command
deploy <PATH_TO_ORACLE_DRIVER_JAR>
I could see the server log as below after the driver is deployed
01:25:53,338 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = ojdbc6.jar
01:25:53,747 INFO [org.jboss.as.server] (management-handler-thread - 6) WFLYSRV0010: Deployed "ojdbc6.jar" (runtime-name : "ojdbc6.jar")
But when I try to create oracle data source(oracle.jdbc.driver.OracleDriver as driver class) from Jboss management console, I get the below error
01:31:35,084 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 66) WFLYCTL0013: Operation ("add") failed - address: ([
("subsystem" => "datasources"),
("data-source" => "OracleDS")
]) - failure description: {"WFLYCTL0180: Services with missing/unavailable dependencies" => [
"jboss.driver-demander.java:/OracleDS is missing [jboss.jdbc-driver.oracle]",
"org.wildfly.data-source.OracleDS is missing [jboss.jdbc-driver.oracle]"
]}
01:31:35,092 INFO [org.jboss.as.controller] (ServerService Thread Pool -- 66) WFLYCTL0183: Service status report
WFLYCTL0184: New missing/unsatisfied dependencies:
service jboss.data-source.reference-factory.OracleDS (missing) dependents: [service jboss.naming.context.java.OracleDS]
service jboss.data-source-config.OracleDS (missing) dependents: [service org.wildfly.data-source.OracleDS]
service jboss.jdbc-driver.oracle (missing) dependents: [service jboss.driver-demander.java:/OracleDS, service org.wildfly.data-source.OracleDS]
1) May I know what is causing this issue? What dependency is missing?
2) Is installing JDBC driver as a JAR deployment for data source creation correct method?

Install a JDBC Driver as a Core Module
Create a directory under $JBOSS_HOME/modules. In this example: "$JBOSS_HOME/modules/com/oracle/jdbc/main".
Put the the JDBC driver jar (ojdbc6.jar) in this directory.
Create a module configuration file module.xml:
Example Oracle JDBC Driver module.xml File
<?xml version="1.0" ?>
<module xmlns="urn:jboss:module:1.1" name="com.oracle">
<resources>
<resource-root path="ojdbc6.jar"/>
</resources>
<dependencies>
<module name="javax.api"/>
<module name="javax.transaction.api"/>
</dependencies>
</module>
Example Oracle Datasource Configuration
<datasources>
<datasource jndi-name="java:jboss/OracleDS" pool-name="OracleDS">
<connection-url>jdbc:oracle:thin:#localhost:1521:XE</connection-url>
<driver>oracle</driver>
<security>
<user-name>admin</user-name>
<password>admin</password>
</security>
<validation>
<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker"/>
<validate-on-match>true</validate-on-match>
<background-validation>false</background-validation>
<stale-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleStaleConnectionChecker"/>
<exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter"/>
</validation>
</datasource>
<drivers>
<driver name="oracle" module="com.oracle">
<xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
</driver>
</drivers>
</datasources>
Example Management CLI Commands
This example configuration can be achieved by using the following management CLI commands.
1# Add the Oracle JDBC driver as a core module.
module add --name=com.oracle --resources=/path/to/misc/jdbc_drivers/oracle/ojdbc7.jar --dependencies=javax.api,javax.transaction.api
2#Register the Oracle JDBC driver.
/subsystem=datasources/jdbc-driver=oracle:add(driver-name=oracle,driver-module-name=com.oracle,driver-xa-datasource-class-name=oracle.jdbc.xa.client.OracleXADataSource)
3#Add the Oracle datasource.
data-source add --name=OracleDS --jndi-name=java:jboss/OracleDS --driver-name=oracle --connection-url=jdbc:oracle:thin:#localhost:1521:XE --user-name=admin --password=admin --validate-on-match=true --background-validation=false --valid-connection-checker-class-name=org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker --exception-sorter-class-name=org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter --stale-connection-checker-class-name=org.jboss.jca.adapters.jdbc.extensions.oracle.OracleStaleConnectionChecker
Refer the link https://access.redhat.com/documentation/en/red-hat-jboss-enterprise-application-platform/version-7.0/configuration-guide/#datasource_management

Wildfly - "Connector 'netty' not found on the main configuration file"

I've added a connection factory at the standalone-full file of Wildfly:
<jms-connection-factories>
<connection-factory name="K19Factory">
<connectors>
<connector-ref connector-name="netty"/>
</connectors>
<entries>
<entry name="K19Factory"/>
<entry name="java:jboss/exported/jms/K19Factory"/>
</entries>
</connection-factory>
...
</jms-connection-factories>
And I have the following code to use it:
Properties props = new Properties();
props.setProperty("java.naming.factory.initial","org.jboss.naming.remote.client.InitialContextFactory");
props.setProperty("java.naming.provider.url", "http-remoting://127.0.0.1:8080/");
/*props.setProperty("java.naming.provider.url","remote://localhost:8080");*/
props.setProperty("java.naming.security.principal","k19");
props.setProperty("java.naming.security.credentials","1234");
InitialContext ic = new InitialContext(props);
// factory of JMS connections
ConnectionFactory factory = (ConnectionFactory)ic.lookup("jms/K19Factory");
But I'm facing problems. Widfly is saying it can't find the Netty connector, so it can't create the connection factory.
...
17:07:02,316 INFO [org.jboss.as.messaging] (ServerService Thread Pool -- 62) JBAS011601: Bound messaging object to jndi name queue/pedidos
17:07:02,331 INFO [org.hornetq.core.server] (ServerService Thread Pool -- 57) HQ221003: trying to deploy queue jms.topic.noticias
17:07:02,347 INFO [org.jboss.as.messaging] (ServerService Thread Pool -- 57) JBAS011601: Bound messaging object to jndi name topic/noticias
17:07:02,331 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 61) MSC000001: Failed to start service jboss.messaging.default.jms.connection-factory.K19Factory: org.jboss.msc.service.StartException in service jboss.messaging.default.jms.connection-factory.K19Factory: JBAS011639: Failed to create connection-factory
at org.jboss.as.messaging.jms.ConnectionFactoryService$1.run(ConnectionFactoryService.java:69) [wildfly-messaging-8.2.0.Final.jar:8.2.0.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_31]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_31]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_31]
at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final.jar:2.1.1.Final]
Caused by: HornetQIllegalStateException[errorType=ILLEGAL_STATE message=HQ129005: Connector 'netty' not found on the main configuration file]
at org.hornetq.jms.server.impl.JMSServerManagerImpl.internalCreateCFPOJO(JMSServerManagerImpl.java:1421) [hornetq-jms-server-2.4.5.Final.jar:]
at org.hornetq.jms.server.impl.JMSServerManagerImpl.internalCreateCF(JMSServerManagerImpl.java:1368) [hornetq-jms-server-2.4.5.Final.jar:]
at org.hornetq.jms.server.impl.JMSServerManagerImpl.access$1300(JMSServerManagerImpl.java:107) [hornetq-jms-server-2.4.5.Final.jar:]
at org.hornetq.jms.server.impl.JMSServerManagerImpl$5.runException(JMSServerManagerImpl.java:1215) [hornetq-jms-server-2.4.5.Final.jar:]
at org.hornetq.jms.server.impl.JMSServerManagerImpl.runAfterActive(JMSServerManagerImpl.java:1906) [hornetq-jms-server-2.4.5.Final.jar:]
at org.hornetq.jms.server.impl.JMSServerManagerImpl.createConnectionFactory(JMSServerManagerImpl.java:1201) [hornetq-jms-server-2.4.5.Final.jar:]
at org.jboss.as.messaging.jms.ConnectionFactoryService$1.run(ConnectionFactoryService.java:66) [wildfly-messaging-8.2.0.Final.jar:8.2.0.Final]
... 4 more
17:07:02,347 INFO [org.jboss.as.messaging] (ServerService Thread Pool -- 57) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/topic/noticias
17:07:02,347 INFO [org.hornetq.core.server] (ServerService Thread Pool -- 59) HQ221003: trying to deploy queue jms.queue.DLQ
17:07:02,347 INFO [org.jboss.as.messaging] (ServerService Thread Pool -- 59) JBAS011601: Bound messaging object to jndi name java:/jms/queue/DLQ
17:07:02,347 INFO [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221003: trying to deploy queue jms.queue.ExpiryQueue
17:07:02,347 INFO [org.jboss.as.messaging] (ServerService Thread Pool -- 60) JBAS011601: Bound messaging object to jndi name java:/jms/queue/ExpiryQueue
17:07:02,347 INFO [org.jboss.as.messaging] (ServerService Thread Pool -- 58) JBAS011601: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory
17:07:02,472 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-8) JBAS010406: Registered connection factory java:/JmsXA
17:07:02,488 INFO [org.jboss.ws.common.management] (MSC service thread 1-6) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.3.2.Final
17:07:02,545 INFO [org.hornetq.ra] (MSC service thread 1-8) HornetQ resource adaptor started
17:07:02,545 INFO [org.jboss.as.connector.services.resourceadapters.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-8) IJ020002: Deployed: file://RaActivatorhornetq-ra
17:07:02,545 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-3) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
17:07:02,545 INFO [org.jboss.as.messaging] (MSC service thread 1-5) JBAS011601: Bound messaging object to jndi name java:jboss/DefaultJMSConnectionFactory
17:07:02,545 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([
("subsystem" => "messaging"),
("hornetq-server" => "default"),
("connection-factory" => "K19Factory")
]) - failure description: {"JBAS014671: Failed services" => {"jboss.messaging.default.jms.connection-factory.K19Factory" => "org.jboss.msc.service.StartException in service jboss.messaging.default.jms.connection-factory.K19Factory: JBAS011639: Failed to create connection-factory
Caused by: HornetQIllegalStateException[errorType=ILLEGAL_STATE message=HQ129005: Connector 'netty' not found on the main configuration file]"}}
17:07:02,623 INFO [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014777: Services which failed to start: service jboss.messaging.default.jms.connection-factory.K19Factory: org.jboss.msc.service.StartException in service jboss.messaging.default.jms.connection-factory.K19Factory: JBAS011639: Failed to create connection-factory
Can someone give me a hand on it?

This error is due the fact that you are referencing a non-existent connector. By default does not exist a netty-connector, but other such as http-connector.
'Cause you're using standalone-full, I'll consider that org.jboss.as.messaging module is enabled.
First, we have to include an acceptor, responsible by accepting connections that will be made to the server. For this, in the messaging subsystem (XML namespace urn:jboss:domain:messaging:2.0) find acceptorstag and add this:
<netty-acceptor name="netty" socket-binding="messaging" />
<netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
<param key="batch-delay" value="50"/>
<param key="direct-deliver" value="false"/>
</netty-acceptor>
After this, we have to include a connector, responsible by transport configurations (how to connect) on the server. For this, in the messaging subsystem (XML namespace urn:jboss:domain:messaging:2.0) find connectorstag and add this:
<netty-connector name="netty" socket-binding="messaging" />
<netty-connector name="netty-throughput" socket-binding="messaging-throughput">
<param key="batch-delay" value="50"/>
</netty-connector>
Finally you have to configure the socket binding. Find socket-binding-group tag and include this:
<socket-binding name="messaging" port="5445"/>
<socket-binding name="messaging-throughput" port="5455"/>
See also the documentation of JBoss EAP (Obs.: there are differences in the EAP settings for Wildfly/AS Community, especially directories, but overall it is a good reference) and of Wildfly for more detailed configuration of messaging system.
Given this setting and your connection-factory, this code should works:
final Properties props = new Properties();
props.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
props.setProperty(Context.PROVIDER_URL, "http-remoting://127.0.0.1:8080");
props.setProperty(Context.SECURITY_PRINCIPAL, "user"); // add an application user before
props.setProperty(Context.SECURITY_CREDENTIALS, "user1234");
final InitialContext ic = new InitialContext(props);
final ConnectionFactory factory = (ConnectionFactory) ic.lookup("jms/K19Factory");
System.out.println(factory != null ? "Factory is not null" : "Factory is null");
This is the log of lookup (source above):
Jun 02, 2015 7:18:13 PM org.xnio.Xnio <clinit>
INFO: XNIO version 3.2.0.Final
Jun 02, 2015 7:18:13 PM org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.3.1.Final
Jun 02, 2015 7:18:13 PM org.jboss.remoting3.EndpointImpl <clinit>
INFO: JBoss Remoting version 4.0.0.Final
Factory is not null
And this is the log in WildFly:
19:18:13,731 INFO [org.jboss.as.naming] (default task-35) JBAS011806: Channel end notification received, closing channel Channel ID 24a74dfb (inbound) of Remoting connection 29a8f328 to /127.0.0.1:63595
To run this example you may need these dependencies in yout classpath:
jboss-remote-naming
hornetq-jms-client
a XNIO provider, like xnio-nio

WildFly leverages Servlet's protocol upgrade feature to drive all traffic over http/https. You should see by default some connectors in your xml like this
<connectors>
<http-connector name="http-connector" socket-binding="http">
<param key="http-upgrade-endpoint" value="http-acceptor"/>
</http-connector>
<http-connector name="http-connector-throughput" socket-binding="http">
<param key="http-upgrade-endpoint" value="http-acceptor-throughput"/>
<param key="batch-delay" value="50"/>
</http-connector>
<in-vm-connector name="in-vm" server-id="0"/>
</connectors>
With these, you get a http connector and in-vm connector. Based on the contents here I don't see a netty connector available. You should ideally stick with the http and in-vm connectors, for performance reasons.

JDBC_PING on infinispan server not working

I'm in phase of experimentation with infinispan server and I have a problem in set-up a infinispan cluster on my local machine.
Configuration files:
I change configuration file cluster.xml by changing part of jgroup stack with udp stack and jdbc_ping discovery.
<subsystem xmlns="urn:infinispan:server:jgroups:7.0" default-stack="${jboss.default.jgroups.stack:udp-jdbc}">
<stack name="udp-jdbc">
<transport type="UDP" socket-binding="jgroups-udp">
<property name="ip_mcast">false</property>
</transport>
<protocol type="JDBC_PING">
<property name="connection_url">jdbc:mysql://localhost/jgroups</property>
<property name="connection_username">root</property>
<property name="connection_password">root</property>
<property name="connection_driver">com.mysql.jdbc.Driver</property>
</protocol>
<protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="UFC"/>
<protocol type="MFC"/>
<protocol type="FRAG2"/>
<protocol type="RSVP"/>
</stack>
I put mysql driver in folder standalone/deployments, run the server and here is what I have as output
20:13:15,389 INFO [org.jboss.as.server.deployment] (MSC service thread 1-8) JBAS015876: Starting deployment of "mysql-connector-java-5.1.34-bin.jar" (runtime-name: "mysql-connector-java-5.1.34-bin.jar")
20:13:15,461 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.jgroups.channel.clustered: org.jboss.msc.service.StartException in service jboss.jgroups.channel.clustered: java.lang.IllegalArgumentException: JDBC Driver required for JDBC_PING protocol could not be loaded: 'com.mysql.jdbc.Driver'
at org.jboss.as.clustering.jgroups.subsystem.ChannelService.start(ChannelService.java:74)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_25]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_25]
Caused by: java.lang.IllegalArgumentException: JDBC Driver required for JDBC_PING protocol could not be loaded: 'com.mysql.jdbc.Driver'
at org.jgroups.protocols.JDBC_PING.loadDriver(JDBC_PING.java:267)
at org.jgroups.protocols.JDBC_PING.init(JDBC_PING.java:93)
at org.jgroups.stack.ProtocolStack.initProtocolStack(ProtocolStack.java:860)
at org.jgroups.stack.ProtocolStack.setup(ProtocolStack.java:481)
at org.jgroups.JChannel.init(JChannel.java:848)
at org.jgroups.JChannel.<init>(JChannel.java:159)
at org.jboss.as.clustering.jgroups.JChannelFactory.createChannel(JChannelFactory.java:87)
at org.jboss.as.clustering.jgroups.subsystem.ChannelService.start(ChannelService.java:69)
... 5 more
20:13:15,559 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-3) JBAS010404: Deploying non-JDBC-compliant driver class com.mysql.jdbc.Driver (version 5.1)
20:13:15,561 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-3) JBAS010404: Deploying non-JDBC-compliant driver class com.mysql.fabric.jdbc.FabricMySQLDriver (version 5.1)
20:13:15,563 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) JBAS010417: Started Driver service with driver-name = mysql-connector-java-5.1.34-bin.jar_com.mysql.jdbc.Driver_5_1
20:13:15,564 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-5) JBAS010417: Started Driver service with driver-name = mysql-connector-java-5.1.34-bin.jar_com.mysql.fabric.jdbc.FabricMySQLDriver_5_1
20:13:15,597 INFO [org.jboss.as.server] (ServerService Thread Pool -- 18) JBAS018559: Deployed "mysql-connector-java-5.1.34-bin.jar" (runtime-name : "mysql-connector-java-5.1.34-bin.jar")
20:13:15,599 INFO [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
JBAS014777: Services which failed to start: service jboss.jgroups.channel.clustered: org.jboss.msc.service.StartException in service jboss.jgroups.channel.clustered: java.lang.IllegalArgumentException: JDBC Driver required for JDBC_PING protocol could not be loaded: 'com.mysql.jdbc.Driver'
As you can see from the jboss log, the subsystem infinispan:server:jgroups failed to start because the JDBC driver isn't loaded (required by JDBC_PING), but after that step infinispan server successfully deployed mysql jdbc driver. This same error occupy in conditions where I work with datasource.
Is there any way to tell the infinispan:server:jgroups subsystem to be started before driver loading or before jndi naming creation ?
Or any workaround that will help
Thanks in advance

For direct JDBC use, you need to allow JGroups module to load your JDBC driver module. To do that, install JDBC driver jar as a module, then open modules/system/layers/base/org/jgroups/main/module.xml, add your JDBC driver module under the dependencies section.
For datasource lookup with EAGER caches, I think it's currently not possible to ensure that the datasource is available before JGroups tries to use it. The services boot order is controlled by programmatic dependencies but there is no way to declare the dependencies through config files so JGroups doesn't wait for the datasource.

JBoss AS 7 shutdown - connection manager closed before WAR deployment stopped

Env: JBoss AS 7.1.1.Final.
I have a WAR application using a data source taken from JBoss AS JNDI. When I shut down the server (Ctrl+C in the console), the application receives a shutdown command and starts to destroy its Spring context. However, I use a scheduler to perform some DB operations. When the application is closing, I want the tasks that are currently in the queue to be finished (but no new tasks are accepted - standard JDK Executor.shutdown() behaviour). This works fine when I undeploy the application without stopping the server. However, when I stop the whole server, the connection manager is closed before the application undeployment, which results in
14:31:51,604 INFO [org.jboss.as.logging] JBAS011503: Restored bootstrap log handlers
14:31:51,617 INFO [org.apache.coyote.http11.Http11Protocol] Stopping Coyote HTTP/1.1 on http-127.0.0.1-127.0.0.1-18080
14:31:51,638 INFO [org.hornetq.ra.HornetQResourceAdapter] HornetQ resource adapter stopped
14:31:51,653 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/MY-APP]] Closing Spring root WebApplicationContext
14:31:51,656 INFO [org.springframework.web.context.support.XmlWebApplicationContext] Closing Root WebApplicationContext: startup date [Tue Jul 09 14:30:56 CST 2013]; root of context hierarchy
14:31:51,659 INFO [org.springframework.beans.factory.support.DefaultListableBeanFactory] Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory#2c591927: defining beans [<snipped>]; root of factory hierarchy
14:31:51,662 INFO [org.hornetq.core.server.impl.HornetQServerImpl] HornetQ Server version 2.2.13.Final (HQ_2_2_13_FINAL_AS7, 122) [5f713ff6-5f86-11e2-a25d-1f3857764d50] stopped
14:31:51,673 INFO [MY-APP.Shutdown] Initializing shutdown. Already running tasks will be finished, new tasks will not be executed.
14:31:53,626 ERROR [org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler] Unexpected error occurred in scheduled task.: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/my/DS1
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:575) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:818) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:874) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:882) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at com.my.app.scanner.db.SyncEntryRepository.deleteById(SyncEntryRepository.java:26) [classes:]
at com.my.app.rules.orphanentries.OrphanedEntriesProcessor.process(OrphanedEntriesProcessor.java:22) [classes:]
at com.my.app.routing.Router$RoutingWorker.performRouting(Router.java:49) [classes:]
at com.my.app.routing.Router$RoutingWorker.route(Router.java:32) [classes:]
at com.my.app.routing.Router.route(Router.java:18) [classes:]
at com.my.app.transformation.Transformation.perform(Transformation.java:21) [classes:]
at com.my.app.MyApp.run(MyApp.java:18) [classes:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45]
at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:64) [spring-context-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:53) [spring-context-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) [rt.jar:1.6.0_45]
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) [rt.jar:1.6.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) [rt.jar:1.6.0_45]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) [rt.jar:1.6.0_45]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) [rt.jar:1.6.0_45]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) [rt.jar:1.6.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [rt.jar:1.6.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [rt.jar:1.6.0_45]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45]
Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/my/DS1
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]
... 26 more
Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/my/DS1
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:321)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129)
... 28 more
14:31:53,640 INFO [MY-APP.Shutdown] Shutdown complete
14:31:53,651 INFO [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean] Closing JPA EntityManagerFactory for persistence unit 'default'
14:31:53,656 INFO [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean] Closing JPA EntityManagerFactory for persistence unit 'default'
14:31:53,837 INFO [org.jboss.as.server.deployment] JBAS015877: Stopped deployment MY-APP.war in 2259ms
14:31:56,158 INFO [com.arjuna.ats.jbossatx] ARJUNA032018: Destroying TransactionManagerService
14:31:56,158 INFO [com.arjuna.ats.jbossatx] ARJUNA032014: Stopping transaction recovery manager
14:31:56,160 INFO [org.jboss.as] JBAS015950: JBoss AS 7.1.1.Final "Brontes" stopped in 4567ms
In my standalone.xml I have
<subsystem xmlns="urn:jboss:domain:datasources:1.0">
<datasources>
<datasource jndi-name="java:/my/DS1" pool-name="My1" enabled="true" use-java-context="true">
<connection-url>jdbc:oracle:thin:#10.172.1.1:1521:ABCD</connection-url>
<driver>oracle</driver>
<pool>
<min-pool-size>1</min-pool-size>
<max-pool-size>10</max-pool-size>
<prefill>true</prefill>
<use-strict-min>false</use-strict-min>
</pool>
<security>
<user-name>xxx</user-name>
<password>xxx</password>
</security>
<timeout>
<idle-timeout-minutes>5</idle-timeout-minutes>
</timeout>
<statement>
<prepared-statement-cache-size>500</prepared-statement-cache-size>
<share-prepared-statements>true</share-prepared-statements>
</statement>
</datasource>
<drivers>
<driver name="oracle" module="com.oracle"/>
</drivers>
</datasources>
</subsystem>
then, I lookup the datasources in the code:
DataSource ds = new InitialContext().lookup("java:/my/DS1");
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
The datasource has to be looked up programmatically and not hard-coded in spring.xml since there can be multiple data sources and their JNDI names are configured in external properties file that the application scans on startup.
Then, the jdbcTemplate is passed to a TableScanner that is invoked with Spring scheduling:
public TableScanner(JdbTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
#Scheduled(fixedDelay = 5000)
public void run() {
// query a table using jdbcTemplate and process the retrieved records, deleting them at the end of processing
}
When stopping of the server occurs during run() method invocation, all the retrieved records should be processed and deleted. When the application tries to delete them, the connection manager is already closed, hence the exception.
Is there any way to make the connection manager (or any other JBoss AS subsystem I might need) to wait until the application is stopped?

I had the same problem, and I have now found the cause described in https://issues.jboss.org/browse/WFLY-944 . The solution proposed there in order to avoid it, is to declare the jndi as a resource in your web application, so that it does not unbind from the jboss server before your application terminates.
To achieve this add the following part to your web.xml which sets a reference to your jboss jndi with name "jdbc/myDS".
<resource-ref>
<res-ref-name>jdbc/myDS</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
<lookup-name>java:/my/DS1</lookup-name>
</resource-ref>
Then instead of looking up "java:/my/DS1" from your code part that points to the actual jndi, you should now instead look up "java:comp/env/jdbc/myDS" which will reference the resource you put in the web.xml that actually points to the actual jndi (that we previously specified as the 'lookup-name' xml property in the part added in the web.xml):
DataSource ds = (DataSource)
ctx.lookup("java:comp/env/jdbc/myDS");

I don't know which functionality is offered by Spring Scheduling, but if you had used an executor from the standard API, the solution would have been to add a ServletContextListener to your web application, which is invoked by the container when undeploying or stopping your application. In the contextDestroyed(ServletContextEvent sce) method, you would then invoke something like executor.shutdown(); executor.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS), which will stop the executor and wait for all pending tasks to finish. You may want to consider using a shorter timeout when invoking awaitTermination.
IMHO the better solution may be to avoid using Spring within Java EE applications, at least when exactly the same functionality is offered by the Java EE application server as well, as in this case.

Have you considered using EJB Scheduled bean? This way jboss will spawn the worker thread.

I found this JBoss AS issue, which reflects my problem. It appears that if the data sources are not bound in a static way, via #Resource, then the server does not know that the data source is still in use. I went with letting Spring manage the data sources on its own, which may not be the Java EE way, but gets the work done and is appropriate in my situation.