JBoss : Timeout Exception during deployment of War File - java

I'm trying to deploy a war on my remote server which is continuously failing with an error like TimeOutException.
I've checked over the web for similar issue but is out of luck as of now:
Please have a look below of the Logs generated out of it as I'm not sure what I'm missing here. Also I've updated my standalone.xml (added the deployment-timeout="240L" ) file as below:
<subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0">
<deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000" deployment-timeout="240L" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/>
</subsystem>
But still I'm getting a same exception seems like I'm missing something here, Logs are as below:
http://pastebin.com/BS6hrJ4z

Your setting for deploy-timeout is incorrect.
<subsystem xmlns="urn:jboss:domain:deployment-scanner:1.0">
<deployment-scanner scan-interval="5000" relative-to="jboss.server.base.dir" path="deployments" deployment-timeout="240" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/>
</subsystem>
Remove the 'L' from the deploy timeout setting, which should be numeric only, then it should work fine.
NOTE: This deployment-timeout is in seconds
Additionally, you are getting a couple of Spring initialization errors during deployment. To get more information you need to increase the log level to see what the root cause may be:
2016-03-28 09:10:44,422 WARN [org.jboss.as.ee] (MSC service thread 1-7) WFLYEE0007: Not installing optional component org.springframework.http.server.ServletServerHttpAsyncRequestControl due to an exception (enable DEBUG log level to see the cause)
2016-03-28 09:10:44,423 WARN [org.jboss.as.ee] (MSC service thread 1-7) WFLYEE0007: Not installing optional component org.springframework.web.context.request.async.StandardServletAsyncWebRequest due to an exception (enable DEBUG log level to see the cause)

After looking at your logs, it seems that your deployment is taking more than 5 minutes to finish. Increasing the timeout is not the solution.
Instead, you should be breaking your deployment into smaller pieces and checking what's going on.
Things I'd check:
Assess your dependency on Spring. I know Spring folks will hate me for that, but most applications nowadays don't need "the full Spring". Pretty much everything can be achieved with Java EE standards, which are provided to you by your container (Wildfly, in this case). This reduces the overall footprint for your application, making the WAR file slimmer and deployment faster.
Answer to yourself: why do I need ehcache? You are using a very capable application server, providing you with JPA and second-level caching "for free" to you. You don't need to ship Hibernate and ehcache, unless you need to use very specific versions/features of Hibernate+ehcache.
Try to deploy some applications from the Java EE samples into your Wildfly. Are they also taking "long", comparing to, say, a reasonable machine on EC2/OpenShift? If so, this could indicate that the problem is on the environment, not on your application. Java EE samples: https://github.com/javaee-samples/javaee7-samples

Related

Wildfly datasource Enable is not working at runtime when configured through CLI

We are migrating from jboss Eap 6.4 to Wildfly. During migration, as part of our application installation, we will configure different xa-datasources in domain.xml. This configuration will be done through jboss-cli.sh.
After configuring the data sources, when deploying the ears, (particularly at *-ejb.jar) we are getting an error saying that the 'persistent unit' is not available and the deployment is rolled back.
Loggers are showing the below:
WFLYCTL0180: Services with missing/unavailable dependencies" => ["jboss.persistenceunit.\"xxyy.ear/xxyy-ejb.jar#xxyyDBUnit\" is missing [jboss.naming.context.java.jboss.datasources.xxyyDS]"]
Post restart of the jboss service, the same deployment works fine. So, we believe that the datasources configured in the domain.xml is not activated or enabled (although we have set the attribute 'enabled=true' for the xa-datasource. We could see the following warning while adding the xa-datasource in jboss.
2021-09-21 16:57:48,308 INFO [org.jboss.as.connector] (Host Controller Service Threads - 8) WFLYJCA0093: The 'enable' operation is deprecated. Use of the 'add' or 'remove' operations is preferred, or if required the 'write-attribute' operation can used to set the deprecated 'enabled' attribute
With this warning, we understood that the enable/disable operation has been removed in the jboss wildfly. Is there anyway we can make it enabled on the runtime without restart of the jboss service.
Below are the commands which we use to create the xa-datasource in wildfly.
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS:add(use-java-context=true,use-ccm=false,driver-name=com.mysql,transaction-isolation=TRANSACTION_READ_COMMITTED,min-pool-size=15,max-pool-size=150,pool-prefill=true,flush-strategy=IdleConnections,valid-connection-checker-class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker",stale-connection-checker-class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker",exception-sorter-class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter",set-tx-query-timeout=true,blocking-timeout-wait-millis=5000,idle-timeout-minutes=1,query-timeout=300,prepared-statements-cache-size=100,share-prepared-statements=true,jndi-name="java:jboss/datasources/xxyyDS",user-name=jboss,password="${VAULT::ds_MySqlDS::password::1}")
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=DatabaseName:add(value=xxyy_db)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=ServerName:add(value=xx-cluster)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=AutoReconnectForPools:add(value=true)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=AutoReconnect:add(value=true)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=useSSL:add(value=false)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS:enable
Any inputs on how to enable the xa-datasource on runtime would help us.
You can enable the datasource via the CLI:
/subsystem=datasources/xa-data-source=xxyyDS:write-attribute(name=enabled,value=true)
It might be required to reload:
:reload
After running through many trial and error method, we found that when adding the datasources by default it will be in enabled state. So no need to do explicitly enable when adding a datasource.
So we removed the below line from the list of CLIs.
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS:enable
Post removing the line, the configuration of datasources through CLI and the deployment of ears went fine without issues. Posting this solution as it will be helpful for others who are facing similar kind of issue.
Finally our datasource CLIs will be like the ones below.
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS:add(use-java-context=true,use-ccm=false,driver-name=com.mysql,transaction-isolation=TRANSACTION_READ_COMMITTED,min-pool-size=15,max-pool-size=150,pool-prefill=true,flush-strategy=IdleConnections,valid-connection-checker-class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker",stale-connection-checker-class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker",exception-sorter-class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter",set-tx-query-timeout=true,blocking-timeout-wait-millis=5000,idle-timeout-minutes=1,query-timeout=300,prepared-statements-cache-size=100,share-prepared-statements=true,jndi-name="java:jboss/datasources/xxyyDS",user-name=jboss,password="${VAULT::ds_MySqlDS::password::1}")
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=DatabaseName:add(value=xxyy_db)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=ServerName:add(value=xx-cluster)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=AutoReconnectForPools:add(value=true)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=AutoReconnect:add(value=true)
/profile=full-ha/subsystem=datasources/xa-data-source=xxyyDS/xa-datasource-properties=useSSL:add(value=false)

Hibernate Search Configuration Problems

I'm very new to Hibernate Search and trying to integrate it with my application.
Facing a memory leak issue (threads going into wait/park state).
Hibernate Search configuration (very minimal) is annotation driven. Using :
#Indexed(index = "<index_name>")
#IndexedEmbedded
AND
#Field(name = "title", store = Store.YES, analyzer = #Analyzer(definition = "standardAnalyzer")
Hibernate Search properties:
<property name="hibernate.search.default.indexBase" value="../lucene/indexes" />
<property name="hibernate.search.default.directory_provider" value="filesystem" />
<property name="hibernate.search.default.exclusive_index_use"
value="false" />
Using Tomcat 8 to deploy the application.
My main problem is that
VisualVm clearly shows that Hibernate search creates a sync consumer thread for every index. It does so for each and every call I make to my server (in my case, 2 new threads spawned and in parked state on every call). Eventually the number of threads increase till my server becomes unresponsive.
On server shutdown, i get the error:
09-Aug-2017 17:15:28.151 WARNING [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [procurewise] appears to have started a thread named [Hibernate Search sync consumer thread for index Category] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:338)
org.hibernate.search.backend.impl.lucene.SyncWorkProcessor.parkCurrentThread(SyncWorkProcessor.java:175)
org.hibernate.search.backend.impl.lucene.SyncWorkProcessor.access$300(SyncWorkProcessor.java:35)
org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:147)
java.lang.Thread.run(Thread.java:745)
I'm using Spring, Hibernate, JPA, AOP and JAX-RS, Jersey .
Environment Details:
Hibernate version 5.2.10.Final
Hibernate Search version 5.7.0.Final
Spring version 4.3.6.RELEASE
JPA 2.1
JAVA 8
Jersey 1.8
This thread should stop automatically when Hibernate Search shuts down, which happens automatically when Hibernate ORM shuts down.
I see two reasons for this warning: either Tomcat checked the threads while Hibernate ORM was still shutting down, or you didn't stop Hibernate ORM at all.
I would suggest to check your shutdown process, see if Hibernate ORM is closed correctly (normally you use org.hibernate.SessionFactory.close(), but I'd expect Spring to take care of that itself), and if it is, see if it's closed synchronously (so that your Tomcat will only perform checks after all the closing is done).

Keeping HTTP session between redeploys in WildFly

Is possible to keep HTTP session between redeploys on WildFly?
You might try configuring session persistence. Here is a blog on how to do it.
<servlet-container name="default">
<persistent-sessions path="session" relative-to="jboss.server.temp.dir"/>
<jsp-config/>
</servlet-container>
However, it seems that this feature is available startinng from v8.2. I'll quote a note from this blog
You can also achieve session passivation, when you are using a non-ha profile, by adding <distributable/> to your web.xml (for those using WildFly 8.1.0 or less).
<distributable/> in web.xml deployed to standalone may lead you to WFLY-3715 (logs polluted with undertow exceptions).
Alternative option can be to configure persistent sessions in standalone.xml or with jboss-cli.sh. JBoss CLI command worth to mention, as it can be used within maven plugin for WildFly.
/subsystem=undertow/servlet-container=default/setting=persistent-sessions:add(path="session", relative-to="jboss.server.temp.dir")
Log pollution example:
2016-12-29 09:39:48,464 ERROR [io.undertow.request] (default task-26) UT005023: Exception handling request to /resources/js/holder.js: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key SessionCreationMetaDataKey(EkmgBEYcvqFxNF_T2dDwEfyXkh7Nzcv2nTKBYy9G) and requestor GlobalTransaction::13:local. Lock is held by GlobalTransaction::12:local
at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:238)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAndRecord(AbstractLockingInterceptor.java:193)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockKey(AbstractTxLockingInterceptor.java:193)
at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockOrRegisterBackupLock(AbstractTxLockingInterceptor.java:116)
at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitDataReadCommand(PessimisticLockingInterceptor.java:71)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitGetKeyValueCommand(AbstractLockingInterceptor.java:80)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:346)
at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:331)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:99)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:114)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:83)
at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:85)
at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:43)
........

Exception "NoClassDefFoundError for javax/mail/Authenticator

I am trying to send mail in my web application. I am using the libraries javax.mail-1.4.4 and activation-1.1. My configuration is set up like described in the Tomcat Docs here http://tomcat.apache.org/tomcat-5.5-doc/jndi-resources-howto.html#JavaMail_Sessions
The web.xml file is configured properly according to this information. The context.xml where I store the settings is configured properly as well (or at least should be) with mail.*, auth, name, type info set properly.
When trying to send a mail this way, I get the following error:
Exception in thread "Thread-27" java.lang.NoClassDefFoundError: javax/mail/Authenticator
in this line:
session = (Session) envCtx.lookup("mail/Session");
I've looked through some solutions, like missing mail/activation libraries, checked up on those but none I could find were able to help me. Any idea what I could do to fix this or get in closer on the error?
"should be" is the operative phrase. I'd recommend that you believe Tomcat when it tells you it's missing things. You have not configured things properly until Tomcat says you have.
I'd recommend that you put them into the Tomcat /lib folder and see if that makes a difference.

"Persistence provider caller does not implement the EJB3 spec correctly." warning when deploying on JBoss 5.1.0.GA

i get this hibernate warning pretty much every time i deploy a persistence unit on JBoss. its never the cause of any issues, but i was wondering if anyone knows what exactly this means, and can it be fixed/slicenced/worked-around in some way.
According to this comment from Jaikiran Pai (RedHat employee member of the JBoss EJB3 development team):
Daniel Winz wrote:
There is a warning that may stick together with the error:
2010-09-12 12:10:52,759 WARN [org.hibernate.ejb.Ejb3Configuration] (HDScanner) Persistence provider caller does not implements the EJB3 spec correctly. PersistenceUnitInfo.getNewTempClassLoader() is null.
You can ignore that log message.
That's the best answer I've found so far about the warning (which is reported in EJBTHREE-710 without any particular comment...).
Check that you datasource is correctly and validate credential! I had same problem and I solved this way.

Categories