We Keep Coding

Configuring Mysql DataSource with IBM WebSphere Application Server Liberty Profile

I have a spring mvc application which i am deploying on IBM WebSphere Application Server Liberty Profile, the application is supposed to access a mysql database server that is hosted locally. I have added the datasource configuration as follows in the sever.xml file
<dataSource id="springdb" jndiName="jdbc/springdb">
<jdbcDriver javax.sql.XADataSource="com.mysql.cj.jdbc.Driver" libraryRef="mysqlJDBCLib"/>
<properties databaseName="spring_db" password="**********" portNumber="3306" serverName="localhost" user="root"/>
</dataSource>
<library id="mysqlJDBCLib">
<fileset dir="/opt/IBM/WebSphere/Liberty_16.0.0.4/usr/shared/resources/mysql" includes="mysql-connector-java-6.0.6.jar"/>
</library>
I am getting the following stack trace
Caused by: java.lang.RuntimeException: java.sql.SQLNonTransientException: DSRA4000E: A valid JDBC driver implementation class was not found for the jdbcDriver dataSource[springdb]/jdbcDriver[default-0] using the library mysqlJDBCLib. [/opt/IBM/WebSphere/Liberty_16.0.0.4/usr/shared/resources/mysql/mysql-connector-java-6.0.6.jar]
at com.ibm.ws.resource.internal.ResourceFactoryTrackerData$1.getService(ResourceFactoryTrackerData.java:123)
... 77 more
Caused by: java.sql.SQLNonTransientException: DSRA4000E: A valid JDBC driver implementation class was not found for the jdbcDriver dataSource[springdb]/jdbcDriver[default-0] using the library mysqlJDBCLib. [/opt/IBM/WebSphere/Liberty_16.0.0.4/usr/shared/resources/mysql/mysql-connector-java-6.0.6.jar]
at com.ibm.ws.jdbc.internal.JDBCDriverService.classNotFound(JDBCDriverService.java:196)
... 77 more
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource
at com.ibm.ws.classloading.internal.AppClassLoader.findClassCommonLibraryClassLoaders(AppClassLoader.java:499)
... 77 more
I have tried the following
Replacing com.mysql.cj.jdbc.Driver with com.mysql.jdbc.Driver and string got the same error
Replacing javax.sql.XADataSource with javax.sql.DataSource and still got the same error
Replacing javax.sql.XADataSource with javax.sql.MysqlConnectionPoolDataSource, same error
Switching out the mysql connector jar from mysql-connector-java-6.0.6.jar to mysql-connector-java-5.1.45-bin.jar, still same error (both the jars are there in the specified file path)

Background info about datasource config:
For all datasources (except for id="DefaultDataSource") the type of DataSource is selected in the following priority:
Use the type class configured on the <dataSource> element, if configured
javax.sql.ConnectionPoolDataSource This option is active in your case
javax.sql.DataSource
javax.sql.XADataSource
(According to Configuring relational database connectivity in Liberty)
If no class is defined for a type, then the next lowest priority will be checked.
By default, Liberty will scan for use the following priority for locating data source implementation class names:
Use the corresponding value configured on the <jdbcDriver> element
Use the default classes for a <properties.DRIVER_TYPE> element. Note that only some JDBC drivers have their own element types. For JDBC drivers that do not have their own properties element (such as MySQL) use the generic <properties> element
Based on the JDBC driver jar name, Liberty will try to guess the implementation class name. This option is active in your case
Reason why your configuration is not working
You've configured your <jdbcDriver> element to say "Use this specific class for XADataSource's", however, the <dataSource> element you've configured is not attempting to create an XADataSource. Using the priority order mentioned above, it will first try to create a ConnectionPoolDataSource (which Liberty has internally mapped to com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource.
To fix the problem:
Specify <dataSource .. type="javax.sql.XADataSource"> in your configuration so that Liberty will try to create an XADataSource instead of a ConnectionPoolDataSource. Then when it tries to create the XADataSource it will look at the class name you've configured on the <jdbcDriver> element.
Specify MySQL's XADataSource class name for the javax.sql.XADataSource property (which is com.mysql.cj.jdbc.MysqlXADataSource) instead of their java.sql.Driver implementation class name.
So your final configuration would look like this:
<dataSource id="springdb" jndiName="jdbc/springdb" type="javax.sql.XADataSource">
<jdbcDriver javax.sql.XADataSource="com.mysql.cj.jdbc.MysqlXADataSource" libraryRef="mysqlJDBCLib"/>
<properties databaseName="spring_db" password="**********" portNumber="3306" serverName="localhost" user="root"/>
</dataSource>
<library id="mysqlJDBCLib">
<fileset dir="/opt/IBM/WebSphere/Liberty_16.0.0.4/usr/shared/resources/mysql" includes="mysql-connector-java-6.0.6.jar"/>
</library>

If you don't specify the type attribute in the dataSource config element, Liberty will try to infer the datasource class to load based on the driver jar filename. In your example, it is incorrectly inferring implementation class names based on a previous mySQL driver. It appears that mySQL has changed the package names of their implementations for DataSource, ConnectionPoolDataSource and XADataSource. I'll open an git issue for this. In the meantime, you can simply specify the type of datasource to create using the type attribute of the dataSource config element and then update your jdbcDriver config element with javax.sql.DataSource="com.mysql.cj.jdbc.MysqlDataSource" to point to the proper mysql driver class. If instead you need a conn pool or XA datasource instead, just update the type attribute of dataSource to identify the type and update jdbcDriver with the driver class impl.

Skipped XML bean definition file due to specified profiles [default] not matching

We have a Spring application deployed on tomcat, and it does not start.
The application error is as follows:
Caused by: org.springframework.beans.PropertyBatchUpdateException; nested
PropertyAccessExceptions (1) are:
PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'driverClassName' threw exception; nested exception is java.lang.IllegalStateException: Could not load JDBC driver class [${hibernate.connection.driver_class}]
at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:121)
at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:75)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1564)
... 49 more
We do have a properties file defined at:
/opt/tomcat/appname/tomcat8-2
with three files: config.properties, core-ws.proprties, and log4j-config.xml
We know ${TOMCAT_CONFIG_HOME} is defined to be: /opt/tomcat/appname/tomcat8-2
We know that the file permissions are correct, the user running tomcat has read permissions to the directories and files in this config directory. I can as that tomcat user edit/read the properties file. So, we know it's not a permissions issue.
From the app log, we have this error message:
Skipped XML bean definition file due to specified profiles [default] not matching: class path resource [spring/app-platform-entity-context.xml]
The Spring application context file has this for properties:
<beans profile="default">
<context:property-placeholder location="file:${TOMCAT_CONFIG_HOME}/core-ws.properties" />
</beans>
Finally here is what the core-ws.properties looks like:
hibernate.connection.url=jdbc:oracle:thin:#//123.456.789/APP_DB
hibernate.connection.driver_class=oracle.jdbc.driver.OracleDriver
hibernate.connection.username=some_db_user
hibernate.connection.password=some_db_password
So, I have looked into this for a coupe of days, and I am at my wits end. I can tell you we have no problem with the log4j-config.xml file. The app can find it, and read it just fine. it is just this one parameter that seems to be crashing the app ...
Any help with this matter would be much appreciated. Thanks!

The reason that XML file was skipped was because we had a -Dspring.profiles.active=prod in our CATALINA_OPTS setup.
We didn't find it until we started exporting what the values were as tomcat started.
Once we removed this line, our [default] profile tookover as it should.
Thanks!

Answering this for stand-alone apps:
In my case it was because of the order of setting up ApplicationContext, I was loading the xml file, before setting the profile.
The correct order is:
GenericXmlApplicationContext ctx =
new GenericXmlApplicationContext();
ctx.getEnvironment().setActiveProfiles("dev");
ctx.load("com/profiles/*-config.xml");
ctx.refresh();

override values in application.properties files from command line

I have apllication.properties files in java web application which contains these properties.
spring.datasource.driverClassName=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://localhost:19095/test
spring.datasource.username=test
spring.datasource.password=test123
But I want to override these properties (except driverClassName) while starting tomcat server.
I'm trying to set these variables from command line, but it's not working.
tomcat version : 7.0.63

Why don't you use a property placeholder with config directory specified by a system parameter:
<context:property-placeholder location="file:${configLocation}/database.properties:defaultDatabase.properties" />
Then start the tomcat with:
-DconfigLocation=/opt/config
see http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/context/annotation/PropertySource.html and also Loading property file from system properties with default path in Spring context

How to get a tomcat Environment variable in a PropertyPlaceholderConfigurer

I try to get an Environment variable specified in Tomcat's server.xml in a 'PropertyPlaceholderConfigurer' located in my jpa-spring.xml file.
So far, the setup looks as follows:
Tomcat server.xml
<Environment description="Identifies the server environement"
name="server-env"
type="java.lang.String"
value="dev" />
The in WebContent/META-INF/context.xml:
<Context>
<ResourceLink name="server-env" global="server-env" type="java.lang.String"/>
</Context>
Which is referenced in WebContent/WEB-INF/web.xml like:
<resource-env-ref>
<description>Identifies server environement</description>
<resource-env-ref-name>server-env</resource-env-ref-name>
<resource-env-ref-type>java.lang.String</resource-env-ref-type>
</resource-env-ref>
<!-- Spring Integration -->
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>
/WEB-INF/config/jpa-spring.xml
</param-value>
</context-param>
And in /WEB-INF/config/jpa-spring.xml I try to get that variable as a replacement:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>WEB-INF/config/db.${server-env}.properties</value>
</list>
</property>
</bean>
This is a setup I put together using information from several articles found on the web.
However, I get an error like ...
Could not resolve placeholder 'server-env' in [WEB-INF/config/db.${server-env}.properties] as system property: neither system property nor environment variable found
05 Nov 2011 14:45:13,385 org.springframework.web.context.ContextLoader
ERROR Context initialization failed
org.springframework.beans.factory.BeanInitializationException: Could not load properties; nested exception is java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/config/db.${server-env}.properties]
... when starting tomcat.
What is the right approach to achieve what I am looking for?
I know that this question is similar to this, and this question. However, I even couldn't figure it out with the information from these answers.

Here are my suggestion
With the current set up, its really going to be complicated to read JNDI property server-env and use the same to load the property file.
The way you have assembled the spring application (and PropertyPlaceholderConfigurer), spring will try to look for the property server-env first in OS environment then in java system properties (passed from command using -D option). It finds it at neither of these places and hence fails.
So currently the easiest way out right now is to pass the value of server-env form command prompt of your application server (where you invoke java ; typical syntax would be -Dserver-env=dev). I leave this to you to figure out.
if above option appears a bit complicated, another easier way out is set an environment variable with name server-env to its appropriate values (on Windows its set server-env=dev. Plz check respect OS documentations for this).

Those Environment elements are setting up JNDI. Getting values out of JNDI isn't supported, by default, by any simple syntactic sugar in Spring.
http://www.theserverside.com/news/thread.tss?thread_id=35474#179220
might give you some useful ideas.

problem configure JBoss to work with JNDI

I am trying to bind connection to the DB using JNDI in my application that runs on JBoss. I did the following:
I created the datasource file oracle-ds.xml filled it with the relevant xml elements:
<datasources>
<local-tx-datasource>
<jndi-name>bilby</jndi-name>
...
</local-tx-datasource>
</datasources>
and put it in the folder \server\default\deploy
Added the relevant oracle jar file
than in my application I performed:
JndiObjectFactoryBean factory = new
JndiObjectFactoryBean();
factory.setJndiName("bilby");
try{
factory.afterPropertiesSet();
dataSource = factory.getObject();
}
catch(NamingException ne) {
ne.printStackTrace();
}
and this cause the error:
javax.naming.NameNotFoundException:
bilby not bound
then in the output after this error occured I saw the line:
18:37:56,560 INFO
[ConnectionFactoryBindingService]
Bound ConnectionManager 'jb
oss.jca:service=DataSourceBinding,name=bilby'
to JNDI name 'java:bilby'
So what is my configuration problem? I think that it may be that JBoss first loads and runs the .war file of my application and only then it loads the oracle-ds.xml that contain my data-source definition.
The problem is that they are both located in the same folder.
Is there a way to define priority of loading them, or maybe this is not the problem at all.
Any idea?

You should use such construction to call Datasource: java:bilby.
You can read more about that here:
Naming and Directory (JNDI) - JBOSS jndi Datasource: jdbc not bound

To check how the datasource is bound in the JNDI tree you should use the jmx-console
http://localhost8080/jmx-console/HtmlAdaptor?action=inspectMBean&name=jboss%3Aservice%3DJNDIView
and invoke the list() method.
Datasources are registered under "jdbc". In your case "jdbc/bilby"
EDIT: That was an example that works for me without spring.
Now found this example which injects a more complete JNDI name.
<bean id="idDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jdbc/bilby" />
</bean>