Given a very simple Camel bundle for Karaf, generated with the camel-archetype-blueprint, I want to add a datasource that is configured via properties and not within the blueprint.xml.
I tried configuring a PropertiesComponent and accessing the property within the property value for the MySQL datasource in various ways, but none seems to work. When logging a message though, the properties are accessible.
How can a datasource be configured using parameter values from a properties file?
I especially need this to use the same datasource configuration for multiple bundles and distinguish between production/test environments. I thought about writing the properties with Maven during the build, depending on the target environment. Are there any other best practices on how to solve this datasource issue?
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<bean id="properties" class="org.apache.camel.component.properties.PropertiesComponent">
<property name="location" value="classpath:config.properties" />
</bean>
<bean id="dataSourceMySQL" class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
<property name="url" value="jdbc:mysql://127.0.0.1/test_database" />
<!-- This causes an error, as it tries to connect with
`${mysqlUser}`#`localhost` without any evaluation -->
<property name="user" value="${mysqlUser}" />
<property name="password" value="${mysqlPassword}" />
</bean>
<service interface="javax.sql.DataSource" ref="dataSourceMySQL">
<service-properties>
<entry key="osgi.jndi.service.name" value="jdbc/mysqlDatasource" />
</service-properties>
</service>
<bean id="sql" class="org.apache.camel.component.sql.SqlComponent">
<property name="dataSource" ref="dataSourceMySQL" />
</bean>
<camelContext xmlns="http://camel.apache.org/schema/blueprint">
<route id="messageQuery">
<from uri="sql:SELECT * FROM messages" />
<log message="The user property is: {{mysqlUser}}, the query result is: ${body}" />
</route>
</camelContext>
</blueprint>
Just for an overview, the project layout looks like this:
a) Datasource in bundle with xml and bundle properties
You could use bundle properties. The example below optionally uses a bundle configuration in etc/org.camel.demo.cfg:
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<!-- etc/org.camel.demo.cfg -->
<cm:property-placeholder persistent-id="org.camel.demo" xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0">
<cm:default-properties>
<cm:property name="mysqlUser" value="root"/>
<cm:property name="mysqlPassword" value=""/>
</cm:default-properties>
</cm:property-placeholder>
<bean id="properties" class="org.apache.camel.component.properties.PropertiesComponent">
<property name="location" value="classpath:config.properties" />
</bean>
<bean id="dataSourceMySQL" class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource">
<property name="url" value="jdbc:mysql://127.0.0.1/test_database" />
<property name="user" value="${mysqlUser}" />
<property name="password" value="${mysqlPassword}" />
</bean>
</blueprint>
b) Shared datasource
Another option would be to use a shared datasource. Just deploy a blueprint file containing only the datasource (example below uses postgres).
You can combine this also with config properties as shown above.
Providing the OSGi service
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
<!-- use config properties if needed -->
<bean id="dataSource" class="org.postgresql.ds.PGPoolingDataSource" destroy-method="close">
<property name="serverName" value=":"/>
<property name="user" value="postgres"/>
<property name="password" value="postgres"/>
<property name="dataSourceName" value="demo"/>
<property name="initialConnections" value="2"/>
<property name="maxConnections" value="4" />
</bean>
<service interface="javax.sql.DataSource" ref="dataSource">
<service-properties>
<entry key="osgi.jndi.service.name" value="jdbc/demo"/>
</service-properties>
</service>
</blueprint>
Referencing from another bundle
In your bundle you can then lookup the datasource osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/demo)
To use this datasource in a SqlComponent, a reference must be created like this:
<reference id="dataSource" interface="javax.sql.DataSource"
filter="(osgi.jndi.service.name=jdbc/mysql)">
</reference>
<bean id="sql" class="org.apache.camel.component.sql.SqlComponent">
<property name="dataSource" ref="dataSource" />
</bean>
Using the persistence.xml
Make sure you import org.demo.osgi.datasource.**. Here is a usage example with persistence.xml:
<persistence version="2.0"
xmlns="http://java.sun.com/xml/ns/persistence" >
<persistence-unit name="demo" transaction-type="JTA">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<jta-data-source>osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/demo)</jta-data-source>
<mapping-file>META-INF/foo.xml</mapping-file>
</persistence-unit>
</persistence>
c) Use jdbc-feature (optional)
The xml file from above can be created and managed using the jdbc feature. It depends a bit on your version if it is available or not:
JBossFuse:admin#545074693af1> features:install jdbc hibernate jndi
JBossFuse:admin#545074693af1> install mvn:org.postgresql/postgresql/9.4.1208
Bundle ID: 292
JBossFuse:admin#545074693af1> resolve 292
JBossFuse:admin#545074693af1> jdbc:create -t postgres -u postgres -p postgres -url ${postgres.addr}:${postgres.port} demo
P.S.: If you want to remove the cleartext password from the config file use something like jasypt.
Hello I can see two things. The first one is that you have not said in which folder in the class path the config file reside. Unless the OSGI-INF folder is not already scanned you should have :
The second thing is about how you reference the properties. Unless otherwise defined the property prefixToken and suffixToken are by default "{{" and "}}"
http://camel.apache.org/properties.html
Which means that you need to replace your ${mysqlUser} with {{mysqlUser}}
Related
i'm using Apache Camel and i try to load datasource properties from this file
config.properties:
url = my_url
user = user_name
password = user_pass
this is dataSource (blueprint.xml):
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource">
<property name="URL" value="my_url"/>
<property name="user" value="user_name"/>
<property name="password" value="user_pass"/>
</bean>
How can i read values from config.properties and insert them into dataSource properties ?
You talk about blueprint.xml, and camel, so I assume you are in an osgi container like Karaf/ServiceMix, and you are using Aries Blueprint.
Then you can use the cm namespace and a property-placeholder. If you use camel and want your properties to be dynamically reloaded, then you can use too an update strategy reload, which start/stop the blueprint container when the configuration change. This will load the configuration with pid "datasource" (ie, in karaf, the file etc/datasource.cfg) :
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.2.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.2.0">
<cm:cm-properties id="myProps" persistent-id="datasource" update-strategy="reload"/>
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource">
<property name="URL" value="${url}"/>
<property name="user" value="${user}"/>
<property name="password" value="${password}"/>
</bean>
</blueprint>
If you want to use your configuration file without using ConfigurationAdmin or dynamically reload your bundle, then you can use the ext namespace :
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.2.0">
<ext:property-placeholder>
<ext:location>file:config.properties</ext:location>
</ext:property-placeholder>
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource">
<property name="URL" value="${url}"/>
<property name="user" value="${user}"/>
<property name="password" value="${password}"/>
</bean>
</blueprint>
According to code I assume you use probably spring as container. General solution in spring is to use PropertyPlaceHolder, your configuration will look like this:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location">
<value>config.properties</value>
</property>
</bean>
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource">
<property name="URL" value="${jdbc.myUrl}"/>
<property name="user" value="${jdbc.user_name}"/>
<property name="password" value="${jdbc.user_pass}"/>
</bean>
Please check the example for details.
I have 2 different applications deployed in application server (glassfish). One is a jar file and other is a war application. Both the applications refer to a single properties file (data.properties). To read the properties file, I have created a instance of Springs PropertyPlaceholderConfigurer in respective context files (business-beans.xml and applicationContext.xml). After deploying the applications, I am able to load the properties file in one application while the other web application throws "Could not resolve placeholder 'sw.throttle.enable'
Question -
How to solve the issue?
Is it incorrect load the same properties file at two locations?
Is there a way I load the properties file in one context and in the other bean definition file use the reference of the first one?
SnapShot of business.beans
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
<property name="placeholderPrefix" value="${sw." />
<property name="location" value="file:///etc/data.properties" />
<property name="ignoreResourceNotFound" value="true" />
</bean>
Property referenced as below in business.beans
<bean id="mService" class=" com.test.business.mService">
<property name="throttlingEnabled" value="${sw.throttle.enable}"/>
</bean>
Snapshot of applicationContext.xml
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
<property name="placeholderPrefix" value="${sw." />
<property name="location" value="file:///etc/data.properties" />
<property name="ignoreResourceNotFound" value="true" />
</bean>
Property referenced as below in applicationContext.xml
<bean id="downloadService" class="com.test.downloadService"
init-method="startUp" destroy-method="shutDown"
p:throttlingEnabled="${sw.throttle.enable}" />
The application containing business.beans deploys well, but the application containing applicationContext.xml throw run time error "could not resolve placeholder sw.throttle.enable"
Note -
Both the applications are deployed in a OsGi Context.
Spring version is 3.0.1
Edit -
The applicationContext.xml has another bean defined as below. Could this be the cause?
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/>
</bean>
The issue was resolved by setting "ignoreUnresolvablePlaceholders" to "true". Apparently business.beans had nothing to do with the issue
Below is the modified configuration which solved the issue
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
<property name="placeholderPrefix" value="${sw." />
<property name="location" value="file:///etc/data.properties" />
<property name="ignoreResourceNotFound" value="true" />
<property name="ignoreUnresolvablePlaceHolders" value="true"
</bean>
Thanks StackOverflow for the answer +1
I need CXF server monitoring with JMX and followed CXF Documentation. I use tomcat and following is the content of my cxf.xml file located at /home/kalpa/applications/apache-tomcat-7.0.54/webapps/java_first_jaxws/WEB-INF/classes
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
<bean id="org.apache.cxf.management.InstrumentationManager" class="org.apache.cxf.management.jmx.InstrumentationManagerImpl">
<property name="bus" ref="cxf" />
<property name="enabled" value="true" />
<property name="JMXServiceURL " value="service:jmx:rmi:///jndi/rmi://localhost:9914/jmxrmi" />
</bean>
</beans>
But I cannot monitor the data with jconsole. There is no local entry for the connection.
What could be the issue here?
I don't know if your question is still an ongoing problem but this is my configuration :
<cxf:bus bus="cxf">
<cxf:properties>
<entry key="bus.jmx.enabled" value="true"/>
<entry key="bus.jmx.JMXServiceURL" value="service:jmx:rmi:///jndi/rmi://localhost:9914/jmxrmi"/>
</cxf:properties>
</cxf:bus>
Instead of yours :
<bean id="org.apache.cxf.management.InstrumentationManager" class="org.apache.cxf.management.jmx.InstrumentationManagerImpl">
<property name="bus" ref="cxf" />
<property name="enabled" value="true" />
<property name="JMXServiceURL " value="service:jmx:rmi:///jndi/rmi://localhost:9914/jmxrmi" />
</bean>
I'm working with CXF 3.0.2, and a jconsole in JRE 1.6.0_27 ...
This is working fine :
jconsole connection string
I have a Spring Web Application with Maven 3 and Datasources/Beans configured for local development on a h2 Database. For Testing there exists a JBoss AS 7.1 Server.
Is it possible to override a Bean / DataSource in a special JBoss XML-File, so that for local (IDE) matters a DriverManagerDataSource is used (see first code posting), and a JndiObjectFactoryBean (second one), if my Application is deployed on a JBoss?
applicationContext.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:tx="http://www.springframework.org/schema/tx" xmlns="http://www.springframework.org/schema/beans"
xsi:schemaLocation="http://www.springframework.org/schema/beans
<!-- And so on -->">
<!--GENERAL-->
<context:annotation-config/>
<context:component-scan base-package="com.mysuperapp"/>
<!--DATASOURCES-->
<bean id="activitiDataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver"/>
<property name="url" value="jdbc:h2:file:h2\activiti;MVCC=TRUE"/>
<property name="username" value="sa"/>
<property name="password" value=""/>
</bean>
<bean id="hibernateDataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver"/>
<property name="url" value="jdbc:h2:file:h2\hibernate;AUTO_SERVER=TRUE"/>
<property name="username" value="sa"/>
<property name="password" value=""/>
</bean>
<!--TXMANAGEMENT-->
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceUnitName" value="mysuperappPersistenceUnit"/>
<property name="dataSource" ref="hibernateDataSource"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"/>
</property>
<property name="packagesToScan" value="com.mysuperapp.model"/>
<property name="jpaPropertyMap">
<map>
<entry key="hibernate.dialect" value="org.hibernate.dialect.H2Dialect"/>
<entry key="hibernate.hbm2ddl.auto" value="update"/>
</map>
</property>
</bean>
<!-- And so on -->
custom applicationContext.xml datasources for JBoss
<bean id="activitiDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jboss/datasources/activitiDataSource"/>
</bean>
<bean id="hibernateDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:jboss/datasources/hibernateDataSource"/>
</bean>
Any help is greatly appreciated.
Your application creates the database connection on its own as i can see on your config.
Instead of doing this you could just define the datasource on the app-server and obtain it by a jndi lookup. So you can configure the database connection for each system separately.
<!-- DataSource-LookUp -->
<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean" scope="singleton">
<property name="jndiName" value="java:jboss/datasources/MyDataSourceDS" />
<property name="resourceRef" value="true" />
</bean>
Your JBoss has a file <jboss_home>/standalone/configuration/standalone.xml which is used if you don't provide a different one.
For testing purposes you could try and make a copy of that file in the same directory, e.g. naming it test.xml and add your datasource definition there. Then use it by starting JBoss with standalone.bat --server-config=test or standalone.bat -c test.
I managed to do this with a SVN patch, which is added prior to the automatic deployment. It changes the DataSource definitions.
Pay attention when creating the patch with windows and applying under linux with /usr/bin/patch (CR+LF problem, apply dos2unix filename to both patch and files to be patched).
Currently we are loading our JDBC source values from properties file as per following:
<context:property-placeholder location="classpath:master.properties" ignore-unresolvable="true" />
<bean id="mainDataSource" class="com.jolbox.bonecp.BoneCPDataSource"
destroy-method="close">
<property name="driverClass" value="${database.driver}" />
<property name="jdbcUrl" value="${database.url}" />
<property name="username" value="${database.user}" />
<property name="password" value="${database.password}" />
<property name="idleConnectionTestPeriod" value="60" />
<property name="idleMaxAge" value="240" />
<property name="maxConnectionsPerPartition" value="2" />
<property name="minConnectionsPerPartition" value="2" />
<property name="partitionCount" value="3" />
<property name="acquireIncrement" value="10" />
<property name="statementsCacheSize" value="50" />
<property name="releaseHelperThreads" value="3" />
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy"
scope="singleton">
<property name="targetDataSource">
<ref local="mainDataSource" />
</property>
</bean>
This seg. works fine with classpath based app.properties file and get rid of app.properties.
We would like to load this values from context.xml (either placed in META-INF or $CATALINA_HOME/conf/context.xml). It will help us load proper values at prod/staging servers.
Will appreciate any help or alternate method/suggestions. (If similar question is already answered, please share the link)
Thanks!
As Alan Hay mentioned you could externalize the datasource configuration into Tomcat's own context.xml and then have Spring do a JNDI lookup to retrieve it. This is an approach that I've commonly used on some of the projects I've worked on.
The pieces you need to put in place to achieve would be:
1. Add the datasource configuration to $CATALINA_HOME/conf/context.xml
<GlobalNamingResources>
<Resource type="javax.sql.DataSource"
name="dsName"
factory="com.jolbox.bonecp.BoneCPDataSource"
driverClassName="your.driver.classname"
jdbcUrl="your:driver:url"
username="username"
password="password"
idleMaxAge="240"
idleConnectionTestPeriod="60"
partitionCount="3"
acquireIncrement="10"
maxConnectionsPerPartition="2"
minConnectionsPerPartition="2"
statementsCacheSize="50"
releaseHelperThreads="3" />
</GlobalNamingResources>
2. Add a resource link in the application's META-INF/context.xml
<Context path="/YourApp">
<ResourceLink description="Datasource for YourApp"
global="jdbc/dsName"
name="jdbc/dsName"
type="javax.sql.DataSource" />
</Context>
3. Modify the Spring config to lookup the datasource in JNDI
<beans xmlns:jee="http://www.springframework.org/schema/jee"
xsi:schemaLocation="http://www.springframework.org/schema/jee classpath:/org/springframework/ejb/config/spring-jee-3.0.xsd">
<jee:jndi-lookup id="dataSource"
jndi-name="java:comp/env/jdbc/dsName" />
4. Move the driver and datasource jars
Since the datasource configuration is now container managed, you should place the database driver and datasource jars into $CATALINA_HOME/lib so they are available to Tomcat when it creates the datasource. These jars should no longer need to reside in the WEB-INF/lib of your application.
The way I do it, I have default properties file in my classpath, and I have one properties file that I use to override those default (development) properties:
in my spring file:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreResourceNotFound" value="true"/>
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
<value>classpath:camel.properties</value>
<value>classpath:email.properties</value>
<value>${external_config}</value>
</list>
</property>
</bean>
In development, I use the properties file in the classpath.
On the different envionment, staging, integration, production we start the application with the following parameter:
-Dexternal_config=file:c:/staging.properties
That way I have the same build for all environments and can choose the location of my overriding properties file.
So, ${external_config} is a place holder for a configuration file that depends on the environment.
Spring looks for an environment variable named external_config, that variable should point to a configuration file. If the file is not found, Spring ignores it with ignoreResourceNotFound = true;
Spring 3.1 introduced support for environment specific profiles. Define some profiles, associate them with some properties files e.g. test.properties, live.properties and set an environment variable on server start -Dspring.profiles.active=Test. Clean and simple.
http://spring.io/blog/2011/02/11/spring-framework-3-1-m1-released/
Other option as hinted at above is to shift to a container managed Connection Pool looked up via JNDI i.e define a JNDI datasource named xyz the same on each server and then: