I'm currently developing a Java DSL route, which will fetch a message from a JMS queue, process it, and put it in a database using JPA. Fairly simple really:
public void configure() {
from("{{ribMessage.source}}")
.split(xpath("/RibMessages/*"))
.streaming()
.process(new RibMessageToEntityProcessor())
.to("{{ribMessage.destination}}");
}
As you can see, I'm trying to use camel properties here, which I have defined in a properties file on my classpath:
ribMessage.source=activemq:queue:in.item_q
ribMessage.destination=jpa:com.axstores.aim.entities.XMLImport
The properties file is defined in spring like so:
<bean id="properties" class="org.apache.camel.component.properties.PropertiesComponent">
<property name="location" value="classpath:ribmessage.properties"/>
</bean>
I've updated the activemq config to include a camel.xml, and I've added my route package to the ActiveMQ camel.xml. And during startup, ActiveMQ is finding my route, but then it looks like it isn't able to find my property, and looks up URI {{ribMessage.source}} instead. This fails of course, and the next line in the log says that Camel is shutting down.
2014-02-01 23:31:20,278 | DEBUG | Searching for implementations of org.apache.camel.RoutesBuilder in packages: [com.axstores.aim] | org.apache.camel.impl.DefaultPackageScanClassResolver | main
2014-02-01 23:31:20,374 | DEBUG | Found: [class com.axstores.aim.routes.RibMessageToAimRoute] | org.apache.camel.impl.DefaultPackageScanClassResolver | main
...
2014-02-01 23:46:52,410 | TRACE | Starting service | org.apache.camel.support.ServiceSupport | main
2014-02-01 23:46:52,414 | TRACE | Getting endpoint with uri: {{ribMessage.source}} | org.apache.camel.spring.SpringCamelContext | main
2014-02-01 23:46:52,416 | INFO | Apache Camel 2.12.1 (CamelContext: camel) is shutting down | org.apache.camel.spring.SpringCamelContext | main
I suspect that I'm missing something in my config, because it seems to me like my spring config file is not being read at all.
Any clues? More info needed?
Full log for reference
Spring config
You need to add the camel-jpa component JAR to the classpath of the ActiveMQ broker. And as well what additional JARs you may need for JPA such as the JPA implementation and JDBC drivers and whatnot.
So with some help from the Claus and Peter gave, and some trial and error, I was finally able to get going ActiveMQ up and running, with my routes.
There was one major thing I missed in my config, which basically screwed up everything else. Active MQ was able to find my route through the config, but it still didn't seem to load my spring config.
I finally figured out that I had to add the following import statement to my ActiveMQ camel.xml, in order for it to load the spring config from my jar file.
<import resource="classpath:META-INF/spring/camel-context.xml"/>
Adding this allowed ActiveMQ to resolve my ribmessage.properties, as Peter mentioned in a comment above, this was not being done at all initially. This also loaded my JPA config, which revealed a bunch of missing jars, as Claus pointed out in the comment above.
For this project I'm using EclipseLink and Oracle 11g, so besides camel-jpa, I also had to add the follwing jars to the ActiveMQ classpath:
ojdbc7-12.1.0.1.jar
eclipselink-2.5.1.jar
spring-orm-3.2.4.RELEASE.jar
persistence-api-1.0.jar
spring-jdbc-3.2.4.RELEASE.jar
Hopefully someone can benefit from my (in)experiences.
I really only discovered Camel one week ago, and it seems to be an awesome framework for building integrations :-)
Related
I'm deploying monitoring solution with Decanter Karaf and i've installed prometheus features: collectors and appenders. And i have java errors
ERROR | tures-3-thread-1 | PrometheusCollector | 581 - org.apache.karaf.decanter.collector.prometheus - 2.9.0 | bundle org.apache.karaf.decanter.collector.prometheus:2.9.0 (581)[org.apache.karaf.decanter.collector.prometheus.PrometheusCollector(30)] : The activate method has thrown an exception
java.lang.IllegalArgumentException: prometheus.url is mandatory in the configuration and the bundle org.apache.karaf.decanter.collector.prometheus is always in waiting state.
I added the prometheus url in the cfg file etc/org.apache.karaf.decanter.collector.prometheus.cfg but i have always the same problem.
According to the documentation that is the correct configuration.
Based on this little info I can only advise to debug, you might have made a simple configuration mistake.
I'm running some Camel 2.12.2 routes deployed inside ActiveMQ 5.9.0.
The ActiveMQ log4j is working as expected, but I'm not able to get a log for my camel application when deployed inside ActiveMQ. My Camel log4j.properties looks like this:
#
# The logging properties used
#
log4j.rootLogger=INFO, console, logger
# uncomment the following line to turn on Camel debugging
#log4j.logger.org.apache.camel=DEBUG
# Console appender
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%5p | %m%n
log4j.appender.console.threshold=INFO
# File appender
log4j.appender.logger=org.apache.log4j.RollingFileAppender
log4j.appender.logger.file=camel.log
log4j.appender.logger.maxFileSize=1024KB
log4j.appender.logger.maxBackupIndex=5
log4j.appender.logger.append=true
log4j.appender.logger.layout=org.apache.log4j.PatternLayout
log4j.appender.logger.layout.ConversionPattern=%d | %-5p | %m | %c | %t%n
My camel project is using JavaDSL with SpringRouteBuilders, and when running the application with the maven camel plugin and running with
maven camel:run
the camel.log file is produced as expeceted.
However, when packaging the camel routes and moving the jar to ActiveMQ, the log file is no longer created. I've checked the jars for slf4j, log4j and slf4j-log4j, and the versions are the same in ActiveMQ as in my pom.xml.
Any clues as to what I'm missing here?
EDIT: Just to clarify what I'm after here.
I've experienced issues on more than one occasaion where ActiveMQ is shutting down because there is something wrong with my camel routes, but I don't see any exceptions in the activemq logs, which makes debugging extremely time consuming. Today I had such an issue, and realised that while there was still no trace of error in the log, I got a stacktrace when just doing maven camel:run on my project. A simple example (really just an example, I know why this is happening, but I want it logged when running in ActiveMQ aswell!)
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) is starting
INFO | StreamCaching is enabled on CamelContext: camel-1
INFO | JMX is enabled
INFO | Using EntityManagerFactory configured: org.springframework.orm.jpa.LocalEntityManagerFactoryBean#2127ee90
INFO | Using TransactionManager found in registry with id [transactionTemplate] org.springframework.orm.jpa.JpaTransactionManager#449f40f1
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) is shutting down
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) uptime 0.477 seconds
INFO | Apache Camel 2.12.2 (CamelContext: camel-1) is shutdown in 0.010 seconds
When deployed in ActiveMQ, this is the last line I see from Camel
But when running maven camel:run, I'm also seeing this:
INFO | Apache Camel 2.12.2 stopping
[ERROR] *************************************
[ERROR] Error occurred while running main from: org.apache.camel.spring.Main
[ERROR]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.camel.maven.RunMojo$1.run(RunMojo.java:486)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.camel.RuntimeCamelException: org.apache.camel.FailedToStartRouteException: Failed to start route XPriceChangeToRibRoute because of duplicate id detected: XPriceChangeToRibRoute. Please correct ids to be unique among all your routes.
at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1352)
...
So, what I really want is for the last part to be visible also when deployed in ActiveMQ, either in my activemq.log if possible, or as a separate log, e.g. camel.log.
Have a look at wrapper.log. That's where the STDOUT and STDERR should be found. Try to start ActiveMQ with the wrapper, e.g. for MacOS you could use:
apache-activemq-5.9.0/bin/macosx/activemq
Or if that's not an option, redirect STDOUT/STDERR to a log file.
I'm having a problem with the following setup:
Neo4j 1.9.4 database with REST service
Web-app connecting to the said REST service, built on top of Spring Data Neo4j 2.3.1.RELEASE. Other technologies are not noteworthy, worth to note that the transitive dependency of SDN is at 1.9. See below.
The said web-app is deployed to WebLogic for production, Tomcat used for local development
Excerpt from Maven dependency tree
[INFO] +- org.springframework.data:spring-data-neo4j-rest:jar:2.3.1.RELEASE:compile
[INFO] | +- org.neo4j:neo4j-rest-graphdb:jar:1.9:compile
[INFO] | | \- org.neo4j:server-api:jar:1.9:compile
[INFO] | | +- org.neo4j.3rdparty.javax.ws.rs:jsr311-api:jar:1.1.2.r612:compile
[INFO] | | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | | \- commons-digester:commons-digester:jar:1.8.1:compile
[INFO] | | \- commons-beanutils:commons-beanutils:jar:1.8.0:compile
The problem is: SDN's save API doesn't work when deployed to WebLogic. The web-app still has no problem when trying to retrieve any #NodeEntity. But when the app tries to do a save (list of APIs I've tried, below), it always throws an error.
What I've tried (both yield the same result, error):
repository.save(U entity);
neo4jTemplate.save(T entity);
The weird things that I noticed are:
When my app connects to Neo4j server via GraphDatabaseFactory.databaseFor(url) it works, I can operate a save on a Node (#NodeEntity of SDN still doesn't work). When connected via new SpringRestGraphDatabase(restUri) both Node and #NodeEntity doesn't work. Both APIs are returning descendants of org.neo4j.graphdb.GraphDatabaseService
This is easily replicable even in local environment. I had assumed it to be a network problem, but when I tried to have both the Neo4j server and WebLogic server in my local, the problem still exhibits itself.
I had thought this is a transactional problem since retrieval is has no problem, but as mentioned by Michael Hunger here, there is no real transactions in Neo4j Server. So it shouldn't cause the problem.
Stacktrace:
]] Root cause of ServletException.
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.api.client.ClientResponse.close(ClientResponse.java:603)
at org.neo4j.rest.graphdb.RequestResult.extractFrom(RequestResult.java:83)
at org.neo4j.rest.graphdb.ExecutingRestRequest.put(ExecutingRestRequest.java:151)
at org.neo4j.rest.graphdb.ExecutingRestAPI.setPropertyOnEntity(ExecutingRestAPI.java:340)
at org.neo4j.rest.graphdb.RestAPIFacade.setPropertyOnEntity(RestAPIFacade.java:135)
Truncated. see log file for complete stacktrace
Caused By: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at weblogic.net.http.KeepAliveStream.close(KeepAliveStream.java:115)
Truncated. see log file for complete stacktrace
>
Everything works fine in Tomcat (both REST and Embedded).
Everything works fine in Embedded db (even in WebLogic).
Any help is greatly appreciated.
That exception throw in com.sun.jersey.api.client.ClientResponse
/**
* Close the response.
* <p>
* The entity input stream is closed.
*
* #throws ClientHandlerException if there is an error closing the response.
*/
public void close() throws ClientHandlerException {
try {
entity.close();
} catch (IOException e) {
throw new ClientHandlerException(e);
}
}
The InputStream implementation in Weblogic 12 by this time is weblogic.net.http.KeepAliveStream
The code 204 return No Content then weblogic mark it as not complete downloading the content.
The solution here is use -DUseSunHttpHandler=true when start your weblogic (bin/startWebLogic.cmd)
set JAVA_OPTIONS=%JAVA_OPTIONS% -DUseSunHttpHandler=true
Regards,
Tung
Below is the error from the log that i see after starting the service... can you help resolving this issue.
2013-08-22 10:35:37,111 | DEBUG | l Console Thread | AbstractServiceReferenceRecipe | r.AbstractServiceReferenceRecipe 143 | 7 - org.apache.aries.blueprint.core - 1.1.0 | Found initial references null for OSGi service (&(&(org.apache.aries.jpa.proxy.factory=true)(osgi.unit.name=tenant))(objectClass=javax.persistence.EntityManagerFactory))
2013-08-22 10:35:37,111 | DEBUG | l Console Thread | BlueprintContainerImpl | container.BlueprintContainerImpl 280 | 7 - org.apache.aries.blueprint.core - 1.1.0 | Running blueprint container for bundle com.igt.arcus.framework.jta.arcus-framework-feature-service in state WaitForInitialReferences
2013-08-22 10:35:37,111 | INFO | l Console Thread | BlueprintContainerImpl | container.BlueprintContainerImpl 344 | 7 - org.apache.aries.blueprint.core - 1.1.0 | Bundle com.igt.arcus.framework.jta.arcus-framework-feature-service is waiting for dependencies [(&(&(org.apache.aries.jpa.proxy.factory=true)(osgi.unit.name=tenant))(objectClass=javax.persistence.EntityManagerFactory))]
2013-08-22 10:35:37,112 | DEBUG | l Console Thread | BlueprintEventDispatcher | ntainer.BlueprintEventDispatcher 136 | 7 - org.apache.aries.blueprint.core - 1.1.0 | Sending blueprint container event BlueprintEvent[type=GRACE_PERIOD, dependencies=[(&(&(org.apache.aries.jpa.proxy.factory=true)(osgi.unit.name=tenant))(objectClass=javax.persistence.EntityManagerFactory))]] for bundle com.igt.arcus.framework.jta.arcus-framework-feature-service
In case you use Aries JPA you should see two EntityManagerFactory services:
jpa-container picks up your bundle with the persistence.xml and creates an Entity manager factory
jpa-container-context picks up every EntityManagerFactory service and makes a new managed EntityManagerFactory service with the (org.apache.aries.jpa.proxy.factory=true) service property key-value pair
The second does not exist in your environment. There can be two causes:
aries-jpa-container-context is not in your OSGi environment or it is not in ACTIVE state
The original EntityManagerFactory service is not registered as some dependency is missing
You can check it if you open an OSGi console and check what services available. If there is no EntityManagerFactory service at all, the second is your problem.
Check if all of your bundles are active! If they are, check if you have all the services that jpa-container needs: TransactionManager, DataSource or DataSourceFactory and a javax.persistence.spi.PersistenceProvider service for Hibernate. If any of the services are missing aries-jpa-container will pick up your bundle but will never create the EntityManagerFactory.
I implemented an own jpa-container that works almost the same way as aries-jpa-container. If you replace the aries-jpa-container (only that one, the jpa-container-context should be left there) it will log more messages at INFO level for you what is missing. The container is available here. A sample application is available that uses hibernate is available at https://github.com/everit-org/osgi-hibernate. After running "mvn install" you will find itests/core/target/eosgi-itests-dist/equinox a subfolder where you can start the working application with on equinox server with bin/runconsole.sh.
The current aries jpa container jar 1.0.0 has a bug that makes it swallow exceptions on creating the EntityManager. A colleague of me opened the aries bug: https://issues.apache.org/jira/browse/ARIES-1160
I created a second patch that can be applied to the 1.0.0 sources to create a jar that correctly logs the exception. Check out the 1.0.0 tag of aries jpa container, apply the patch and build.
When you then run in karaf you should see the real source of the error.
here is the current logging which I am seeing in Server:
12:40:10,190 | INFO | -549263035-19951 | ServiceImpl | Upload started
12:40:12,912 | INFO | -549263035-19960 | ServiceImpl | Upload started
12:40:12,915 | INFO | -549263035-19958 | ServiceImpl | Upload started
My application is using log4j to implement logging. here is the log4j.properties file which is sitting inside META-INF.
log4j.rootCategory=DEBUG, O
log4j.appender.O=org.apache.log4j.ConsoleAppender
log4j.appender.O.layout=org.apache.log4j.PatternLayout
log4j.appender.O.layout.ConversionPattern=[%d{ISO8601}]%5p%6.6r[%t]%x - %C.%M(%F:%L) - %m%n
As we can clearly see, current logging has the time information for every event. I wanted to know if there is a way I could make some changes in this properties file or elsewhere to include the full timestamp i.e. Date and time both.
something like :
2013-07-18 12:40:12 | INFO | -549263035-19958 | ServiceImpl | Upload started
Thanks for suggestion.
Found my answer.
this application was deployed in Servicemix container. Apparantly inside servicemix's installation directory I found this configuration file.
"/usr/local/servicemix/etc/org.ops4j.pax.logging.cfg".
Inside this file, there was a SiftingAppender which was using %d[ABSOLUTE] as a conversion pattern. So basically every bundle which is deployed under servicemix is going to use this "siftingappender" properties. I just removed [ABSOLUTE] from this configuration file and started getting the Date in my logs.