I'm fairly new to Java and am currently writing a web application using a number of Dropwizard based micro services and Java8 SE. I now want to integrate a Message Queue for Async communication between the services and want to start by using a cloud based queue like Amazon SQS. However I don't want to lock myself into a particular cloud provider so would like the option of easily switching to another provider or using something like RabbitMQ or ActiveMQ later on. So my question is, is there a framework in Java that allows this? For example with Hibernate I can switch between databases with a simple config change, is there an equivalent for Message queues?
I've spent quite a bit of time researching this but haven't been able to find a definitive answer anywhere, so far I've found;
JMS, Which looks a bit like what I'm after but looks to only be available in the Java EE edition and may require and app server? Is that correct?
AMQP, Which looks like a low level protocol for message queue interoperability. There is also Apache Qpid Proton which looks like a pure AMQP message library but all the documentation and examples for Java seem to use the JMS.
All the tutorials I can find for specific MQs (Rabbit, etc) use those Queues specific client libraries.
Obviously I could add my own abstraction layer but don't want to re-invent the wheel and I suspect I'm not the first developer to want to do something like this.
Just as Hibernate or just JDBC for that matter allows you to switch amongst differing Database providers the JMS API allows you to switch amongst message Brokers or message Providers at will without breaking you code provided you are not using any specific vendor extensions in your code.
JMS is just an API, there is no JMS protocol only the API that various vendors implement and provide you a client to use with their messaging provider. You can use the JMS API from your Java 8 code just fine, you just need to pull in the JMS API jar using whatever build management tool you happen to have chosen along with the client jar from the vendor you happen to be using at that time. To see how to grab an Apache licensed version of the JMS API jar see the answer to this question.
From what I can see Amazon does offer a JMS implementation, the documentation here seems to cover it well.
When or if you decide to switch to another messaging product such as ActiveMQ or RabbitMQ there are JMS implementations offered by each that allow you to swap out the client and not need to change any existing code (again provided you aren't using any vendor extensions). If you switch to a messaging solutiuon that offers AMQP 1.0 support than there is a JMS over AMQP 1.0 implementation offered by the Apache Qpid project here.
I think you need to spend some time reading up on the JMS specification and some tutorials to get a handle on what JMS is and how leveraging JMS and JNDI you can create provider agnostic code.
JMS in the java world is one of the most common API for producing/consumming messages over queues. When using JMS you are free to use any JMS provider (activeMQ, rabbitMQ ...), and if do not make any direct call to your provider (only calling the JMS API) you can switch from one to another easily.
In order for the message to travel from a producer tu a consumer you need a broker ( a software that will handle them). Brokers can be hosted on a dedicated servers or embbeded in your application (I would not recommend the second option).
AMQP is a more recent protocol and is wire-level. Some Brokers are able to handle both AMQP and JMS.
Both AMQP and JMS can provide you with decent abstraction. However they both have their limits. On one hand with JMS you may be tempted to use some feature/fine configuration tuning, and then you may become implementation-depend due to a specific set of behaviors. On the other hand with AMPQ given the AMQP version you've choosen (0.9 or 1.0+), you only may be able to select only a few broker because those versions differs heavily and at the moment most broker only supports one of them.
Check your provider carefully if you're looking for JMS 2.0, some - i.e., ActiveMQ - only support JMS 1.1.
Related
Regarding the question above: Is there a Java library for AMQP 1.0 which fully supports peer-to-peer (point-to-point, brokerless) functionality?
The part that I am missing is the functionality for opening a local port for messages incoming directly without going through a broker.
So far I have used the SwiftMQ AMQP 1.0 Client library which is fine for sending and receiving through a broker (used RabbitMQ server here without any flaws) and also for sending directly to another endpoint.
The other endpoint is written in C#, using the AMQPNetLite library. I found out that the AMQPNetLite lib provided the mentioned functionality through the ContainerHost class (see also here).
My best guess is that the QPID Proton-J somehow supports what I am looking for, but documentation and examples are rare. The Python variant of the QPID Proton library also shows the Container class in this example. However, neither the Container class nor something that seems similar is not contained in the QPID Proton-J (Java) lib.
Yes, I also saw this question, which makes me doubt that what I search for exists at all...
Any help would be greatly appreciated.
There aren't any actively maintained Java peer-to-peer capable implementations that I'm aware of. The proton-j project has a rather old experimental API called reactor that can do that but is not maintained and likely has a fair amount of bugs. The vertx-proton project has a relatively naive server based implementation that might work for you but I don't know how actively maintained it is so you'd have to try it out and see how it goes.
You could write your own although the AMQP protocol has many complexities so you need to weigh how important peer-to-peer is vs something like an embedded broker or simple message router. There is some basic AMQP server code here that be a start on writing your own proton-j based mechanism.
Don’t mess with the AMQP 1.0 protocol. It‘s just too complicated. The SwiftMQ AMQP 1.0 Client works flawlessly because everybody used it for testing their implementation.
An embedded broker that you launch intravm is what you need. Which one you use doesn’t matter but it should be Java I guess. So go for Artemis or even SwiftMQ CE which is open source as well.
Then connect your client to the local embedded broker and configure it to open the AMQP port.
I am using rabbitmq, and can perform all the required functions like message routing according to bindings.
I have never used apache camel but have used rabbitmq.
My Question is what additional features with camel-rabbitmq provide that are not provided by rabbitmq alone.
Camel is an implementation of enterprise integration patterns whereas RabbitMQ is a messaging architecture.
Camel provides an abstraction so that endpoints are swappable with a minimum of fuss. If you were to use camel-rabitmq you could easily swap to jms for instance, or perhaps write to a file instead (or aswell).
You can also add routers, filters etc etc to your channels.
I'm new to JMS and now trying to integrate it with my applications. I've decided to use apachemq jars at the client side.
So, if I avoid the communication through the message broker ApacheMQ and communicate between the parts of the application directly, will it make a sense? Is the main benefit of the JMS API lost in that case?
Couldn't someone explain it in a nutshell?
JMS is a API specification that describes how applications can send and receive messages through messaging middle-ware in a standard way and is portable across multiple JMS providers. Without a middle-ware (or broker), just the JMS APIs is of no use. So you need both, broker and JMS API implementation provided by that broker.
In one of our recent projects, we have decided to use a message based solution to integrate an existing .Net app to a new JAVA based application. There is a requirement to not use a Application Server. So I am trying to look into some alternative options. Currently we are planning to use ActiveMQ as the JMS provider. Also the Java application needs to interact with an backend database. So, when trying to figure out options, I found that the Spring framework would be quite helpful. It appears that Spring may need some additional components (e.g, Atomikos) to provide transaction support for JMS & JDBC operations.
I am trying to limit the user of additional components as much as possible.
So, Is this the right approach to use ActiveMQ with Spring. If so, can I just use the built-in transaction module available in Spring to support transactions?
Any advice would be very helpful?
Leo
Use WebSphere MQ .NET interface (.NET native or XMS .NET) for sending/receiving messages from your .NET application. At the Java app end use MQ JMS interface. MQ JMS interface supports two phase commit, so you can synchronize your message gets and database updates.
Keep the integration through MQ and WS. More tight coupling may lead to problems and complexity.
I'd go with a messagequeue that has good .NET and Java clients - e.g. RabbitMQ That builds on AMQP. As well as serialization technology that has good cross-language capabilities (Thrift, Protocol-buffers etc.)
I have an application in C++, but it'll need to 'talk' to Java based message-service. In the past we used WebSphere MQ and used their C++ libraries to do the 'talking'.
So I am in search of (ideally) free C++ to Java solution which doesn't hold the whole JVM in memory.
The other option I've looked into is SOAP.
I've looked into Axis2-C but it gives me the whole server implementation, which I don't need.
I've seen talk about gSOAP but saw mixed comments here. And again it seems to be providing me with a whole server.
I could write the code myself - but it goes against my (Java based) belief that excellent free code exists out there.
Thanks!
A'z
There are a couple of points here which don't make sense to me, JMS is a java specific abstraction over a generic messaging API, much the same way that JDBC is a java specific abstraction over a generic database API.
I can't imagine anyone wanting a JDBC driver for a C++ application, they would rather use an ODBC driver.
So if I assume that your objective is to send messages without using Websphere MQ from a C++ application, then I can recommend that you consider the following:
Do you need asynchronous messaging? i.e. store message on queue until message is consumed?
If yes, then web-services will not work for your application, unless you are prepared to host a web-server to receive the responses, and call back to your application.
You haven't mentioned whether the underlying java based message service is going to be JMS or WebServices.
You could consider using ActiveMQ as a messaging provider, it provides an implementation of the JMS API and also implements the STOMP protocol, which has client libraries for a number of languages including C++.
You could leave open your decision for end-point protocols while you try out various options, by implementing an integration layer using something like Mule.
You can quickly develop small integrations on Mule e.g. to accept a Message on ActiveMQ, and post it to a WebService and put the WebService response on a different ActiveMQ response queue. Or vice-versa, accept web-service call and post SOAP onto JMS queue, wait for JMS response and build SOAP response.
There are many ESB-like frameworks which can facilitate these sort of integrations to various degrees :
Mule
ServiceMix
Fuse
Apache Camel
Spring Integration
JBoss ESB
EDIT:
Given the clarification I will refine my answer:
You need a common message broker that is accessible to C++ and Java such as ActiveMQ.
You need a small / lightweight integration layer such as some of those above to accept from ActiveMQ and forward to SonicMQ, and vice-versa.
2.1 From what I know of Sonic, they have an ESB stack that should be able to do this instead of using one of the containers/frameworks mentioned above, but that will open up issues of integration ownership between you and the client.
I found 3 links showing Sonic support for C and C++ :
- http://www.sonicsoftware.com/products/docs/sonicmq_app_server_ds.pdf
- http://www.sonicsoftware.com/developer/documentation/docs/sonicmq_c_v60.pdf
- http://communities.progress.com/pcom/servlet/JiveServlet/download/10809-3-10161/cclients_readme_76.htm (dodgy mime type on this link)
The SonicMQ site seems to indicate that they support C++.
It would appear that this is suitable.
There are C++ libraries for SonicMQ. The main constraint is that you have to be using the same build (ie. STL libraries, etc) that Sonic used to compile the libraries.
http://web.progress.com/en/sonic/sonicmq-clients.html
Of course you asked this question over half a year ago, so this information is probably a bit late. :-)