I need to enhance the JMX interfaces of a distributed java application. The reason for choosing JMX is the simplicity of exposing data. These distributed apps exist in several different machines connected to a JMS server (activemq 5.7) (which in turn is connected to another JMS server to bridge 2 networks, also activemq 5.7).
What I would like to do, is to be able to access the remote JMX interfaces on the individual servers from anywhere on the JMS network. I would nee full JMX access as if accessing through the usual RMI interface. That means every type of action.
I understand I could use lingo to make the remote jmx interfaces talk to the JMS server, and from there my bridge should allow me access to them (assuming its configured correctly).
Is this a good approach? has anyone tried lingo for this purpose? Are there other options out there I may have not found?
A plan B could be to use apache camel RMI module, but it seems like if lingo is an option, it will be much more plug & play than this.
I think it's not a bad way to go. The one downside of using JMS that I can think of, off the top of my head, is the dependency on a broker, which most JMS implementations rely on.
On the other hand, it does present some interesting capabilities like discovery, asynchronous JMX invocation and pub/sub multicast style JMX operations where you could issue one operation request and receive back a response from all your MBeanServers.
I am not aware of any actual implementations, but it's probably not too difficult to implement. You simply need a configured client on each target JVM that will:
Listen for JMX requests: The listener will unmarshall the request (which should be an encoding of an MBeanServerConnection method invocation). Use a common topic for pub/sub style invocations, returning the marshalled result
to the destination specified in the JMSReplyTo property in the request message. Otherwise, you could allocate a queue per JVM, or pick a unique identifier for each JVM and use message selectors.
If you want to implement JMX notifications, you will need to implement a proxy NotificationListener that registers for the desired notifications and forwards them to the designated JMS destination on receipt.
You may also consider implementing a full blown javax.management.remote implementation which may integrate more smoothly into your environment by virtue of the standard adherence.
I have found the OpenDMK project very useful for extending/implementing JMX servers and clients. The library provides the basic building blocks for implementing a standard JMX remoting solution using a "custom" protocol. Basically, you implement a javax.management.remote.generic.MessageConnection which serves as the transport and invocation mechanism. All JMX invocations, responses and callbacks are serialized
into instances of javax.management.remote.message.Message, and they're all Serializable so you should not have any issues writing them into and reading them from JMS ObjectMessages.
A couple of additional benefits you will get from this approach are:
Provided you configiure the classpath correctly, you should be able to connect to your JVMs using any standard JMX tool such as JConsole.
The OpenDMK also provides the ability to federate MBeanServers which makes all your MBeanServer instances appear, and be accessible through one central MBeanServer. This feature requires a standard JMX remoting
implementation.
The OpenDMK also implements an interesting service discovery protocol, and it comes in a couple of different flavours including raw multicast and a "phone-home" approach which would mesh nicely with your JMS protocol.
I posted a mavenised project of the OpenDMK here if you're interested.
I am implementing a basic JMX client for java-agents using netty, and it optionally supports asynchronous JMX requests. Responses are delivered through a registered listener which is like a "reverse" MBeanServerConnection. In case this is useful, find the source here.
Jolokia (http://www.jolokia.org/) is also a great project for remote accessing JMX using REST and JSON. It does this automatic for you. And support batch operations as well.
I suggest to take a look at that.
If you use JMX to get AMQ statistics then it offers a plugin, so you can just using JMS messaging to get the stats instead of JMX: http://activemq.apache.org/statisticsplugin.html
I would use JMS to discover the URLs of the servers I am interested in and use plain JMX from then on. I don't see the advantage of sending every RMI call over JMS.
Related
I am currently testing out JMS queue (first time using JMS) and message driven beans.
I have created a queue to provide other applications with state updates for one of our projects.
Logic is written in native JMS, deployed on a JBOSS7 using the ActiveMQ implementation.
It depends on a selector to deliver the messages to the right client, and while I can just place good faith in my colleagues, preferably I would like to enforce the use of the selector so the clients don't consume messages not meant for them.
So basically I would prefer that no messages are delivered to a client which has not specified a selector.
When I deploy a consumer without any selector it just consumes all messages available on the queue.
Otherwise everything works as expected.
I have looked and haven't been able to find anything I am looking for, maybe it's possible by configuring ActiveMQ itself but I am not really at home in that ecosystem.
So the problem is resolved by using a system I wasn't that aware about and I thought I'd share it here if someone need it:
The JMS clients are on different physical machines so originally the plan was to do manual JNDI remote lookup to access the queue but this caused some problems. Mainly having to write retry logic when the Queue is unavailable.
I threw that plan out the window to opt for a Bridge instead, following the guide found here: http://www.mastertheboss.com/howto/jboss-jms6/configuring-jms-bridge-with-wildfly-10
This has multiple advantages (both the producer system or the client can go down without causing too many problems) but most notably this solves my problem: I can define a selector on the bridge per Client. So the responsibility of choosing who receives which messages is back in my court.
I will have a crack at implementing this.
I'm fairly new to Java and am currently writing a web application using a number of Dropwizard based micro services and Java8 SE. I now want to integrate a Message Queue for Async communication between the services and want to start by using a cloud based queue like Amazon SQS. However I don't want to lock myself into a particular cloud provider so would like the option of easily switching to another provider or using something like RabbitMQ or ActiveMQ later on. So my question is, is there a framework in Java that allows this? For example with Hibernate I can switch between databases with a simple config change, is there an equivalent for Message queues?
I've spent quite a bit of time researching this but haven't been able to find a definitive answer anywhere, so far I've found;
JMS, Which looks a bit like what I'm after but looks to only be available in the Java EE edition and may require and app server? Is that correct?
AMQP, Which looks like a low level protocol for message queue interoperability. There is also Apache Qpid Proton which looks like a pure AMQP message library but all the documentation and examples for Java seem to use the JMS.
All the tutorials I can find for specific MQs (Rabbit, etc) use those Queues specific client libraries.
Obviously I could add my own abstraction layer but don't want to re-invent the wheel and I suspect I'm not the first developer to want to do something like this.
Just as Hibernate or just JDBC for that matter allows you to switch amongst differing Database providers the JMS API allows you to switch amongst message Brokers or message Providers at will without breaking you code provided you are not using any specific vendor extensions in your code.
JMS is just an API, there is no JMS protocol only the API that various vendors implement and provide you a client to use with their messaging provider. You can use the JMS API from your Java 8 code just fine, you just need to pull in the JMS API jar using whatever build management tool you happen to have chosen along with the client jar from the vendor you happen to be using at that time. To see how to grab an Apache licensed version of the JMS API jar see the answer to this question.
From what I can see Amazon does offer a JMS implementation, the documentation here seems to cover it well.
When or if you decide to switch to another messaging product such as ActiveMQ or RabbitMQ there are JMS implementations offered by each that allow you to swap out the client and not need to change any existing code (again provided you aren't using any vendor extensions). If you switch to a messaging solutiuon that offers AMQP 1.0 support than there is a JMS over AMQP 1.0 implementation offered by the Apache Qpid project here.
I think you need to spend some time reading up on the JMS specification and some tutorials to get a handle on what JMS is and how leveraging JMS and JNDI you can create provider agnostic code.
JMS in the java world is one of the most common API for producing/consumming messages over queues. When using JMS you are free to use any JMS provider (activeMQ, rabbitMQ ...), and if do not make any direct call to your provider (only calling the JMS API) you can switch from one to another easily.
In order for the message to travel from a producer tu a consumer you need a broker ( a software that will handle them). Brokers can be hosted on a dedicated servers or embbeded in your application (I would not recommend the second option).
AMQP is a more recent protocol and is wire-level. Some Brokers are able to handle both AMQP and JMS.
Both AMQP and JMS can provide you with decent abstraction. However they both have their limits. On one hand with JMS you may be tempted to use some feature/fine configuration tuning, and then you may become implementation-depend due to a specific set of behaviors. On the other hand with AMPQ given the AMQP version you've choosen (0.9 or 1.0+), you only may be able to select only a few broker because those versions differs heavily and at the moment most broker only supports one of them.
Check your provider carefully if you're looking for JMS 2.0, some - i.e., ActiveMQ - only support JMS 1.1.
I am working with several different services which have been written in Python and Java. I have now come to the point where I need to have these services communicate with each other and ActiveMQ seems to be a sensible choice.
However, I am not sure I quite follow the concept of the different protocols available for ActiveMQ. The main question being, do I have to implement all my services to use the same protocol for accessing the bus?
Is it possible to run the same bus with multiple different connection protocols concurrently?
If yes to 1. Is it just connection to the bus which is governed by a Protocol or does the protocol choice also influence the type of the actual message content. I.e. will it be possible to post a message from Java JMS to a topic and have a Python STOMP client read that message correctly?
Inter language interopability can be made by different clients using the same protocol, such as JMS for Java, CMS for C++ and NMS for .NET (when it comes to OpenWire). I don't think there is a great OpenWire Python client out there, except some CMS wrappers.
On the other hand, ActiveMQ does a great job to bridge between different protocols so that they can share the same queues/topics. It works really well to read stomp, MQTT and AMQP messages sent from OpenWire/JMS. Please be aware that you may want to avoid advanced features of say JMS like MapMessage, rely on specific JMS headers and stick to simple text messages or byte messages. The bridge is not 100% transparent, but as I said, does a great job.
So, you can go with stomp or even AMQP 1.0, some instruction from Microsoft here.
I am currently evaluating JMS and I don't get what I could use it for.
Currently, I believe this would be a Usecase: I want to create a SalesInvoice PDF and print it when an SalesOrder leaves the Warehouse, so during the Delivery transaction I could send a transactional print request which just begins when the SalesOrder transaction completes successfully.
Now I found out most JMS products are standalone server.
Why would a need a Standalone Server for Message Processing, vs. e.g. some simple inproc processing with Quartz scheduler?
How does it interact with my application?
Isn't it much too slow?
What are Usecases you already implemented successfully?
JMS is an amazingly useful system, but not for every purpose.
It's essentially a high-level framework for sending messages between nodes, with options for discovery, robustness, etc.
One useful use case is when you want a client and a server to talk to one another, but without the client actually having the server's address (E.g., you may have more than one server). The client only needs to know the broker and the queue/topic name, and the server can connect as well.
JMS also adds robustness. For instance, you can configure it so that if the server dies while the client sends messages or the other way around, you can still send messages from the client or poll messages from the server. If you ever tried implementing this directly with sockets - it's a nightmare.
The scenario you describe sounds like a classic J2EE problem, why are you not using a J2EE framework? JMS is often used inside J2EE for communications, but you got all the other benefits.
What ist Java Message Service (JMS) for
JMS is a messaging standard that allows Java EE applications to create, send, receive, and consume messages in a loosely coupled, reliable, and asynchronous way. I'd suggest to read the Java Message Service API Overview for more details.
Why would a need a Standalone Server for Message Processing, vs. e.g. some simple inproc processing with Quartz scheduler?
Sure, in your case, Quartz is an option. But what if the invoice system is a remote system? What if you don't want to wait for the answer? What if the remote system is down when you want to communicate with it? What if the network is not always available? This is where JMS comes in. JMS allows to send a message guaranteed to be delivered and to consume it in a transactional way (sending or consuming a message can be part of a global transaction).
How does it interact with my application?
JMS supports two communication modes: point-to-point and publish/subscribe (if this answers the question).
Isn't it much too slow?
The MOMs I've been working with were blazing fast.
What are Usecases you already implemented successfully?
Used in system such as a reservation application, a banking back-office (processing market data), or more simply to send emails.
See also
EJB Message-Driven Beans
Why would a need a Standalone Server
for Message Processing, vs. e.g. some
simple inproc processing with Quartz
scheduler?
The strength of JMS lies in the fact that you can have multiple producers and multiple consumers for the same queue, and the JMS broker manages the load.
If you have multiple producers but a single consumer, you can use other approaches as well, such as a quartz scheduler and a database table. But as soon as you have multiple consumer, the locking scheme become very hard to design; better go for already approved messaging solution. See these other answers from me for a few more details: Why choosing JMS for asynchronous solution ? and Producer/consumer system using database
The other points are just too vague to be answered.
I've used it on a number of projects. It can help with scalability, decoupling of services, high availability. Here's a description of how I used it on a project several years ago:
http://coders-log.blogspot.com/2008/12/favorite-projects-series-installment-2.html
The description explains what JMS brought to the table for this particular project, but other projects will use messaging systems for a variety of reasons.
Messaging is usually used to interconnect different systems and send requests/commands asynchronously. A common example is a bank client application requesting an approval for a transaction. The server is located in another bank's system. Both systems are connected in an Enterprise Service Bus. The request goes into the messaging bus, which instantly acknowledges the reception of the message. The client can go on with processing. Whenever the server system becomes available, the bus forwards the message to it. Of course there needs to be a second path, for the server to inform the client that the transaction executed successfully or failed. This again can be implemented with JMS.
Please note that the two systems need not to implement JMS. One can use JMS and the other one MSMQ. The bus will take care of the interconnection.
JMS is a message-oriented middleware.
Why would a need a Standalone Server for Message Processing, vs. e.g. some simple inproc processing with Quartz scheduler?
It depends on what other components you may have. I guess. But I don't know anything about Quartz
How does it interact with my application?
You send messages to the broker.
Isn't it much too slow?
Compare to what ?
What are Usecases you already implemented successfully?
I've used JMS to implement a SIP application server, to communicate between the various components.
From the Javadoc:
The Java Message Service (JMS) API provides a common way for Java programs to create, send, receive and read an enterprise messaging system's messages.
In other words, and contrary to every other answer here, JMS is nothing more than an API, which wraps access to third-party Message Brokers, via 'JMS Providers' implemented by the vendor. Those Message Brokers, such as IBM MQ and dozens of others, have the features of reliability, asynchronicity, etc. that have been mentioned in other answers. JMS itself provides exactly none of them. It is to Message Brokers what JDBC is to SQL databases, or JNDI is to LDAP servers (among other things).
I have found a very good explanation of JMS with an example.
That is a simple chat application with JMS queues are used to communicate messages between users and messages stay in the queue if the receiver is offline.
In this example implementation they have used
XSD to generate domain classes.
Eclipse EE as IDE.
JBoss as web/application server.
HTML/JavaScript/JQuery for UI.
Servlet as controller.
MySQL as DB.
The JBoss configuration step for queue is explained nicely
Its available at http://coder2design.com/messaging-service/
Even the downloadable code is also available there.
I'm in the process of writing a client/server application which should work message based. I would like re-use as much as possible instead of writing another implementation and curious what others are using.
Features the library should offer:
client and server side functionality
should work message based
support multi-threading
should work behind load balancer / firewalls
I did several tests with HTTPCore, but the bottom line is that one has to implement both client and server, only the transport layer would be covered. RMI is not an option either due to the network related requirements.
Any ideas are highly appreciated.
Details
My idea is to implement a client/server wrapper which handles the client communication (including user/password validation) and writes incoming requests to a JMS queue:
#1 User --> Wrapper (Check for user/password) --> JMS --> "Server"
#2 User polls Wrapper which polls JMS
Separate processes will handle the requests and can reply via wrapper to the clients. I'd like to use JMS because:
it handles persistence quite well
load balancing - it's easy to handle peaks by adding additional servers as consumer
JMSTimeToLive comes in handy too
Unfortunately I don't see a way to use JMS on it's own, because clients should only have access to their messages and the setup of different users on JMS side doesn't sound feasible either.
Well, HTTP is probably the best supported in terms of client and server code implementing it - but it may well be completely inappropriate based on your requirements. We'll need to actually see some requirements (or at least a vague idea of what the application is like) before we can really advise you properly.
RMI works nicely for us. There are limitations, such as not being able to call back to the client unless you can connect directly to that computer (does not work if client is behind a firewall). You can also easily wrap your communication in SSL or tunnel it over HTTP which can be wrapped in SSL.
If you do end up using this remember to always set the serial version of a class that is distributed to the client. You can set it to 1L when you create it, or if the client already has the class use serialver.exe to discover the existing class's serial. Otherwise as soon as you change or add a public method or variable compatibility with existing clients will break.
static final long serialVersionUID = 1L
EDIT: Each RMI request that comes into the server gets its own thread. You don't have to handle this yourself.
EDIT: I think some details were added later in the question. You can tunnel RMI over HTTP, then you could use a load balancer with it.
I've recently started playing with Hessian and it shows a lot of promise. It natively uses HTTP which makes it simpler than RMI over HTTP and it's a binary protocol which means it's faster than all the XML-based protocols. It's very easy to get Hessian going. I recently did this by embedding Jetty in our app, configuring the Hessian Servlet and making it implement our API interface. The great thing about Hessian is it's simplicity... nothing like JMS or RMI over HTTP. There are also libraries for Hessian in other languages.
I'd say the best-supported, if not best-implemented, client/server communications package for Java is Sun's RMI (Remote Method Invocation). It's included with the standard Java class library, and gets the job done, even if it's not the fastest option out there. And, of course, it's supported by Sun. I implemented a turn-based gaming framework with it several years ago, and it was quite stable.
It is difficult to make a suggestion based on the information given but possibly the use of TemporaryQueues e.g. dynamically created PTP destinations on a per client basis might fit the problem?
Here is a reasonable overview.
Did you tried RMI or CORBA? With both of them you can distribute your logic and create Sessions
Use Spring....Then pick and choose the protocol.
We're standardizing on Adobe's AMF as we're using Adobe Flex/AIR in the client-tier and Java6/Tomcat6/BlazeDS/Spring-Framework2.5/iBATIS2.3.4/ActiveMQ-JMS5.2 in our middle-tier stack (Oracle 10g back-end).
Because we're standardizing on Flex client-side development, AMF and BlazeDS (now better coupled to Spring thanks to Adobe and SpringSource cooperating on the integration), are the most efficient and convenient means we can employ to interact with the server-side.
We also heavily build on JMS messaging in the data center - BlazeDS enables us to bridge our Flex clients as JMS topic subscribers. That is extremely powerful and effective.
Our Flex .swf and Java .class code is bundled into the same .jar file for deployment. That way the correct version of the client code will be deployed to interact with the corresponding middle-tier java code that will process client service calls (or messaging operations). That has always been a bane of client-server computing - making sure the correct versions of the respective tiers are hooked up to each other. We've effectively solved that age-old problem with our particular approach to packaging and deployment.
All of our client-server interactions work over HTTP/HTTPS ports 80 and 443. Even the server-side messaging push we do with BlazeDS bridged to our ActiveMQ JMS message broker.