I'm in the process of writing a client/server application which should work message based. I would like re-use as much as possible instead of writing another implementation and curious what others are using.
Features the library should offer:
client and server side functionality
should work message based
support multi-threading
should work behind load balancer / firewalls
I did several tests with HTTPCore, but the bottom line is that one has to implement both client and server, only the transport layer would be covered. RMI is not an option either due to the network related requirements.
Any ideas are highly appreciated.
Details
My idea is to implement a client/server wrapper which handles the client communication (including user/password validation) and writes incoming requests to a JMS queue:
#1 User --> Wrapper (Check for user/password) --> JMS --> "Server"
#2 User polls Wrapper which polls JMS
Separate processes will handle the requests and can reply via wrapper to the clients. I'd like to use JMS because:
it handles persistence quite well
load balancing - it's easy to handle peaks by adding additional servers as consumer
JMSTimeToLive comes in handy too
Unfortunately I don't see a way to use JMS on it's own, because clients should only have access to their messages and the setup of different users on JMS side doesn't sound feasible either.
Well, HTTP is probably the best supported in terms of client and server code implementing it - but it may well be completely inappropriate based on your requirements. We'll need to actually see some requirements (or at least a vague idea of what the application is like) before we can really advise you properly.
RMI works nicely for us. There are limitations, such as not being able to call back to the client unless you can connect directly to that computer (does not work if client is behind a firewall). You can also easily wrap your communication in SSL or tunnel it over HTTP which can be wrapped in SSL.
If you do end up using this remember to always set the serial version of a class that is distributed to the client. You can set it to 1L when you create it, or if the client already has the class use serialver.exe to discover the existing class's serial. Otherwise as soon as you change or add a public method or variable compatibility with existing clients will break.
static final long serialVersionUID = 1L
EDIT: Each RMI request that comes into the server gets its own thread. You don't have to handle this yourself.
EDIT: I think some details were added later in the question. You can tunnel RMI over HTTP, then you could use a load balancer with it.
I've recently started playing with Hessian and it shows a lot of promise. It natively uses HTTP which makes it simpler than RMI over HTTP and it's a binary protocol which means it's faster than all the XML-based protocols. It's very easy to get Hessian going. I recently did this by embedding Jetty in our app, configuring the Hessian Servlet and making it implement our API interface. The great thing about Hessian is it's simplicity... nothing like JMS or RMI over HTTP. There are also libraries for Hessian in other languages.
I'd say the best-supported, if not best-implemented, client/server communications package for Java is Sun's RMI (Remote Method Invocation). It's included with the standard Java class library, and gets the job done, even if it's not the fastest option out there. And, of course, it's supported by Sun. I implemented a turn-based gaming framework with it several years ago, and it was quite stable.
It is difficult to make a suggestion based on the information given but possibly the use of TemporaryQueues e.g. dynamically created PTP destinations on a per client basis might fit the problem?
Here is a reasonable overview.
Did you tried RMI or CORBA? With both of them you can distribute your logic and create Sessions
Use Spring....Then pick and choose the protocol.
We're standardizing on Adobe's AMF as we're using Adobe Flex/AIR in the client-tier and Java6/Tomcat6/BlazeDS/Spring-Framework2.5/iBATIS2.3.4/ActiveMQ-JMS5.2 in our middle-tier stack (Oracle 10g back-end).
Because we're standardizing on Flex client-side development, AMF and BlazeDS (now better coupled to Spring thanks to Adobe and SpringSource cooperating on the integration), are the most efficient and convenient means we can employ to interact with the server-side.
We also heavily build on JMS messaging in the data center - BlazeDS enables us to bridge our Flex clients as JMS topic subscribers. That is extremely powerful and effective.
Our Flex .swf and Java .class code is bundled into the same .jar file for deployment. That way the correct version of the client code will be deployed to interact with the corresponding middle-tier java code that will process client service calls (or messaging operations). That has always been a bane of client-server computing - making sure the correct versions of the respective tiers are hooked up to each other. We've effectively solved that age-old problem with our particular approach to packaging and deployment.
All of our client-server interactions work over HTTP/HTTPS ports 80 and 443. Even the server-side messaging push we do with BlazeDS bridged to our ActiveMQ JMS message broker.
Related
I'm looking for a full duplex streaming solution with Java EE.
The situation: client applications (JavaFX) read data from a peripheral device. This data needs to be transferred in near real-time to a server for processing and also get the response back asynchronously, all while it keeps sending new data for processing.
Communication with the server needs to have an overhead as low as possible. Data coming in is basically some sensor data and after processing it is turned in what can be described as a set of commands.
What I've looked into:
A TCP/IP server (this is a non-Java EE approach).This would be the obvious solution. Two connections opened in parallel from each client app: one for upstream data and one for downstream data.
Remote & stateless EJBs. This would mean that there's no streaming involved and that I pack sensor data in smaller windows (1-2 seconds worth of sensor data) which I then send to the server for processing and get the processing result as a response. For this approach, while it is scalable, I am not sure how fast it will be considering I have to make a request each 1-2 seconds. I still need to test this but I have my doubts.
RMI. Is this any different than EJBs, technically?
Two servlets (up/down) with long polling. I've not done this before, so it's something to be tested.
For now I would like to test the performance for my approach #2. The first solution will work for sure, but I'm not too fond of having a separate server (next to Tomcat, where I already have something running).
However, meanwhile, it would be worth knowing if there are any other Java specific (EE or not) technologies that could easily solve this. If anyone has an idea, then please share it.
This looks like a good place for using JMS. Instead of stateless EJBs, you will probably be using Message-Driven Beans.
This gives you an approach similar to your first solution, using two message queues instead of TCP/IP connections. JMS makes your communications fully asynchronous and is low-overhead in the sense that your clients can send messages as fast as they can regardless of how fast your server can consume them. You also get delivery guarantees and other JMS goodness.
Tomcat does not come with JMS, however. You might try TomEE or integrate your existing Tomcat with a JMS implementation like ActiveMQ.
There are numerous options you could try. Appropriate solutions depend on the nature of your application, communication protocol, data transfer type, control you have over the client and server and firewall restrictions on client server routes.
There's not much info on this in your question, but given what you have provided, you may like to look at netty as it is quite general purpose and flexible and seems to fit your requirements. Netty also includes a duplex websocket implementation. Note that a netty based solution may be more complex to implement and require more background study than some other solutions (such as jms).
Yet another possible solution in GraniteDS, which advertises a JavaFX client integration and multiple server integrations for full duplex client/server communication, though I have not used it. GraniteDS uses comet (your two asynchronous servlets with long polling model) with the Active Message Format for data which you may be familiar with from Flex/Flash.
Have you looked at websockets as a solution? They are known to keep persistent connections and hence the asynchronous response will be quick.
I'm designing a replacement for a custom messaging system that is currently used to notify a JavaScript web application about changed content from the server side (Java). This legacy messaging system works through Flash XMLSocket by use of custom text based protocol and plain Java sockets.
The replacement will be used not only by the web application (through web sockets instead of Flash) but by an additional desktop client application written in C# too.
My requirements are:
user authentication
transport encryption (SSL/TLS)
bidirectional message exchange
some sort of (automatic) publish/subscribe so that users get only the messages they are allowed to receive
message exchange based on an established protocol (so that we may use existing libraries where possible)
cluster-able server components
At this time the messaging system will only be used to publish updates down to the clients. The clients will react to these messages and acquire further information directly from the server (not via the messaging system). If this new messaging system is established successfully it may be used for more advanced use cases in the future. Some possibilities may include users chatting, file exchange and remote control of the server components.
I did a little research about viable technology to implement these requirements and I think my choices boil down to either using ejabberd (XMPP) or RabbitMQ (AMQP). What are the main pros and cons of these two systems regarding my requirements? We already use RabbitMQ for other parts of our system infrastructure so it was my natural choice. I'm just not sure if it would be a good idea to have client applications connect directly to such an critical main component. This may be mitigated though by just using a different RabbitMQ installation for the client notifications.
Well you could use both protocols to fulfill your needs, xmpp is an extensible protocol so no doubt the things that you're looking for does allready exist as a plugin or whatever is the correct term for a protocol. However some, me included, might actually see this as a downside, and they add extra complexity to the protocol. Another thing to keep in mind is that xmpp was primarily designed to be a instant messaging protocol. For example publish/subscribe is an extension of xmpp and not part of the protocol itself.
That being said xmpp is backed by organisations such as Google, which means that there are some major players using this protocol, so no doubt some of the extensions are really good and well written/thought out.
On the other hand you have AMQP, a protocol almost specifically designed for what you're after. It is backed by such organisations as JP Morgan, Cisco, Credit Suisse etc. so no doubt AMQP is a protocol to be counted on, even though early versions of it has been critized
When it comes to using RabbitMQ there seem to be some issues with memory, however I cannot speak much of that since I was only notified about this issue, and never really got into actually fix it or even understand it. However there seem to be quite a few people experiencing this on different versions of RMQ.
But for me RabbitMQ has never crashed (hey if erlang is known for one thing, it's stability), it has been a joy to setup and it is really easy to setup in a cluster, and you can have your queues easily mirrored on several instances of RMQ and you can have an extra layer of security by having one or more instances write messages to disk.
So I'd say go with RabbitMQ and AMQP, I believe it's a protocol well suited for your needs, but having said that xmpp could probably do the work just fine aswell.
I've read this book, which is a nice introduction to AMQP and RabbitMQ but I find it lacking in the technical aspect, it's basically a good tutorial.
PS: I feel I should be honest and say that I'm not really sure what bidirectional message exchange entails, but if it means sending and receiving messages, you're in the clear on that point also with AMQP. :)
I hope this helped shed some light on which protocol to choose.
Edit
RabbitMQ has something called virtual hosts which acts like an own instance of RabbitMQ so you don't have to start with setting up a cluster just to handle separate responsibilities. Depending on how you set up your queues and exchanges I don't see a problem with clients connecting to the RabbitMQ server, but clustering is without a doubt a good idea. It also seems that setting up RabbitMQ with HAProxy is very easy, but yet again that is something I have no experience of.
I have an application in C++, but it'll need to 'talk' to Java based message-service. In the past we used WebSphere MQ and used their C++ libraries to do the 'talking'.
So I am in search of (ideally) free C++ to Java solution which doesn't hold the whole JVM in memory.
The other option I've looked into is SOAP.
I've looked into Axis2-C but it gives me the whole server implementation, which I don't need.
I've seen talk about gSOAP but saw mixed comments here. And again it seems to be providing me with a whole server.
I could write the code myself - but it goes against my (Java based) belief that excellent free code exists out there.
Thanks!
A'z
There are a couple of points here which don't make sense to me, JMS is a java specific abstraction over a generic messaging API, much the same way that JDBC is a java specific abstraction over a generic database API.
I can't imagine anyone wanting a JDBC driver for a C++ application, they would rather use an ODBC driver.
So if I assume that your objective is to send messages without using Websphere MQ from a C++ application, then I can recommend that you consider the following:
Do you need asynchronous messaging? i.e. store message on queue until message is consumed?
If yes, then web-services will not work for your application, unless you are prepared to host a web-server to receive the responses, and call back to your application.
You haven't mentioned whether the underlying java based message service is going to be JMS or WebServices.
You could consider using ActiveMQ as a messaging provider, it provides an implementation of the JMS API and also implements the STOMP protocol, which has client libraries for a number of languages including C++.
You could leave open your decision for end-point protocols while you try out various options, by implementing an integration layer using something like Mule.
You can quickly develop small integrations on Mule e.g. to accept a Message on ActiveMQ, and post it to a WebService and put the WebService response on a different ActiveMQ response queue. Or vice-versa, accept web-service call and post SOAP onto JMS queue, wait for JMS response and build SOAP response.
There are many ESB-like frameworks which can facilitate these sort of integrations to various degrees :
Mule
ServiceMix
Fuse
Apache Camel
Spring Integration
JBoss ESB
EDIT:
Given the clarification I will refine my answer:
You need a common message broker that is accessible to C++ and Java such as ActiveMQ.
You need a small / lightweight integration layer such as some of those above to accept from ActiveMQ and forward to SonicMQ, and vice-versa.
2.1 From what I know of Sonic, they have an ESB stack that should be able to do this instead of using one of the containers/frameworks mentioned above, but that will open up issues of integration ownership between you and the client.
I found 3 links showing Sonic support for C and C++ :
- http://www.sonicsoftware.com/products/docs/sonicmq_app_server_ds.pdf
- http://www.sonicsoftware.com/developer/documentation/docs/sonicmq_c_v60.pdf
- http://communities.progress.com/pcom/servlet/JiveServlet/download/10809-3-10161/cclients_readme_76.htm (dodgy mime type on this link)
The SonicMQ site seems to indicate that they support C++.
It would appear that this is suitable.
There are C++ libraries for SonicMQ. The main constraint is that you have to be using the same build (ie. STL libraries, etc) that Sonic used to compile the libraries.
http://web.progress.com/en/sonic/sonicmq-clients.html
Of course you asked this question over half a year ago, so this information is probably a bit late. :-)
I've seen a bunch of screencasts demonstrating the integration between blazeds and flex, also some lcds tutorials, model driven or not.
I've seen that some of them the presenter opens 2 browsers and once you change one value in a grid, it propagates to all other grids that presents the data.
I am wondering how the heck this is done, and how to reproduce.
Does this feature depends of the Edge / LCDS solution? I don't think so, but I've never seen some code explaining about it.
I feel it may or may not rely on JMS / MQ / messaging protocols or if this is some sort of 2 way sync and propagation of collection between instances of the same service result.
Thanks for any inputs.
Cheers,
Ernani
You can implement this feature both with BlazeDS and LiveCycle Data Services. BlazeDS provides remote and messaging features (the messaging features is the one allowing you to synchronize the data between the clients), LCDS extends BlazeDS adding new features like data management (productivity improvements), PDF generation, EDGE server for dealing with DMZ zones), MDA development, portal integration etc. It also adds some advanced messaging features like message conflation, throttling, reliability.
BlazeDS is free and open source and in my opinion a robust solution, you can use it if you want to synchronize the data between clients. LCDS adds a lot of things, but the LCDS customers should have a large budget.
How does it work? There is no JMS behind for this feature (however BlazeDS can integrate with a JMS provider so you can have one client in broswser and the second one running a SWING application). Instead there are some message queues on the server and a publisher - subscriber graph. In order to push the data from the clients to the server there are several choices, the more advanced are available only on LCDS: HTTP polling, HTTP long polling, HTTP streaming, RTMP sockets (LCDS only). All of them are described in details on Damon blog.
If you want to see some code go and download BlazeDS and take a look on the samples, there are several ones showing the messaging features. Also there a tomcat server is bundled in the download, and the samples are already deployed in it.
To do this you need to keep an open socket connection between the client and the server so that the server can push data back to the client.
I believe that the RTMP protocol was used for this two-way communication.
I understand that this is the primary reason to use LiveCycle Data Services over BlazeDS. WebORB also has push functionality, as does GraniteDS. I've also seen demos where this is done with ColdFusion.
If none of those options are available to you, you're stuck doing some kind of polling to the remote server.
Unfortunately, I do not have specific code samples to share.
A simple sample showing how to do this using WebORB can be found here:
http://www.kensodev.com/2009/11/01/synchronize-client-application-using-flexweborb-net/
That sample is based on WebORB 3 for .NET; WebORB 4 is now available, for both .NET and Java.
Point being: This is brain-dead simple using WebORB, which is FREE (although a paid Enterprise version is also available). God forbid that anyone should shell out $30K for LCDS just to get this feature.
--- Jim Plamondon, Midnight Coders (makers of WebORB)
I decided that it is time for me to dig into the whole Java EE stuff. I am using EE some techniques whithin Java SE like JPA or JMS, but i still messing around with Java SE and i believe Java EE and an application server will solve some of my problems i have.
BUT: I have still some questions after reading some articles on the web.
1st: Am i limited to request-response applications? I have an application which serves XML documents via HTTP. All delivered objects are added to a queue which will be dispatched in a different thread. Some validation is made for this objectes, including the opening of sockets to a remote machine (I heard EJ-Beans are not allowed to do this, is this true?). So, is is possible to do this within an application server?
2nd: I know there are Message driven beans, is it possible to send JMS messages to a MDB from outside of the application server? I have a service which sends JMS messages, but runs, as a legacy system, not inside the same application server.
3rd: How can the System Adminstrator or User configure my application? I know that some things like database connections are configured within the application server and my application can lookup them via JNDI or get them via DI. But what about application specific configuration?
Yeah, these are quite noobish questions, but maybe someone has the time to explain me how all this stuff is working. :)
regards,
Posix
PS:
4th: It seems EJBs are not allowed to do anything with files, so Java EE seems to be no option for a Service which receives Files, pushes them around to different systems and want them to write to a Socket (see question 1)?
I can say that Java EE can be used without any doubts in your case. Let me drill a little bit more into your specific questions:
You can open socket connection from your EJB. There is nothing that prevents you from doing that. However this kind of operation is not advised for Java EE applications. In my opinion the better option is to implement Java EE Connector (JCA) that would manage pool of socket connections to your proprietary system. This is the model way to implement such a integration as per specification.
Yes! It is perfectly possible to receive messages send from external application/system (outside the AS). This is main idea of integration using messaging :) In many cases your application being Java EE application receives messages via MDB from JMS channel, but JMS is only an API and can be implemented by any messaging system e.g. IBM MQ. In this architecture the external system puts an MQ message onto the queue and your Java EE application that listens to the very queue receives the message via JMS API!
Generally speaking Application Server gives the Administrator great tools to manage Java EE resources i.e. data sources, JMS connection factories, JMS destinations, JTA transaction manager, etc. If you require the ability to change your specific Java EE application the best options seems to be JMX. Just implement a few MBeans, export those to the JMX server embedded within your Application Server and you are done. This task is really trivial in, say, JBoss, but most of the modern Application Servers offer extensive JMX capabilities these days.
For the first glance, EJB doesn't seem to be the best for dealing with files. But remember that implementation of your EJBs is still written in pure Java, so nothing prevents you from reading/streaming files and so on. I have experience with large Java EE applications that are handling large files as input files and can assure you that Java EE is is a good technology choice :)
Here are the restrictions on EJB 1.1 spec.
Here's my take on your questions:
I believe an EJB can open a socket on a remote machine, but I would say that opening sockets is too low level an operation. I'd think about exposing whatever that socket is doing for you as another EJB.
An MDB is just a listener that's registered with a particular topic or queue. It doesn't say anything about sending. If your client knows how to get a message to the queue it's possible. They just have to know the queue URL and be able to create a connection.
The admin sets up connection pools, JNDI names, etc. - everything. They do it using the admin console for the app server.
It's a violation of the EE spec to do anything with files (to ensure that an EE app is portable and distributable). However since it's all just plain Java code, yopu can choose to do anything that you want. As long as you know how your target environment looks (eg the system is for internal use) I wouldn't hesitate modifying files just because the spec says so.
In an application server like Tomcat (others too, probably, but I've never worked with them) you can not only execute things upon receiving a request, but also do things (including starting long running threads) on server startup. Basically you can do anything that you can with "normal" Java. In fact, you could put a normal Java app in application server if you just include a piece of code which calls the appropriate main() on server startup.
I would suggest applying each technology to the appropriate points where you are currently feeling pain. Regarding your specific points,
In an EE context, you would add the messages to a JMS queue, that has MDBs which would do the actual processing. Regarding the management of the HTTP request/response lifecycle, you would manage this the same way you do now, or use an existing library to do if for you. By moving to an EE app server, you would allow the app server to manage the threading, transactions, etc. instead of having to manage it manually.
As duffymo stated, MDBs are responsible for receiving messages, they do not care where the message originated from.
The system administrator can configure the app server as duffymo stated. Additionally, you can expose JMX beans to other systems or to the end user to allow them to configure services if you so desire.