I'm evaluating solutions for managing 400+ instances of a kiosk-like application. Each instance runs a custom Java application that displays information and interacts with users. We are pondering if we should develop a custom solution, try to include JMX support for it or find something else.
The requirements are simple:
It must be a free and open source solution;
It must be able to manage more than 400 instances (for example: one manager managing 400 JMX-enabled clients);
It should be preferably programmed in Java;
We need standard metrics (for example: available resources, running times, current status, etc);
Optionally we would like to send some control messages to the instances.
It seems that JMX features and flexibility is what we were looking for. However, I haven't found much about JMX scalability. How many "clients" can one JMX manager handle? Is it possible to manage 400 JMX instances concurrently? Is there any recommended architecture or workaround if it doesn't handle so many clients directly?
Thanks for any hints
JMX is exactly what you want.
As far as how many clients you can connect to, I'd imagine that for the monitoring portions of it, you'd only really need to fetch data from the remote clients every few seconds, at the most, right?
I'm not sure if a solution already exists which will allow you to monitor N clients and also control them, so you'd likely want to develop something yourself - interacting with remote MBeans with JMX code seems to be trivial, which then turns this into an exercise of being able to write a program which can concurrently communicate with 400 nodes successfully.
Related
I am starting with a new application, that is intended to be run in a more "cloud" way following the idea of serverless infrastructure, where there is no notion of a "machine" anymore, but just an application running somewhere.
As Graphite only knows "absolute" values, typically, the host name of the server would be encoded in the metric path to deal with concurrency. (e.g. applicationname.webserver_1.requestCount). Using Graphite functions, data can be merged easily when building graphs.
However when I run in a server less environment, hostnames do not exist in the classical sense anymore - every time the application is restarted or deployed a new, random hostname will be generated for it. What I am wondering now, is - how to best deal with this. I can't use the hostname, because this way I will end up with hundreds of different hostnames quite quickly and a completely unusable graphite backend. What are the best practices in this use case?
As a sidenote - this is not only to measure counters, but also to measure response times (where p99 functions will be applied).
I'd like to use dropwizard-metrics (formally known as codahale metrics) to send messages to grahite, however that should be only an implementation detail.
Thanks a lot for you help.
The plan for now is to not directly sent to Graphite, but to go via StatsD, which is able to do the necessary aggregation. In this case there's no need to embed the hostname.
thanks for your suggestions! :)
I need to implement a proof of concept application Swing application where there's a server having a list of users and several clients which connect to the server and do CRUD operations on the database and hence on the list of users.
I have an obvious synchronization dilemma of keeping all clients lists updated so that if one client removed a user another one who still has it in his list cannot change its name.
Now I know a protocol in which before updating a user the client asks the server whether it still exists would work.
However this is just a simple example but in the real application I might have junction tables and complex references between objects which need always to be kept consistent and I don't want to reinvent the wheel.
What I'm wondering if there's some ready made solutions or some library which does this job which doesn't require me to change database or load extremely complex dependencies.
I did some research on Google but nothing seems to fit and the most similar example of client server synchronization I found was "chat programs", however chat programs are inherently simple because a message is never modified or deleted and all you have to keep consistent is the chronological order. I would need something more involved than that or some useful hints on the subject.
What you need is some sort of messaging between server and clients. Clients will either long-poll the server, asking for the updates, or subscribe to some streaming endpoint on your server.
You can take a look on Comet model, long-polling itself, websockets, etc.
Alternatively - there are couple of data management servies - BlazeDS, GraniteDS or any similar purpose solution. You can integrate one of those in your application and use for complete data management cycle.
I'm looking for a full duplex streaming solution with Java EE.
The situation: client applications (JavaFX) read data from a peripheral device. This data needs to be transferred in near real-time to a server for processing and also get the response back asynchronously, all while it keeps sending new data for processing.
Communication with the server needs to have an overhead as low as possible. Data coming in is basically some sensor data and after processing it is turned in what can be described as a set of commands.
What I've looked into:
A TCP/IP server (this is a non-Java EE approach).This would be the obvious solution. Two connections opened in parallel from each client app: one for upstream data and one for downstream data.
Remote & stateless EJBs. This would mean that there's no streaming involved and that I pack sensor data in smaller windows (1-2 seconds worth of sensor data) which I then send to the server for processing and get the processing result as a response. For this approach, while it is scalable, I am not sure how fast it will be considering I have to make a request each 1-2 seconds. I still need to test this but I have my doubts.
RMI. Is this any different than EJBs, technically?
Two servlets (up/down) with long polling. I've not done this before, so it's something to be tested.
For now I would like to test the performance for my approach #2. The first solution will work for sure, but I'm not too fond of having a separate server (next to Tomcat, where I already have something running).
However, meanwhile, it would be worth knowing if there are any other Java specific (EE or not) technologies that could easily solve this. If anyone has an idea, then please share it.
This looks like a good place for using JMS. Instead of stateless EJBs, you will probably be using Message-Driven Beans.
This gives you an approach similar to your first solution, using two message queues instead of TCP/IP connections. JMS makes your communications fully asynchronous and is low-overhead in the sense that your clients can send messages as fast as they can regardless of how fast your server can consume them. You also get delivery guarantees and other JMS goodness.
Tomcat does not come with JMS, however. You might try TomEE or integrate your existing Tomcat with a JMS implementation like ActiveMQ.
There are numerous options you could try. Appropriate solutions depend on the nature of your application, communication protocol, data transfer type, control you have over the client and server and firewall restrictions on client server routes.
There's not much info on this in your question, but given what you have provided, you may like to look at netty as it is quite general purpose and flexible and seems to fit your requirements. Netty also includes a duplex websocket implementation. Note that a netty based solution may be more complex to implement and require more background study than some other solutions (such as jms).
Yet another possible solution in GraniteDS, which advertises a JavaFX client integration and multiple server integrations for full duplex client/server communication, though I have not used it. GraniteDS uses comet (your two asynchronous servlets with long polling model) with the Active Message Format for data which you may be familiar with from Flex/Flash.
Have you looked at websockets as a solution? They are known to keep persistent connections and hence the asynchronous response will be quick.
We're trying to design a new addition to our application. Basically we need to submit very basic queries to various remote databases accessed over the internet and not owned or controlled by us.
Our proposal is to install a small client app on each of the foreign systems, tiered in 2 basic layers, 1 that is tailored to the particular database its talking to, to handle the actual query in SQL or whatever, the other tier would be the communication tier to handle incoming requests and send back responses. This communication interface would be the same over all of the foreign systems, ie all requests and responses have the same structure.
In terms of java remoting I guess this small client app would be the 'server' and our webapp (normally referred to as the server) is the 'client'.
I've looked at various java remoting solutions (Hessian, Burlap, RMI, SOAP/REST WebServices). However am I correct in thinking that with all of these the 'server' must run in a container, ie in a tomcat/jetty etc instance?
I was really hoping to avoid having to battle all the IT departments controlling the foreign systems to get them to install very much. The whole idea is that its thin/small/easy to install/pain free. Are there any solutions that do not require running in a container / webserver?
The communication really is the smallest part of this design, no more than 10 string input params (that have no meaning other than to the db) and one true/false output. There are no complex object models required. The only complexity would be from security/encryption etc.
I wamly suggest somethig based on Jetty, the embedded HTTP server. You package a simple runnable JAR with dependency JARs into a ZIP file, add a startup script, and you have your product. See for example here.
I often use Sprint-Remoting in my projects and here you find a description how to use without a container. The guy is starting the jetty from within his application:
http://forum.springsource.org/showthread.php?12852-HttpInvoker-without-web-container
http://static.springsource.org/spring/docs/2.0.x/reference/remoting.html
Regards,
Boskop
Yes, most of them runs a standard servlet container. But containers like Jetty have very low footprint and you may configure and run Jetty completely out of your code while you stay with servlet standards.
Do not fail to estimate initial minimal requirements that may grow with project enhancement over time. Then have a standard container makes things much more easier.
As you have tagged this question with [rmi], RMI does not require any form of container. All you need is the appropriate TCP ports to be open.
I have a web service, that takes an input xml message, transforms it, and then forwards it to another web service.
The application is deployed to two web logic app servers for performance, and resilience reasons.
I would like a single website monitoring page that allows two things
ability to stop/ start forwarding of messages
ability to monitor throughput of number of messages in the last hour etc. Number of different senders into the webservice etc.
I was wondering what the best way to implement this was.
My current idea is to have an in memory database (eg Debry or HSQL) replicating data to share the information between the two (or more) instances of my application that are running in different instances of the app server. I imagine I would have to setup some sort of master/ slave configuration.
I would love a link to an article that discusses how to solve this problem.
(Note, this is a simple spring application using spring MVC)
thanks,
David.
This sounds like a good match for Java Management Extensions (JMX)
JMX allows you to expose certain operations (eg: start/stop forwarding messages)
JMX allows you to monitor certain performance indicators (eg: moving average of messages processed)
Spring has good support for exposing beans as JMX MBeans. See here for more information.
Then you could use an open-source web-based JMX console, such as jManage
Hope this helps.
Sounds like you are looking for a Message Queue, some MDBs and a configurable design would let you do all these. Spring has support for JMS Queues if I'm not wrong
I think you are looking for a message queue. If you need additional monitoring, using a web service as the end point may not suffice - with regards to stop/start or forwarding of messages; monitoring http requests to web service is more cumbersome than tracking messages to a queue (even though you can do it).
If you are exposing this service to third party, then the web service will sit on top of the message queue and delegate to to it.
In my experience, RabbitMQ is a fine messaging queue service with a relatively simple learning curve.