I have a web service, that takes an input xml message, transforms it, and then forwards it to another web service.
The application is deployed to two web logic app servers for performance, and resilience reasons.
I would like a single website monitoring page that allows two things
ability to stop/ start forwarding of messages
ability to monitor throughput of number of messages in the last hour etc. Number of different senders into the webservice etc.
I was wondering what the best way to implement this was.
My current idea is to have an in memory database (eg Debry or HSQL) replicating data to share the information between the two (or more) instances of my application that are running in different instances of the app server. I imagine I would have to setup some sort of master/ slave configuration.
I would love a link to an article that discusses how to solve this problem.
(Note, this is a simple spring application using spring MVC)
thanks,
David.
This sounds like a good match for Java Management Extensions (JMX)
JMX allows you to expose certain operations (eg: start/stop forwarding messages)
JMX allows you to monitor certain performance indicators (eg: moving average of messages processed)
Spring has good support for exposing beans as JMX MBeans. See here for more information.
Then you could use an open-source web-based JMX console, such as jManage
Hope this helps.
Sounds like you are looking for a Message Queue, some MDBs and a configurable design would let you do all these. Spring has support for JMS Queues if I'm not wrong
I think you are looking for a message queue. If you need additional monitoring, using a web service as the end point may not suffice - with regards to stop/start or forwarding of messages; monitoring http requests to web service is more cumbersome than tracking messages to a queue (even though you can do it).
If you are exposing this service to third party, then the web service will sit on top of the message queue and delegate to to it.
In my experience, RabbitMQ is a fine messaging queue service with a relatively simple learning curve.
Related
We have a number of related Java Spring applications running on our servers. Lets call them App1, App2 & App3. As is standard all these use the common code in our-common-utils.jar
I want these applications(App1, App2 & App3) to broadcast their state to one or more remote listeners. For e.g.
App1: I failed to read file abc.
App2: I am using more than 90% of my heap space etc.
The listener/s of these events will take specific actions such as send emails to support and/or clients based on the notifications received.
The best solution I can think of is to have a NotificationSender JMX enabled(implements NotificationBroadcasterSupport) bean in our-common-utils.jar. This will have a thread consuming from a queue of Notifications and firing off sendNotification() to the listeners for each Notification. This will be done by each of the Apps in our eco system but using common code from common-utils.
Do you see any flaws in this design? Any more efficient ways/frameworks of doing it?
Many Thanks :)
Alternative solution is to use any distributed coordination service zookeeper for example. I used it in my very first micro service project. As I can see you are using spring. Spring cloud provides necessary solutions that you can use in declarative way. I would pay your attention to #FeignClient. It is very simple in use and flexible in spring world.
If I would work on this issue now, I would use spring hystrix based solution. To simplify integration between your java services I would recommend to check service-registration-and-discovery.
Ignore my opinion if spring is not general engine part in your projects (may be you need other vendor solutions, there are a lot of alternatives). I concentrate my attention on spring solutions because spring is not restricted in my projects and I can use anything I wish if it's reasonable.
We are using a Core Java APP with no WEBSERVER, which is MULTI-Threaded. We have a requirement where in ,our app is to be made highly available in the customers environment.
All the transactions in our app are majorly ActiveMQ(Java Messaging Services TCP connections) based i.e. we communicate with other apps using message Queues. We also have HTTP connections
For High-availability of ActiveMQ ,we have implemented it in Master/Slave Configuration(Active/Passive)
For High-availability of our App(Active/Active),we thought of deploying two instances of the app which will consume the messages parallelly,
but this implementaion will rule out our internal feature of retaining the message . We are acknowledging the message from ActiveMQ queue only if they are proccessed.
Hence having two instances running might result in duplication of the proccess for the corresponding message.
Please advice us on how to make our App Highly-available.
Does a load Balancer in Place solve our issue? Also,
Should we have to convert our Core Java App into services?
Thanks in advance
Whenever you want high availability of app, AND IF high availability of your app directly depends on high availability of ActiveMQ, then what you should really be doing is, having a single instance of App and multiple instances of activeMQ,
what this does is, even if one instance of ActiveMQ goes down, other might takeover(typical Master-slave configuration) and App will function as expected.
This topology will also not result in duplication of message processing,because at any point of time only one ActiveMQ instance will be associated with your app.
for load balncing you can have a look here(if it suits your requirements).
https://docs.oracle.com/cd/E19823-01/819-0215/loadb.html
hope this helps!
Good luck!
I make the web-application using Spring MVC and there is a posibility of users to send messages to each other. How can I realize such feature: when the user is on his messages page and he gets the message from someone else, this message adds to his messages list without refreshing the page. Some kind of push notifications, but I can't come up with the right idea, how to realize it.
there's two pieces to consider in this; the 'messaging' framework, and the client-side notification.
typically, the 'messaging' framework would be designed/constructed to be independant of the view layer (Spring MVC piece) and might consist of either a reliable messaging platform (JMS, AMQP, etc.) or some service that allows events to be pushed into the framework. this allows for users to be 'connected' to a JVM instance independant of each other (say a clustered Tomcat environment or some such).
a simpler - old school solution to this was to use a shared database and write/read messages to a shared table with a user identifier; something like from=userA,to=userB,message=... the you could use a polling mechanism to retrieve the messages.
on the client-side, there are a number of patterns including long polling, ajax, websockets etc. that are intended to solve this design question. to marry into the polling solution, an AJAX timed poller (coupled with event) would allow you to continually update a section of your page by requesting from a service "do i (userB) have any new messages?"
the polling solution is "old school" and there are many more options than this. have a dig on topics such as websockets that were developed with this exact challenge in mind. (and have a look at Tomcat8's websocket support too)
I'm looking for a full duplex streaming solution with Java EE.
The situation: client applications (JavaFX) read data from a peripheral device. This data needs to be transferred in near real-time to a server for processing and also get the response back asynchronously, all while it keeps sending new data for processing.
Communication with the server needs to have an overhead as low as possible. Data coming in is basically some sensor data and after processing it is turned in what can be described as a set of commands.
What I've looked into:
A TCP/IP server (this is a non-Java EE approach).This would be the obvious solution. Two connections opened in parallel from each client app: one for upstream data and one for downstream data.
Remote & stateless EJBs. This would mean that there's no streaming involved and that I pack sensor data in smaller windows (1-2 seconds worth of sensor data) which I then send to the server for processing and get the processing result as a response. For this approach, while it is scalable, I am not sure how fast it will be considering I have to make a request each 1-2 seconds. I still need to test this but I have my doubts.
RMI. Is this any different than EJBs, technically?
Two servlets (up/down) with long polling. I've not done this before, so it's something to be tested.
For now I would like to test the performance for my approach #2. The first solution will work for sure, but I'm not too fond of having a separate server (next to Tomcat, where I already have something running).
However, meanwhile, it would be worth knowing if there are any other Java specific (EE or not) technologies that could easily solve this. If anyone has an idea, then please share it.
This looks like a good place for using JMS. Instead of stateless EJBs, you will probably be using Message-Driven Beans.
This gives you an approach similar to your first solution, using two message queues instead of TCP/IP connections. JMS makes your communications fully asynchronous and is low-overhead in the sense that your clients can send messages as fast as they can regardless of how fast your server can consume them. You also get delivery guarantees and other JMS goodness.
Tomcat does not come with JMS, however. You might try TomEE or integrate your existing Tomcat with a JMS implementation like ActiveMQ.
There are numerous options you could try. Appropriate solutions depend on the nature of your application, communication protocol, data transfer type, control you have over the client and server and firewall restrictions on client server routes.
There's not much info on this in your question, but given what you have provided, you may like to look at netty as it is quite general purpose and flexible and seems to fit your requirements. Netty also includes a duplex websocket implementation. Note that a netty based solution may be more complex to implement and require more background study than some other solutions (such as jms).
Yet another possible solution in GraniteDS, which advertises a JavaFX client integration and multiple server integrations for full duplex client/server communication, though I have not used it. GraniteDS uses comet (your two asynchronous servlets with long polling model) with the Active Message Format for data which you may be familiar with from Flex/Flash.
Have you looked at websockets as a solution? They are known to keep persistent connections and hence the asynchronous response will be quick.
I am currently evaluating JMS and I don't get what I could use it for.
Currently, I believe this would be a Usecase: I want to create a SalesInvoice PDF and print it when an SalesOrder leaves the Warehouse, so during the Delivery transaction I could send a transactional print request which just begins when the SalesOrder transaction completes successfully.
Now I found out most JMS products are standalone server.
Why would a need a Standalone Server for Message Processing, vs. e.g. some simple inproc processing with Quartz scheduler?
How does it interact with my application?
Isn't it much too slow?
What are Usecases you already implemented successfully?
JMS is an amazingly useful system, but not for every purpose.
It's essentially a high-level framework for sending messages between nodes, with options for discovery, robustness, etc.
One useful use case is when you want a client and a server to talk to one another, but without the client actually having the server's address (E.g., you may have more than one server). The client only needs to know the broker and the queue/topic name, and the server can connect as well.
JMS also adds robustness. For instance, you can configure it so that if the server dies while the client sends messages or the other way around, you can still send messages from the client or poll messages from the server. If you ever tried implementing this directly with sockets - it's a nightmare.
The scenario you describe sounds like a classic J2EE problem, why are you not using a J2EE framework? JMS is often used inside J2EE for communications, but you got all the other benefits.
What ist Java Message Service (JMS) for
JMS is a messaging standard that allows Java EE applications to create, send, receive, and consume messages in a loosely coupled, reliable, and asynchronous way. I'd suggest to read the Java Message Service API Overview for more details.
Why would a need a Standalone Server for Message Processing, vs. e.g. some simple inproc processing with Quartz scheduler?
Sure, in your case, Quartz is an option. But what if the invoice system is a remote system? What if you don't want to wait for the answer? What if the remote system is down when you want to communicate with it? What if the network is not always available? This is where JMS comes in. JMS allows to send a message guaranteed to be delivered and to consume it in a transactional way (sending or consuming a message can be part of a global transaction).
How does it interact with my application?
JMS supports two communication modes: point-to-point and publish/subscribe (if this answers the question).
Isn't it much too slow?
The MOMs I've been working with were blazing fast.
What are Usecases you already implemented successfully?
Used in system such as a reservation application, a banking back-office (processing market data), or more simply to send emails.
See also
EJB Message-Driven Beans
Why would a need a Standalone Server
for Message Processing, vs. e.g. some
simple inproc processing with Quartz
scheduler?
The strength of JMS lies in the fact that you can have multiple producers and multiple consumers for the same queue, and the JMS broker manages the load.
If you have multiple producers but a single consumer, you can use other approaches as well, such as a quartz scheduler and a database table. But as soon as you have multiple consumer, the locking scheme become very hard to design; better go for already approved messaging solution. See these other answers from me for a few more details: Why choosing JMS for asynchronous solution ? and Producer/consumer system using database
The other points are just too vague to be answered.
I've used it on a number of projects. It can help with scalability, decoupling of services, high availability. Here's a description of how I used it on a project several years ago:
http://coders-log.blogspot.com/2008/12/favorite-projects-series-installment-2.html
The description explains what JMS brought to the table for this particular project, but other projects will use messaging systems for a variety of reasons.
Messaging is usually used to interconnect different systems and send requests/commands asynchronously. A common example is a bank client application requesting an approval for a transaction. The server is located in another bank's system. Both systems are connected in an Enterprise Service Bus. The request goes into the messaging bus, which instantly acknowledges the reception of the message. The client can go on with processing. Whenever the server system becomes available, the bus forwards the message to it. Of course there needs to be a second path, for the server to inform the client that the transaction executed successfully or failed. This again can be implemented with JMS.
Please note that the two systems need not to implement JMS. One can use JMS and the other one MSMQ. The bus will take care of the interconnection.
JMS is a message-oriented middleware.
Why would a need a Standalone Server for Message Processing, vs. e.g. some simple inproc processing with Quartz scheduler?
It depends on what other components you may have. I guess. But I don't know anything about Quartz
How does it interact with my application?
You send messages to the broker.
Isn't it much too slow?
Compare to what ?
What are Usecases you already implemented successfully?
I've used JMS to implement a SIP application server, to communicate between the various components.
From the Javadoc:
The Java Message Service (JMS) API provides a common way for Java programs to create, send, receive and read an enterprise messaging system's messages.
In other words, and contrary to every other answer here, JMS is nothing more than an API, which wraps access to third-party Message Brokers, via 'JMS Providers' implemented by the vendor. Those Message Brokers, such as IBM MQ and dozens of others, have the features of reliability, asynchronicity, etc. that have been mentioned in other answers. JMS itself provides exactly none of them. It is to Message Brokers what JDBC is to SQL databases, or JNDI is to LDAP servers (among other things).
I have found a very good explanation of JMS with an example.
That is a simple chat application with JMS queues are used to communicate messages between users and messages stay in the queue if the receiver is offline.
In this example implementation they have used
XSD to generate domain classes.
Eclipse EE as IDE.
JBoss as web/application server.
HTML/JavaScript/JQuery for UI.
Servlet as controller.
MySQL as DB.
The JBoss configuration step for queue is explained nicely
Its available at http://coder2design.com/messaging-service/
Even the downloadable code is also available there.