I am using Java, spring-boot and ActiveMQ.
I need to send a large bunch of messages in shortest time.
Right now it takes lot of time to send message one by one using JMSTemplate.
Is there any way I can bunch the messages and send if to activemq at once with guarantee to maintain the order of messages?
thanks in advance
Default ActiveMQ configuration can be slow for large message flow. We use following configuration for improving the message rates -
connection.setOptimizeAcknowledge(true);
consumerSession = connection.createSession(false, Session.DUPS_OK_ACKNOWLEDGE);
setOptimizeAcknowledge configures optimized acknowledgement of received messages while Session.DUPS_OK_ACKNOWLEDGE allows batched acknowledgements.
Spring's JMSTemplate is notorious for bad performance outside a Java EE container (or some other environment which provides pooled connection resources). Read more on the Apache ActiveMQ website. Therefore, you need to use a connection pool or ditch the JMSTemplate for something else.
Related
I am currently testing out JMS queue (first time using JMS) and message driven beans.
I have created a queue to provide other applications with state updates for one of our projects.
Logic is written in native JMS, deployed on a JBOSS7 using the ActiveMQ implementation.
It depends on a selector to deliver the messages to the right client, and while I can just place good faith in my colleagues, preferably I would like to enforce the use of the selector so the clients don't consume messages not meant for them.
So basically I would prefer that no messages are delivered to a client which has not specified a selector.
When I deploy a consumer without any selector it just consumes all messages available on the queue.
Otherwise everything works as expected.
I have looked and haven't been able to find anything I am looking for, maybe it's possible by configuring ActiveMQ itself but I am not really at home in that ecosystem.
So the problem is resolved by using a system I wasn't that aware about and I thought I'd share it here if someone need it:
The JMS clients are on different physical machines so originally the plan was to do manual JNDI remote lookup to access the queue but this caused some problems. Mainly having to write retry logic when the Queue is unavailable.
I threw that plan out the window to opt for a Bridge instead, following the guide found here: http://www.mastertheboss.com/howto/jboss-jms6/configuring-jms-bridge-with-wildfly-10
This has multiple advantages (both the producer system or the client can go down without causing too many problems) but most notably this solves my problem: I can define a selector on the bridge per Client. So the responsibility of choosing who receives which messages is back in my court.
I will have a crack at implementing this.
I have a standalone Spring application that reads messages from a Weblogic cluster. It is not a MDP, rather it runs multiple threads that each use a JMSTemplate to browse the queue and retrieve messages based on specific criteria.
I would like to cache the JMS connections, while also ensuring that I open enough connections that I always am retrieving messages from each server in the cluster. My issue is the default ConnectionFactory does not cache at all, but the Spring wrappers SingleConnectionFactory and CachingConnectionFactory do not allow for multiple connections open at once.
Should I implement my own ConnectionFactory that caches on a limited basis? Or what is the recommended approach.
Use case seemed to be quite clumsy to do in Spring, so resolved the issue by directly managing the JMS resources directly.
I have a web application i am rewriting that currently performs a large amount of audit data sql writes. Every step of user interaction results in a method being executed that writes some information to a database.
This has the potential to impact users by causing the interaction to stop due to database problems.
Ideally I want to move this is a message based approach where if data needs to be written it is fired off too a queue, where a consumer picks these up and writes them to the database. It is not essential data, and loss is acceptable if the server goes down.
I'm just a little confused if I should try and use an embedded JMS queue and broker, or a Java queue. Or something I'm not familiar with (suggestions?)
What is the best approach?
More info:
The app uses spring and is running on websphere 6. All message communication is local, it will not talk to another server.
I think logging with JMS is overkill, and especially if loggin is the only reason for using JMS.
Have a look at DBAppender, you can log directly to the database. If performance is your concern you can log asynchronously using Logback.
If you still want to go JMS way then Logback has JMS Queue & Topic appenders
A plain queue will suffice based on your description of the problem. You can have a fixed size queue and discard messages if it fills too quickly since you say they are not critical.
Things to consider:
Is this functionality required by other apps too, now or in the
future.
Are the rate of producing messages so huge that it can start
consuming lot of heap memory when large number of users are logged
in. Important if messages should not be lost.
I'm not sure if that is best practice inside a Java EE container however.
Since you already run on a WebSphere machine, you do have a JMS broker going (SIBus). The easiest way to kick off asynchronous things are to send JMS messages and have a MDB reading them off - and doing database insertions. You might have some issues spawning own threads in WebSphere can still utilise the initial context for JNDI resources.
In a non Java EE case, I would have used a something like a plain LinkedBlockingQueue or any blocking queue, and just have a thread polling that queue for new messages to insert into a database.
I would uses JMS queue only if there are different servers involved. So in your case I would do it in simple plain pure java with some Java queue.
I have two systems: HQ on Linux and Active MQ on windows
Both systems need to send and receive message between each other.
Anyone implemented a way of integrating between them?? In this case I would like to have an example
Thanks,
ray.
JMS providers in general are not interoperable, because there is no common internal message format, or the connection protocol.
The perfect solution would be unifying the providers, so that both systems use the same (could be different instances — don't know about HQ, but ActiveMQ can send to another ActiveMQ).
If this is not acceptable, you can always write adapters yourself, with message-driven beans. One MDB would listen on an MQ queue, to repack the message and forward to ActiveMQ; the other MDB would do the same other way round. The exact setup and configuration of connection factories and queues depends on the application server.
ActiveMQ provides a solution for this scenario, its called a JMS to JMS bridge this allows you to bridge destinations between JMS brokers either inbound, outbound or both. Have a look at the documentation at the link above.
I know I might sounds ridiculous for some experts, however, it's been in my head for quite a while and still no concrete answer found.
In PUBLISH/SUBSCRIBE MESSAGING WITH JMS TOPICS: JMS publisher sends a msg to JMS provider, and JMS provider sends the msg to JMS subscribers and receives their acknowledgement.
Is it possible that I can somehow modify the JMS provider, so that the JMS producer only sends out every other message it receives from JMS publisher?
Totally newbie in this field, so any suggestion is welcomed.
If what you want is for the subscriber to be able to configure to receive messages in batches, where each subscriber can have a different batch size, then JMS will not provide this functionality. This is not a typical pubsub type scenario.
If you want to accomplish this, I would suggest you add some custom buffering on your subscriber side that will queue up the incoming messages and then do a batch notify when your queue is full. This could then be easily configured on a per subscriber basis.
The only messaging system that I know provides a similar functionality is pubsub in XMPP, but even then the batches are determined by a timed interval instead of number.
You could look at filtering at your JMS subscriptions using JMS API Message Selectors. You can then only read/process messages that match a certain criteria.
With more information about what you are trying to accomplish (filtering? testing dropped messages? load balancing? something else entirely?) you might get a better answer.
Why would you want to do this? Would it not defeat the whole gambit of messaging, which is not to lose any messages? Or is it that you want to control exactly how the message gets distributed to subscribers? Even this would go against the basic JMS specifications.