Good morning,
I have a ambience wso2 MESSAGE BROKER where I shoot and consume messages inside a queue.
I would like to create a console (in addition to the MB) that allows me to consult what is inside the queue.
Is it possible?
tnks
Yes this is possible, inside the MB you can see all the content of message (like the image below)
But if for some reason you still want to create your own message monitor, I advise you to take a look at the product administration API, you might be able to monitor it via API. Another possibility is to create a monitor via the same AMQP protocol and check all the queue.
Related
I am not able to find a way to send/broadcast a message to all application instances in Pivotal Cloud Foundry. How can we notify to all app instances of some events? If we use the HTTP request, PCF router will dispatch it to a single instance of the app. How can we solve this problem?
What #Florian said is probably the safer option, but if you want something quick and easy, you can send HTTP requests directly to an app instance by using the X-CF-APP-INSTANCE header. The format for the header is YOUR-APP-GUID:YOUR-INSTANCE-INDEX.
https://docs.cloudfoundry.org/concepts/http-routing.html#app-instance-routing
So given an app guid, you could iterate over the number of instances, say 0 to 5, and send an HTTP request to each one. Make sure to check the response to confirm that each one succeeded.
This also requires that you know the app guid for your app (i.e. cf app <name> --guid) and the number of instances of your app.
CF, out of the box, does not provide any event queue mechanism where apps can subscribe to.
What I would do (assuming you've two app instances A and B):
Provide an event endpoint in your application code, e.g. POST /api/event (alternatively, if the event should arise from another app (e.g. another microservice), this one could directly send messages onto the queue)
All app instances are listening on an internal event queue for new events
instance A receives the call from the CF router and processes it by issuing an event on an internal event queue, the instance will not react to the event, yet
When A publishes the event, A and B receives the event and processes it accordingly
Now, the internal event queue you can use highly depends on your deployment. On AWS you probably can use SQS or SNS or something similar. PCF, as I know, may also provide a messaging system which would suit here as well, rabbitmq. You could also use features of other services that would allow you to subscribe to events, such as redis (pub/sub commands) or similar.
If you provide more information about what you want to achieve more concretely, more detailed answer would be possible, though.
I'm trying to implement a service which consumes a google pubsub subscription at its own pace. By that, I mean I need fine control on when I need to consume messages i.e get a batch of messages, pause for a while, do not get more than X messages...
Using google client libraries I did not find a way to do it as the MessageReceiver is running in its own thread and I don't have any control on what exactly happens.
Basically, being able to consume messages in a synchronous way should solve my issue.
Do you know how I can use the google client libs synchronously ? Or is there another way in the API I missed ?
You might try using setFlowControlSettings when you build your subscriber. In particular, you can use setMaxOutstandingElementCount or setMaxOutstandingRequestBytes to limit the messages sent to your MessageReceiver. When you have enough messages outstanding, i.e., messages for which you have not called Ack() or Nack(), to exceed these limits, then your MessageReceiver will not be called until messages have been acked or nacked.
We are developing a webapp that needs to send out emails written in Java/Groovy. Currently, we are persisting each email to a database before we call the Java Mail APIs to send the mail to our SMTP server.
I want to send out email asynchronously. I want to persist the email and then have another process pick up the email and send it (and send it only once). Ideally, this process is running outside of my webapp.
Are there any tools that do that?
Update: This solution needs to prevent duplicate emails and it needs to handle spikes in email. I was hoping someone already wrote an offline email processor. (I'd rather not implement this myself.)
The suggestions to use a cron job to read the database are quite workable.
Another good approach here is to use a Java Message Service (JMS) message queue. These are persistent (backed up by a database) and reliable. You can have one or more producer programs enqueue messages with the relevant data in them, and then one or more consumers process the messages and dequeue them. All of this is set up for very high reliability, and you gain the flexibility of asynchronously decoupling the operations, which means during email spikes the message queue can grow larger until the consumers catch up with the spike. Another benefit is that the email goes out as soon as a consumer gets to it instead of on a timer. Plus, if you require high availability, you can have multiple consumers in case one goes down.
Check out Apache's ActiveMQ for a good open source implementation of JMS.
If you're using Linux/Unix you could create a cron job to run every few minutes which calls a program to grab the email from the database and send it out. You could also have a field in the database to indicate whether the message has been sent. The downside of this approach is that there may be a delay of a few minutes from when your webapp persists the email and when the cron job is run.
Setup a cron job and use scripts to query the db and send out emails via sendmail.
On the off chance it's an Oracle DB, you can use the UTL_MAIL package to write PL/SQL to send the mail through your SMTP server. Then create a scheduled job to execute on your desired schedule.
http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14258/u_mail.htm
Since you are already using groovy, this might be an interesting tool to solve your problem
http://gaq.sourceforge.net/
You could use Quartz, a scheduling library (similar to cron), to schedule a recurring task that reads the DB and sends the emails. If you're using Grails, there's a Quartz plugin that makes working with Quartz a bit more Groovy.
How can we keep track of every message that gets into our Java Message Queue? We need to save the message for later reference. We already log it into an application log (log4j) but we need to query them later.
You can store them
in memory - in a collection or in an in-memory database
in a standalone database
You could create a database logging table for the messages, storing the message as is in a BLOB column, the timestamp that it was created / posted to the MQ and a simple counter as primary key. You can also add fields like message type etc if you want to create statistical reports on messages sent.
Cleanup of the tabe can be done simply by deleting all message older than the retention period by using the timestamp column.
I implemented such a solution in the past, we chose to store messages with all their characteristics in a database and developed a search, replay and cancel application on top of it. This is the Message Store pattern:
(source: eaipatterns.com)
We also used this application for the Dead Letter Channel.
(source: eaipatterns.com)
If you don't want to build a custom solution, have a look at the ReplayService for JMS from CodeStreet.
The best way to do this is to use whatever tracing facility your middleware provider offers. Or possibly, you could set up an intermediate listener whose only job was to log messages and forward on to your existing application.
In most cases, you will find that the middleware provider already has the ability to do this for you with no changes or awareness by your application.
I would change the queue to a topic, and then keep the original consumer that processes the messages, and add another consumer for auditing the messages to a database.
Some JMS providers cater for topic-to-queue-bridge definitions, the consumers then receive from their own dedicated queues, and don't have to read past messages that are left on the queue due to other consumers being inactive.
Alternatively, you could write a log4j appender, which writes your logged messages to a database.
I have a constantly-running Java program that needs to send an email whenever it encounters a problem. However it is possible that the mail server it uses could be down at the time it tries to send the email.
What is the best way to ensure that the email will be delivered when the mail server comes back up?
Queue up the requests. Have a separate thread which merely waits for something to enter the queue, then tries to email it. If it fails, it waits a few hours and tries again. Once it sends a message, it goes back to the queue to get the next message.
Put the email object into a stack or list when it fails to send, when the email server comes back up, pop each email out until it is empty.
You may want to save the email in a file, perhaps an xml file, so that should the application crash you won't lose this information.
This file is loaded when the application starts, and it keeps everything in memory, so that while there are pending emails then it keeps checking every 5 minutes or so, then, as it sends each email it will resave the xml file, so that should it crash after sending 3 emails out of 10 it won't resend those three when it starts up.
But, how you handle that is really going to depend on the specification for how to handle error conditions.
If you go from "forward everything to this SMTP server which is always there" to a situation where you need to handle all kinds of conditions normally handled by a full SMTP-server like retry later, retransmit if connection closed, use MX-hosts in their stated order and similar, you may want to consider simply having a SMTP-server inside your client (but one that does not accept incoming connections) since this moves all the dirty logic away from your applications.
I believe that the James email server - http://james.apache.org/ - is easily embeddable, but I have not actually tried.
The suggestion of using James is a good one but I've had some issues in the past of James being a bit flaky and needing to be restarted.
You could use something like Quartz to have a scheduler check for messages that need to be sent. If the message can't be sent (eg. smtp server isn't available), then that message is rescheduled to be sent at a later time. You could either have a task per message or have a persistent task that checks for messages and available mail server then sends the messages. The persistent task would give you email batching.
If you are in a Unix/Linux world, then consider the alternative of sending your alerts using syslog, and dealing with the generation of emails on that side. For example, nsyslogd has a module called ommail for generating emails natively.
IIRC, there are adapters for log4j and the like that can bridge between the Java and syslog worlds with a minimum of (zero ?) coding.
Apache James - http://james.apache.org/ will let you run your own mailserver as a proxy, not only that but is written in 100% java, so you can figure out what its doing,
and as an extra bonus James uses databases to queue the mail, so you can even inject mail directly into the queues by inserting into a database, then leave whole business of sending the mail up to James.