Scenario 1 :
Setup a JMS Queue in your server
Java code to send Messages to Producer
Create a JMS Producer, which when invoked, should receive the email data (subject, body, to , cc etc) and post it to the Queue setup in step 1
Create a JMS Consumer, which subscribes to the Queue created in Step 1, and its onMessage should call the JavaMail API to send the email.
Scenario 2 :
Directly call the JavaMail API to send the email.
I know about how to use and what JMS and Java Mail are doing.Thing is why we have to go from Scenario 2 to Scenario 1 for sending mails.Initially we did Scenario 2.Now we are using Scenario 1.From Different parts of the Big Application are sending mails so we use JMS Queue ,there will be Consumer of Queue from there sending mails.Please help me to understand.
You would use this mechanism in a large application for 2 reasons:
1) You don't want your clients to have to wait for the mail to be sent.
2) You don't want to lose mails if you lose connectivity to your mail server for any reason.
You would do this if you don't have a relyable MTA near your local machine but need to be sure your mail will be send. For example if there is a network outage but you rely on Java Mail to send your mail without additional logic, your mail will not be send at all.
Using JMS you can reschedule the mail for transfer as soon as the real MTA will become available again.
Besides:
the conversation with the mail provider (SMTP und POP3) is
asynchronous and close to the JMS/MDB api. So why should i use a
different API than JMS ?
You can keep the mail handling in one transaction, together with some database changes other activities. I remember too many Spring .. sic' projects, where the customer demmands for a atomic operation, that included a state change in a db ;-)
Image, the messages you send become more compulsory and you have to connect to a X400 service. Simply think of the slight code change (and the change of the RA) and you will discover to met the right architectual descision.
Related
Hy all,
in the software I'm developing, I have different camel routes that work on data, that is (in this case) loaded from an imap server using the camel-mail component.
Each of those routes does something with the data and then gives the data to the next route. They are dynamically configured at runtime.
In between those routes is an embedded ActiveMQ server which is used by each route to load the data from and save the data to (for the next route to pick it up).
Because of this structure I'm having a special case with the camel-mail consumer.
When loading a mail and sending it to the first ActiveMQ queue, it is immediatelly deleted/marked as read (depending on the settings on the mail consumer), but the actual processing of the mail has not concluded yet, as the next routes still have to process it.
This is a simplified view:
from("imaps://imap.server.com?...")
// Format mail in a way the other routes understand
.to("activemq:queue1"); // After this the mail is delete on the imap server
from("activemq:queue1")
// do some processing
.to("activemq:queue2");
from("activemq:queue2")
// Do some final processing
.to("..."); // NOW the mail should be delete on the imap server
This issue is even more a problem with the error handling I do.
Ever route in this "chain" sends failed exchanges to a deadLetterQueue on the ActiveMQ server. This way there is one error handling route, which picks up the failed exchanges and deals with them, on matter where it crashed.
In case there is a problem I want the email on the imap server to be handled differently (maybe even do nothing an try again on the next poll)
As camels InOut MEP returns the exchange to the (mail)consumer when the route ends i.e. when the exchange is given to the queue, I can't use the consumer to delete the mails after the whole process has ended.
Unfortunatelly I also don't see a delete option on the mail producer (which makes sense I guess, because its not how imap works).
I could also use smtp for this if thats necessary.
Does anybody have an idea how I could achieve this using no other connector then the camel component to connect to the mail server?
Greets and thanks in advance
Chris
Edit:
Adding the parameter "exchangePattern=InOut" to the jms queues (.to("activemq:queue1?exchangePattern=InOut")) lets the mail component wait for the whole process to finish.
The problem with that is, that we lose the big advantage with ActiveMQ that all routes are independent of each other. This is important so we are don't run into issues with consuming the mail when a later route takes a long time to process, which is very likely to happen.
So idealy we find a solution, where the mail is deleted without any component waiting for something to finish
I want to run a Java based message broker that will route messages to web clients. Web client connections are handled on our server using our custom Java websocket code, which authenticates users against the user database.
I think my server side websocket handler code would connect to ActiveMQ and perform subscription management via AQMP.
I have a specific requirement however:
route messages for a topic specifically to one or more web clients
Note that I don't need to retain messages if a client is not connected. Messages are being used to inform the web client applications of actions they need to take.
I'm considering ActiveMQ but I was hoping people with experience of the product could clarify if it supports this requirement?
If ActiveMQ isn't the best option, could you recommend something else?
Thanks
Yes, ActiveMQ is a great choice for this.
As far as specific approach goes, it depends on your data model and message flow.
You have several options, including:
Produce and consume to a topic-per-client
a. Messages for Client ABC go to topic://CLIENTS.ABC, for Client XYZ go to topic://CLIENTS.XYZ, and the subscribers connect accordingly.
Produce a message with a header and use a consumer-side selector (aka 'filters' in AMQP) to filter messages on a per-client basis. (abc client subscribes to-- ClientId = ABC, xyz client subscribe to-- ClientId = XYZ)
When using WebSockets, you might also look to STOMP which is text-based protocol. (Just depends on your programming language and available libraries that you had in mind)
After googling how message is sent/received in chat messenger like whatsapp, i came across they use queues based messaging system. I am just trying
to figure out what can be high level design of this feature
HLD per mine understanding :-
Say Friend 1 and Friend 2 are online . Friend 1 has established HTTP web connection to web server 1 and Friend 2 has established HTTP web connection to web server 2. Friend 1 send the message to Friend 2.
Now as soon as message comes to web server 1, i need to convey the message to web server 2 so that message can be pushed back to friend 2 through already established web connection.
I believe distributed custom java queues can be used here to propagate the message from one server to another. As soon as message comes to one server , it will push it to distributed queue(distribute queue because of load balancing and high availability) with message content, fromUserId, toUserId. There will be listener on queue which will see destination userId of just poppedIn message and find on which webserver destination userId is active . If user is active pop out the message and push it to client otherwise store it in db so that it can be pulled
once once gets online. To see which user is active on which server, there we can maintain the treemap with userId as key and value as serverName for efficient look up
Probably actual design must be more complex/scalable than above brief . Would like to know if this is the right direction for scalable chat messenger?
Also i believe we need to have multiple distributed queues instead of one for such a scalable application. But if we have multiple distributed queues how system will ensure the FIFO message delivery across distributed queues ?
Would like to know if this is the right direction for scalable chat
messenger?
Designing this application using message queues has the following benefits:
Decoupling of client-server and reduce of failure blast: Queues can gracefully handle traffic peaks, by just having a temporarily increased queue size, which will be back to normal as long as traffic normal again (or any transient failures have been fixed)
In a messaging application, clients (mobiles) can be offline for long periods. As a result, a synchronous design would not work, since the clients might not be accessible for message delivery. However, with an asynchronous design as with message queues, the responsibility of message delivery is on the client side. As a result, the client can poll for new messages as soon as it gets online.
So, yes this design could be quite scalable in terms of performance and usability. The only thing to have in mind is that this design would require a separate queue for each user, so the number of queues would scale linearly with the number of the application's users (which could be a significant financial & scalability issue).
But if we have multiple distributed queues how system will ensure the
FIFO message delivery across distributed queues ?
Many queues, either open-source (rabbitMQ, activeMQ) or commercial (AWS SQS), support FIFO ordering. However, the FIFO guarantee inside the queue is not enough, since the messages sent by a single client could be delivered to the queue in different order due to asynchronicity issues in the network (unless you are using a single, not-distributed queue and TCP which guarantees ordered delivery).
However, you could implement FIFO ordering on the client side. Following this approach, the messages would include a timestamp, which would be used by each client to sort the messages when receiving them. The only side-effect of that is that a client could see a message, without having seen all the previous messages first. However, when previous messages arive, they will be shown in the correct order in the client's UI, so eventually the user would see all the messages and in the correct order.
Would like to know if this is the right direction for scalable chat messenger?
I would probably prefer a slightly different approach. Your ideas are correct, but I would like to add up a bit more to the same. I happened to create such a chat messenger a few years ago, and it was supposed to be quite similar to watsapp. I am sure that when you googled, you would have come across XMPP Extensible messaging and presence protocol. we were using openfire as the server that maintains connections . The concept that you explained where
Say Friend 1 and Friend 2 are online . Friend 1 has established HTTP web connection to web server 1 and Friend 2 has established HTTP web connection to web server 2. Friend 1 send the message to Friend 2.
is called federation, and openfire can be run in a federated mode. After reading through your comments, i came across the one queue per user point. I am sure that you already know that this approach is not scalable as its very resource intensive. A good approach would be use an Actor framework such as akka. Each actor is like a light weight thread in java and each actor has an inbox. so messaging is taken care of in this case.
So your scenario transforms to Friend 1 opens a connection to openfire xmpp server and initializes a Friend 1 Actor.When he types a message, it is transferred to the Friend 1 actor's in-box ( Each actor in akka has an in memory inbox). This is communicated to the xmpp server. The server has a database of its own, and since it is federated with other xmpp servers, it will try to find if friend 2 is online. The xmpp server will keep the message in its db until the friend 2 comes online. Once friend 2 establishes a connection to any of the xmpp server a friend 2 actor is created and its presence is propagated to all other servers and the xmpp server 1 will notify Friend 2's actor. Friend 2's actor inbox will now get the message
Optional: There is also a option of delivery receipt. Once Friend2 reads the message, a delivery receipt can be sent to friend 1 to indicate the status of the message i.e read, unread, delivered, not delivered etc.
I understand JMS as depicted by the following diagram:
(source: techhive.com)
Is there any way for me to access the underlying database using JMS or some other thing? Further, the JDBC connections that the JMS server maintains, can I add new connections in it so as to access other databases also and do CRUD operations on them? If yes, how?
Where did you get this from?
Normally JMS is used to send messages to queue (or topics). You have message producers that push messages in the queue and message consumers consume them and process it.
In your exemple it seems that you have multiple queues. One for the messages that need to be processed, and one for each client to retrieve the result the processing of its messages.
With JMS Server you don't necessarily have a database behind. Everything can stay in memory, or can be written to files. You will need database server behind only if you configure your JMS server to be persistent (and to assure that even if server/application crash your messages won't be lost). But in that case you will never have to interact with the database. Only the JMS server will and you will interact with the JMS server sending and consuming messages.
I am looking to build an instant messenger in Java.
Clients will connect to the server to log in.
They will start a conversation with one or more other clients.
They will then post messages to the server that will relay the messages to all the clients.
The client needs to be continually updated when users post messages or log in.
so the way I see it, the client needs to run a server itself in a separate thread so that the main server can send stuff to it. Otherwise the client will have to the poll the main server every xyz seconds to get the latest updates. And that would need a separate thread anayway, as that would be purely for getting updates whereas the 'main' thread would be used for when the client initiates actions such as posting messages/inviting others to conversations etc...
So anyone recommendations on how to write this instant messenger? Does it sound like a good idea to make the connection a 'two-way' connection where both the client and server act as servers? Or is polling a better option? Anyone know how the IRC protocol does this?
There's no real advantage of having 2 connections unless they can be handled independently (for example receiving / sending a file usually done in a separate connection). A connection itself is already a two-way communication channel so it can be used to both send and receive messages, events etc. You don't need to poll server since client is able to maintain persistent connection and just wait for data to appear (optionally sending periodic PING-like message to ensure connection is alive).
IRC uses a single connection to server to exchange text commands. For example one of the main commands:
PRIVMSG <msgtarget> <message>
This command can be originated either by client or by server. Client sends PRIVMSG to notify that it wants to deliver message to one or more destination (in IRC this either user(s) or channel(s)). Server's task here is to properly broadcast this message to appropriate clients.
If you're using raw InputOutput streams then yes this is a good way of doing it. You create one thread on the clientside that acts in a similar fashion as the server thread - waits for any incoming updates and when it does it updates the client. I wouldn't call it a server though. So you'd ideally have 2 TCP/UDP connections one for requests made by the client and one to notify the client of server changes.
This solution in an enterprise environment would probably be done through some kind of messaging framework such as Spring Integration but dig deep enough and it will essentially be a similar way to how you mentioned.
Do you need a fully custom protocol or would it be sufficient to use the XMPP? There are several open source libraries implementing XMPP.
http://xmpp.org/xmpp-software/libraries/
e.g. http://www.igniterealtime.org/projects/smack/
For me, to develop instant messaging service, I will use websocket protocol instead of normal java socket because the normal socket can not work well with HTTP protocol and moreover some network providers and firewalls banned custom ports. If you develop it in normal socket, your service could not be accessed by web clients.
Did you plan to develop the instant messaging service yourself? How about using other protocols such as Jabber?