In our java mail (using Java Mail API) application we first connect to the mail server, fetch messages, process headers and then afterwards process the message bodies and attachments using pop3 as usual.
Session session = Session.getInstance(props, null);
Store store = session.getStore(urln);
store.connect();
Folder f = store.getFolder("INBOX");
f.open(READ);
Messages m = f.getMessages(..);
for (Message m : messages) {
if (!store.isConnected()) {
//raise exception
}
processSubject();
processFrom();
processBodyAndAttachments();
..
}
The implementation works fine on most environments, but on some customer the storeconnection gets lost during the process in the for loop. We can see the raises exception in the logs. My questions:
AFAIK, the mail server can sometimes reject new connections, but does
it terminate current living connections (may be becasue of too much
connections or disconnects old ones to give access to the new ones?)
When the store is disconnected, does the folder gets closed too?
Is it better to check the folder?
The connection may be lost everywhere in the for loop and it does not
seem to be a good practise to put isConnected check everywhere in the
loop, it will make the code dirty and also cause performance issues,
is it a good practise to put in a try catch block and check for
IOExceptions? (Folder closed) Or other suggestions? Which exceptions
should be handled? There may be some cases where the message is not
parseable but connection is healthy.
What about adding a disconnect listener?
Network connections can be broken for a variety of reasons. Your program always has to be prepared for the connection to drop at any time.
With POP3, there is only one connection, so if the connection is dropped the store should be disconnected and the folder should be closed.
If the Folder is open, check the Folder. Otherwise check the Store.
You need a strategy for handling failures. If you keep track of what messages have been successfully processed you may be able to restart the processing at the next message after a failure. A lot of the details depend on your environment and your application requirements.
A disconnect listener won't make this easier.
Related
In my Java application I am using the failover transport to connect to a local ActiveMQ broker:
failover:(tcp://0.0.0.0:61616)
I create one single connection that I reuse in the rest of the application:
ActiveMQConnection connection = (ActiveMQConnection) connectionFactory.createConnection();
In another part of the application when I receive some external call I need to send a message to the broker, and so, for doing that I create a new "Session":
Session locSession = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
When the broker is down my app tries to reconnect to the broker forever (this is the expected behavior I really want to have).
However, the problem is that if the broker is down and I receive a call that invokes the code that executes the connection.createSession(false, Session.AUTO_ACKNOWLEDGE) then my app hangs forever on this line of code waiting for the app to reconnect successfully to the broker and then create the session.
Please, do you know any way to check before I execute createSession if the connection object is trying to reconnect or it is really connected? If I am able to know this I could avoid the creation of the session if the app is not connected to the broker (only trying to reconnect) and therefore I would avoid to hanging on connection.createSession forever (I would raise an exception).
I wasn't able to find any property or method on ActiveMQConnection to gather this information.
The failover: url provides a setting startupMaxReconnectAttempts to prevent infinite retry when connecting to the broker the first time.
Also note-- If you want an exception to bubble up, that conflicts with requirement to have infinite retry. You would need to adjust the failover settings to match your intended behavior, by setting a max count or max time to perform retry, then throw an exception and unblock your caller.
For example, you could indicate you only want to retry for 5 minutes, then receive an exception to handle in the code to prevent the infinite blocking.
Thank you all for your help and suggestions. They helped me a lot in re-focusing the problem.
However I f found the answer to my question using the method "getTransport().isConnected()".
I have a distributed system application that uses JBoss as an application server. I have a client application that serves as a simulation engine. When client is up, it sends an registration message(JMS message) to Server, then some field is set in the database. When Server is up, it sends a message ( a topic) to all clients to check that they are alive. If clients are alive, they can read message and send a response to server (queue) that it is alive.
If user close client normally, client send a message to server that I will unregister. Then server unregisters it. This is done in database side.
If user close client abnormally(kill) , then client can not send a message to server for unregistration. Then server does not know this client is not alive anymore. This causes inconsistency in my application. So I need a way to understand that client subscribed a topic is not subscribed anymore.
Server sends a message to topic to check that clients are alive.
#Schedule(hour = "*", minute = "*", second = "30", persistent = false)
public void sendNodeStatusRequest() {
Message msg = MessageFactory.createStatusRequestMessage();
publishNodeMessage(msg);
}
After a time, Server show following logs. Could I catch this warning from Java?
07:17:00,698 WARN [org.hornetq.core.protocol.core.impl.RemotingConnectionImpl] Connection failure
has been detected: Did not receive ping from /127.0.0.1:61888. It is likely
the client has exited or crashed without closing its connection, or the
network between the server and client has failed. The connection will now be closed. [code=3]
07:17:00,698 WARN [org.hornetq.core.server.impl.ServerSessionImpl] Client
connection failed, clearing up resources for session 4e4e9dc6-153e-11e7-
80fa-742b62812c29
To me the whole point of messaging system is decoupled communication. The sender (server in your case) send its stuff to the topic without actually knowing who will get the message. The clients come and go, and they should be able to read the message whenever it (still) resides in the topic.
Now from your question I understand that the server keeps track of all the connected clients by means of receiving the message back to the dedicated queue.
So I'm asking myself - maybe its something wrong with the design here.
Let me propose slightly different way of implementation.
The server should not be aware of any client, at most (because your system seems to work this way) it should know that client A, B and C are alive now only because these clients passed to the server this knowledge.
Why just don't make clients sending the "keep-alive" message every, say 1 minute (or less, depending on your needs) to the server queue without prior message from the server.
The message can include some client identifier and probably time if its not added by the infrastructure or something)
So the server will just get this message and it will keep track in memory the list of available clients along with the last time they've sent something.
So if some client disconnects "gracefully" - it can send a special message to the server like "I'm client A and consider me disconnected". Otherwise (abnormal termination/network outage/whatever) - it just won't send anything, the server will have a special process that will check whether there are stale clients on the list and if it finds them - it knows that something went wrong.
If you still want to stick with JMS way of doing, then you can try to send the message synchronously, meaning the producer will wait until it hears from the consumer. More information here : http://docs.oracle.com/javaee/6/tutorial/doc/bncfa.html
I need some help with the following problem:
I open a tcp-socket in the constructor then proceed to provide a object over an object output-stream to the server. I have no control over the server and don't get any response back.
How can I detect that the connection was lost? Will I always get the IOExeption-Error when trying to write? Because according to javadoc once a connection was successfully made most of the checks are basically useless to me.
Additionally what is the best way to reconnect a socket? Set the reference to "Null" then create a new one?
Here is my current approach:
I have a status-list in which I have the following statuses:
SocketSuccess; SocketFailure; MessageSuccess; MessageFailure;
My idea is kind of like a state-machine so check first what the last status was. If the connection was successfull or the last message was successfull then try to send the message. When I get a IOExeption then set the status MessageFailure, save the Message locally till I get a successfull connection again.
Or are there any recommended patterns for this kind of situation?
Clearing all your douts. If the connection with the server is lost then the client will throw IOException and that will kill the application but if you have handled the exception and tried to reconnect with the server and Re-establish the input output stream the your message function will start again. The predefined messages you are using will travel only when there is a connection between server and client. So when the connection is lost you will get IOException and when you handle that exception and try to reconnect a new input output stream should be established that will carry your messaging service.
I am writing a Rest Service that connects to an FTP server to read some files, then do some operations over the read data to serve the service request. I am using Apache commons FTPClient.
As a temporary solution, I am creating an FTPClient object - then connecting it - and then logging in with the credentials - inside a method (the client is local to this method - doing this as FTPClient is not thread safe) in my data access layer and then disconnecting it before coming out of the methods(ie.. after reading the file). The issue is, FTPClient is taking some 3-7 seconds for logging in which is very high. So I am thinking of implementing an FTPClientPool that can provide an already prepared client in the data access method.
Do any such ClientPools already exist?
If yes, then what one should I opt for?
If no, the difficulty in implementing is once created and connected, How long does an apache FTPClient stay alive? for infinite time?? (what I mean is what is the default keep alive time for an FTPClient - idle time after which client gets disconnected - coz I see various kind of times in the java docs. :( ) And next questions is How do you keep it alive always?? (may be sending the NOOPS after regular intervals in a separate thread??) Any kind of help regarding how should I move forward is really helpful.
Thanks & Regards
Idle timeout for clients is generally determined server side.
Here's some of the more non obvious client parameters:
soTimeout - Determines how long the client blocks waiting for a message. Generally you poll a socket every so often and this determines how long you wait during a poll.
soLinger - Determines how long to keep the connection after close() has been called.
From my experience of using FTP, they normally just reconnect if the connection closes - it's not normally vital to have a constant uninterrupted connection unlike in other applications.
What are you using FTP for - it's normally not that time critical a service ...
As for ClientPools, I happened to write a demo project.
commons-pool-ftp
I am getting a little bit annoyed by the ftp protocol,
in our experience, it would meet broken pipe when testing on the client that just getting from the pool.
testOnBorrow=true
Configure
protected static ThreadLocal<FTPClient> ftpClientContainer = new
ThreadLocal<>();
Then use:
//login() will be your login method to FTP:
ftpClientContainer.set(ftpLogin());
Then in each method add:
FTPClient ftpClient = ftpClientContainer.get();
and finely when done:
//ftpDisconnect () will be your disconnect method to FTP:
ftpDisconnect(ftpClientContainer.get());
In JMS it is easy to find out if a connection is lost, a exception happens. But how do I find out if the connection is there again?
Scenario: I use JMS to communicate with my server. Now my connection breaks (server is down), which results in a exception. So far so good. If the server is up again and the connection is reestablished, how do I know that?
I don't see any Listeners which would facilitate such information.
Ahhh...the old exception handling/reconnection conundrum.
There are some transport providers that will automatically reconnect your application for you and some who make the app drive reconnection. In general the reconnections hide the exception from the application. The down side is that you don't want the app to hang forever if all the remote messaging nodes are down so ultimately, you must include some reconnection logic.
Now here's the interesting part - how do you handle the exceptions in a provider neutral way? The JMS exception is practically worthless. For example, a "security exception" can be that the Java security policies are too restrictive, that the file system permissions are too restrictive, that the LDAP credentials failed, that the connection to the transport failed, that the open of the queue or topic failed or any of dozens of other security-related problems. It's the linked exception that has the details from the transport provider that really help debug the problem. My clients have generally taken one of three different approaches here...
Treat all errors the same. Close all objects and reinitialize them. this is JMS portable.
Allow the app to inspect the linked exceptions to distinguish between fatal and transient errors (i.e. auth error vs. queue full). Not provider portable.
Provider-specific error-handling classes. A hybrid of the other two.
In your case, the queue and topic objects are probably only valid in the context of the original connection. Assuming a provider who reconnects automatically the fact that you got an exception means reconnect failed and the context for queue and topic objects could not be restored. Close all objects and reconnect.
Whether you want to do something more provider-specific such as distinguish between transient and permanent errors is one of those "it depends" things and you'll have to figure that out on a case-by-case basis.
The best way to monitor for connection exception is setting an exception listener, for example:
ConnectionFactory connectionFactory = (ConnectionFactory) context.lookup("jmsContextName");
connection = connectionFactory.createConnection();
connection.setExceptionListener(new ExceptionListener() {
#Override
public void onException(JMSException exception) {
logger.error("ExceptionListener triggered: " + exception.getMessage(), exception);
try {
Thread.sleep(5000); // Wait 5 seconds (JMS server restarted?)
restartJSMConnection();
} catch (InterruptedException e) {
logger.error("Error pausing thread" + e.getMessage());
}
}
});
connection.start();
JMS spec does not describe any transport protocol, it does not say anything about connections (i.e. should broker keep them alive or establish a new connection for every session). So, I think what you mean by
Now my connection breaks (server is down), which results in a exception.
is that you are trying to send a message and you are getting a JmsException.
I think, the only way to see if broker is up is to try to send a message.
Your only option in the case of a Connection based JMSException is to attempt to reestablish the connection in your exception handler, and retry the operation.