We Keep Coding

JavaMail connection timeout is not working as per properties

Javax mail version used 1.6.2
manually setting JavaMailSender
Timeout thing I tried with mail.smtp.timeout & mail.smtps.timeout.
And, I tried with both String & Integer value 3000.
String timeOut = "3000";
Properties pros = new Properties();
pros.put("mail.smtp.timeout", timeOut);
pros.put("mail.smtp.connectiontimeout", timeOut);
pros.put("mail.smtp.writetimeout", timeOut);
pros.put("mail.smtp.auth", "true");
pros.put("mail.smtp.starttls.enable", "true");
jmailSender.setJavaMailProperties(pros);
return jmailSender;
It's taking around 7 seconds without any fail.
Since by default is infinite, so most probably it is not setting somehow
Are any properties missing or something else?

The properties mail.smtp.connectiontimeout and `mail.smtps.connectiontimeout only apply while establishing the connection. It is not related to any timeouts during transport.
The properties mail.smtp.timeout and mail.smtps.timeout are related to the time blocked waiting for a read. This is related to reading SMTP response codes.
The properties mail.smtp.writetimeout and mail.smtps.writetimeout are related to writing chunks of data which can vary in size.
None of these timeouts represent a deadline for a single transaction of sending a mime message. What is happening is that there is no single action (connect, read, write) that is exceeding the 3000ms.
For example, connect could take 1000ms, followed by say 30 requests (write) and response parsing (reads) that take 100ms, and set of say 3 writes to send the message that take 1000ms each due to the speed of the network and size of the message. That is 1000 + (30 * 100) + (3 * 1000) = 7000ms total time but no single action exceeded the timeouts.
In a test environment
Set all timeouts to 3000.
Set connectimeout to 1 and test. You should see the connection fail.
Restart the test by setting it back to 3000 and set timeout to 1. You should see the reads fail.
Restart the test by setting it back to 3000 and set writetimeout to 1. You should see the transport fail.
If the test doesn't act this way you either haven't set the properties correctly (typo or smtp vs. smtps). Or you are really lucky to have such low latency.

Invoking Seagull Diameter Client using Java

i need to send some messages from my java web application to some servers using Diameter protocol, in particular CCR-CCA scenario. I had a look at jdiameter opensource project, but my usecase does not require such complexity, since that i just need to send a single request and log the response (actually i don't even need the CER-CEA part).
So i thought i could just have used Seagull running under my webapp. I downloaded Seagull (for Windows), and what i'm trying to do is basically to run the .bat file coming from Seagull for the diameter environment from my java environment.
That's what i've done till now..
1) A simple test to invoke the client.. Here wrapper simply sets working dir and starts the process
public static void main(String[] args) {
List<String> cmd=new ArrayList<>();
cmd.add("cmd.exe");
cmd.add("/c");
cmd.add("my_start_client.bat");
JavaProcessBuilderWrapper wrapper = new JavaProcessBuilderWrapper();
Process p = wrapper.createProcess(RedirectErrorsTo.STDERR,
new HashMap<>(), new File("my_working_dir"), cmd);
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
StringBuilder output = new StringBuilder();
String line;
try {
while ((line = reader.readLine()) != null) {
output.append(line);
}
System.out.println(line);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
2) I modified the client's and server's .bat files coming from Seagull to use CCR-CCA protocol.
Running Java main with this configuration caused a
Fatal: Keyboard saved configuration failure error
on my logs.
3) So, as mentioned here i further modified my client's .bat file to run in background mode, adding -bg at the end. Now my client's bat look like this
#ECHO OFF
rem
"Diameter Start Script Sample"
"Local env"
SET RUN_DIR=C:\Program Files\Seagull
set PATH=%PATH%;%RUN_DIR%
set LD_LIBRARY_PATH=%RUN_DIR%
set RUN_DIR=%RUN_DIR%\diameter-env\run
cd %RUN_DIR%
cls
mode 81,25
echo "Seagull Diameter Client Sample Start"
seagull -conf ..\config\conf.client.xml -dico ..\config\base_ro_3gpp.xml -scen ..\scenario\ccr-cca.ro.client.xml -log ..\logs\ccr-cca.client.log -llevel ETM -bg
pause
Since i was facing some troubles, to keep things simple, i just tried to make it work at least via cmd (not using my java method), but i think background mode is messing around, because now when i start my server and then my client in bg mode, sometimes i get a
Fatal: Forking error
but the most of the times, the client send a single message and then on my console i see that my software is causing connection abort (error code -1), and from the log i see that the channel just get closed, and my client does not even receive an answer. (NB for now i left the configuration files untouched)
Has any of you faced this behaviour? Is something else closing the connection (firewall perhaps)? Do i have to provide other configurations to make this work?
Once i can get this working, can i use my java web app (with a method similar to the one i already mentioned) to make diameter calls?
Thanks in advance, any help is really welcomed.

Connecting to an MQ with Jmeter

I've been assigned the task of using Apache Jmeter to connect to an MQ. Unfortunately, I'm not the admin of the MQ, and all my attempts to get more information about it have gotten me nowhere. What I have now is a queue name (call it q), a queue manager (its name, anyway; call it v), a host (w), a port (x), a channel (y), a user (z), and a test message I'm supposed to send across. The object of the game is straightforward: send the test message from Apache Jmeter to the MQ (after which I'll ask the admins whether the message went through or not). In other words, I need help figuring out what to do with Jmeter.
The MQ is version 8.0.0.4. I already have Jmeter installed, so I don't need advice on that (unless there's some special way it should have been installed for this task).
The links provided in this question's answer didn't get me very far. They seemed largely unrelated to what I was trying to do (and also imprecise in their instructions).

Download 8.0.0.4-WS-MQ-Install-Java-All file
Run it like java -jar 8.0.0.4-WS-MQ-Install-Java-All.jar and accept the license agreement
Add all .jars from wmq/JavaSE/ folder to JMeter Classpath
Restart JMeter to pick the .jars up
Add JSR223 Sampler to your Test Plan and put the following code into "Script" area:
import com.ibm.msg.client.jms.JmsFactoryFactory
import com.ibm.msg.client.wmq.WMQConstants
import javax.jms.Session
// 1
def hostName = "127.0.0.1"
def hostPort = 1414
def channelName = "DEV.APP.SVRCONN"
def queueManagerName = "QM1"
def queueName = "DEV.QUEUE.1"
// 2
def ff = JmsFactoryFactory.getInstance(WMQConstants.WMQ_PROVIDER)
def cf = ff.createConnectionFactory()
// 3
cf.setStringProperty(WMQConstants.WMQ_HOST_NAME, hostName)
cf.setIntProperty(WMQConstants.WMQ_PORT, hostPort)
cf.setStringProperty(WMQConstants.WMQ_CHANNEL, channelName)
cf.setIntProperty(WMQConstants.WMQ_CONNECTION_MODE, WMQConstants.WMQ_CM_CLIENT)
cf.setStringProperty(WMQConstants.WMQ_QUEUE_MANAGER, queueManagerName)
// 4
def conn = cf.createConnection("app", "")
def sess = conn.createSession(false, Session.AUTO_ACKNOWLEDGE)
// 5
def destination = sess.createQueue(queueName)
conn.start()
See IBM MQ testing with JMeter - Learn How article for more information if needed.

FTP to 1and1.com

The code below works to two other sites I've tried, but will not work with my domain hosted by 1and1. The return code is always 500 - Permanent Negative Completion reply.
I know I'm connecting because FTPReply.isPositiveCompletion(reply) returns true. I tried it with both reading file off phone storage and sending it from a byte array populated early in the code. This is the byte array. Both return 500. Both work on the other sites.
If I don't use enterLocalPassiveMode() the code stops executing on the storeFile call. No exception, no socket time-out. It just ends there and the async task will not call again in that session. The code does not do that on the other sites.
I've tried both ASCII and BINARY file types. Both return 500. 1and1 site says to use my domain and port 21. I can connect with CoreFTP and read and write using both of the accounts I've set up.
I also tired ftp4j and had the same response with all scenarios so went back to Apache because the code was already written with robust error trapping.
I've tried both mydomain.com and ftp.mydomian.com. 500 on both. I also tried the dot quad I can see in the CoreFTP window, but i get "cannot resolve host name" with the Apache Java code. Maybe not a static IP?
This is what CoreFTP does. It connects on port 21 and then goes in to passive mode and an ASCII data connection.
It's a long shot, but has anyone else ever FTPed to their 1and1 domain using Java in Android Studio?
Greg
Resolving mydomain.com...
Connect socket #5684 to xx.xx.xx.xxx, port 21...
220 Microsoft FTP Service
USER ftp79815757-0
331 Password required for ftp79815757-0.
PASS **********
230 User logged in.
SYST
215 Windows_NT
Keep alive off...
PWD
257 "/ftp79815757-0" is current directory.
PASV
227 Entering Passive Mode (xx,xxx,xx,xxx,xxx,xxx).
LIST
Connect socket #5700 to xx.xx.xx.xx, port 62894...
150 Opening ASCII mode data connection.
226 Transfer complete.
Transferred 51 bytes in 0.094 seconds
FTPClient mFtpClient = new FTPClient();
String ip = "my domain dot com";
String userName = "ftp79815757-0";
String pass = "password";
mFtpClient.connect(InetAddress.getByName(ip));
mFtpClient.login(userName, pass);
int reply = mFtpClient.getReplyCode();
if (FTPReply.isPositiveCompletion(reply)) {
mFtpClient.setFileType(FTP.BINARY_FILE_TYPE);
//one thread said this would do the trick
mFtpClient.enterLocalPassiveMode();
mFtpClient.enterRemotePassiveMode();
InputStream stream = new ByteArrayInputStream(imageData);
//I have two accounts. One points to images_in
/*if (!mFtpClient.changeWorkingDirectory("images_in")) {
Log.e("ChangeDir", String.valueOf(mFtpClient.getReplyCode()));
}*/
if (!mFtpClient.storeFile("remoteName.jpg", stream)) {
Log.e("FTPUpload", String.valueOf(mFtpClient.getReplyCode()));
}
stream.close();
mFtpClient.disconnect();
}

Finally got it. The main problem was that I was using an old version of the Apache library. The Jar I was using was commons-net-1.4.jar. Someone in another thread pointed me to commons-net-3.3.jar.
I commented out both mFtpClient.enterLocalPassiveMode() and mFtpClient.enterRemotePassiveMode(), and with some trail and error it worked with FTP.BINARY_FILE_TYPE and not ASCII_FILE_TYPE. ASCII got the file there, but it was garbage.

How can I ensure that my HttpClient 4.1 does not leak sockets?

My server uses data from an internal web service to construct its response, on a per request basis. I'm using Apache HttpClient 4.1 to make the requests. Each initial request will result in about 30 requests to the web service. Of these, 4 - 8 will end up with sockets stuck in CLOSE_WAIT, which never get released. Eventually these stuck sockets exceed my ulimit and my process runs out of file descriptors.
I don't want to just raise my ulimit (1024), because that will just mask the problem.
The reason I've moved to HttpClient is that java.net.HttpUrlConnection was behaving the same way.
I have tried moving to a SingleClientConnManager per request, and calling client.getConnectionManager().shutdown() on it, but sockets still end up stuck.
Should I be trying to solve this so that I end up with 0 open sockets while there are no running requests, or should I be concentrating on request persistence and pooling?
For clarity I'm including some details which may be relevant:
OS: Ubuntu 10.10
JRE: 1.6.0_22
Language: Scala 2.8
Sample code:
val cleaner = Executors.newScheduledThreadPool(1)
private val client = {
val ssl_ctx = SSLContext.getInstance("TLS")
val managers = Array[TrustManager](TrustingTrustManager)
ssl_ctx.init(null, managers, new java.security.SecureRandom())
val sslSf = new org.apache.http.conn.ssl.SSLSocketFactory(ssl_ctx, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER)
val schemeRegistry = new SchemeRegistry()
schemeRegistry.register(new Scheme("https", 443, sslSf))
val connection = new ThreadSafeClientConnManager(schemeRegistry)
object clean extends Runnable{
override def run = {
connection.closeExpiredConnections
connection.closeIdleConnections(30, SECONDS)
}
}
cleaner.scheduleAtFixedRate(clean,10,10,SECONDS)
val httpClient = new DefaultHttpClient(connection)
httpClient.getCredentialsProvider().setCredentials(new AuthScope(AuthScope.ANY), new UsernamePasswordCredentials(username,password))
httpClient
}
val get = new HttpGet(uri)
val entity = client.execute(get).getEntity
val stream = entity.getContent
val justForTheExample = IOUtils.toString(stream)
stream.close()
Test: netstat -a | grep {myInternalWebServiceName} | grep CLOSE_WAIT
(Lists sockets for my process that are in CLOSE_WAIT state)
Post comment discussion:
This code now demonstrates correct usage.

One needs to pro-actively evict expired / idle connections from the connection pool, as in the blocking I/O model connections cannot react to I/O events unless they are being read from / written to. For details see
http://hc.apache.org/httpcomponents-client-dev/tutorial/html/connmgmt.html#d4e631

I've marked oleg's answer as correct, as it highlights an important usage point about HttpClient's connection pooling.
To answer my specific original question, though, which was "Should I be trying to solve for 0 unused sockets or trying to maximize pooling?"
Now that the pooling solution is in place and working correctly the application throughput has increased by about 150%. I attribute this to not having to renegotiate SSL and multiple handshakes, instead reusing persistent connections in accordance with HTTP 1.1.
It is definitely worth working to utilize pooling as intended, rather than trying to hack around with calling ThreadSafeClientConnManager.shutdown() after each request etcetera. If, on the other hand, you were calling arbitrary hosts and not reusing routes the way I am you might easily find that it becomes necessary to do that sort of hackery, as the JVM might surprise you with the long life of CLOSE_WAIT designated sockets if you're not garbage collecting very often.

I had the same issue and solved it using the suggesting found here: here. The author touches on some TCP basics:
When a TCP connection is about to close, its finalization is negotiated by both parties. Think of it as breaking a contract in a civilized manner. Both parties sign the paper and it’s all good. In geek talk, this is done via the FIN/ACK messages. Party A sends a FIN message to indicate it wants to close the socket. Party B sends an ACK saying it received the message and is considering the demand. Party B then cleans up and sends a FIN to Party A. Party A responds with the ACK and everyone walks away.
The problem comes in
when B doesn’t send its FIN. A is kinda stuck waiting for it. It has
initiated its finalization sequence and is waiting for the other party
to do the same.
He then mentions RFC 2616, 14.10 to suggest setting up an http header to solve this issue:
postMethod.addHeader("Connection", "close");
Honestly, I don't really know the implications of setting this header. But it did stop CLOSE_WAIT from happening on my unit tests.