We Keep Coding

Is there a minimum timeout to InetAddress isReachable method?

I'm having a weird behavior with the isReachable method of InetAddress class.
Method prototype is :
public boolean isReachable(int timeout)
When using a timeout > 1500 (ms), the method waits the exact time
given as argument (if the target IP is not reachable of course...).
When using timeout < 1500, the method waits 1000ms maximum...
The code is quite simple :
InetAddress addr = null;
String ip = "10.48.2.169";
try {
addr = InetAddress.getByName(ip);
} catch (UnknownHostException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Timestamp s = new Timestamp(System.currentTimeMillis());
System.out.println(s + "\t Starting tests :");
pingTest(addr, 100);
pingTest(addr, 500);
pingTest(addr, 1000);
pingTest(addr, 1500);
pingTest(addr, 2000);
pingTest(addr, 2500);
Where pingTest is defined by :
public static void pingTest(InetAddress addr, int timeout) {
boolean result = false;
try {
result = addr.isReachable(timeout);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Timestamp s = new Timestamp(System.currentTimeMillis());
System.out.println(s + "\t (" + timeout + ") " + addr.toString() + " " + result);
}
Then the output is :
2017-09-07 16:45:41.573 Starting tests :
2017-09-07 16:45:42.542 (100) /10.48.2.169 false
2017-09-07 16:45:43.542 (500) /10.48.2.169 false
2017-09-07 16:45:44.541 (1000) /10.48.2.169 false
2017-09-07 16:45:46.041 (1500) /10.48.2.169 false
2017-09-07 16:45:48.041 (2000) /10.48.2.169 false
2017-09-07 16:45:50.541 (2500) /10.48.2.169 false
So the question is : Is there a minimum timeout to InetAddress isReachable method ? (like 1500 in my doubt, but I doubt, huge timeout...)
Or maybe I just made a huge mistake that I still miss...
Tell me if this isn't clear enough.
Thanks for your help and thoughts.

First you should notice that the behavior of INetAddress.isReachable is not the same on each platform supported by Java. I will assume you work on Windows.
When undocumented behavior happens you should always look at the source if they are available. The java.net implementation for windows is here for the OpenJDK (it should be quite similar for the Oracle JVM, but I am not sure of this).
What we saw in the isReachable method implementation is:
they don't rely on ping because they find the Windows ICMP protocol implementation too unreliable
they pass the timeout value to the NET_Wait function
So the isReachable method doesn't perform a ping and we need to check what the NET_Wait do with the timeout to understand why a less than 1 second timeout isn't possible.
The NET_Wait function is defined here: src/windows/native/java/net/net_util_md.c
It consist in an infinite loop which break when these events occurs during the select function call:
NET_WAIT_CONNECT on the socket file descriptor (socket is connected to the remote host)
The timeout ends
The select function is documented in a man page you may consult here. This man page tells us that the timeout can "be rounded up to the system clock granularity, and kernel scheduling delays mean that the blocking interval may overrun by a small amount".
This is why there is no guarantee on the minimal timeout value. Also, I think that the documentation doesn't state any minimal timeout value because the implementation differs on OSs supported by the JVM.
Hope this helps you understanding why.
However, to achieve a wanted timeout you may test the reachability in a separate task. You wait until the task returns the result, or if you wait more than your timeout you cancel the task or ignore its results.

SSH Server Identification never received - Handshake Deadlock [SSHJ]

We're having some trouble trying to implement a Pool of SftpConnections for our application.
We're currently using SSHJ (Schmizz) as the transport library, and facing an issue we simply cannot simulate in our development environment (but the error keeps showing randomly in production, sometimes after three days, sometimes after just 10 minutes).
The problem is, when trying to send a file via SFTP, the thread gets locked in the init method from schmizz' TransportImpl class:
#Override
public void init(String remoteHost, int remotePort, InputStream in, OutputStream out)
throws TransportException {
connInfo = new ConnInfo(remoteHost, remotePort, in, out);
try {
if (config.isWaitForServerIdentBeforeSendingClientIdent()) {
receiveServerIdent();
sendClientIdent();
} else {
sendClientIdent();
receiveServerIdent();
}
log.info("Server identity string: {}", serverID);
} catch (IOException e) {
throw new TransportException(e);
}
reader.start();
}
isWaitForServerIdentBeforeSendingClientIdent is FALSE for us, so first of all the client (we) send our identification, as appears in logs:
"Client identity String: blabla"
Then it's turn for the receiveServerIdent:
private void receiveServerIdent() throws IOException
{
final Buffer.PlainBuffer buf = new Buffer.PlainBuffer();
while ((serverID = readIdentification(buf)).isEmpty()) {
int b = connInfo.in.read();
if (b == -1)
throw new TransportException("Server closed connection during identification exchange");
buf.putByte((byte) b);
}
}
The thread never gets the control back, as the server never replies with its identity. Seems like the code is stuck in this While loop. No timeouts, or SSH exceptions are thrown, my client just keeps waiting forever, and the thread gets deadlocked.
This is the readIdentification method's impl:
private String readIdentification(Buffer.PlainBuffer buffer)
throws IOException {
String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
if (ident.isEmpty()) {
return ident;
}
if (!ident.startsWith("SSH-2.0-") && !ident.startsWith("SSH-1.99-"))
throw new TransportException(DisconnectReason.PROTOCOL_VERSION_NOT_SUPPORTED,
"Server does not support SSHv2, identified as: " + ident);
return ident;
}
Seems like ConnectionInfo's inputstream never gets data to read, as if the server closed the connection (even if, as said earlier, no exception is thrown).
I've tried to simulate this error by saturating the negotiation, closing sockets while connecting, using conntrack to kill established connections while the handshake is being made, but with no luck at all, so any help would be HIGHLY appreciated.
: )

I bet following code creates a problem:
String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
if (ident.isEmpty()) {
return ident;
}
If the IdentificationStringParser.parseIdentificationString() returns empty string, it will be returned to the caller method. The caller method will keep calling the while ((serverID = readIdentification(buf)).isEmpty()) since the string is always empty. The only way to break the loop would be if call to int b = connInfo.in.read(); returns -1... but if server keeps sending the data (or resending the data) this condition is never met.
If this is the case I would add some kind of artificial way to detect this like:
private String readIdentification(Buffer.PlainBuffer buffer, AtomicInteger numberOfAttempts)
throws IOException {
String ident = new IdentificationStringParser(buffer, loggerFactory).parseIdentificationString();
numberOfAttempts.incrementAndGet();
if (ident.isEmpty() && numberOfAttempts.intValue() < 1000) { // 1000
return ident;
} else if (numberOfAttempts.intValue() >= 1000) {
throw new TransportException("To many attempts to read the server ident").
}
if (!ident.startsWith("SSH-2.0-") && !ident.startsWith("SSH-1.99-"))
throw new TransportException(DisconnectReason.PROTOCOL_VERSION_NOT_SUPPORTED,
"Server does not support SSHv2, identified as: " + ident);
return ident;
}
This way you would at least confirm that this is the case and can dig further why .parseIdentificationString() returns empty string.

Faced a similar issue where we would see:
INFO [net.schmizz.sshj.transport.TransportImpl : pool-6-thread-2] - Client identity string: blablabla
INFO [net.schmizz.sshj.transport.TransportImpl : pool-6-thread-2] - Server identity string: blablabla
But on some occasions, there were no server response.
Our service would typically wake up and transfer several files simultaneously, one file per connection / thread.
The issue was in the sshd server config, we increased maxStartups from default value 10
(we noticed the problems started shortly after batch sizes increased to above 10)
Default in /etc/ssh/sshd_config:
MaxStartups 10:30:100
Changed to:
MaxStartups 30:30:100
MaxStartups
Specifies the maximum number of concurrent unauthenticated connections to the SSH daemon. Additional connections will be dropped until authentication succeeds or the LoginGraceTime expires for a connection. The default is 10:30:100. Alternatively, random early drop can be enabled by specifying the three colon separated values start:rate:full (e.g. "10:30:60"). sshd will refuse connection attempts with a probability of rate/100 (30%) if there are currently start (10) unauthenticated connections. The probability increases linearly and all connection attempts are refused if the number of unauthenticated connections reaches full (60).
If you cannot control the server, you might have to find a way to limit your concurrent connection attempts in your client code instead.

spymemcached: how can I tell whether memcached had connected successfully?

I'm using the following library: compile 'net.spy:spymemcached:2.12.0'
So if I do this, if memcachedclient is not connected successfully, it gives me an error:
mc.get(myvariable)
I want to check the connection status before I call mc.get() to prevent the error.
I thought maybe I could check this: mc.getConnection().connectionsStatus() but all it does it return a string that says:
Connection Status { pub-memcache-XXX active: false, authed: false, last read: 7 ms ago }
I thought it would just return a SUCCESS or a FAILURE but it returns a string.
The doc for the method is here: https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/MemcachedConnection.java
I have copied the relevant method below:
/**
* Construct a String containing information about all nodes and their state.
*
* #return a stringified representation of the connection status.
*/
public String connectionsStatus() {
StringBuilder connStatus = new StringBuilder();
connStatus.append("Connection Status {");
for (MemcachedNode node : locator.getAll()) {
connStatus
.append(" ")
.append(node.getSocketAddress())
.append(" active: ")
.append(node.isActive())
.append(", authed: ")
.append(node.isAuthenticated())
.append(MessageFormat.format(", last read: {0} ms ago",
node.lastReadDelta()));
}
connStatus.append(" }");
return connStatus.toString();
}
I don't know, I could parse out the active variable and check that but is there an easier way to check whether spymemcached was connected before I call get on it?

Instead of parsing the active variable, you can directly check if the node is active or not by accessing the MemcachedNode object for your MemcachedClient. You can implement it as shown below:
for (MemcachedNode node : mc.getNodeLocator().getAll()) {
if (!node.isActive()) {
System.out.println("Failed to connect to Memcached server");
mc.shutdown();
//Handle accordingly
}
}
You can use mc.shutdown() as MemcachedClient continues to reconnect to the server causing your program to hang.

How to write multithreaded algorithm to discovery available socket

For now I have sequential single threaded algorithm like this:
boolean conn = false;
for (InetSocketAddress addr : addrs) {
while (!conn) {
SocketChannel ch = SocketChannel.open();
ch.configureBlocking(true);
ch.socket().setTcpNoDelay(tcpNoDelay);
ch.socket().setKeepAlive(true);
try {
ch.socket().connect(addr, connTimeout);
conn = true;
....
}
catch (...) {
log("Not available: " + addr);
}
if (conn)
break;
}
}
But in some situations I could have quite a large list of addresses and available address could be in the end of this list - this will result into slow discovery of available address because I need to check all addresses one by one.
So my question - how I could implement algorithm that will try to connect to several addresses from different threads and will exit once first available address is found?
Also this algorithm should exit if no address available.
Any ideas how to code this?

Do all the connects in non-blocking mode, and then use a Selector on OP_CONNECT to find out when and if the connections succeed.

Java ProgramCall.run hangs

Busy trying to Call RPG function from Java and got this example from JamesA. But now I am having trouble, here is my code:
AS400 system = new AS400("MachineName");
ProgramCall program = new ProgramCall(system);
try
{
// Initialise the name of the program to run.
String programName = "/QSYS.LIB/LIBNAME.LIB/FUNNAME.PGM";
// Set up the 3 parameters.
ProgramParameter[] parameterList = new ProgramParameter[2];
// First parameter is to input a name.
AS400Text OperationsItemId = new AS400Text(20);
parameterList[0] = new ProgramParameter(OperationsItemId.toBytes("TestID"));
AS400Text CaseMarkingValue = new AS400Text(20);
parameterList[1] = new ProgramParameter(CaseMarkingValue.toBytes("TestData"));
// Set the program name and parameter list.
program.setProgram(programName, parameterList);
// Run the program.
if (program.run() != true)
{
// Report failure.
System.out.println("Program failed!");
// Show the messages.
AS400Message[] messagelist = program.getMessageList();
for (int i = 0; i < messagelist.length; ++i)
{
// Show each message.
System.out.println(messagelist[i]);
}
}
// Else no error, get output data.
else
{
AS400Text text = new AS400Text(50);
System.out.println(text.toObject(parameterList[1].getOutputData()));
System.out.println(text.toObject(parameterList[2].getOutputData()));
}
}
catch (Exception e)
{
//System.out.println("Program " + program.getProgram() + " issued an exception!");
e.printStackTrace();
}
// Done with the system.
system.disconnectAllServices();
The application Hangs at this lineif (program.run() != true), and I wait for about 10 minutes and then I terminate the application.
Any idea what I am doing wrong?
Edit
Here is the message on the job log:
Client request - run program QSYS/QWCRTVCA.
Client request - run program LIBNAME/FUNNAME.
File P6CASEL2 in library *LIBL not found or inline data file missing.
Error message CPF4101 appeared during OPEN.
Cannot resolve to object YOBPSSR. Type and Subtype X'0201' Authority
FUNNAME insert a row into table P6CASEPF through a view called P6CASEL2. P6CASEL2 is in a different library lets say LIBNAME2. Is there away to maybe set the JobDescription?

Are you sure FUNNAME.PGM is terminating and not hung with a MSGW? Check QSYSOPR for any messages.
Class ProgramCall:
NOTE: When the program runs within the host server job, the library list will be the initial library list specified in the job description in the user profile.

So I saw that my problem is that my library list is not setup, and for some reason, the user we are using, does not have a Job Description. So to over come this I added the following code before calling the program.run()
CommandCall command = new CommandCall(system);
command.run("ADDLIBLE LIB(LIBNAME)");
command.run("ADDLIBLE LIB(LIBNAME2)");
This simply add this LIBNAME, and LIBNAME2 to the user's library list.

Oh yes, the problem is Library list not set ... take a look at this discussion on Midrange.com, there are different work-around ...
http://archive.midrange.com/java400-l/200909/msg00032.html
...
Depe