We Keep Coding

Connection reset - Java (underlying socket status remain established) Azure VM

I am debugging one problem of connection reset and need some help.
Here is the background
Using java version 8, apache httpClient 4.5.2
I have a following program, which runs successfully on windows 10, 7 but end up with connection reset on Azure windows server 2016 VM.
import java.io.IOException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import org.apache.commons.codec.binary.Base64;
import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;
public class TestConnectionReset
{
static PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
static {
connManager.setMaxTotal(10);
connManager.setDefaultMaxPerRoute(2);
}
public static void main(String[] args) throws ClientProtocolException, IOException, InterruptedException {
while (true) {
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
RequestConfig config = RequestConfig.custom().setConnectTimeout(1800000).setConnectionRequestTimeout(1800000)
.setSocketTimeout(1800000).build();
clientBuilder.setDefaultRequestConfig(config);
clientBuilder.setConnectionManager(connManager);
String userName = "xxxxx";
String password = "xxxxx";
String userNamePasswordPair = String.valueOf(userName) + ":" + password;
String authenticationData = "Basic " + new String((new Base64()).encode(userNamePasswordPair.getBytes()));
HttpPost post = new HttpPost("https://url/rest/oauth/token");
Map<String, String> requestBodyMap = new HashMap<String, String>();
requestBodyMap.put("grant_type", "client_credentials");
String req = getFormUrlEncodedBodyFromMap(requestBodyMap);
StringEntity stringEntity = new StringEntity(req);
post.setEntity(stringEntity);
post.setHeader("Authorization", authenticationData);
post.setHeader("Content-Type", "application/x-www-form-urlencoded");
CloseableHttpClient closeableHttpClient = clientBuilder.build();
HttpResponse response = closeableHttpClient.execute(post);
Header[] hs = response.getAllHeaders();
for (Header header : hs) {
System.out.println(header.toString());
}
System.out.println(EntityUtils.toString(response.getEntity()));
Thread.sleep(10*60*1000L);
}
}
public static String getFormUrlEncodedBodyFromMap(Map<String, String> formData) {
StringBuilder requestBody = new StringBuilder();
Iterator<Map.Entry<String, String>> itrFormData = formData.entrySet().iterator();
while (itrFormData.hasNext()) {
Map.Entry<?, ?> entry = (Map.Entry)itrFormData.next();
requestBody.append(entry.getKey()).append("=").append(entry.getValue());
if (itrFormData.hasNext()) {
requestBody.append("&");
}
}
return requestBody.toString();
}
}
I am using pooling httpclient connection manager. 1st request in 1st time loop execution succeeded but subsequent iteration of for loop with next request fails.
My findings
If we see underlying socket connection on windows 10, after 1st request socket goes into CLOSE_WAIT state and next request executes with closing the existing connection and creating new connection.
Actually server closes the connection in duration of 5 minutes. But windows 10 able to detect it and re-initiate the connection when next request is triggered.
Now, on windows server 2016, I can see that netstat shows socket ESTABLISHED state. Means connection is ready to use and in that, it picks up the same connection and finally server has already closed it so results into connection reset error.
I suspect its an environmental issue, where server 2016 is keeping socket ESTABLISHED even after server has terminated it, but on windows 10 socket status changed to CLOSE_WAIT.
Help on this is much appreciated

Finally got the root cause,
Its issue with microsoft azure. They are using SNAT and closing outbound TCP connections after 4 minute idle time. This wasted my 5 days to figureout.
Means if you are connected with server with keep-alive and hope that you can reuse the connection till server time out and sends FIN. But before that if idle period reaches to 4 minutes, azure kills it. BOOM!!. Worst part is, it is not even notifying server or client with RST, means violating TCP and questioning its reliability.
clientBuilder.setKeepAliveStrategy(new ConnectionKeepAliveStrategy() {
#Override
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
// TODO Auto-generated method stub
return 3*60*1000;
}
});
Using above code, I managed to close connection on 3 minute expiry and close it before azure kills it.

How to listen and generate respones for client in server in java?

This is my server side code in java listening on port 1880 and making http client request from browser/client.java code in localhost which gives connection refused error.I copied that server code from a webpage.I am not sure what exactly this code is doing.
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
public class ServerHandlingHttpRequest {
public void start() throws IOException
{
InetSocketAddress addr=new InetSocketAddress(1880);
System.out.println(addr.getPort());
System.out.println(addr.getAddress());
System.out.println(addr.getHostName());
System.out.println(addr.toString());
HttpServer server=HttpServer.create(addr,0);
server.createContext("/", new MyHandler());
server.setExecutor(null);
server.start();
System.out.println("Server Listening");
System.out.println(server.getAddress());
server.stop(60);
}
class MyHandler implements HttpHandler {
public void handle(HttpExchange t) throws IOException {
System.out.println("Request Arrived");
String response = "This is the response";
t.sendResponseHeaders(200, response.length());
OutputStream os = t.getResponseBody();
os.write(response.getBytes());
os.close();
}
}
public static void main(String[] args) throws IOException
{
ServerHandlingHttpRequest w=new ServerHandlingHttpRequest();
w.start();
}
}

That code simply starts listening to the port that you have specified, as an HTTP listener. Which means, any HTTP Request that might come into that port will be intercepted by your program. And when it intercepts, it looks for the context path that you have configured to handle the response to the request.
So, when you run this program, it starts listening at port 1880. When a URL http://localhost:1880/ is hit, it will be intercepted by the handler, MyHandler. The response will be written to the OutputStream which will send back the request back to the client.
One of the problems that I see is that you are stopping the server after printing some statements, while the example that you have refered is not doing so. May be, remove server.stop(60) statement and check whether the issue still persists.

About detecting proxy setting using rome api

The below provides an error when in lan behind a proxy ,but works properly outside lan/proxy.
Please let me know how I can rectify it
I used the code to detect proxy setting and it gave
Detecting Windows/IE proxy setting using Java
I am getting: proxy hostname : DIRECT No Proxy Does this mean I am not behind a Proxy Server?
I'm trying to use java rome-fetcher to acquire rss feeds for processing. Everything works fine when I have direct internet access.
However, I need to be able to run my application behind a proxy server.
The below provides an error when in lan ,but works properly outside lan
Exception in thread "main" java.net.ConnectException: Connection timed out: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
import java.util.Properties;
import java.net.*;
import java.io.*;
import java.io.FileWriter;
import java.io.Writer;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Iterator;
import java.util.List;
import com.sun.syndication.feed.synd.SyndEntry;
import com.sun.syndication.feed.synd.SyndFeed;
import com.sun.syndication.io.SyndFeedInput;
import com.sun.syndication.io.SyndFeedOutput;
import com.sun.syndication.io.XmlReader;
public class RomeLibraryExample {
#SuppressWarnings("unchecked")
public static void main(String[] args) throws Exception {
URL url = new URL("http://rss.cnn.com/rss/cnn_topstories.rss");
//System.setProperty("http.proxyHost", "DIRECT");
// System.setProperty("http.proxyPort", "8080");
HttpURLConnection httpcon = (HttpURLConnection)url.openConnection(Proxy.NO_PROXY);
// Reading the feed
SyndFeedInput input = new SyndFeedInput();
SyndFeed feed = input.build(new XmlReader(httpcon));
List<SyndEntry> entries = feed.getEntries();
Iterator<SyndEntry> itEntries = entries.iterator();
while (itEntries.hasNext()) {
SyndEntry entry = itEntries.next();
System.out.println("Title: " + entry.getTitle());
System.out.println("Link: " + entry.getLink());
System.out.println("Author: " + entry.getAuthor());
System.out.println("Publish Date: " + entry.getPublishedDate());
System.out.println("Description: " + entry.getDescription().getValue());
System.out.println();
}
}
}

Checking the type of a web proxy

How do I determine whether a web proxy IP is of type HTTP or SOCKS4/5 with java?
Thank you.

As mentioned in the comments from my other answer, if you know the IP address of a proxy server and want to detect what type it is, you could try each proxy type in Java until one works.
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.SocketException;
import java.net.URL;
import java.net.URLConnection;
import java.util.Arrays;
import java.util.List;
public class ProxyTest
{
public static void main(String... args)
throws IOException
{
InetSocketAddress proxyAddress = new InetSocketAddress("myproxyaddress", 1234);
Proxy.Type proxyType = detectProxyType(proxyAddress);
System.out.println(proxyAddress + " is a " + proxyType + " proxy.");
}
public static Proxy.Type detectProxyType(InetSocketAddress proxyAddress)
throws IOException
{
URL url = new URL("http://www.google.com");
List<Proxy.Type> proxyTypesToTry = Arrays.asList(Proxy.Type.SOCKS, Proxy.Type.HTTP);
for (Proxy.Type proxyType : proxyTypesToTry)
{
Proxy proxy = new Proxy(proxyType, proxyAddress);
//Try with SOCKS
URLConnection connection = null;
try
{
connection = url.openConnection(proxy);
//Can modify timeouts if default timeout is taking too long
//connection.setConnectTimeout(1000);
//connection.setReadTimeout(1000);
connection.getContent();
//If we get here we made a successful connection
return(proxyType);
}
catch (SocketException e) //or possibly more generic IOException?
{
//Proxy connection failed
}
}
//No proxies worked if we get here
return(null);
}
}
In this code, it first tries to connect to www.google.com using the proxy at myproxyaddress with SOCKS, and if that fails it will try using it as an HTTP proxy, returning the method that worked, or null if none worked.

If you want to determine the type of proxy being used from Java, you can use ProxySelector and Proxy.
e.g.
import java.net.Proxy;
import java.net.ProxySelector;
import java.net.URI;
import java.util.List;
public class ProxyTest
{
public static void main(String... args)
{
System.setProperty("java.net.useSystemProxies", "true");
List<Proxy> proxyList = ProxySelector.getDefault().select(URI.create("http://www.google.com"));
if (!proxyList.isEmpty())
{
Proxy proxy = proxyList.get(0);
switch (proxy.type())
{
case DIRECT:
System.out.println("Direct connection - no proxy.");
break;
case HTTP:
System.out.println("HTTP proxy: " + proxy.address());
break;
case SOCKS:
System.out.println("SOCKS proxy: " + proxy.address());
break;
}
}
}
}

NoClassDefFoundError and Netty

First to say I'm n00b in Java. I can understand most concepts but in my situation I want somebody to help me. I'm using JBoss Netty to handle simple http request and using MemCachedClient check existence of client ip in memcached.
import org.jboss.netty.channel.ChannelHandler;
import static org.jboss.netty.handler.codec.http.HttpHeaders.*;
import static org.jboss.netty.handler.codec.http.HttpHeaders.Names.*;
import static org.jboss.netty.handler.codec.http.HttpResponseStatus.*;
import static org.jboss.netty.handler.codec.http.HttpVersion.*;
import com.danga.MemCached.*;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
import org.jboss.netty.buffer.ChannelBuffer;
import org.jboss.netty.buffer.ChannelBuffers;
import org.jboss.netty.channel.ChannelFuture;
import org.jboss.netty.channel.ChannelFutureListener;
import org.jboss.netty.channel.ChannelHandlerContext;
import org.jboss.netty.channel.ExceptionEvent;
import org.jboss.netty.channel.MessageEvent;
import org.jboss.netty.channel.SimpleChannelUpstreamHandler;
import org.jboss.netty.handler.codec.http.Cookie;
import org.jboss.netty.handler.codec.http.CookieDecoder;
import org.jboss.netty.handler.codec.http.CookieEncoder;
import org.jboss.netty.handler.codec.http.DefaultHttpResponse;
import org.jboss.netty.handler.codec.http.HttpChunk;
import org.jboss.netty.handler.codec.http.HttpChunkTrailer;
import org.jboss.netty.handler.codec.http.HttpRequest;
import org.jboss.netty.handler.codec.http.HttpResponse;
import org.jboss.netty.handler.codec.http.HttpResponseStatus;
import org.jboss.netty.handler.codec.http.QueryStringDecoder;
import org.jboss.netty.util.CharsetUtil;
/**
* #author The Netty Project
* #author Andy Taylor (andy.taylor#jboss.org)
* #author Trustin Lee
*
* #version $Rev: 2368 $, $Date: 2010-10-18 17:19:03 +0900 (Mon, 18 Oct 2010) $
*/
#SuppressWarnings({"ALL"})
public class HttpRequestHandler extends SimpleChannelUpstreamHandler {
private HttpRequest request;
private boolean readingChunks;
/** Buffer that stores the response content */
private final StringBuilder buf = new StringBuilder();
protected MemCachedClient mcc = new MemCachedClient();
private static SockIOPool poolInstance = null;
static {
// server list and weights
String[] servers =
{
"lcalhost:11211"
};
//Integer[] weights = { 3, 3, 2 };
Integer[] weights = {1};
// grab an instance of our connection pool
SockIOPool pool = SockIOPool.getInstance();
// set the servers and the weights
pool.setServers(servers);
pool.setWeights(weights);
// set some basic pool settings
// 5 initial, 5 min, and 250 max conns
// and set the max idle time for a conn
// to 6 hours
pool.setInitConn(5);
pool.setMinConn(5);
pool.setMaxConn(250);
pool.setMaxIdle(21600000); //1000 * 60 * 60 * 6
// set the sleep for the maint thread
// it will wake up every x seconds and
// maintain the pool size
pool.setMaintSleep(30);
// set some TCP settings
// disable nagle
// set the read timeout to 3 secs
// and don't set a connect timeout
pool.setNagle(false);
pool.setSocketTO(3000);
pool.setSocketConnectTO(0);
// initialize the connection pool
pool.initialize();
// lets set some compression on for the client
// compress anything larger than 64k
//mcc.setCompressEnable(true);
//mcc.setCompressThreshold(64 * 1024);
}
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) throws Exception {
HttpRequest request = this.request = (HttpRequest) e.getMessage();
if(mcc.get(request.getHeader("X-Real-Ip")) != null)
{
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK);
response.setHeader("X-Accel-Redirect", request.getUri());
ctx.getChannel().write(response).addListener(ChannelFutureListener.CLOSE);
}
else {
sendError(ctx, NOT_FOUND);
}
}
private void writeResponse(MessageEvent e) {
// Decide whether to close the connection or not.
boolean keepAlive = isKeepAlive(request);
// Build the response object.
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK);
response.setContent(ChannelBuffers.copiedBuffer(buf.toString(), CharsetUtil.UTF_8));
response.setHeader(CONTENT_TYPE, "text/plain; charset=UTF-8");
if (keepAlive) {
// Add 'Content-Length' header only for a keep-alive connection.
response.setHeader(CONTENT_LENGTH, response.getContent().readableBytes());
}
// Encode the cookie.
String cookieString = request.getHeader(COOKIE);
if (cookieString != null) {
CookieDecoder cookieDecoder = new CookieDecoder();
Set<Cookie> cookies = cookieDecoder.decode(cookieString);
if(!cookies.isEmpty()) {
// Reset the cookies if necessary.
CookieEncoder cookieEncoder = new CookieEncoder(true);
for (Cookie cookie : cookies) {
cookieEncoder.addCookie(cookie);
}
response.addHeader(SET_COOKIE, cookieEncoder.encode());
}
}
// Write the response.
ChannelFuture future = e.getChannel().write(response);
// Close the non-keep-alive connection after the write operation is done.
if (!keepAlive) {
future.addListener(ChannelFutureListener.CLOSE);
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e)
throws Exception {
e.getCause().printStackTrace();
e.getChannel().close();
}
private void sendError(ChannelHandlerContext ctx, HttpResponseStatus status) {
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, status);
response.setHeader(CONTENT_TYPE, "text/plain; charset=UTF-8");
response.setContent(ChannelBuffers.copiedBuffer(
"Failure: " + status.toString() + "\r\n",
CharsetUtil.UTF_8));
// Close the connection as soon as the error message is sent.
ctx.getChannel().write(response).addListener(ChannelFutureListener.CLOSE);
}
}
When I try to send request like http://127.0.0.1:8090/1/2/3
I'm getting
java.lang.NoClassDefFoundError: com/danga/MemCached/MemCachedClient
at httpClientValidator.server.HttpRequestHandler.<clinit>(HttpRequestHandler.java:66)
I believe it's not related to classpath. May be it's related to context in which mcc doesn't exist.
Any help appreciated
EDIT:
Original code http://docs.jboss.org/netty/3.2/xref/org/jboss/netty/example/http/snoop/package-summary.html
I've modified some parts to fit my needs.

Why do you think this is not classpath related? That's the kind of error you get when the jar you need is not available. How do you start your app?
EDIT
Sorry - i loaded and tried the java_memcached-release_2.5.2 bundle in eclipse and found no issue so far. Debugging the class loading revealed nothing unusual. I can't help besides some more hints to double check:
make sure your download is correct. download and unpack again. (are the com.schooner.* classes available?)
make sure you use > java 1.5
make sure your classpath is correct and complete. The example you have shown does not include netty. Where is it.
I'm not familiar with interactions stemming from adding a classpath to the manifest. Maybe revert to plain style, add all jars needed (memcached, netty, yours) to the classpath and reference the main class to start, not a startable jar file