We Keep Coding

How to validate is PoolingHttpClientConnectionManager is applied on jersey client

Below is the code snippet I am using for jersey client connection pooling.
ClientConfig clientConfig = new ClientConfig();
clientConfig.property(ClientProperties.CONNECT_TIMEOUT, defaultConnectTimeout);
clientConfig.property(ClientProperties.READ_TIMEOUT, defaultReadTimeout);
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(50);
cm.setDefaultMaxPerRoute(5);
clientConfig.property(ApacheClientProperties.CONNECTION_MANAGER, cm);
clientConfig.connectorProvider(new ApacheConnectorProvider());
How can I validate that my client is using connection pooling? Is poolStats.getAvailable() count is valid way of making sure ? In my case this available count is 1 when I tested client.

Yes, the count can be 1, but to confirm you can try following steps.
You can first add a thread that keep running in background and prints the existing state of poolstats at some interval, lets say every 60 sec. You can use below logic. Ensure you are referring to same PoolingHttpClientConnectionManager object instance in below logic code running as a part of background thread.
Then, try calling the logic which internally makes call to external service using the mentioned jersey client in continuation (may be in for loop)
You should see different logs (in your thread logic) getting printed which would confirm that the jersey client is actually using the pooled configuration.
Logic:
PoolStats poolStats = cm.getTotalStats();
Set<HttpRoute> routes = cm.getRoutes();
if(CollectionUtils.isNotEmpty(routes)) {
for (HttpRoute route : routes) {
PoolStats routeStats = poolingHttpClientConnectionManager.getStats(route);
int routeAvailable = routeStats.getAvailable();
int routeLeased = routeStats.getLeased();
int routeIdle = (routeAvailable - routeLeased);
log.info("Pool Stats for Route - Host = {}, Available = {} , Leased = {}, Idle = {}, Pending = {}, Max = {} " ,
route.getTargetHost(), routeAvailable, routeLeased, routeIdle, poolStats.getPending(), poolStats.getMax());
}
}
int available = poolStats.getAvailable();
int leased = poolStats.getLeased();
int idle = (available - leased);
log.info("Pool Stats - Available = {} , Leased = {}, Idle = {}, Pending = {}, Max = {} " ,
available, leased, idle, poolStats.getPending(), poolStats.getMax());

httpClient connection pool manager validateAfterInactivity

I am looking at the documentation of PoolingHttpClientConnectionManager https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html
There is an API setValidateAfterInactivity. validateAfterInactivity is not very clear to me. It says - "Defines period of inactivity in milliseconds after which persistent connections must be re-validated prior to being leased to the consumer"
How exactly does it re-validate the connection? Wanted to understand the process. Does it send any http request to server or something to re-validate, or its something else?
What is the criteria/mechanism it uses to revalidate the connection? How does it all work?

It use JDBC connection to do validate.
final ManagedHttpClientConnection conn = poolEntry.getConnection();
if (conn != null) {
conn.activate();
} else {
poolEntry.assignConnection(connFactory.createConnection(null));
}
if (log.isDebugEnabled()) {
log.debug("Connection leased: " + ConnPoolSupport.formatStats(
poolEntry.getConnection(), route, state, pool));
}
source code here

How to set time limit for creating HBase connection

I'm currently using HBase v0.98.6. I would like to check the current connection status from an external Java program. Right now, I'm doing something like this to check:
connectionSuccess = true;
try {
HConnection hConnection = createConnection(config);
} catch (Exception ex) {
connectionSuccess = false;
}
When the connection is working, this returns fairly quickly. The problem is when the connection is not working, and it takes 20 minutes for it to finally return connectionSuccess=false. Is there a way to reduce this time limit, as I'm just interested in getting the connection status at the current time?

The reason it takes so long is that by default if the connection fails it will retry multiple times (I think 6? don't quote me), and each connection attempt takes a while. Try a combination of these commands to limit time per connection before timeout, and number of permitted retry attempts.
hbase.client.retries.number = 3
hbase.client.pause = 1000
zookeeper.recovery.retry = 1 (i.e. no retry)
Credit to Lars from http://hadoop-hbase.blogspot.com/2012/09/hbase-client-timeouts.html

You can set the retries value to 1 to get the status of the connection at the current time.
Configuration conf = HBaseConfiguration.create();
conf.setInt("hbase.client.retries.number",1);
conf.setInt("zookeeper.recovery.retry",0);
Or you can use the below in-built HbaseAdmin method which does the same thing.
connectionSuccess = true;
try
{
HBaseAdmin.checkHBaseAvailable(config);
}
catch(MasterNotRunningException e)
{
connectionSuccess = false;
}

My org.apache.hadoop.conf.Configuration object contains following key value pairs:
Configuration conf = HBaseConfiguration.create();
//configuring timeout and retry parameters
conf.set("hbase.rpc.timeout", "10000");
conf.set("hbase.client.scanner.timeout.period", "10000");
conf.set("hbase.cells.scanned.per.heartbeat.check", "10000");
conf.set("zookeeper.session.timeout", "10000");
conf.set("phoenix.query.timeoutMs", "10000");
conf.set("phoenix.query.keepAliveMs", "10000");
conf.set("hbase.client.retries.number", "3");
conf.set("hbase.client.pause", "1000");
conf.set("zookeeper.recovery.retry", "1");

How to maintain Channel API connection when internet is offline

I am using Google AppEngine's Channel API. I am having some issue to re-start the lost connection due to the user's network connection. When you loose the internet connection, channel call onError but it will not call onClose. As far as JavaScript object is concerned, the channel socket is open.
How do you handle lost connection due to the internet issue? I am thinking of 1) by trigger to close the channel and re-open it when RPC unrelated to the channel somewhere in the application succeeds for the first time (which indicates the internet is alive again) or 2) Use timer that runs all the time and pings the server for network status (which was the point of introducing the long polling to avoid consuming unwanted resource this way). Any other ideas would be great.
Observation:
When the internet connection is dead, onError is called in incremental interval (10sec, 20sec, 40sec) twice. Once the internet connection is back, channel does not resume connection. It stops working without any indication that it is dead.
Thanks.

When you see the javascript console, presumably you will see "400 Unknown SID Error".
If so, here is my workaround for this. This is a service module for AngularJS, but please look at the onerror callback. Please try this workaround and let me know if it works or not.
Added: I neglected to answer your main question, but in my opinion, it is hard to determine if you're connected to the internet unless actually pinging the "internet". So probably you may want to use some retry logic similar to the following code with some tweak. In the following example, I just retrying 3 times, but you can do it more with some back offs. However, I think the best way to handle this is, when the app consume the retry max count, you can indicate the user that the app lost the connection, ideally showing a button or a link to re-connect to the channel service.
And, you can also track the connection on the server side, see:
https://developers.google.com/appengine/docs/java/channel/#Java_Tracking_client_connections_and_disconnections
app.factory('channelService', ['$http', '$rootScope', '$timeout',
function($http, $rootScope, $timeout) {
var service = {};
var isConnectionAlive = false;
var callbacks = new Array();
var retryCount = 0;
var MAX_RETRY_COUNT = 3;
service.registerCallback = function(pattern, callback) {
callbacks.push({pattern: pattern, func: callback});
};
service.messageCallback = function(message) {
for (var i = 0; i < callbacks.length; i++) {
var callback = callbacks[i];
if (message.data.match(callback.pattern)) {
$rootScope.$apply(function() {
callback.func(message);
});
}
}
};
service.channelTokenCallback = function(channelToken) {
var channel = new goog.appengine.Channel(channelToken);
service.socket = channel.open();
isConnectionAlive = false;
service.socket.onmessage = service.messageCallback;
service.socket.onerror = function(error) {
console.log('Detected an error on the channel.');
console.log('Channel Error: ' + error.description + '.');
console.log('Http Error Code: ' + error.code);
isConnectionAlive = false;
if (error.description == 'Invalid+token.' || error.description == 'Token+timed+out.') {
console.log('It should be recovered with onclose handler.');
} else {
// In this case, we need to manually close the socket.
// See also: https://code.google.com/p/googleappengine/issues/detail?id=4940
console.log('Presumably it is "Unknown SID Error". Try closing the socket manually.');
service.socket.close();
}
};
service.socket.onclose = function() {
isConnectionAlive = false;
console.log('Reconnecting to a new channel');
openNewChannel();
};
console.log('A channel was opened successfully. Will check the ping in 20 secs.');
$timeout(checkConnection, 20000, false);
};
function openNewChannel(isRetry) {
console.log('Retrieving a clientId.');
if (isRetry) {
retryCount++;
} else {
retryCount = 0;
}
$http.get('/rest/channel')
.success(service.channelTokenCallback)
.error(function(data, status) {
console.log('Can not retrieve a clientId');
if (status != 403 && retryCount <= MAX_RETRY_COUNT) {
console.log('Retrying to obtain a client id')
openNewChannel(true);
}
})
}
function pingCallback() {
console.log('Got a ping from the server.');
isConnectionAlive = true;
}
function checkConnection() {
if (isConnectionAlive) {
console.log('Connection is alive.');
return;
}
if (service.socket == undefined) {
console.log('will open a new connection in 1 sec');
$timeout(openNewChannel, 1000, false);
return;
}
// Ping didn't arrive
// Assuming onclose handler automatically open a new channel.
console.log('Not receiving a ping, closing the connection');
service.socket.close();
}
service.registerCallback(/P/, pingCallback);
openNewChannel();
return service;
}]);

How can i get already occupied ports?

I am developing eclipse plugin for our organization . We are opening multiple servers[minimum 10 servers] on a user machine using this plugin via eclipse . For starting servers we want port numbers which has been not already binded . For that , I am using serversocket to check this . I think it's a costly operation to open a serversocket object . Internally serversocket will check the port is already binded or not It takes minimum 50 milliseconds . Here is my code to return a free port . Is there any way to find already occupied ports without using OS Commands and opening ServerSocket ?
/**
*Tries 100 times
* #param port
* modes
* 1.increment - 1
* This mode increment the port with your start value . But it's costly operation because each time we open a socket and check the port is free .
* 2.decrement - 2
* Invert of increment.
* 3.random - 3
* Randomly choose based on your starting point
* #return
*/
public static String getDefaultPort(int port , int mode){
int retry = 100;
int random = 3;
int increment = 1;
int decrement = 2;
while(true){
//this is for preventing stack overflow error.
if(retry < 1){ //retries 100 times .
break;
}
if(mode==increment){
port++;
}else if(mode == decrement){
port--;
}else if(mode == random){
port = (int) (port+Math.floor((Math.random()*1000)));
}
if(validate(port+"")){
long end = System.currentTimeMillis();
return port+"";
}
}
return "";
}
public boolean validate(String input) {
boolean status = true;
try {
int port = Integer.parseInt(input);
ServerSocket ss = new ServerSocket(port);
ss.close();
}
catch (Exception e) {
status = false;
}
return status;
}

The quickest way would be to run native netstat command and parse the output. It's available on Windows as well and Linux platform. A typical netstat command output is as follows
Proto Local Address Foreign Address State
TCP MYHOST:8080 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:9090 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:3389 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:7717 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:51114 MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:netbios-ssn MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:netbios-ssn MYHOST.mydomain.co.in:0 LISTENING
TCP MYHOST:2573 hj-lyncfe.mydomain.co.in:5061 ESTABLISHED
TCP MYHOST:2591 mail.mydomain.co.in:8502 ESTABLISHED
TCP MYHOST:2593 mail.mydomain.co.in:8502 ESTABLISHED
The ports of your interest are in the column Local Address with State in LISTENING