Using connection pool with JSch - java

I'm using JSch for file upload over SFTP. In its current state each thread opens and closes connection when needed.
If it possible to use connection pooling with JSch in order to avoid overhead caused by large number of connection opening and closing?
Here is a example of function called from inside of thread
public static void file_upload(String filename) throws IOException {
JSch jsch = new JSch();
Session session = null;
try {
session = jsch.getSession("user", "server_name", 22);
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword("super_secre_password");
session.connect();
Channel channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftpChannel = (ChannelSftp) channel;
FileInputStream inputSrr = new FileInputStream(filename);
try {
sftpChannel.put(inputSrr, "/var/temp/"+filename);
} catch (SftpException e) {
e.printStackTrace();
} finally {
if (inputSrr != null) {
inputSrr.close();
}
}
sftpChannel.exit();
session.disconnect();
} catch (JSchException e) {
e.printStackTrace();
} catch (SftpException e) {
e.printStackTrace();
}
}

For that I would prefer commons-pool. ;)

Here's an implementation of Ssh Connection pool
http://www.javacodegeeks.com/2013/02/pool-of-ssh-connections-using-apache-keyedobjectpool.html
you can use grep4j to use this pool
https://code.google.com/p/grep4j/source/browse/trunk/src/main/java/org/grep4j/core/command/linux/SessionFactory.java?r=354
Also make sure you can access the server from the execution machine. For instance if the target server is not in your reach. It'll throw connection timeout.

I wold like to share with you our implementation, We have used Session Manager of jsch-extension library
First of all you need to implement pool object factory that is responsible for lifecycle of pooled objects:
public class ChannelSftpConnectionsFactory extends BasePooledObjectFactory<ChannelSftp> {
private SessionManager sessionManager;
public ChannelSftpConnectionsFactory(final SessionManager sessionManager) {
this.sessionManager = sessionManager;
}
//Create and open channel
#Override
public ChannelSftp create() throws JSchException {
ChannelSftp channelSftp = (ChannelSftp) sessionManager.getSession().openChannel("sftp");
channelSftp.connect();
return channelSftp;
}
//wrapping
#Override
public PooledObject<ChannelSftp> wrap(final ChannelSftp channelSftp) {
return new DefaultPooledObject<>(channelSftp);
}
#Override
//disconnect channel on destroy
public void destroyObject(final PooledObject<ChannelSftp> pooledObject) {
ChannelSftp sftp = pooledObject.getObject();
disconnectChannel(sftp);
}
void disconnectChannel(final ChannelSftp sftp) {
if (sftp.isConnected()) {
sftp.disconnect();
}
}
#Override
//reset channel current folder to home if someone was walking on another folders
public void passivateObject(final PooledObject<ChannelSftp> p) {
ChannelSftp sftp = p.getObject();
try {
sftp.cd(sftp.getHome());
} catch (SftpException ex) {
log.error("Could not reset channel to home folder, closing it");
disconnectChannel(sftp);
}
}
#Override
//validate object before it is borrowed from pool. If false object will be removed from pool
public boolean validateObject(final PooledObject<ChannelSftp> p) {
ChannelSftp sftp = p.getObject();
return sftp.isConnected() && !sftp.isClosed();
}
}
Now you could create pool using configured factory:
ObjectPool<ChannelSftp> createPool(final SessionManager sessionManager, final GenericObjectPoolConfig<ChannelSftp> poolConfig) {
return PoolUtils.synchronizedPool(new GenericObjectPool<>(buildFactory(sessionManager), poolConfig));
}
PooledObjectFactory<ChannelSftp> buildFactory(final SessionManager sessionManager) {
return PoolUtils.synchronizedPooledFactory(new ChannelSftpConnectionsFactory(sessionManager));
}
This java doc would help you to configure pool properly : https://commons.apache.org/proper/commons-pool/api-2.6.0/org/apache/commons/pool2/impl/BaseGenericObjectPool.html
Do not forget about correct borrowing and returning of object into pool: https://commons.apache.org/proper/commons-pool/api-2.6.0/org/apache/commons/pool2/ObjectPool.html
Object obj = null;
try {
obj = pool.borrowObject();
try {
//...use the object...
} catch(Exception e) {
// invalidate the object
pool.invalidateObject(obj);
// do not return the object to the pool twice
obj = null;
} finally {
// make sure the object is returned to the pool
if(null != obj) {
pool.returnObject(obj);
}
}
} catch(Exception e) {
// failed to borrow an object
}

Related

How to stream data between server and client

I am using Apache Mina SSHD to communicate with two Android devices. I have server-side code and client-side I'd like to establish a connection and start streaming data (byte array packet) from server to client and client to server until a button pressed to disconnect. How can I do this?
SERVER
I use this link SSHD SERVER stackOverflow
public void startSSHServer() {
int port = 22;
SshServer sshd = SshServer.setUpDefaultServer();
sshd.setPort(port);
sshd.setKeyPairProvider(new SimpleGeneratorHostKeyProvider(
"src/test/resources/hostkey.ser"));
sshd.setSubsystemFactories(Arrays
.<NamedFactory<Command>>asList(new SftpSubsystem.Factory()));
sshd.setCommandFactory(new ScpCommandFactory());
sshd.setShellFactory(new ProcessShellFactory(new String[]{"/system/bin/sh", "-i", "-l"})); // necessary if you want to type commands over ssh
sshd.setPasswordAuthenticator(new PasswordAuthenticator() {
#Override
public boolean authenticate(String u, String p, ServerSession s) {
return ("sftptest".equals(u) && "sftptest".equals(p));
}
});
try {
sshd.start();
} catch (IOException e) {
e.printStackTrace();
}
}
CLIENT
I use this link SSHD CLIENT geeksforgeeks
Thread thread = new Thread(new Runnable() {
#Override
public void run() {
try {
// Connection establishment and authentication
try (ClientSession session = client.connect(username, host, port).verify(10000).getSession()) {
session.addPasswordIdentity(password);
session.auth().verify(50000);
System.out.println("Connection establihed");
// Create a channel to communicate
channel = session.createChannel(Channel.CHANNEL_SHELL);
System.out.println("Starting shell");
ByteArrayOutputStream responseStream = new ByteArrayOutputStream();
channel.setOut(responseStream);
// Open channel
channel.open().verify(5, TimeUnit.SECONDS);
try (OutputStream pipedIn = channel.getInvertedIn()) {
pipedIn.write(command.getBytes());
pipedIn.flush();
}
// Close channel
channel.waitFor(EnumSet.of(ClientChannelEvent.CLOSED),
TimeUnit.SECONDS.toMillis(5));
// Output after converting to string type
String responseString = new String(responseStream.toByteArray());
System.out.println(responseString);
shellOutput.setText(responseString);
} catch (IOException e) {
e.printStackTrace();
} finally {
client.stop();
}
} catch (Exception e) {
e.printStackTrace();
}
}
});
thread.start();

Cannot start a ChannelSftp channel connection to local apache sshd server (java)

I am trying to test files arriving in an SFTP server. I set up a local sshd server using the following code
#Before
public void setUp() throws Exception{
SshServer sshd = SshServer.setUpDefaultServer();
sshd.setKeyPairProvider(new SimpleGeneratorHostKeyProvider());
sshd.setPasswordAuthenticator(new PasswordAuthenticator() {
public boolean authenticate(String username, String password, ServerSession session) {
return true;
}
});
sshd.setPort(22);
sshd.setFileSystemFactory(new FileSystemFactory() {
#Override
public FileSystem createFileSystem(org.apache.sshd.common.session.Session session) throws IOException {
return null;
}
});
sshd.start();
}
I use Jsch as a client to try and connect to this server. Unfortunately, whenever I try to call channel.connect(), I get com.jcraft.jsch.JSchException: failed to send channel request exception. The code for the Jsch is shown below:
public void testServer(){
try{
JSch jSch = new JSch();
Session session = jSch.getSession("user", "localhost",22);
Properties configTemp = new Properties();
configTemp.put("StrictHostKeyChecking", "no");
session.setConfig(configTemp);
session.connect();
ChannelSftp channel = (ChannelSftp)session.openChannel("sftp");
channel.connect();
if(channel.isConnected()){
System.out.println("Connected");
}
}
catch (Exception e){
e.printStackTrace();
}
}
Can anyone help with what I am doing wrong and how to fix this? Thank you
You have to implement a subsystem factory, like:
sshd.setSubsystemFactories(Collections.singletonList(new SftpSubsystemFactory()));

How can i keep a Jsch Session alive on android

I am writing an app to connect over SSH to a server. My intention is to give the user of the app an internet connection as long as they are connected to the server (The SSH Script runs as Android Service). The Problem is, when I start a session and create a channel everything works fine. But after about 20-30 minutes (sometimes up to several hours) the channel and the session closes.
Connect function:
public String connecting(
String username,
final String password,
String hostname,
int port) {
try {
Log.d("MainActivity", "Start JSch session and connect");
jsch = new JSch();
session = jsch.getSession(username, hostname, port);
session.setPassword(password);
// Avoid asking for key confirmation
Properties prop = new Properties();
prop.put("StrictHostKeyChecking", "no");
session.setConfig(prop);
session.connect();
session.setServerAliveInterval(15);
session.setServerAliveCountMax(100);
Channel channel = session.openChannel("shell");
channel.setInputStream(System.in);
channel.setOutputStream(System.out);
channel.connect();
InputStream in = channel.getInputStream();
serviceStatus = true;
streamtext = "";
byte[] tmp = new byte[1024];
while (true) {
while (in.available() > 0) {
int i = in.read(tmp, 0, 1024);
if (i < 0) break;
streamtext = new String(tmp, 0, i);
}
}
if (channel.isClosed()) {
if (in.available() > 0) continue;
Log.d(TAG, "exit-status: " + channel.getExitStatus());
break;
}
try {
Thread.sleep(1000);
} catch (Exception ee) {
ee.printStackTrace();
}
}
return streamtext;
} catch (Exception except){
except.printStackTrace();
passErrorToActivity("Error: Connection error");
return "Error: Connection error";
}
}
Start function:
public void start(){
try {
new AsyncTask<Integer, Void, Void>() {
#Override
protected Void doInBackground(Integer... params) {
try {
passMessageToActivity(connecting(user, password, host, port));
} catch (Exception ex) {
ex.printStackTrace();
}
return null;
}
}.execute(1);
} catch (Exception exc) {
exc.printStackTrace();
}
"passMessageToActivity" Just creats an intent and sends the "streamtext" to the MainActivity
I've already tryed it with Session#setServerAliveInterval(int milliseconds) but it didn't work. Is there any posibility to keep the session and channel alive?
I've seen the solution of this user but this doesn't work for me because it's important that the connection between server and service is always up.
If setting keepalives only does not help, you need to keep the session busy with more high-level actions. Like sending a dummy command, like pwd. Though you may need to explain, why you have "shell" session open, that look suspicious.
Anyway, there's no way you can guarantee that a connection stays alive. You would have to deal with losing connection occasionally anyway.

Execute multiple ssh commands and get result for each command

I have opened session with jsch on Android, this way:
SshObjects Connect(String username, String password, String hostname, int port)
{
JSch jsch=new JSch();
try
{
sshObjects._session = jsch.getSession(username, hostname, port);
}
catch (JSchException e)
{
e.printStackTrace();
return null;
}
sshObjects._session.setPassword(password);
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
sshObjects._session.setConfig(config);
try
{
sshObjects._session.connect();
}
catch (JSchException e)
{
e.printStackTrace();
return null;
}
try
{
sshObjects._channel = (ChannelExec) sshObjects._session.openChannel("exec");
}
catch (JSchException e)
{
e.printStackTrace();
return null;
}
connected = true;
return sshObjects;
}
And then, to execute some command on opened session and get result, I did this:
private String ExecuteCommand(SshCommandsEnum cmdType)
{
String result = "";
switch (cmdType)
{
case SERVER_INFO:
sshObjects._channel.setCommand("uname --all");
break;
.......
}
try
{
BufferedReader in=new BufferedReader(new InputStreamReader(sshObjects._channel.getInputStream()));
//sshObjects._channel.disconnect();
try
{
sshObjects._channel.connect();
}
catch (JSchException e)
{
e.printStackTrace();
}
String msg=null;
try
{
while((msg=in.readLine())!=null)
{
System.out.println(msg);
result += msg;
}
sshObjects._channel.disconnect();
}
catch (IOException e)
{
e.printStackTrace();
}
}
catch (IOException e)
{
return "";
}
return result;
}
So I want to open my session only once. And then execute commands as "exec" on it. It works for first command executed after connect - everything seems to be ok and I can get result succesfully. But when I call "Execute Command" again, it doesn't work anymore. My thread hangs on sshObjects._channel.connect(); and nothing works. When I try to disconnect (close channel and session) and connect again - the same. I can connect and disconnect without any problems only if I don't even try to execute command.
However, I don't experience this issue without this:
BufferedReader in=new BufferedReader(new InputStreamReader(sshObjects._channel.getInputStream()));
But, obviously I need it to get my command output. So what's the problem? Do you have any idea what am I doing wrong?
The ChannelExec is not re-usable, so you need to instantiate it for each command.

Why does an SFTP connection still exist after the JSCH Channel has been closed?

When the code below has finished running, netstat -a|grep sftp shows an open SFTP connection. It also shows up as an open connection in JProfiler.
channel.isConnected() in the finally block prints false. Any ideas why the connections is not being closed as I'm at a loss?
public static void clean() {
com.jcraft.jsch.ChannelSftp channel = null;
try {
channel = Helper.openNewTLSftpChannel();
channel.connect();
channel.cd(remoteFileDirectory);
List<ChannelSftp.LsEntry> list = channel.ls("*." + fileType);
for (ChannelSftp.LsEntry file : list) {
String fileName = file.getFilename();
DateTime fileDate = new DateTime(parseDateFromFileName(fileName));
//if this file is older than the cutoff date, delete from the SFTP share
if (fileDate.compareTo(cleanupCutoffdate) < 0) {
channel.rm(fileName);
}
}
} catch (Exception exception) {
exception.printStackTrace();
} finally {
if (channel != null) {
channel.disconnect();
System.out.println(channel.isConnected());
}
}
}
Adding openNewTLSftpChannel() below:
public static ChannelSftp openNewSftpChannel(String privateKeyFileName, String password, String username, String host, int port)
throws ConfigurationErrorException {
JSch jsch = new JSch();
File sftpPrivateFile = new File(privateKeyFileName);
Channel channel;
try {
if (!sftpPrivateFile.canRead()) {
throw new ConfigurationErrorException("File access error: " + sftpPrivateFile.getAbsolutePath());
}
jsch.addIdentity(sftpPrivateFile.getAbsolutePath(), password);
Session session = jsch.getSession(username, host, port);
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect();
channel = session.openChannel("sftp");
} catch (JSchException jschException) {
throw new ConfigurationErrorException("File access error: " + sftpPrivateFile.getAbsolutePath());
}
return (ChannelSftp) channel;
}
If you take a look at the JSCH examples for SFTP you'll see how the session is terminated:
//setup Session here
...
session.connect();
...
Channel channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftpChannel = (ChannelSftp) channel;
...run sftp logic...
//close sessions here
sftpChannel.exit();
session.disconnect();
You'll notice that there are two parts to the connection and disconnection; the Session object and the Channel object.
In my code I use the Session object to set my authentication information, and the Channel object to execute the sftp commands I need.
In your instance, you're creating the Session object in your openNewSftpChannel method, but it is never closed, hence your session stays alive.
For further context, check out the examples.
Robert H is correct, you need to exit your channel and disconnect your session. I wanted to add that the session exists even when the channel has been closed. Since you create your session within a try block inside a method, it seems you have lost your session, but you can get it back using 'getSession' on your sftpChannel channel.
You can change your finally block to this:
} finally {
if (channel != null) {
Session session = channel.getSession();
channel.disconnect();
session.disconnect();
System.out.println(channel.isConnected());
}
}

Categories