I'm using JSch to pull data from a router, however the way the router's prompt works is preventing me from successfully reading the data.
The flow should go something like this:
router>drop_into_privileged_mode
Password: password_entered_here
router#give_me_data
...Data goes here...
router#
The problem I'm running into is that since the router drops back to a shell prompt after give_me_data, the InputStream never dies. The natural choice would be to run a command like give_me_data; exit; or give_me_data && exit, but unfortunately the router's operating system doesn't allow me to chain commands like that. Equally frustratingly, the command give_me_data takes so long to run that putting the exit command into the OutputBuffer does nothing (as it gets sent while the other command is still running and thus not interpreted).
Is there a way for me to preserve the connects of the InputStream, but kill the connection? If that's possible, then the buffer will have a natural end (as the connection is dead), and I can just copy that to a variable. Alternatively, if there's a way to transfer all information currently present in the stream out, then kill the connection, that would work too.
I've tried the solutions proposed in similar StackOverflow questions, such as this one and this one to no avail.
Additionally, here is the code that I'm currently using (It's a solution I devised that suffers from the same blocking problem):
InputStream in = channel.getInputStream(); //This gets the output from the router
byte[] buffer = new byte[1024];
StringBuffer stringBuffer = new StringBuffer();
while (true) {
while (in.available() > 0) {
int i = in.read(buffer, 0, 1024);
if (i < 0) {
break;
}
stringBuffer.append(new String(buffer, "UTF-8"));
}
System.out.println("done");
channel.disconnect(); // this closes the jsch channel
if (channel.isClosed()) {
if (in.available() > 0) {
continue;
}
System.out.println("exit-status: " + channel.getExitStatus());
break;
}
try{
Thread.sleep(1000);
}catch(InterruptedException e){
e.printStackTrace();
}
}
Related
I've gone through so many related StackOverflow questions for this that I'm getting lost in them, and I've coded this multiple ways, but none seem to solve this problem in a way that works for me: How can I send output to the same command and process multiple times while at the same time receiving input from this same process?
(See Input various strings to same process in Java for a similar question, but this ended with only a theoretical answer.)
The command (command line, from a C++ executable) loads a large file, and then I want to send input to it very quickly, get back the answer, do other stuff in between, then send different input and get the corresponding answer. Multiply this by thousands or millions of times.
One implementation, with threads:
ProcessBuilder pb = new ProcessBuilder(command.split(" "));
kenLMProcess = pb.start();
KenLMInThread lmInput = new KenLMInThread(kenLMProcess.getInputStream());
KenLMInThread lmError = new KenLMInThread(kenLMProcess.getErrorStream());
KenLMOutThread lmOutput = new KenLMOutThread(kenLMProcess.getOutputStream());
lmOutput.inStr = "Test . \n";
lmInput.start();
lmOutput.start();
lmError.start();
lmOutput.join();
lmInput.join();
lmError.join();
outStr = lmInput.newStr;
But join waits until the thread ends. What if I don't want to wait for it to end? I can't seem to figure out how to use wait() for that purpose. For one I'd prefer to not have to keep opening and closing a new output stream and input stream every time I query the command. But at least that's better than starting a new ProcessBuilder every time.
Here's what run() looks like for KenLMOutThread:
public void run() {
try {
pw.write(inStr+"\n");
pw.write('\n');
} catch (Exception e) {
System.out.println("Error while inputting to KenLM.");
e.printStackTrace();
} finally {
pw.flush();
try {
pw.flush();
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Supposedly flush is supposed to let it move on, and "\n" at the end is supposed to help, but it just hangs unless I use close. And if I use close, I can't use the OutputStream anymore. I'm also then unable to make a new OutputStream from the Process.
If it helps, here's a more simple implementation with everything together (taken from How to send EOF to a process in Java?):
Note that close() is used, and using flush() without close() causes the program to hang.
public static String pipe(String str, String command2) throws IOException, InterruptedException {
Process p2 = Runtime.getRuntime().exec(command2);
OutputStream out = p2.getOutputStream();
out.write(str.getBytes());
out.close();
p2.waitFor();
BufferedReader reader
= new BufferedReader(new InputStreamReader(p2.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
return sb.toString();
}
Other things I've tried:
Using exec(): Process kenLMProcess=Runtime.getRuntime().exec(command);
Putting the command process in its own thread: KenLMProcessThread procThread = new KenLMProcessThread(pb.start());
If the target process is hanging unless you close the output stream, the problem is at that end: it is reading until end of stream before doing anything. Nothing you can do about that at the sending end.
I'm building a web server and client which I will use on a Raspberry Pi. The server is written in Python and every time a client connects to it, it will return data to the client. This is working well and when I access the server through my browser, I see the correct data and when I refresh the page I get new data, so that is working.
The problem is that I'm now writing a client in Java to access the server and collect the data. The reason I use here Java is that I'm more experienced in it and I find building a GUI more convenient; I use python on the raspberry pi because it offers me a simpler way of interacting with the hardware.
The java client code I have is this:
while (true) {
Socket socket = new Socket("192.168.0.37", 9315);
PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
try {
System.out.println("Getting");
out.write("GIMME DATA!");
out.flush();
System.out.println("receiving");
String data;
while ((data = in.readLine()) != null) {
System.out.println("Line: " + data);
}
socket.close();
Thread.sleep(5000);
} catch (Exception e) {
e.printStackTrace();
}
}
This all works well, but I want to take the initial 3 objects out of the while loop (and obviously not close the connection). When I do that, I only can access the server once (it will give me the correct data) and on the second run in.readling() will return null which is weird. It seems that the inputstream only has the data from the initial run and no more bytes are written to it. To be clear, when putting the initial 3 objects IN the while loop, everything works well.
How can I take the objects out of the loop? It seems wasteful, time consuming and possibly memory consuming to create them every time...
Once you have reached to end of the file or BufferedReader reached to end of data, then how can it print again data from beginning. For example :
int i = 0;
BufferedReader br = new BufferedReader(new FileReader(new File("C:\\data.txt")));
while (true) {
String data = "";
while ((data = br.readLine()) != null) {
System.out.println(" data :" + data);
}
Thread.sleep(1000);
i++;
if (i == 2) {
break;
}
}
br.close();
In the above program, br reached to end of file, then it cannot point to the beginning. You have to create BufferedReder object every time when you enter while loop.
I am having a socket listener program running(eclipse) on a mac machine and iOS client app is sending image to it in Bytes format. Normally, Image bytes will be 40 K and above.
I am facing a strange issue while reading the image bytes in socket. I have checked many links, they are suggesting to use like below code for reading all the bytes. The issue is, its reading all the bytes and NOT coming out of 'While' loop. After reading all the bytes, just struck inside the while loop only. I don't know what to do? Could someone please help me to solve this issue?
InputStream input = socket.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] bufferr = new byte[1024];
int read = 0;
long numWritten = 0;
try {
// Tried both the below while conditions, both are giving same issue
// while ((read = input.read(bufferr, 0, bufferr.length)) != -1)
while ((read = input.read(bufferr)) > 0) {
baos.write(bufferr, 0, read);
numWritten += read;
System.out.println("numWritten: " + numWritten);
}
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
baos.flush();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
byte[] data = baos.toByteArray();
The below is my iOS code. I am closing the stream, still the same issue.
-(void) shareImage
{
AppDelegate *appDelegate = [UIApplication sharedApplication].delegate;
UIGraphicsBeginImageContext(appDelegate.window.bounds.size);
[appDelegate.window.layer renderInContext:UIGraphicsGetCurrentContext()];
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
NSData * data = UIImagePNGRepresentation(image);
//[data writeToFile:#"screenshot.png" atomically:YES];
NSLog(#"[data length] %i: ", [data length]);
self.sentPing = YES;
int num = [self.outputStream write:[data bytes] maxLength:([data length])];
if (-1 == num) {
NSLog(#"Error writing to stream %#: %#", self.outputStream, [self.outputStream streamError]);
}else{
NSLog(#"Wrote %i bytes to stream %#.", num, self.outputStream);
[self.outputStream close];
//NSTimer *myRegularTime = [NSTimer scheduledTimerWithTimeInterval:5.0 target:self selector:#selector(ShareNextScreen:) userInfo:nil repeats:NO];
}
}
input.read(buffer) will block until data is received. If the stream is closed, it will return -1 as you are testing for. But, since the stream is still open and is waiting for data to arrive, it'll block.
Since you did update your question, I will update my answer. Closing a stream is not the same as terminating a TCP session.
Closing a stream will put the connection into FIN_WAIT_1 or FIN_WAIT_2 and it needs to finish and reset to be fully closed. You need to tell your server that you're shutting down your client and then shut down, or tell the client you're shutting down the server, and then close. Basically, both sides need to close when they wish to terminate the connection. Closing also may, depending on your environment, not even do anything but release references.
In most implementations of low level socket APIs, you have socket_shutdown(2) which actually sends the FIN TCP packet for a mutual shutdown initiation.
Basically both parties need to close, or the connection will be stuck in a waiting state. This is a defined behavior in various RFCs. An explanation can be found here.
From the post I linked, you can review the diagram here.
You are reading to end of stream but the peer hasn't closed the connection. So, you block.
I am developing a tool to get client information, send to a server, and receive the information again (a proxy). I'm also trying to dump the data being received from the server. I can read the Integer representation of the inputStream, but I am not able to read the String format. I've tried the below example, but it hangs and never connects to the server. Also, System.out.println(inputStream.nextLine()) displays only one line and hangs.
public void run() {
try {
int i;
while ((i = inputStream.read()) != -1){
System.out.println(IOUtils.toString(inputStream));
outputStream.write(i);
}
} catch (IOException e) {
System.out.println("Lost connection to the client.");
}
}
My guess at this is that you're reading from the input stream, and then using the IOUtils library to read from the stream too. My suspicion is that your application is reading the first byte from the input stream, then reading the remainder of the inputstream with the IOUtils library, and then printing out the initial byte that was read.
It doesn't make any sense to call IOUtils.toString(inputstream) from within a loop. That method call will put all the data from the inputstream into a string. Why have the loop at all in this case?
You might want to try not using the IOUtils library for this. Just read a byte of data, push it into a StringBuilder, and then print that byte. In this approach, the loop would be necessary, and you'll probably get what you're looking for.
Try something like this, but modify it as necessary to print the data at the same time to your output stream:
public static String inputStreamToString(final InputStream is, final int bufferSize)
{
final char[] buffer = new char[bufferSize];
final StringBuilder out = new StringBuilder();
try {
final Reader in = new InputStreamReader(is, "UTF-8");
try {
for (;;) {
int rsz = in.read(buffer, 0, buffer.length);
if (rsz < 0)
break;
out.append(buffer, 0, rsz);
}
}
finally {
in.close();
}
}
catch (UnsupportedEncodingException ex) {
/* ... */
}
catch (IOException ex) {
/* ... */
}
return out.toString();
}
The code you posted doesn't attempt to connect to the server, but if any of it executes you must already have connected.
If your program is hanging in this code, either the server hasn't sent any data yet, or the IOUtils.toString() method probably tries to read to EOS, so if the peer doesn't close the connection you will block here forever.
If your program hangs at a readLine() call it means the peer hasn't sent a line to read.
Question edited following first comment.
My problem is mostly with java socket performance, and especially reading from the target server.
The server is a simple serversocket.accept() loop that create a client thread for every connection from firefox
Main problem is socket input stream reading that blocks for enormous amounts of time.
Client thread is as follows :
//Take an httpRequest (hc.apache.org), raw string http request, and the firefox socket outputstream
private void handle(httpRequest req, String raw, Outputstream out)
{
InputStream targetIn =null;
OutputStream targetOut = null;
Socket target = null;
try {
System.out.println("HANDLE HTTP");
String host = req.getHeaders("Host")[0].getValue();
URI uri = new URI(req.getRequestLine().getUri());
int port = uri.getPort() != -1 ? uri.getPort() : 80;
target = new Socket(host, port);
//**I have tried to play around with these but cannot seem to get a difference in performance**
target.setTcpNoDelay(true);
// target.setReceiveBufferSize(1024 *1024);
// target.setSendBufferSize(1024 * 1024);
//Get your plain old in/out streams
targetIn = target.getInputStream();
targetOut = target.getOutputStream();
//Send the request to the target
System.out.println("---------------Start response---------------");
targetOut.write(raw.getBytes());
System.out.println("request sent to target");
////Same as membrane
byte[] buffer = new byte[8 * 1024];
int length = 0;
try {
while((length = targetIn.read(buffer)) > 0) {
out.write(buffer, 0, length);
out.flush();
}
} catch(Exception e) {
e.printStackTrace();
}
System.out.println("closing out + target socket");
//IOUTILS
// long count = IOUtils.copyLarge(targetIn, out, 0L, 1048576L);
// int count = IOUtils.copy(targetIn, out);
// System.out.println("transfered : " + count );
//CHANNEL COPY
//
// ReadableByteChannel input = Channels.newChannel(targetIn);
// WritableByteChannel output = Channels.newChannel(out);
//
// ChannelTools.fastChannelCopy(input, output);
//
// input.close();
// output.close();
//CHAR TO CHAR COPY
// int c;
// while ((c = targetIn.read()) != -1) {
// out.write(c);
// }
target.close();
out.close();
System.out.println("-------------------- end response ------------------------------");
}
catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
The main problem lies in in the appropriate method to copy the target inputstream to the client (firefox) outputstream.
The site i am using to test this out is http://www.ouest-france.fr (new site with a load of images and makes loads of requests).
Ping time from workstation to target : 10ms
Normal Loading in iceweasel (debian firefox, firebug time) : 14 secs, 2.5MB
Loading behind this proxy : 14 minutes (firebug net panel is full of fake 404s, and aborted request that go back to black after a certain time, loads of requests are in blocking or waiting mode)
Now when executing i loadup visual vm, launch profiling with no class filter (to see where the app is really spending its time) and it spends 99% of its time in java.net.SocketInputStream.read(byte[], int, int), which is reading on the target socket input stream.
I think i have done my homework and been searching a testing different solutions juste about anywhere i could.
but performance never seems to improve.
I What i have already tried :
-Putting input and output streams into their buffered version, no change at all
-int to int copy, no change at all,
-classic byte[] array copy with variable sized arrays, no change at all
fiddling around with settcpnodelay, setsendbuffersize, setreceivebuffersize, could not get any change.
Was thinking of trying out nio socketchannels , but cannot find a way to do the socket to sslsocket hijacking.
So at the moment i am a bit stuck and searching for solutions.
I look at the source code of open sources proxies and cannot seem to find a fundamental difference in logic so i am completely lost with this.
Tried a other test :
export http_proxy="localhost:4242"
wget debiandvd.iso
Throughput gets to 2MB/s.
And threads seems to spend 66% time reading from target an 33% time writing to client
I am thinking that maybe i have to many threads running but running a test on www.google.com has much less requests going through but still the sames problems as www.ouest-france.fr
With the debian iso test i was thinking i had to many threads running (ouest-france is around 270 requests) but the google test (10 request) test seems to confirm that thread numbers are not the problem.
Any help will be appreciated.
Environment is debian, sun java 1.6, dev with eclipse and visualvm
I can provide the rest of the code as needed.
Thank you
Partial solution found :
Not a very clean solution but works.
I still have a throughput problem.
What I do is set the socket timer to a normal timeout (30000ms).
When the first read has come in the loop I reset the timer to something a lot lower (1000ms at the moment).
That allows me to wait for the server to start sending data, and if I get 1 second without any new data coming I consider the transfer to be finished.
Response times are still quite slow but way better.