I am using jna module to connect two processes that both perform FFMPEG commands. Send SDTOUT of FFMPEG command on the server side to NampedPipe and receive STDIN from that NampedPipe for other FFMPEG command on the Client side.
this is how I capture STDOUT and send into the pipe in server Side:
InputStream out = inputProcess.getInputStream();
byte[] buffer = new byte[maxBufferSize];
while (inputProcess.isAlive()) {
int no = out.available();
if (no > 0 && no > maxBufferSize) {
int n = out.read(buffer, 0,maxBufferSize);
IntByReference lpNumberOfBytesWritten = new IntByReference(maxBufferSize);
Kernel32.INSTANCE.WriteFile(pipe, buffer, buffer.length, lpNumberOfBytesWritten, null);
}
}
And this is how I capture STDIN and feed it to the Client Side:
OutputStream in = outputProcess.getOutputStream();
while (pipeOpenValue >= 1 && outputProcess.isAlive() && ServerIdState) {
// read from pipe
resp = Kernel32.INSTANCE.ReadFile(handle, readBuffer,readBuffer.length, lpNumberOfBytesRead, null);
// Write to StdIn inputProcess
if (outputProcess != null) {
in.write(readBuffer);
in.flush();
}
// check pipe status
Kernel32.INSTANCE.GetNamedPipeHandleState(handle, null,PipeOpenStatus, null, null, null, 2048);
pipeOpenValue = PipeOpenStatus.getValue();
WinDef.ULONGByReference ServerId = new WinDef.ULONGByReference();
ServerIdState = Kernel32.INSTANCE.GetNamedPipeServerProcessId(handle, ServerId);
}
But I faced two problems:
High CPU usage due to iterating two loops in Server and Client. (find by profiling resources by VisualVM)
Slower operation than just connecting two FFMPEG command with regular | in command prompt. Speed depends on buffer size but large buffer size blocks operation and small buffer size reduce speed further.
Questions:
Is there any way not to send and receive in chunks of bytes? Just stream STDOUT to the Namedpipe and capture it in Client. (Eliminate two Loops)
If I cant use NampedPipe, is there any other way to Connect two FFMPEG process that runs in different java modules but in the same machine?
Thanks
Seems like you're constantly polling process states stdin on server side without waiting for any events or sleeping the thread.
Probably you want to take a look here:
Concurrent read/write of named pipe in Java (on windows)
Cheers!
Related
I'm trying to read data from the server with SSH protocol. For this, I'm using the j2ssh library. My server connects with the other server in ssh without any problem. The problem is when I try to read any data from the shell command line. Whatever "command" I send to program "read = in.read(buffer)" never get any data, I tried with "ls" with "cat filename.txt" and other commands.
Only one command works fine and is "tail -f filename.txt". With this command, I can see the buffer is not empty, this contain the text of file, but the tail command does not close and while listening, sends the program in loop.
Can Anyone help me to know why I can't get any data from othere command?
This is my code:
private String exec(String cmd) throws SSHHandlerException {
String result = null;
session = ssh.openSessionChannel();
if(session.startShell())
{
session.getOutputStream().write((cmd+"\n").getBytes());
session.getOutputStream().close();
result = read(session,log);
}
session.close();
ssh.disconnect();
return result;
}
private static String read(SessionChannelClient session, ProcessLogger log) throws Exception{
byte buffer[] = new byte[255];
int read;
StringBuffer out=new StringBuffer();
InputStream in = session.getInputStream();
while((read = in.read(buffer)) > 0) {
out.append(new String(buffer, 0, read));
}
return out.toString();
}
If your goal is to transfer files, you should be using an SFTP client instead. SFTP is exactly what you're looking for: a file transfer protocol on top of SSH. It's much, much more efficient than using some command on the host and redirecting the stream.
J2SSH has an SftpClient implementation that can be constructed with an SshClient. Just use one of the get methods. Javadocs are here.
Edit after learning that you're not trying to transfer files:
You need to request a pseudo-terminal before you start the shell. From the docs:
The remote process may require a pseudo terminal. Call this method before executing a command or starting a shell.
Also, because it appears that you're using a Linux environment, I would recommend using terminal type "xterm" rather than their example of "vt100".
The reason that tail was working and not the other commands was because you were calling tail interactively. The interactive command creates its own pseudo-terminal of sorts. If, instead, you call tail -n 16 filename.txt then you will get the same results as with the other commands because it won't be interactive.
I have created a basic client-server program that transfers file from server to the client. Since the file is about 500MB, the server transfers the file as chunks of bytes to the client over the DataOutputStream object. While this logic works okay when both client and server are running on the same computer, it doesn't work when the two programs are running on separate computers(Both computers are on the same network and I have disabled the firewall for both)
When running on separate computers the problem is that a few bytes get transferred
Server Logic:
byte byteArr[] = new byte[1024];
while((c=fileInputStream.read(byteArr, 0, 1024) != -1))
{
dataOutputStream.writeBoolean(true);
dataOutputStream.flush();
dataOutputStream.write(byteArr, 0, 1024);
dataOutputStream.flush();
}
/*When running on different computers, after a few hundred iterations
it just stops looping the following lines are never executed*/
dataOutputStream.writeBoolean(false);
System.out.println("Transfer complete");
Client Logic
byte byteArr[] = new byte[1024];
while(dataInputStream.readBoolean())
{
dataInputStream.read(byteArr, 0, 1024);
fileOutputStream.write(byteArr, 0, 1024);
}
A read(buf, 0, 1024) call is not guaranteed to read exactly 1024 bytes. This causes bugs in both pieces of code:
the server incorrectly assumes that each chunk read from the file is always exactly 1024 bytes long and sends the whole buffer to the client.
the client might not read the whole chunk in a single iteration. It will then treat the first byte of the remainder as a boolean and get out of sync with the server.
To resolve this, you could:
send the file size (if known) before sending the file,
then just keep reading until you've read that many bytes.
or send c (the chunk size) instead of a single boolean,
then use dataInputStream.readFully() to make sure that
many bytes will be read.
I am working on an assignment for my Networks class and we have to use sockets to download an image from the web. The idea is to open 5 separate sockets, run them concurrently in separate threads and have them each download a part of the file then stitch them back together after getting the different parts. What I am confused about is how would the second thread know to start the stream at the point the first one ended? Like I have:
InputStream is = new BufferedInputStream(socket.getInputStream());
DataOutputStream dout = new DataOutputStream(socket.getOutputStream());
String request = "GET " + path + " HTTP/1.0\n\r\n\r";
dout.write(request.getBytes());
Then the loop that reads the stream:
while ((bytes = is.read(buffer)) != -1)
But if this is happening every time a run a new thread and create a new socket/TCP connection, how can set the stream to start at the point the last stream stopped? To clarify a bit more, the first thread should create a file called image.jpg.part1 then the second image.jpg.part2 and so on. Then we combine the parts into the original image.
Thanks!
I have a client/server application written in Java using non-blocking IO.
There are several message types which are transferred as Json encoding and a message delimiter appended at the end of each message.
The client reads bytes and merges the messages which are coming in chunks. On regular cases it is working but in heavy load cases i get a chunk which includes messages which are not in right order. I mean, lets say I have a message m1="AAABBBCCCDDD" and m2="EEEFFF" and delimiter is "||". When the message is received it is supposed to be "AAABBBCCCDDD||EEEFFF||". But it is received "AAABBBEEEFFF||CCCDDD||". As a result it fails to parse the message.
Actually, I would like to hear the ideas that should be considered while developing network applications using non-blocking IO. what can be the reason of being in the wrong order..?
Reader code is like this:
ByteBuffer buffer = ByteBuffer.allocate(20000);
count = 0;
while ((count = channel.read(buffer)) > 0) {
buffer.flip();
processSocketData(Charset.defaultCharset().decode(buffer));
}
processSocketData() method is like that:
socketData.append(newData);
delIndex = socketData.indexOf(cGlobals.delimiterSequence);
if (delIndex > -1) {
processRawMessage(socketData.substring(0, delIndex));
socketData.delete(0, delIndex + cGlobals.delimiterSize);
}
You need to flip() before processing, as you are doing, and you also need to either compact() or clear() the buffer after you process it.
I'm very new to networking in java and wanted to create a network chat program, I have found a few tutorials online and kind of drifted from that. I have the server of my program working and the only thing that is interfering is when I try to read the chat messages that the server sends over. The server does send the bytes of data over since the print message does work. So the problem is that the while loop never ends, what can this be a problem of?
public String[] updateChatDialog() throws IOException{
String returnString = "";
int accessed = -1;
while((accessed = in.read()) > 0){
System.out.println((char)accessed);
returnString = returnString + (char)accessed;
}
System.out.println(returnString);
return stringToTable(returnString);
}
Any tips on java networking would be helpful!
I do reset the BufferedInputStream every time the chats are rendered into a string, AKA the method above with the return in it.
The 'problem' is that your loop reads until end of stream, and the peer isn't closing the connection so EOS never arrives.
If you want to read messages you have to define them yourself. The easiest thing for you to do in this application is write and read lines, with e.g. PrintWriter.println() and BufferedReader.readLine().