Read data from inputstream without client side "flush()" - java

I have implemented a Java program that reads data from GPS Devices through ServerSocket.
ServerSocket serverSocket = new ServerSocket(13811);
serverSocket.setReceiveBufferSize(receiveBufferSize);
Socket incomingSocket = serverSocket.accept();
InputStream stream = incomingSocket.getInputStream();
byte[] buffer = new byte[1000];
StringBuffer sb = new StringBuffer();
System.out.println("START getting message from TCP stream: " + dateFormat.format(Calendar.getInstance().getTime()));
while (stream.read(buffer) > 0)
{
sb.append(new String(buffer));
System.out.println(sb.toString());
}
System.out.println("[incomingMessage]: " + incomingMessage);
System.out.println("FINISHED getting message from TCP stream: " + dateFormat.format(Calendar.getInstance().getTime()));
However, we found out that there was large delay (i.e. large deviation between Sys out "START..." and "FINISHED..." time above). The time was spent on inputStream.read().
If I use a Java client to connect to the above server port and send data to it, the message is readable by server's inputStream within a few ms. Below shows the Java client code.
Socket socket = new Socket("localhost", 13811);
DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
String tobesend = "testing message 1";
out.writeBytes(tobesend);
out.flush();
out.close();
However, if I add a "Thread.Sleep(10*1000)" before "out.flush()" and "out.close()", the delay at Server side will become 10seconds... Therefore I suspect if the GPS Device did not perform the "flush" and resulting the delay of inputstream.read() at server side...
Unfortunately, we have no control on the GPS Device TCP calls so I can't make any modifications on it to enforce it to "flush" message to my inputstream... Please advice if there is any means that server side can read data from inputstream without such delay even the client side (i.e. the GPS device) do not perform a "flush"?

The receiver cannot read data which has not been sent. It cannot force the other end to send data which has not been sent either.

Thanks for Peter Lawrey's advice and we used TCPDump to proved that the data are flushed to our server a few seconds after they establish the connection. That's why the server program captured the large delay.
But then, we perform some load test with the same Server program by having 4000 testing GPS devices pushing data to it every 5mins, each data is around 300bytes.
We tried to modify the server code by introducing Threadpool to handle the TCP data retrieval and hope that would give us better performance.
We have turned on TCPDump, and found that this time the time deviation was found between the TCPDump timestamp and the "START..." timestamp captured in the Java program. The deviation was around several seconds to less than 20 seconds...
Any suggestion on how to troubleshoot the issue?
Initialization of Threadpool:
blockingQueueForRetriveTCPMsg = new LinkedBlockingQueue<Runnable>(50);
threadPoolExecutorForRetriveTCPMsg = new ThreadPoolExecutor(
50,1200, 0, TimeUnit.SECONDS,
blockingQueueForRetriveTCPMsg,
new ThreadPoolExecutor.CallerRunsPolicy());
ServerSocket.accept() :
ServerSocket serverSocket = new ServerSocket(13811);
serverSocket.setReceiveBufferSize(receiveBufferSize);
Socket incomingSocket = serverSocket.accept();
RetrieveTcpMessage retrieveTcpMessage = new RetrieveTcpMessage(incomingSocket);
Thread retrieveTcpMessageThread = new Thread(retrieveTcpMessage);
threadPoolExecutorForRetriveTCPMsg.execute(retrieveTcpMessageThread);
Inside RetrieveTcpMessage.run(), simuilar to before:
InputStream stream = incomingSocket.getInputStream();
byte[] buffer = new byte[1000];
StringBuffer sb = new StringBuffer();
System.out.println("START getting message from TCP stream: " + dateFormat.format(Calendar.getInstance().getTime()));
while (stream.read(buffer) > 0)
{
sb.append(new String(buffer));
System.out.println(sb.toString());
}
System.out.println("[incomingMessage]: " + incomingMessage);
System.out.println("FINISHED getting message from TCP stream: " + dateFormat.format(Calendar.getInstance().getTime()));

Related

Android socket is connected but doesn't send data

I'm developing an Android client app which talks to server via a plain TCP Socket, let's assume that the server ip 192.168.1.2 and the androdi device ip is 192.168.1.3.
I open the socket, i check if socket is connected (i get true as result) and after that i write a presentation message.
Here is my code
// scoket setup
Sockets = new Socket(addressToConnect, 2015);
s.setSoTimeout(2500);
setTcpNoDelay(true);
// if i'm connected
if (s.isConnected()) {
// wrapping streams
DataInputStream dis = new DataInputStream(s.getInputStream());
DataOutputStream dos = new DataOutputStream(s.getOutputStream());
// sending data
String presentationMessage = "Presentation message content--- TERM";
dos.write(presentationMessage.getBytes("UTF-8");
dos.flush();
// buffers
byte[] readBuffer = new byte[4096];
StringBuffer responseBuffer = new StringBuffer();
// read data until command terminator string is found
boolean readResponse = true;
while (readResponse) {
int dataBufferLength = dis.read(readBuffer);
String chunk = new String(readBuffer, 0, dataBufferLength, "UTF-8"));
responseBuffer.append(chunk);
readResponse = ! chunk.endWith("--- TERM");
}
// Process data here
} else {
// Notify missing connection here
}
// here i close the socket
When i execute this code the execution seems working like a charme until the first read which timesout.
Sniffing the used WIFI network with a third machine i can't see the connection establishment and the written stream even if the code doesn't throw any exception before the read timeout.
I granted the android.permission.INTERNET in manifest.
Are there some other permissions to grant? or what i'm doing wrong?
Executing the same code in a standard Java SE environment everything goes fine.
I'm testing the code on a Nexus 5, Nexus 9 and Samsung S3 and S4 and the project is compatible with API 14+
Edit: Fixed typo in code example
You are reading from dos and writing to dis. You should reverse that.

input.read() never returns -1

I am writing a proxy server in Java.
Initially, I do (simplified)
server = new ServerSocket(5568);
incoming = server.accept();
input = incoming.getInputStream();
...
outgoing = new Socket(host, 80);
output = outgoing.getOutputStream();
output.write(inputbuffer, 0, i);
where inputbuffer is some collection of bytes received so far (I read the incoming data up until the part where I know the host header, and then open a connection to the server and send what I have so far). So server is my welcome socket, input is the data coming to my proxy from the client, and output is the data to the serve from my proxy.
Next, I want the output from the server to be written to the client in parallel with the client still possibly writing stuff to the server. So I create a separate thread to read from the client:
final InputStream finalInput = input;
final OutputStream finalOutput = output;
Thread sendingStuff = new Thread(){
public void run(){
int c;
while ((c = finalInput.read()) != -1){
finalOutput.write((byte)c);
finalOutput.flush();
}
finalInput.close();
finalOutput.close();
}
}
sendingStuff.start();
Finally, I have a different section in the main thread to read from the server and write that to the client.
InputStream reverseInput = outgoing.getInputStream();
OutputStream reverseOutput = incoming.getOutputStream();
int c;
while ((c = reverseInput.read()) != -1){
reverseOutput.write((byte)c);
reverseOutput.flush();
}
reverseInput.close();
reverseOutput.close();
What happens is I get input, and send output, but the browser spins forever and the line in the thread that's reading from the client never gets a -1 signal.
A lot of the time I get errors that say things like "invalid header name" or "your browser sent a request that the server could not understand" and I think it has to do with this problem I'm having. One time I even got an IOException: Socket Closed on the line that reads from the client.
So why isn't the client sending an EOF? And is this the right way to go about doing this?
"I think it's because my HTTP request has Connection: keep-alive. How do I handle this?"
I think maybe you can just open your socket once for one connection.
Try to have flag like isNewConnection. set it to true at first and after the connection is initiated, set it to false.

readLine() never stops reading

inFromClientR.readLine() never stops. any ideas? Am I forgetting something?
Server:
/*{ some code:
send a file with a dataoutputstream to client using a new port(4000) and when transfer is done i want a responce message (e.g. OK) send back to server in the old port(6000)
}*/
ServerSocket listenTransferSocket = new ServerSocket(6000);
Socket connectionTransferSocket = listenTransferSocket.accept();
BufferedReader inFromClientR =
new BufferedReader(new InputStreamReader(connectionTransferSocket.getInputStream()));
System.out.println("Client's response to Transfer: " +inFromClientR.readLine());
Client:
/*{ some code:
receive the file on port (4000) and then the responce is sent to server using the following commands
}*/
Socket fileTransferSocket = new Socket("localhost", 6000);
DataOutputStream outToServerR =
new DataOutputStream(fileTransferSocket.getOutputStream());
outToServerR.writeBytes("Transfer completed " +'\n');
BufferedReader#readLine() tries to fill its buffer with 8192 bytes, regradless of any linefeeds it find meanwhile. Since you have the connection open, the receiving side will wait until 1) you have sent 8192 bytes, or 2) closes the connection.
You would be better off using some other framing mechanism, maybe an ObjectOutputStream/ObjectInputStream.
String line = null;
while ((line = inFromClientR.readLine()) != null) {
// do sth
}

How to read from socket using datainputstream

I can display a single image received, but now I want to receive and write multiple images on my disk which will be sent from Android client after every 5 seconds.
Socket sock = servsock.accept();
dataInputStream = new DataInputStream(sock.getInputStream());
dataOutputStream = new DataOutputStream(sock.getOutputStream());
System.out.println("Accepted connection : " + sock);
dataInputStream = new DataInputStream(sock.getInputStream());
byte[] base64=dataInputStream.readUTF().getBytes();
byte[] arr=Base64.decodeBase64(base64);
FileOutputStream imageOutFile = new FileOutputStream("E:\\image.jpeg");
imageOutFile.write(arr);
You'll need to build a protocol between your client and the server.
If the image size is constant you just have to loop over the input-stream and keep buffering until you get the fixed number of bytes and write all the already buffered image.
Otherwise you'll need to add some metadata indicating the size of the images; e.g. you could use a simple format like this :
[size1][image1][size2][image2][size3][image3]
with [size-i] occupying a fixed amount of space and [image-i] the size of the ith image.
But above all don't be tempted to do such a naive protocol as processing an image, sleeping 5 seconds and retrieving the next one because the exact time could vary a lot between each image due to client, network or the server (your code and/or the file-system).
Little changes:
Socket sock = servsock.accept();
dataInputStream = new DataInputStream(sock.getInputStream());
dataOutputStream = new DataOutputStream(sock.getOutputStream());
System.out.println("Accepted connection : " + sock);
int imagesCount = dataInputStream.readInt();
for (int imgNum = 0; imgNum < imagesCount; imgNum++) {
int imgLen = dataInputStream.readInt();
byte[] base64 = new byte[imgLen];
dataInputStream.readFully(base64);
byte[] arr = Base64.decodeBase64(base64);
FileOutputStream imageOutFile = new FileOutputStream("E:\\image"+imgNum+".jpeg");
imageOutFile.write(arr);
}
If you are having multiple clients and a single server. Try multi-threading. Sample piece of code:
public static void main(String[] args){
try {
servSocket = new ServerSocket(xxxx);
while(true)
{
socket = servSocket.accept();
ClientNum++;
for(int i=0;i<10;i++){ //this for eg accepts 10 clients, with each client assigned an ID.
if(threads[i]==null){
sockets[i]=socket;
In your "run", you can call each of these clients by their respective ID and perform the reqd function you need. Not sure how clear this is. If needed, I can send you the entire code I have which is kinda similar to what you are doing.

C++ Client and Java Server

I did a Java Socket server, and a C++ Client.
However, the client connects to the server, without problems.
But when I write something client-server, the server doesn't catch the message.
What I'm doing wrong?
A little bit of the code of the Java Server:
DataInputStream dis=new DataInputStream(usrSocket.getInputStream());
ByteArrayOutputStream out = new ByteArrayOutputStream();
int data;
while((data = dis.read())>=0) {
out.write(data);
}
byte[] bytes = out.toByteArray();
String decrypt = new String(bytes);
if(decrypt.equals("status")){
System.out.println("Status emitted.");
}
System.out.println("Received a message.");
C++ Client writing:
QByteArray qba;
qba.append(text);
sock->write(qba.data());
qDebug() << "Send status";
I need help with that, thank you very much.
(that variable "text" it's a QString)
EDIT
Java server: That's only one part of all the code, the main thread waits for connections (Socket sock = server.accept()) and create a new thread for each user.
The code that I published of the java server, its one part of that threads for the users.
If you need ALL the code, plese tell me.
I will be waiting the answers!
Thank u very much!
Sorry if I answer ya late.
Try this code for Java Server.
ServerSocket ss = new ServerSocket(Port_No);
Socket incomingClient = ss.accept();
InputStream i = incomingClient.getInputStream();
OutputStream o = incomingClient.getOutputStream(); // Use it write to the Client Socket
InputStreamReader isr = new InputStreamReader(i);
BufferedReader br = new BufferedReader(isr);
String str = new String();
while ((str = br.readLine())!=null){
// do what you want with the data received in str.
}
As youre using QTcpSocket, it highly likely that you are running the client in the default asynchronous mode. This means after when you write after calling connectToHost, nothing will be sent as the socket is not connected.
Try using:
socket->connectToHost(hostAddress, hostPort, QIODevice::ReadWrite);
if (socket->waitForConnected()) {
QString text = "test string";
QByteArray array;
array.append(string);
qDebug() << socket->write(array);
} else {
// connect error!
}
Your Java code reads the socket until EOS and then prints something, which by the way is not a decryption operation. Your C++ client writes something and never closes the socket. So the server can never get out of the read loop.
If I read it correctly it is caused by the fact that your client is still running. Read() returns number >= 0 until the client socket is closed.

Categories