We Keep Coding

reading TCP in java issue

I am reading TCP packets that I use to display an image. Each packet is supposed to have a 1025 length. And each time, I get 128 lines, I draw an image.
SO I begin by initializing
s = new Socket(ip, port);
stream = s.getInputStream();
byte[] tmpBuffer = new byte[1025];
byte[] finalBuffer = new byte[128 * 1025];
int count_lines =0
and then I read stream by stream of length 1025,
while((read = stream.read(tmpBuffer, 0, tmpBuffer.length)) != -1){
System.arraycopy(tmpBuffer, 1, finalBuffer, count_lines * 1025, 1025);
count_lines++;
if(count_lines == 128) break;
}
The problem is when I log the read integer, I get a bunch of 1025 but sometimes (apparently randomly) 423 or 602 (noticing that 423+602=1025)
Am I going wrong with the TCP reading or is there a problem on the server side ?

Am I going wrong with the TCP reading or is there a problem on the server side ?
In TCP you have only a stream of bytes. You have messages nor control over packets which are typically no more than 1532 bytes long.
You have to have your own protocol to handle sending of messages.
inputStream.read(buffer) will read between 1 byte and buffer.length and you can't ignore the actual length read as you are doing.
If you want to read into a buffer of a known length, you can use
DataInputStream dis = new DataInputStream(socket.getInputStream());
byte[] finalBuffer = new byte[128 * 1025];
dis.readFully(finalBuffer);
The readFully reads as much as it can at a time until the byte[] has been fully read.
A more flexible protocol is to send the length first.
public static void write(DataOutputStream out, byte[] bytes) {
out.writeInt(bytes.length());
out.write(bytes);
}
public static byte[] read(DataInputStream in) {
int length = in.readInt();
byte[] bytes = new byte[length];
in.readFully(bytes);
return bytes;
}
This way, whatever length of data you send, you will recieve in the same manner.

Java getInputStream Read not Exiting

Hi I have created a server socket for reading byte array from socket using getInputStream, But getInputStream.read is not exiting after endof data reaches. Below is my code.
class imageReciver extends Thread {
private ServerSocket serverSocket;
InputStream in;
public imageReciver(int port) throws IOException
{
serverSocket = new ServerSocket(port);
}
public void run()
{
Socket server = null;
server = serverSocket.accept();
in = server.getInputStream();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte buffer[] = new byte[1024];
while(true){
int s = 0;
s = in.read(buffer); //Not exiting from here
if(s<0) break;
baos.write(buffer, 0, s);
}
server.close();
return;
}
}
From the client if I sent 2048 bytes, the line in.read(buffer) should return -1 after reading two times, but it waiting there to read for the third time. How can I solve this ?
Thanks in advance....

Your server will need to close the connection, basically. If you're trying to send multiple "messages" over the same connection, you'll need some way to indicate the size/end of a message - e.g. length-prefixing or using a message delimiter. Remember that you're using a stream protocol - the abstraction is just that this is a stream of data; it's up to you to break it up as you see fit.
See the "network packets" in Marc Gravell's IO blog post for more information.
EDIT: Now that we know that you have an expected length, you probably want something like this:
int remainingBytes = expectedBytes;
while (remainingBytes > 0) {
int bytesRead = in.read(buffer, 0, Math.min(buffer.length, remainingBytes));
if (bytesRead < 0) {
throw new IOException("Unexpected end of data");
}
baos.write(buffer, 0, bytesRead);
remainingBytes -= bytesRead;
}
Note that this will also avoid overreading, i.e. if the server starts sending the next bit of data, we won't read into that.

If I send 2048 bytes, the line 'in.read(buffer)' should return -1 after reading two times.
You are mistaken on at least two counts here. If you send 2048 bytes, the line 'in.read(buffer)' should execute an indeterminate number of times, to read a total of 2048 bytes, and then block. It should only return -1 when the peer has closed the connection.

Missing data while sending large string over TCP. (client/server)

I have a Client/ server application where the Server is in java and Client is in Vb.net.
When i send large string from client to server am not receiving complete text.
please help.
code attached below.
client-- VB.net-
Try
Dim clientSocket As New System.Net.Sockets.TcpClient()
' msg("Client Started")
clientSocket.Connect(StrIP_Add, intPort)
clientSocket.SendBufferSize=104857600
'6511 6522
' Label1.Text = "Client Socket Program - Server Connected ..."
Dim serverStream As NetworkStream = clientSocket.GetStream()
Dim outStream(104857600) As Byte
' MsgBox(strValidator.Trim.Length)
outStream = System.Text.Encoding.ASCII.GetBytes(strValidator.Trim)
' Dim outStream As Byte() = "sdsfd"
System.Threading.Thread.Sleep(2000)
serverStream.Write(outStream, 0, outStream.Length)
System.Threading.Thread.Sleep(2000)
serverStream.Flush()
Dim inStream(104857600) As Byte
serverStream.Read(inStream, 0, outStream.Length) '104857600) ' CInt(clientSocket.ReceiveBufferSize))
Dim returndata As String = _
System.Text.Encoding.ASCII.GetString(inStream)
' msg("Data from Server : " + returndata)
clientSocket.Close()
Catch ex As Exception
' VikUcMsg.AddMessage("<b><u>" & Page.Title & "</u></b><br><br>" & "No Connectivity on the port :" & intPort, enmMessageType.Error)
End Try
server-- Java
BufferedInputStream RecievedBuffer = new BufferedInputStream(
TCPIP_Client_SOCKET.getInputStream());
InputStreamReader RecievedInputStreamReader = new InputStreamReader(
RecievedBuffer);
System.out.println(RecievedBuffer.toString().length());
//char[] RecievedChars = new char[TCPIP_Client_SOCKET
//.getReceiveBufferSize()];
char[] RecievedChars = new char[100000];
//Thread.sleep(5000);
RecievedInputStreamReader.read(RecievedChars);
//Thread.sleep(5000);
String strRecievedData=null;
//Thread.sleep(5000);
strRecievedData = new String( RecievedChars ).trim();
//strRecievedData = RecievedChars.;
Thread.sleep(5000);
if (strRecievedData!=null)
{
System.out.println(strRecievedData);
}
strRecievedData is only havig 8192 all the time.

Well the short answer is that you must loop when reading from a socket because there is no guarantee how many bytes you will receive on each attempt to read.
Psuedo-code:
while (!msgCompleted && !overallTimeout)
{
bytesRead = netstream.Read(readBuffer);
if (bytesRead > 0)
{
// here append readBuffer to msgBuffer from offset to offset+bytesRead
offset += bytesRead // update offset so you can keep appending
// inspect the msgBuffer to see if the message is completed
}
}
That all being said, you've got nyumerous other problems in your code. For example...
You allocate a 104857601 (not 104857600) byte buffer here:
Dim outStream(104857600) As Byte
And then discard and replace that buffer with whatever contents get reurned from strValidator:
outStream = System.Text.Encoding.ASCII.GetBytes(strValidator.Trim)
No point in pre-allocating it just to replace it.
Another one...
You allocate an input buffer of a certain length:
Dim inStream(104857600) As Byte
But then read into that buffer using the length of a different buffer:
serverStream.Read(inStream, 0, outStream.Length)
This is prone to errors depending on the lengths.
You will also need to loop in this VB read just as for the Java read.

Gunzipping Contents of a URL - Java

So as the title suggests, I'm trying to get and gunzip a string from an HTTP request.
urlConn = url.openConnection();
int len = CONTENT_LENGTH
byte[] gbytes = new byte[len];
gbuffer = new GZIPInputStream(urlConn.getInputStream(), len);
System.out.println(gbuffer.read(gbytes)+"/"+len);
System.out.println(gbytes);
result = new String(gbytes, "UTF-8");
gbuffer.close();
System.out.println(result);
With some URLs, it works fine. I get output like this:
42/42
[B#96e8209
The entire 42 bytes of my data. Abcdefghij.
With others, it gives me something like the following output:
22/77
[B#1d94882
The entire 77 bytes of
As you can see, the first some-odd bytes of data are very similar if not the same, so they shouldn't be causing these issues. I really can't seem to pin it down. Increasing CONTENT_LENGTH doesn't help, and data streams of sizes both larger and smaller than the ones giving me issues work fine.
EDIT: The issue also does not lie within the raw gzipped data, as Cocoa and Python both gunzip it without issue.
EDIT: Solved. Including final code:
urlConn = url.openConnection();
int offset = 0, len = CONTENT_LENGTH
byte[] gbytes = new byte[len];
gbuffer = new GZIPInputStream(urlConn.getInputStream(), len);
while(offset < len)
{
offset += gbuffer.read(gbytes, offset, offset-len);
}
result = new String(gbytes, "UTF-8");
gbuffer.close();

It's possible that the data isn't available in the stream. The first println() you have says you've only read 22 bytes, so only 22 bytes were available when you called read(). You can try looping until you've read CONTENT_LENGTH worth of bytes. Maybe something like:
int index = 0;
int bytesRead = gbuffer.read(gbytes);
while(bytesRead>0 && index<len) {
index += bytesRead;
bytesRead = gbuffer.read(gbytes,index,len-index);
}

GZIPInputStream.read() is not guaranteed to read all data in one call. You should use a loop:
byte[] buf = new byte[1024];
int len = 0, total = 0;
while ((len = gbuffer.read(buf)) > 0) {
total += len;
// do something with data
}

Java TCP Client-send blocked?

I am writing a java TCP client that talks to a C server.
I have to alternate sends and receives between the two.
Here is my code.
The server sends the length of the binary msg(len) to client(java)
Client sends an "ok" string
Server sends the binary and client allocates a byte array of 'len' bytes to recieve it.
It again sends back an "ok".
step 1. works. I get "len" value. However the Client gets "send blocked" and the server waits to receive data.
Can anybody take a look.
In the try block I have defined:
Socket echoSocket = new Socket("192.168.178.20",2400);
OutputStream os = echoSocket.getOutputStream();
InputStream ins = echoSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
String fromPU = null;
if( (fromPU = br.readLine()) != null){
System.out.println("Pu returns as="+fromPU);
len = Integer.parseInt(fromPU.trim());
System.out.println("value of len from PU="+len);
byte[] str = "Ok\n".getBytes();
os.write(str, 0, str.length);
os.flush();
byte[] buffer = new byte[len];
int bytes;
StringBuilder curMsg = new StringBuilder();
bytes =ins.read(buffer);
System.out.println("bytes="+bytes);
curMsg.append(new String(buffer, 0, bytes));
System.out.println("ciphertext="+curMsg);
os.write(str, 0, str.length);
os.flush();
}
UPDATED:
Here is my code. At the moment, there is no recv or send blocking on either sides. However, both with Buffered Reader and DataInput Stream reader, I am unable to send the ok msg. At the server end, I get a large number of bytes instead of the 2 bytes for ok.
Socket echoSocket = new Socket("192.168.178.20",2400);
OutputStream os = echoSocket.getOutputStream();
InputStream ins = echoSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
DataInputStream dis = new DataInputStream(ins);
DataOutputStream dos = new DataOutputStream(os);
if( (fromPU = dis.readLine()) != null){
//if( (fromPU = br.readLine()) != null){
System.out.println("PU Server returns length as="+fromPU);
len = Integer.parseInt(fromPU.trim());
byte[] str = "Ok".getBytes();
System.out.println("str.length="+str.length);
dos.writeInt(str.length);
if (str.length > 0) {
dos.write(str, 0, str.length);
System.out.println("sent ok");
}
byte[] buffer = new byte[len];
int bytes;
StringBuilder curMsg = new StringBuilder();
bytes =ins.read(buffer);
System.out.println("bytes="+bytes);
curMsg.append(new String(buffer, 0, bytes));
System.out.println("binarytext="+curMsg);
dos.writeInt(str.length);
if (str.length > 0) {
dos.write(str, 0, str.length);
System.out.println("sent ok");
}

Using a BufferedReader around a stream and then trying to read binary data from the stream is a bad idea. I wouldn't be surprised if the server has actually sent all the data in one go, and the BufferedReader has read the binary data as well as the line that it's returned.
Are you in control of the protocol? If so, I suggest you change it to send the length of data as binary (e.g. a fixed 4 bytes) so that you don't need to work out how to switch between text and binary (which is basically a pain).
If you can't do that, you'll probably need to just read a byte at a time to start with until you see the byte representing \n, then convert what you've read into text, parse it, and then read the rest as a chunk. That's slightly inefficient (reading a byte at a time instead of reading a buffer at a time) but I'd imagine the amount of data being read at that point is pretty small.

Several thoughts:
len = Integer.parseInt(fromPU.trim());
You should check the given size against a maximum that makes some sense. Your server is unlikely to send a two gigabyte message to the client. (Maybe it will, but there might be a better design. :) You don't typically want to allocate however much memory a remote client asks you to allocate. That's a recipe for easy remote denial of service attacks.
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
/* ... */
bytes =ins.read(buffer);
Maybe your BufferedReader has sucked in too much data? (Does the server wait for the Ok before continuing?) Are you sure that you're allowed to read from the underlying InputStreamReader object after attaching a BufferedReader object?
Note that TCP is free to deliver your data in ten byte chunks over the next two weeks :) -- because encapsulation, differing hardware, and so forth makes it very difficult to tell the size of packets that will eventually be used between two peers, most applications that are looking for a specific amount of data will instead populate their buffers using code somewhat like this (stolen from Advanced Programming in the Unix Environment, an excellent book; pity the code is in C and your code is in Java, but the principle is the same):
ssize_t /* Read "n" bytes from a descriptor */
readn(int fd, void *ptr, size_t n)
{
size_t nleft;
ssize_t nread;
nleft = n;
while (nleft > 0) {
if ((nread = read(fd, ptr, nleft)) < 0) {
if (nleft == n)
return(-1); /* error, return -1 */
else
break; /* error, return amount read so far */
} else if (nread == 0) {
break; /* EOF */
}
nleft -= nread;
ptr += nread;
}
return(n - nleft); /* return >= 0 */
}
The point to take away is that filling your buffer might take one, ten, or one hundred calls to read(), and your code must be resilient against slight changes in network capabilities.