We Keep Coding

Wrting a HTTP proxy in Java using only the Socket class

I'm trying to write an HTTP proxy in Java using only the Socket class. I had attempted to construct one earlier, and I was successfully sending a request by writing to the socket's output stream But I am having a hard time reading the response. the research I have conducted suggests that I should use the input stream and read it line by line, but I have not been able to read any web-pages successfully using this method. Would anyone have any suggestions as to where I could go from here?
My code actually uses a byte buffer to read from the input stream in order to read the page in bytes:
InputStream input = clientSocket.getInputStream()
byte[] buffer = new byte[48*1024];
byte[] redData;
StringBuilder clientData = new StringBuilder();
String redDataText;
int red;
while((red = input.read(buffer)) > -1) {
redData = new byte[red];
System.arraycopy(buffer, 0, redData, 0, red);
redDataText = new String(redData, "UTF-8");
System.out.println("Got message!! " + redDataText);
clientData.append(redDataText);
}

If you are asking for a way to read an InputStream by lines, this one may serve you:
BufferedReader bufferedReader=new BufferedReader(new InputStreamReader(input, "UTF-8"));
String line;
StringBuilder clientData=new StringBuilder();
while ((line=bufferedReader.readLine()) != null)
{
clientData.append(line);
}
You have to be careful not to read an InputStream in this fashion unless you are a priori sure that it contains just plain text (and not binary data).
BTW: For shake of efficiency, I recommend you to pre-size the clientData with an initial size according to the final size (if not, it will start from a default size of 10, and will need to be re-sized more times).

how to send both binary file and text using the same socket

i have to send a short string as text from client to server and then after that send a binary file.
how would I send both binary file and the string using the same socket connection?
the server is a java desktop application and the client is an Android tablet. i have already set it up to send text messages between the client and server in both directions. i have not yet done the binary file sending part.
one idea is to set up two separate servers running at the same time. I think this is possible if i use two different port numbers and set up the servers on two different threads in the application. and i would have to set up two concurrent clients running on two services in the Android app.
the other idea is to somehow use an if else statement to determine which of the two types of files is being sent, either text of binary, and use the appropriate method to receive the file for the file type being sent.
example code for sending text
PrintWriter out;
BufferedReader in;
out = new PrintWriter(new BufferedWriter
(new OutputStreamWriter(Socket.getOutputStream())) true,);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out.println("test out");
String message = in.readLine();
example code for sending binary file
BufferedOutputStream out;
BufferedInputStream in;
byte[] buffer = new byte[];
int length = 0;
out = new BufferedOutputStream(new FileOutputStream("test.pdf));
in = new BufferedInputStream(new FileOutputStream("replacement.pdf"));
while((length = in.read(buffer)) > 0 ){
out.write(buffer, 0, length);
}

I don't think using two threads would be necessary in your case. Simply use the socket's InputStream and OutputStream in order to send binary data after you have sent your text messages.
Server Code
OutputStream stream = socket.getOutputStream();
PrintWriter out = new PrintWriter(
new BufferedWriter(
new OutputStreamWriter(stream)
)
);
out.println("test output");
out.flush(); // ensure that the string is not buffered by the BufferedWriter
byte[] data = getBinaryDataSomehow();
stream.write(data);
Client Code
InputStream stream = socket.getInputStream();
String message = readLineFrom(stream);
int dataSize = getSizeOfBinaryDataSomehow();
int totalBytesRead = 0;
byte[] data = new byte[dataSize];
while (totalBytesRead < dataSize) {
int bytesRemaining = dataSize - totalBytesRead;
int bytesRead = stream.read(data, totalBytesRead, bytesRemaining);
if (bytesRead == -1) {
return; // socket has been closed
}
totalBytesRead += bytesRead;
}
In order to determine the correct dataSize on the client side you have to transmit the size of the binary block somehow. You could send it as a String right before out.flush() in the Server Code or make it part of your binary data. In the latter case the first four or eight bytes could hold the actual length of the binary data in bytes.
Hope this helps.
Edit
As #EJP correctly pointed out, using a BufferedReader on the client side will probably result in corrupted or missing binary data because the BufferedReader "steals" some bytes from the binary data to fill its buffer. Instead you should read the string data yourself and either look for a delimiter or have the length of the string data transmitted by some other means.
/* Reads all bytes from the specified stream until it finds a line feed character (\n).
* For simplicity's sake I'm reading one character at a time.
* It might be better to use a PushbackInputStream, read more bytes at
* once, and push the surplus bytes back into the stream...
*/
private static String readLineFrom(InputStream stream) throws IOException {
InputStreamReader reader = new InputStreamReader(stream);
StringBuffer buffer = new StringBuffer();
for (int character = reader.read(); character != -1; character = reader.read()) {
if (character == '\n')
break;
buffer.append((char)character);
}
return buffer.toString();
}

You can read about how HTTP protocol works which essentially sends 'ascii and human readable' headers (so to speak) and after that any content can be added with appropriate encoding like base64 for example. You may create sth similar yourself.

You need to first send the String, then the size of the byte array then the byte array, use String.startsWith() method to check what is being send.

Missing data while sending large string over TCP. (client/server)

I have a Client/ server application where the Server is in java and Client is in Vb.net.
When i send large string from client to server am not receiving complete text.
please help.
code attached below.
client-- VB.net-
Try
Dim clientSocket As New System.Net.Sockets.TcpClient()
' msg("Client Started")
clientSocket.Connect(StrIP_Add, intPort)
clientSocket.SendBufferSize=104857600
'6511 6522
' Label1.Text = "Client Socket Program - Server Connected ..."
Dim serverStream As NetworkStream = clientSocket.GetStream()
Dim outStream(104857600) As Byte
' MsgBox(strValidator.Trim.Length)
outStream = System.Text.Encoding.ASCII.GetBytes(strValidator.Trim)
' Dim outStream As Byte() = "sdsfd"
System.Threading.Thread.Sleep(2000)
serverStream.Write(outStream, 0, outStream.Length)
System.Threading.Thread.Sleep(2000)
serverStream.Flush()
Dim inStream(104857600) As Byte
serverStream.Read(inStream, 0, outStream.Length) '104857600) ' CInt(clientSocket.ReceiveBufferSize))
Dim returndata As String = _
System.Text.Encoding.ASCII.GetString(inStream)
' msg("Data from Server : " + returndata)
clientSocket.Close()
Catch ex As Exception
' VikUcMsg.AddMessage("<b><u>" & Page.Title & "</u></b><br><br>" & "No Connectivity on the port :" & intPort, enmMessageType.Error)
End Try
server-- Java
BufferedInputStream RecievedBuffer = new BufferedInputStream(
TCPIP_Client_SOCKET.getInputStream());
InputStreamReader RecievedInputStreamReader = new InputStreamReader(
RecievedBuffer);
System.out.println(RecievedBuffer.toString().length());
//char[] RecievedChars = new char[TCPIP_Client_SOCKET
//.getReceiveBufferSize()];
char[] RecievedChars = new char[100000];
//Thread.sleep(5000);
RecievedInputStreamReader.read(RecievedChars);
//Thread.sleep(5000);
String strRecievedData=null;
//Thread.sleep(5000);
strRecievedData = new String( RecievedChars ).trim();
//strRecievedData = RecievedChars.;
Thread.sleep(5000);
if (strRecievedData!=null)
{
System.out.println(strRecievedData);
}
strRecievedData is only havig 8192 all the time.

Well the short answer is that you must loop when reading from a socket because there is no guarantee how many bytes you will receive on each attempt to read.
Psuedo-code:
while (!msgCompleted && !overallTimeout)
{
bytesRead = netstream.Read(readBuffer);
if (bytesRead > 0)
{
// here append readBuffer to msgBuffer from offset to offset+bytesRead
offset += bytesRead // update offset so you can keep appending
// inspect the msgBuffer to see if the message is completed
}
}
That all being said, you've got nyumerous other problems in your code. For example...
You allocate a 104857601 (not 104857600) byte buffer here:
Dim outStream(104857600) As Byte
And then discard and replace that buffer with whatever contents get reurned from strValidator:
outStream = System.Text.Encoding.ASCII.GetBytes(strValidator.Trim)
No point in pre-allocating it just to replace it.
Another one...
You allocate an input buffer of a certain length:
Dim inStream(104857600) As Byte
But then read into that buffer using the length of a different buffer:
serverStream.Read(inStream, 0, outStream.Length)
This is prone to errors depending on the lengths.
You will also need to loop in this VB read just as for the Java read.

Socket: premature end of JPEG file

I'm trying to send an image file from a server to a client via a socket. The socket was previously used to send some strings from the server to the client (with buffered input/output streams).
The trouble is the image file can't be received properly, with "Premature end of JPEG file" error.
The server first sends the file size to the client, the client then creates a byte[] of that size, and starts to receive the file.
Here are the codes:
Server:
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
//Send file size
dos.writeInt((int) file.length());
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
byte[] fileBytes = new byte[bis.available()];
bis.read(fileBytes);
bis.close();
BufferedOutputStream bos = new BufferedOutputStream(socket.getOutputStream());
bos.write(fileBytes);
bos.flush();
Client:
DataInputStream dis = new DataInputStream(socket.getInputStream());
//Receive file size
int size = dis.readInt();
BufferedInputStream bis = new BufferedInputStream(socket.getInputStream());
byte[] fileBytes = new byte[size];
bis.read(fileBytes, 0, fileBytes.length);
More interestingly, if I let server sleep for about 2 seconds between sending the file size and writing the byte[], then the image is received properly. I wonder if there's some kind of race condition between the server and the client

The error is most likely here:
byte[] fileBytes = new byte[bis.available()];
The method available does not return the size of the file. It might return only the size of the input buffer, which is smaller than the size of the file. See the API documentation of the method in BufferedInputStream.
Also, read in the line below is not guaranteed to read the whole file in one go. It returns the number of bytes that were actually read, which can be less than what you asked for. And in the client code, you are using read in the same way, without actually checking if it read all the data.

Please check commons-io with FileUtils and IOUtils. This should make work a lot easier.
http://commons.apache.org/io/

The correct way to copy a stream in Java is as follows:
int count;
byte[] buffer = new byte[8192]; // more if you like, but over a network it won't make much difference
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
Your code fails to logically match this at several points.
Also available() is not a valid way to determine either a file size or the size of an incoming network transmission - see the Javadoc. It has few if any correct uses and these aren't two of them.

Java TCP Client-send blocked?

I am writing a java TCP client that talks to a C server.
I have to alternate sends and receives between the two.
Here is my code.
The server sends the length of the binary msg(len) to client(java)
Client sends an "ok" string
Server sends the binary and client allocates a byte array of 'len' bytes to recieve it.
It again sends back an "ok".
step 1. works. I get "len" value. However the Client gets "send blocked" and the server waits to receive data.
Can anybody take a look.
In the try block I have defined:
Socket echoSocket = new Socket("192.168.178.20",2400);
OutputStream os = echoSocket.getOutputStream();
InputStream ins = echoSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
String fromPU = null;
if( (fromPU = br.readLine()) != null){
System.out.println("Pu returns as="+fromPU);
len = Integer.parseInt(fromPU.trim());
System.out.println("value of len from PU="+len);
byte[] str = "Ok\n".getBytes();
os.write(str, 0, str.length);
os.flush();
byte[] buffer = new byte[len];
int bytes;
StringBuilder curMsg = new StringBuilder();
bytes =ins.read(buffer);
System.out.println("bytes="+bytes);
curMsg.append(new String(buffer, 0, bytes));
System.out.println("ciphertext="+curMsg);
os.write(str, 0, str.length);
os.flush();
}
UPDATED:
Here is my code. At the moment, there is no recv or send blocking on either sides. However, both with Buffered Reader and DataInput Stream reader, I am unable to send the ok msg. At the server end, I get a large number of bytes instead of the 2 bytes for ok.
Socket echoSocket = new Socket("192.168.178.20",2400);
OutputStream os = echoSocket.getOutputStream();
InputStream ins = echoSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
DataInputStream dis = new DataInputStream(ins);
DataOutputStream dos = new DataOutputStream(os);
if( (fromPU = dis.readLine()) != null){
//if( (fromPU = br.readLine()) != null){
System.out.println("PU Server returns length as="+fromPU);
len = Integer.parseInt(fromPU.trim());
byte[] str = "Ok".getBytes();
System.out.println("str.length="+str.length);
dos.writeInt(str.length);
if (str.length > 0) {
dos.write(str, 0, str.length);
System.out.println("sent ok");
}
byte[] buffer = new byte[len];
int bytes;
StringBuilder curMsg = new StringBuilder();
bytes =ins.read(buffer);
System.out.println("bytes="+bytes);
curMsg.append(new String(buffer, 0, bytes));
System.out.println("binarytext="+curMsg);
dos.writeInt(str.length);
if (str.length > 0) {
dos.write(str, 0, str.length);
System.out.println("sent ok");
}

Using a BufferedReader around a stream and then trying to read binary data from the stream is a bad idea. I wouldn't be surprised if the server has actually sent all the data in one go, and the BufferedReader has read the binary data as well as the line that it's returned.
Are you in control of the protocol? If so, I suggest you change it to send the length of data as binary (e.g. a fixed 4 bytes) so that you don't need to work out how to switch between text and binary (which is basically a pain).
If you can't do that, you'll probably need to just read a byte at a time to start with until you see the byte representing \n, then convert what you've read into text, parse it, and then read the rest as a chunk. That's slightly inefficient (reading a byte at a time instead of reading a buffer at a time) but I'd imagine the amount of data being read at that point is pretty small.

Several thoughts:
len = Integer.parseInt(fromPU.trim());
You should check the given size against a maximum that makes some sense. Your server is unlikely to send a two gigabyte message to the client. (Maybe it will, but there might be a better design. :) You don't typically want to allocate however much memory a remote client asks you to allocate. That's a recipe for easy remote denial of service attacks.
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
/* ... */
bytes =ins.read(buffer);
Maybe your BufferedReader has sucked in too much data? (Does the server wait for the Ok before continuing?) Are you sure that you're allowed to read from the underlying InputStreamReader object after attaching a BufferedReader object?
Note that TCP is free to deliver your data in ten byte chunks over the next two weeks :) -- because encapsulation, differing hardware, and so forth makes it very difficult to tell the size of packets that will eventually be used between two peers, most applications that are looking for a specific amount of data will instead populate their buffers using code somewhat like this (stolen from Advanced Programming in the Unix Environment, an excellent book; pity the code is in C and your code is in Java, but the principle is the same):
ssize_t /* Read "n" bytes from a descriptor */
readn(int fd, void *ptr, size_t n)
{
size_t nleft;
ssize_t nread;
nleft = n;
while (nleft > 0) {
if ((nread = read(fd, ptr, nleft)) < 0) {
if (nleft == n)
return(-1); /* error, return -1 */
else
break; /* error, return amount read so far */
} else if (nread == 0) {
break; /* EOF */
}
nleft -= nread;
ptr += nread;
}
return(n - nleft); /* return >= 0 */
}
The point to take away is that filling your buffer might take one, ten, or one hundred calls to read(), and your code must be resilient against slight changes in network capabilities.