We Keep Coding

Java Socket HTTP GET request

I'm trying to create a simple Java program that create an HTTP request to a HTTP server hosted locally, by using Socket.
This is my code:
try
{
//Create Connection
Socket s = new Socket("localhost",80);
System.out.println("[CONNECTED]");
DataOutputStream out = new DataOutputStream(s.getOutputStream());
DataInputStream in = new DataInputStream(s.getInputStream());
String header = "GET / HTTP/1.1\n"
+"Host:localhost\n\n";
byte[] byteHeader = header.getBytes();
out.write(byteHeader,0,header.length());
String res = "";
/////////////READ PROCESS/////////////
byte[] buf = new byte[in.available()];
in.readFully(buf);
System.out.println("\t[READ PROCESS]");
System.out.println("\t\tbuff length->"+buf.length);
for(byte b : buf)
{
res += (char) b;
}
System.out.println("\t[/READ PROCESS]");
/////////////END READ PROCESS/////////////
System.out.println("[RES]");
System.out.println(res);
System.out.println("[CONN CLOSE]");
in.close();
out.close();
s.close();
}catch(Exception e)
{
e.printStackTrace();
}
But by when I run it the Server reponse with a '400 Bad request error'.
What is the problem? Maybe some HTTP headers to add but I don't know which one to add.

There are a couple of issues with your request:
String header = "GET / HTTP/1.1\n"
+ "Host:localhost\n\n";
The line break to be used must be Carriage-Return/Newline, i.e. you should change that to
String header = "GET / HTTP/1.1\r\n"
+ "Host:localhost\r\n\r\n";
Next problem comes when you write the data to the OutputStream:
byte[] byteHeader = header.getBytes();
out.write(byteHeader,0,header.length());
The call of readBytes without the specification of a charset uses the system's charset which might be a different than the one that is needed here, better use getBytes("8859_1"). When writing to the stream, you use header.length() which might be different from the length of the resulting byte-array if the charset being used leads to the conversion of one character into multiple bytes (e.g. with UTF-8 as encoding). Better use byteHeader.length.
out.write(byteHeader,0,header.length());
String res = "";
/////////////READ PROCESS/////////////
byte[] buf = new byte[in.available()];
After sending the header data you should do a flush on the OutputStream to make sure that no internal buffer in the streams being used prevents the data to actually be sent to the server.
in.available() only returns the number of bytes you can read from the InputStream without blocking. It's not the length of the data being returned from the server. As a simple solution for starters, you can add Connection: close\r\n to your header data and simply read the data you're receiving from the server until it closes the connection:
StringBuffer sb = new StringBuffer();
byte[] buf = new byte[4096];
int read;
while ((read = in.read(buf)) != -1) {
sb.append(new String(buf, 0, read, "8859_1"));
}
String res = sb.toString();
Oh and independent form the topic of doing an HTTP request by your own:
String res = "";
for(byte b : buf)
{
res += (char) b;
}
This is a performance and memory nightmare because Java is actually caching all strings in memory in order to reuse them. So the internal cache gets filled with each result of this concatenation. A response of 100 KB size would mean that at least 5 GB of memory are allocated during that time leading to a lot of garbage collection runs in the process.
Oh, and about the response of the server: This most likely comes from the invalid line breaks being used. The server will regard the whole header including the empty line as a single line and complains about the wrong format of the GET-request due to additional data after the HTTP/1.1.

According to HTTP 1.1:
HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all
protocol elements except the entity-body [...].
So, you'll need all of your request to be ending with \r\n.

Parsing int from the start of a byte[] from a socket

I have a Java application that is reading data from a TCP socket that is receiving XML of varying size. The first 5 bytes of a given packet are supposed to indicate the size of the remaining message. I can read the message and xml successfully if I manually create a large byte[] and read the data.
Here are the instructions from the manual for the application that is generating the data:
Each message is preceded by the message size indicator which is a
32-bit unsinged integer using the network bytes order method. For
example: \x05\x00\x00\x00\x30\x31\x30\x32\x00 indicates the message
size of an ack which is 5 bytes included the fifth message byte '\0'. The
size indicator specifies everything following the size indicator
itself.
However I can't figure out how to decode the first 5 bytes to an integer that I can use to correctly size a byte[] for reading the rest of the message. I get random results:
Here is the code I'm using to parse the message:
DataOutputStream out = new DataOutputStream(clientSocket.getOutputStream());
BufferedInputStream inFromServer = new BufferedInputStream(clientSocket.getInputStream());
byte[] data = new byte[10];
inFromServer.read(data);
String result = new String(data, "ISO-8859-1");
Logger.info(data+"");
//PROBLEM AREA: Tried reading different byte lengths but no joy
//This should be a number but it never is. Often strange symbols
byte[] numeric = Arrays.copyOfRange(data,1,5);
String numericString = new String(numeric, "ISO-8859-1");
//Create a huge array to make sure everything gets captured.
//Want to use the parsed value from the start here
byte[] message = new byte[1000000];
inFromServer.read(message);
//This works as expected and returns correctly formatted XML
String fullMessage = new String(message, "ISO-8859-1");
Logger.info("Result "+result+ " Full message "+fullMessage);

The length looks like it's little endian. You can still use DataInputStream but you have to swap the bytes. If you used NIO's SocketChannel and a ByteBuffer you could set the byte order, but this is likely to be harder to use.
// only do this once per socket.
DataInputStream in = new DataInputStream(
new BufferedInputStream(clientSocket.getInputStream()));
// for each message.
int len0 = in.readInt();
int len = Integer.reverseBytes(len0);
assert len < 1 << 24;
byte[] bytes = new byte[len];
in.readFully(bytes);
String text = new String(bytes, "ISO-8859-1").trim();
int number = Integer.parseInt(text);

Network byte order is aka big-endian. But seeing your data it seems, that actually little-endian is used. At least 5 will look like those first 4 bytes in little-endian, but not in big-endian. So you need to read those bytes, consider little-endian and convert to long to consider "unsigned-ness".
public static void main(String[] args) throws IOException {
DataInputStream inFromServer = new DataInputStream(new BufferedInputStream(null));
int iSize = inFromServer.readInt();
iSize = Integer.reverseBytes(iSize); //read as little-endian
long count = Integer.toUnsignedLong(iSize); //unsigned int
}

Byte array message is splitted while sending. I want to send request message in single shot

Using java socket programming, sending byte array. Byte array size is 3500. It is not send as single request, splitted into 3 request while network capture. so server couldn't able to process the splitted request. I want to send the request in single shot. Find below the code snippet which am used for sending byte array request.
byte[] bISOMsg = new byte[9000];
bISOMsg = isoMsg.pack();
int messageLength = (short)bISOMsg.length;
messageLength = messageLength - 16;
/* System.out.println("messageLength --> " + messageLength);
System.out.println("Header --> " + new String(isoMsg.getHeader()));*/
byte[] bHeaderLen = new byte[2];
ByteBuffer bbHeader = ByteBuffer.wrap(bHeaderLen);
bbHeader.putShort((short)messageLength);
isoMsg.setHeader(bbHeader.array());
bISOMsg = isoMsg.pack();
isoMsg.unpack(bISOMsg);
logISOMsg(isoMsg);
System.out.println("bISOMsg....."+new String(bISOMsg));
byte[] BitmapBytVal= new byte[32];
System.arraycopy(bISOMsg, 4,BitmapBytVal, 0, 32);
//System.out.println("BitmapBytVal..."+BitmapBytVal);
ByteArrayOutputStream outputStream1 = new ByteArrayOutputStream();
outputStream1.write(isoHeader.getBytes());
outputStream1.write(bISOMsg, 0,4);
outputStream1.write( HexToByte1(new String(BitmapBytVal)));
outputStream1.write(bISOMsg, 36, bISOMsg.length-36);
TotalMsgBytVal =outputStream1.toByteArray();
outputStream1.close();
System.out.println("TotalMsgBytVal Hex value="+TotalMsgBytVal);
System.out.println("Msg Length ---- " + TotalMsgBytVal.length);
String msgLength= Integer.toHexString(TotalMsgBytVal.length);
msgLength = addZeros(msgLength,4);
System.out.println("Msg Length ----: " + msgLength);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream( );
String MSGIndicater="03NPCI ONC";
outputStream.write(MSGIndicater.getBytes());
outputStream.write(HexToByte1(msgLength));
outputStream.write(TotalMsgBytVal,0,TotalMsgBytVal.length);
outputStream.close();
TotalMsgBytVal = outputStream.toByteArray();
Socket soc = null;
byte []dataRes = new byte[9000];
System.out.println("Gateway IP Address ="+ cbsipaddr);
System.out.println("Gateway Port ="+ cbsport);
soc= new Socket(cbsipaddr,cbsport);
in=soc.getInputStream();
/*
/* Added by Syed on 03/09/15 */
System.out.println("Total Length of Request is = "+ TotalMsgBytVal.length);
DataOutputStream dout = new DataOutputStream(soc.getOutputStream());
dout.writeInt(TotalMsgBytVal.length); // write length of the message
dout.write(TotalMsgBytVal); // write the message
Thread.sleep(1000);
dout.flush();

Well, the MTU for an ethernet network is about 1500 bytes.
What do you think happens when you try to write 3500 bytes over ethernet?
Your code also looks very funky, I think you should look at an existing implementation to see how you can improve your code. If it can't handle messages that are split into multiple packets, it's a pretty bad server.

Java DataInputStream

Client side:
out = new DataOutputStream(connection.getOutputStream());
String process;
System.out.println("Connecting to server on "+ host + " port " + port +" at " + timestamp);
process = "Connection: "+host + ","+port+","+timestamp;
System.out.println("client len " + process.length());
out.write(process.length());
Prints:
Client len 55
Server side:
in = new DataInputStream(connection.getInputStream());
while (true) {
int len = in.readInt();
System.out.println("Length of pkt: "+len);
Prints: Length of pkt: 927166318
What's going on here? I tried writing 0 and it printed 3621743 on the server side. I checked some other sites and a few people had problems with other streams. I read about the issues arising with big vs little endianness, but I am not sure what the problem is here since I am using the data*streams that should work fine with each other.

If you call readInt() on one side, you should call writeInt(int) on the other. Change this
out.write(process.length());
to
out.writeInt(process.length());
From the Javadoc for write(int),
Writes the specified byte (the low eight bits of the argument b) to the underlying output stream.

Use out.writeInt(process.length()); instead of out.write(...); since you read an Integer from the stream afterwards.

Transferring Data Between Client and Server and Dynamically Flush

I've been playing around with transferring data between a test client (written in Java) and a server (written in C#/.NET).
I tried TCP clients and servers, but there has been and current is a problem flushing the stream. I realize flush doesn't always flush the stream, so I'm wondering if there is any way to flush/send a stream without .flush() or in a more reliable way?
Currently, the important part of the client looks like this (message is a string, serverSocket is a Socket object):
OutputStream output = serverSocket.getOutputStream();
byte[] buffer = message.getBytes();
int length = buffer.length;
output.write(ByteBuffer.allocate(4).putInt(length).array());
output.write(buffer);
output.flush();
and the server looks like this:
NetworkStream stream = client.GetStream ();
byte[] sizeBuffer = new byte[4];
int read = stream.Read (sizeBuffer, 0, 4);
int size = BitConverter.ToInt32 (sizeBuffer, 0);
Databaser.log ("recieved byte message denoting size: " + size);
byte[] messageBuffer = new byte[size];
read = stream.Read (messageBuffer, 0, size);
string result = BitConverter.ToString (messageBuffer);
Databaser.log ("\tmessage is as follows: '" + result + "'");
Where, if it's not evident from the code, the client sends 4 bytes, which are combined into a 32 bit integer which is the length of the message. Then I read in the message based on that length and have build in converters translate it into a string.
As I said, I'm wondering how to flush the connection? I know this code isn't perfect, but I can change it back to when I used TCP and UTF exclusive string messaging over the network, but either way, the connection doesn't send anything from the client until the client shuts down or closes the connection.

Maybe the problem is in the byte order. I have an application which send from a tablet (java) to a C# application (Windows Intel), I used similar to what you've done, except in the following
ByteBuffer iLength = ByteBuffer.allocate(4);
iLength.order(ByteOrder.LITTLE_ENDIAN);
iLength.putInt(length);
output.write(iLength.array(), 0, 4);
output.write(buffer);
output.flush();
Java uses BIG-ENDIAN and Intel uses LITTLE-ENDIAN bytes order.