I am trying to send bytes of data from server to client, so i am using the file pointer to point to location the file has read and read set of bytes and send it to client.
Below is server side
byte[] b = readByte()// my function which return bytes of Data
ServletOutputStream stream = httpServletResponse.getOutputStream();
stream.flush();
stream.write(b);
stream.flush();
Below is client side
URL url = new URL("http://localhost:1222/?filePointer=" + fp);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.connect();
InputStream is = conn.getInputStream();
System.out.println("Connection Open");
int pos = 0;
byte[] b = new byte[buffLength];
while (pos != -1) {
pos = is.read(b, pos, b.length - pos);
}
write2File(b);
The above code works for bufferLength of 20Kb, as bufferLength is increased I receive improper values.
Inasmuch as you have clarified that the server and client have a common agreement on the number of bytes to transfer, represented by bufferLength, you are right that they do not need to to exchange that length.
But the client does need to use that length correctly. Your problem is in the client's read loop:
int pos = 0;
byte[] b = new byte[buffLength];
while (pos != -1) {
pos = is.read(b, pos, b.length - pos);
}
There is one definite and one potential major problem with that:
Your receiving code does not populate the destination array correctly on the third and subsequent iterations of the loop (when that many iterations are performed), because pos records only the number of bytes received in the most recent previous read(). You need to use the aggregate number of bytes read in all previous reads into the array up to that point to determine the offset and number of bytes to attempt to read.
InputStream.read() will return -1 only at the end of the stream, which will not normally occur on a network stream until the far end is closed. If the server does not close the connection (at least the output side) from its end after writing the array, then the client will hang trying to read it. In particular, it will go into a tight loop if pos is equal to b.length, so that the number of bytes requested is 0.
Related
I'm trying to create a simple Java program that create an HTTP request to a HTTP server hosted locally, by using Socket.
This is my code:
try
{
//Create Connection
Socket s = new Socket("localhost",80);
System.out.println("[CONNECTED]");
DataOutputStream out = new DataOutputStream(s.getOutputStream());
DataInputStream in = new DataInputStream(s.getInputStream());
String header = "GET / HTTP/1.1\n"
+"Host:localhost\n\n";
byte[] byteHeader = header.getBytes();
out.write(byteHeader,0,header.length());
String res = "";
/////////////READ PROCESS/////////////
byte[] buf = new byte[in.available()];
in.readFully(buf);
System.out.println("\t[READ PROCESS]");
System.out.println("\t\tbuff length->"+buf.length);
for(byte b : buf)
{
res += (char) b;
}
System.out.println("\t[/READ PROCESS]");
/////////////END READ PROCESS/////////////
System.out.println("[RES]");
System.out.println(res);
System.out.println("[CONN CLOSE]");
in.close();
out.close();
s.close();
}catch(Exception e)
{
e.printStackTrace();
}
But by when I run it the Server reponse with a '400 Bad request error'.
What is the problem? Maybe some HTTP headers to add but I don't know which one to add.
There are a couple of issues with your request:
String header = "GET / HTTP/1.1\n"
+ "Host:localhost\n\n";
The line break to be used must be Carriage-Return/Newline, i.e. you should change that to
String header = "GET / HTTP/1.1\r\n"
+ "Host:localhost\r\n\r\n";
Next problem comes when you write the data to the OutputStream:
byte[] byteHeader = header.getBytes();
out.write(byteHeader,0,header.length());
The call of readBytes without the specification of a charset uses the system's charset which might be a different than the one that is needed here, better use getBytes("8859_1"). When writing to the stream, you use header.length() which might be different from the length of the resulting byte-array if the charset being used leads to the conversion of one character into multiple bytes (e.g. with UTF-8 as encoding). Better use byteHeader.length.
out.write(byteHeader,0,header.length());
String res = "";
/////////////READ PROCESS/////////////
byte[] buf = new byte[in.available()];
After sending the header data you should do a flush on the OutputStream to make sure that no internal buffer in the streams being used prevents the data to actually be sent to the server.
in.available() only returns the number of bytes you can read from the InputStream without blocking. It's not the length of the data being returned from the server. As a simple solution for starters, you can add Connection: close\r\n to your header data and simply read the data you're receiving from the server until it closes the connection:
StringBuffer sb = new StringBuffer();
byte[] buf = new byte[4096];
int read;
while ((read = in.read(buf)) != -1) {
sb.append(new String(buf, 0, read, "8859_1"));
}
String res = sb.toString();
Oh and independent form the topic of doing an HTTP request by your own:
String res = "";
for(byte b : buf)
{
res += (char) b;
}
This is a performance and memory nightmare because Java is actually caching all strings in memory in order to reuse them. So the internal cache gets filled with each result of this concatenation. A response of 100 KB size would mean that at least 5 GB of memory are allocated during that time leading to a lot of garbage collection runs in the process.
Oh, and about the response of the server: This most likely comes from the invalid line breaks being used. The server will regard the whole header including the empty line as a single line and complains about the wrong format of the GET-request due to additional data after the HTTP/1.1.
According to HTTP 1.1:
HTTP/1.1 defines the sequence CR LF as the end-of-line marker for all
protocol elements except the entity-body [...].
So, you'll need all of your request to be ending with \r\n.
I've been playing around with transferring data between a test client (written in Java) and a server (written in C#/.NET).
I tried TCP clients and servers, but there has been and current is a problem flushing the stream. I realize flush doesn't always flush the stream, so I'm wondering if there is any way to flush/send a stream without .flush() or in a more reliable way?
Currently, the important part of the client looks like this (message is a string, serverSocket is a Socket object):
OutputStream output = serverSocket.getOutputStream();
byte[] buffer = message.getBytes();
int length = buffer.length;
output.write(ByteBuffer.allocate(4).putInt(length).array());
output.write(buffer);
output.flush();
and the server looks like this:
NetworkStream stream = client.GetStream ();
byte[] sizeBuffer = new byte[4];
int read = stream.Read (sizeBuffer, 0, 4);
int size = BitConverter.ToInt32 (sizeBuffer, 0);
Databaser.log ("recieved byte message denoting size: " + size);
byte[] messageBuffer = new byte[size];
read = stream.Read (messageBuffer, 0, size);
string result = BitConverter.ToString (messageBuffer);
Databaser.log ("\tmessage is as follows: '" + result + "'");
Where, if it's not evident from the code, the client sends 4 bytes, which are combined into a 32 bit integer which is the length of the message. Then I read in the message based on that length and have build in converters translate it into a string.
As I said, I'm wondering how to flush the connection? I know this code isn't perfect, but I can change it back to when I used TCP and UTF exclusive string messaging over the network, but either way, the connection doesn't send anything from the client until the client shuts down or closes the connection.
Maybe the problem is in the byte order. I have an application which send from a tablet (java) to a C# application (Windows Intel), I used similar to what you've done, except in the following
ByteBuffer iLength = ByteBuffer.allocate(4);
iLength.order(ByteOrder.LITTLE_ENDIAN);
iLength.putInt(length);
output.write(iLength.array(), 0, 4);
output.write(buffer);
output.flush();
Java uses BIG-ENDIAN and Intel uses LITTLE-ENDIAN bytes order.
i have to send a short string as text from client to server and then after that send a binary file.
how would I send both binary file and the string using the same socket connection?
the server is a java desktop application and the client is an Android tablet. i have already set it up to send text messages between the client and server in both directions. i have not yet done the binary file sending part.
one idea is to set up two separate servers running at the same time. I think this is possible if i use two different port numbers and set up the servers on two different threads in the application. and i would have to set up two concurrent clients running on two services in the Android app.
the other idea is to somehow use an if else statement to determine which of the two types of files is being sent, either text of binary, and use the appropriate method to receive the file for the file type being sent.
example code for sending text
PrintWriter out;
BufferedReader in;
out = new PrintWriter(new BufferedWriter
(new OutputStreamWriter(Socket.getOutputStream())) true,);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out.println("test out");
String message = in.readLine();
example code for sending binary file
BufferedOutputStream out;
BufferedInputStream in;
byte[] buffer = new byte[];
int length = 0;
out = new BufferedOutputStream(new FileOutputStream("test.pdf));
in = new BufferedInputStream(new FileOutputStream("replacement.pdf"));
while((length = in.read(buffer)) > 0 ){
out.write(buffer, 0, length);
}
I don't think using two threads would be necessary in your case. Simply use the socket's InputStream and OutputStream in order to send binary data after you have sent your text messages.
Server Code
OutputStream stream = socket.getOutputStream();
PrintWriter out = new PrintWriter(
new BufferedWriter(
new OutputStreamWriter(stream)
)
);
out.println("test output");
out.flush(); // ensure that the string is not buffered by the BufferedWriter
byte[] data = getBinaryDataSomehow();
stream.write(data);
Client Code
InputStream stream = socket.getInputStream();
String message = readLineFrom(stream);
int dataSize = getSizeOfBinaryDataSomehow();
int totalBytesRead = 0;
byte[] data = new byte[dataSize];
while (totalBytesRead < dataSize) {
int bytesRemaining = dataSize - totalBytesRead;
int bytesRead = stream.read(data, totalBytesRead, bytesRemaining);
if (bytesRead == -1) {
return; // socket has been closed
}
totalBytesRead += bytesRead;
}
In order to determine the correct dataSize on the client side you have to transmit the size of the binary block somehow. You could send it as a String right before out.flush() in the Server Code or make it part of your binary data. In the latter case the first four or eight bytes could hold the actual length of the binary data in bytes.
Hope this helps.
Edit
As #EJP correctly pointed out, using a BufferedReader on the client side will probably result in corrupted or missing binary data because the BufferedReader "steals" some bytes from the binary data to fill its buffer. Instead you should read the string data yourself and either look for a delimiter or have the length of the string data transmitted by some other means.
/* Reads all bytes from the specified stream until it finds a line feed character (\n).
* For simplicity's sake I'm reading one character at a time.
* It might be better to use a PushbackInputStream, read more bytes at
* once, and push the surplus bytes back into the stream...
*/
private static String readLineFrom(InputStream stream) throws IOException {
InputStreamReader reader = new InputStreamReader(stream);
StringBuffer buffer = new StringBuffer();
for (int character = reader.read(); character != -1; character = reader.read()) {
if (character == '\n')
break;
buffer.append((char)character);
}
return buffer.toString();
}
You can read about how HTTP protocol works which essentially sends 'ascii and human readable' headers (so to speak) and after that any content can be added with appropriate encoding like base64 for example. You may create sth similar yourself.
You need to first send the String, then the size of the byte array then the byte array, use String.startsWith() method to check what is being send.
I am sending data to a server in two steps:
1) Length of what I will send using byte[4]
2) Data.
The server listens to the exact length of the data (shipped first) and then replies.
So I listen to the InputStream and try to get the data.
My Problem:
Whatever I am doing I am getting only the stream I send, but the server definatly sends a new string.
It seems I cannot wait for a -1 (end of string), as the program would time out and I am sure the server does not send anything alike.
Therefore I am using inputStream.available() to find out how many bytes are left in the buffer.
Once I am sending inputStream.read() after reading all the data it will time out with "Network idle timeout".
But I need to listen to the inputStream to make sure I am not missing information.
Why am I only receiving the information I send and not what is send by the server?
How can I listen to the connection for new items coming in?
Here is my code:
private void sendData (byte[] sendBytes){
try {
os.write(sendBytes);
os.flush();
} catch (IOException ex) {
}
}
Please help
THD
This is how you normally read all data from a reader (until the other end closes):
//BufferedReader is
StringBuilder data = new StringBuilder();
char[] buffer = new char[1024 * 32];
int len = 0;
while ((len = is.read(buffer)) != -1) {
data.append(buffer, 0, len);
}
//data will on this line contain all code received from the server
I am writing a java TCP client that talks to a C server.
I have to alternate sends and receives between the two.
Here is my code.
The server sends the length of the binary msg(len) to client(java)
Client sends an "ok" string
Server sends the binary and client allocates a byte array of 'len' bytes to recieve it.
It again sends back an "ok".
step 1. works. I get "len" value. However the Client gets "send blocked" and the server waits to receive data.
Can anybody take a look.
In the try block I have defined:
Socket echoSocket = new Socket("192.168.178.20",2400);
OutputStream os = echoSocket.getOutputStream();
InputStream ins = echoSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
String fromPU = null;
if( (fromPU = br.readLine()) != null){
System.out.println("Pu returns as="+fromPU);
len = Integer.parseInt(fromPU.trim());
System.out.println("value of len from PU="+len);
byte[] str = "Ok\n".getBytes();
os.write(str, 0, str.length);
os.flush();
byte[] buffer = new byte[len];
int bytes;
StringBuilder curMsg = new StringBuilder();
bytes =ins.read(buffer);
System.out.println("bytes="+bytes);
curMsg.append(new String(buffer, 0, bytes));
System.out.println("ciphertext="+curMsg);
os.write(str, 0, str.length);
os.flush();
}
UPDATED:
Here is my code. At the moment, there is no recv or send blocking on either sides. However, both with Buffered Reader and DataInput Stream reader, I am unable to send the ok msg. At the server end, I get a large number of bytes instead of the 2 bytes for ok.
Socket echoSocket = new Socket("192.168.178.20",2400);
OutputStream os = echoSocket.getOutputStream();
InputStream ins = echoSocket.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
DataInputStream dis = new DataInputStream(ins);
DataOutputStream dos = new DataOutputStream(os);
if( (fromPU = dis.readLine()) != null){
//if( (fromPU = br.readLine()) != null){
System.out.println("PU Server returns length as="+fromPU);
len = Integer.parseInt(fromPU.trim());
byte[] str = "Ok".getBytes();
System.out.println("str.length="+str.length);
dos.writeInt(str.length);
if (str.length > 0) {
dos.write(str, 0, str.length);
System.out.println("sent ok");
}
byte[] buffer = new byte[len];
int bytes;
StringBuilder curMsg = new StringBuilder();
bytes =ins.read(buffer);
System.out.println("bytes="+bytes);
curMsg.append(new String(buffer, 0, bytes));
System.out.println("binarytext="+curMsg);
dos.writeInt(str.length);
if (str.length > 0) {
dos.write(str, 0, str.length);
System.out.println("sent ok");
}
Using a BufferedReader around a stream and then trying to read binary data from the stream is a bad idea. I wouldn't be surprised if the server has actually sent all the data in one go, and the BufferedReader has read the binary data as well as the line that it's returned.
Are you in control of the protocol? If so, I suggest you change it to send the length of data as binary (e.g. a fixed 4 bytes) so that you don't need to work out how to switch between text and binary (which is basically a pain).
If you can't do that, you'll probably need to just read a byte at a time to start with until you see the byte representing \n, then convert what you've read into text, parse it, and then read the rest as a chunk. That's slightly inefficient (reading a byte at a time instead of reading a buffer at a time) but I'd imagine the amount of data being read at that point is pretty small.
Several thoughts:
len = Integer.parseInt(fromPU.trim());
You should check the given size against a maximum that makes some sense. Your server is unlikely to send a two gigabyte message to the client. (Maybe it will, but there might be a better design. :) You don't typically want to allocate however much memory a remote client asks you to allocate. That's a recipe for easy remote denial of service attacks.
BufferedReader br = new BufferedReader(new InputStreamReader(ins));
/* ... */
bytes =ins.read(buffer);
Maybe your BufferedReader has sucked in too much data? (Does the server wait for the Ok before continuing?) Are you sure that you're allowed to read from the underlying InputStreamReader object after attaching a BufferedReader object?
Note that TCP is free to deliver your data in ten byte chunks over the next two weeks :) -- because encapsulation, differing hardware, and so forth makes it very difficult to tell the size of packets that will eventually be used between two peers, most applications that are looking for a specific amount of data will instead populate their buffers using code somewhat like this (stolen from Advanced Programming in the Unix Environment, an excellent book; pity the code is in C and your code is in Java, but the principle is the same):
ssize_t /* Read "n" bytes from a descriptor */
readn(int fd, void *ptr, size_t n)
{
size_t nleft;
ssize_t nread;
nleft = n;
while (nleft > 0) {
if ((nread = read(fd, ptr, nleft)) < 0) {
if (nleft == n)
return(-1); /* error, return -1 */
else
break; /* error, return amount read so far */
} else if (nread == 0) {
break; /* EOF */
}
nleft -= nread;
ptr += nread;
}
return(n - nleft); /* return >= 0 */
}
The point to take away is that filling your buffer might take one, ten, or one hundred calls to read(), and your code must be resilient against slight changes in network capabilities.