We Keep Coding

How to send byte array from server(jetty server) to client(RPI)?

I am trying to send bytes of data from server to client, so i am using the file pointer to point to location the file has read and read set of bytes and send it to client.
Below is server side
byte[] b = readByte()// my function which return bytes of Data
ServletOutputStream stream = httpServletResponse.getOutputStream();
stream.flush();
stream.write(b);
stream.flush();
Below is client side
URL url = new URL("http://localhost:1222/?filePointer=" + fp);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.connect();
InputStream is = conn.getInputStream();
System.out.println("Connection Open");
int pos = 0;
byte[] b = new byte[buffLength];
while (pos != -1) {
pos = is.read(b, pos, b.length - pos);
}
write2File(b);
The above code works for bufferLength of 20Kb, as bufferLength is increased I receive improper values.

Inasmuch as you have clarified that the server and client have a common agreement on the number of bytes to transfer, represented by bufferLength, you are right that they do not need to to exchange that length.
But the client does need to use that length correctly. Your problem is in the client's read loop:
int pos = 0;
byte[] b = new byte[buffLength];
while (pos != -1) {
pos = is.read(b, pos, b.length - pos);
}
There is one definite and one potential major problem with that:
Your receiving code does not populate the destination array correctly on the third and subsequent iterations of the loop (when that many iterations are performed), because pos records only the number of bytes received in the most recent previous read(). You need to use the aggregate number of bytes read in all previous reads into the array up to that point to determine the offset and number of bytes to attempt to read.
InputStream.read() will return -1 only at the end of the stream, which will not normally occur on a network stream until the far end is closed. If the server does not close the connection (at least the output side) from its end after writing the array, then the client will hang trying to read it. In particular, it will go into a tight loop if pos is equal to b.length, so that the number of bytes requested is 0.

Wrting a HTTP proxy in Java using only the Socket class

I'm trying to write an HTTP proxy in Java using only the Socket class. I had attempted to construct one earlier, and I was successfully sending a request by writing to the socket's output stream But I am having a hard time reading the response. the research I have conducted suggests that I should use the input stream and read it line by line, but I have not been able to read any web-pages successfully using this method. Would anyone have any suggestions as to where I could go from here?
My code actually uses a byte buffer to read from the input stream in order to read the page in bytes:
InputStream input = clientSocket.getInputStream()
byte[] buffer = new byte[48*1024];
byte[] redData;
StringBuilder clientData = new StringBuilder();
String redDataText;
int red;
while((red = input.read(buffer)) > -1) {
redData = new byte[red];
System.arraycopy(buffer, 0, redData, 0, red);
redDataText = new String(redData, "UTF-8");
System.out.println("Got message!! " + redDataText);
clientData.append(redDataText);
}

If you are asking for a way to read an InputStream by lines, this one may serve you:
BufferedReader bufferedReader=new BufferedReader(new InputStreamReader(input, "UTF-8"));
String line;
StringBuilder clientData=new StringBuilder();
while ((line=bufferedReader.readLine()) != null)
{
clientData.append(line);
}
You have to be careful not to read an InputStream in this fashion unless you are a priori sure that it contains just plain text (and not binary data).
BTW: For shake of efficiency, I recommend you to pre-size the clientData with an initial size according to the final size (if not, it will start from a default size of 10, and will need to be re-sized more times).

how to send both binary file and text using the same socket

i have to send a short string as text from client to server and then after that send a binary file.
how would I send both binary file and the string using the same socket connection?
the server is a java desktop application and the client is an Android tablet. i have already set it up to send text messages between the client and server in both directions. i have not yet done the binary file sending part.
one idea is to set up two separate servers running at the same time. I think this is possible if i use two different port numbers and set up the servers on two different threads in the application. and i would have to set up two concurrent clients running on two services in the Android app.
the other idea is to somehow use an if else statement to determine which of the two types of files is being sent, either text of binary, and use the appropriate method to receive the file for the file type being sent.
example code for sending text
PrintWriter out;
BufferedReader in;
out = new PrintWriter(new BufferedWriter
(new OutputStreamWriter(Socket.getOutputStream())) true,);
in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
out.println("test out");
String message = in.readLine();
example code for sending binary file
BufferedOutputStream out;
BufferedInputStream in;
byte[] buffer = new byte[];
int length = 0;
out = new BufferedOutputStream(new FileOutputStream("test.pdf));
in = new BufferedInputStream(new FileOutputStream("replacement.pdf"));
while((length = in.read(buffer)) > 0 ){
out.write(buffer, 0, length);
}

I don't think using two threads would be necessary in your case. Simply use the socket's InputStream and OutputStream in order to send binary data after you have sent your text messages.
Server Code
OutputStream stream = socket.getOutputStream();
PrintWriter out = new PrintWriter(
new BufferedWriter(
new OutputStreamWriter(stream)
)
);
out.println("test output");
out.flush(); // ensure that the string is not buffered by the BufferedWriter
byte[] data = getBinaryDataSomehow();
stream.write(data);
Client Code
InputStream stream = socket.getInputStream();
String message = readLineFrom(stream);
int dataSize = getSizeOfBinaryDataSomehow();
int totalBytesRead = 0;
byte[] data = new byte[dataSize];
while (totalBytesRead < dataSize) {
int bytesRemaining = dataSize - totalBytesRead;
int bytesRead = stream.read(data, totalBytesRead, bytesRemaining);
if (bytesRead == -1) {
return; // socket has been closed
}
totalBytesRead += bytesRead;
}
In order to determine the correct dataSize on the client side you have to transmit the size of the binary block somehow. You could send it as a String right before out.flush() in the Server Code or make it part of your binary data. In the latter case the first four or eight bytes could hold the actual length of the binary data in bytes.
Hope this helps.
Edit
As #EJP correctly pointed out, using a BufferedReader on the client side will probably result in corrupted or missing binary data because the BufferedReader "steals" some bytes from the binary data to fill its buffer. Instead you should read the string data yourself and either look for a delimiter or have the length of the string data transmitted by some other means.
/* Reads all bytes from the specified stream until it finds a line feed character (\n).
* For simplicity's sake I'm reading one character at a time.
* It might be better to use a PushbackInputStream, read more bytes at
* once, and push the surplus bytes back into the stream...
*/
private static String readLineFrom(InputStream stream) throws IOException {
InputStreamReader reader = new InputStreamReader(stream);
StringBuffer buffer = new StringBuffer();
for (int character = reader.read(); character != -1; character = reader.read()) {
if (character == '\n')
break;
buffer.append((char)character);
}
return buffer.toString();
}

You can read about how HTTP protocol works which essentially sends 'ascii and human readable' headers (so to speak) and after that any content can be added with appropriate encoding like base64 for example. You may create sth similar yourself.

You need to first send the String, then the size of the byte array then the byte array, use String.startsWith() method to check what is being send.

TCP how to send/receive real-time large packets in Java?

I have this TCP server running Because UDP server was able to accept large packets but ISP blocks my UDP packets even i have Public IP and Static services. But now decided to change it to TCP but i have a large stack now to replace with UDP to TCP.
Here is the server which is running but it does not at once receive a large packet, how can i increase it to unlimited or maximum size or etc?
1) Is there any way with this?
public void run()
{
while(true)
{
byte[] buf=new byte[5024]; <<< not helping!!
int bytes_read = 0;
try {
bytes_read = sockInput.read(buf, 0, buf.length);
String data = null;
data = new String(buf, 0, bytes_read);
System.out.println("[TCP]: incomeing data: " + bytes_read + " bytes, data=" +data);
}
}
2) Is there any way with this?
public TCPHandler(Socket sock) throws IOException
{
sock.setReceiveBufferSize(10*1024 +1024); //<<<<< not helping !!!
sock.setSendBufferSize(10*1024+1024);
this.sock = sock;
sockInput = sock.getInputStream();
sockOutput = sock.getOutputStream();
this.myThread = new Thread(this);
}
None is allowing me to handle large so that i can switch from UDP to TCP. Any ideas!!

TCP and UDP are different. TCP is not a packet-based protocol, it is a stream based protocol. Your data will arrive in order, and un-corrupted, but it will not necessarily all arrive at once.
What you should do is prefix each set 'packet' with a length. When you receive data, first read the length value, and then continue reading chunks from the socket until you have the amount of data required.

You shouldn't care how TCP splits your messages into packets and how many times you must read to get a full message. Just read until the end of the message has been reached, and then return all the bytes of the message.
Knowing when the message has been read completely depends on your protocol. You could choose to send only one message per connection, or you could send the length of the message before the message, or use some marker byte.
Note that you shouldn't use new String(byte[]) without specifying an encoding (and use the same when calling String.getBytes()), especially if you transfer from one machine to another, since both machines could have a different default encoding.

The trouble you're having is how InputStreams work. When you call read(buf,offset,length) it doesn't always read length bytes everytime. It simply reads what is available() on the InputStream, and returns the number of bytes it read. So you need to continue to call read until you've actually read length bytes.
int len = 0;
int expectedLength = sockInput.readInt();
byte[] buf = new byte[ expectedLength ];
while( len != buf.length ) {
len += sockInput.read( buf, len, buf.length );
}
// now you have a buffer with expectedLength data in it.
You can also look at using DataInputStream.readFully() by wrapping your
DataInputStream ds = new DataInputStream( sock.getInputStream() );
ds.readFully( buf, 0, buf.length );
http://docs.oracle.com/javase/6/docs/api/java/io/DataInputStream.html
But if you're really just after a String you should do this:
Reader reader = new InputStreamReader( sockInput.getInputStream(), "UTF-8" );
Same rules for read( char[], offset, length ) as read( byte[], offset, length )

A TCP is a stream of bytes you can get a portion of, one or more messages is a single read.
With UDP packets of more than 532 bytes, the packet can be broken up, so you have the same potential problem.
A simple way to encode message is to send the length first.
final DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
final DataInputStream in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
public void writeMessage(String text) {
out.writeUTF(text);
}
public String readMessage() {
return in.readUTF();
}
read/write UTF sends the length first and followed by the content

Java - InputStream - Test for input

I am sending data to a server in two steps:
1) Length of what I will send using byte[4]
2) Data.
The server listens to the exact length of the data (shipped first) and then replies.
So I listen to the InputStream and try to get the data.
My Problem:
Whatever I am doing I am getting only the stream I send, but the server definatly sends a new string.
It seems I cannot wait for a -1 (end of string), as the program would time out and I am sure the server does not send anything alike.
Therefore I am using inputStream.available() to find out how many bytes are left in the buffer.
Once I am sending inputStream.read() after reading all the data it will time out with "Network idle timeout".
But I need to listen to the inputStream to make sure I am not missing information.
Why am I only receiving the information I send and not what is send by the server?
How can I listen to the connection for new items coming in?
Here is my code:
private void sendData (byte[] sendBytes){
try {
os.write(sendBytes);
os.flush();
} catch (IOException ex) {
}
}
Please help
THD

This is how you normally read all data from a reader (until the other end closes):
//BufferedReader is
StringBuilder data = new StringBuilder();
char[] buffer = new char[1024 * 32];
int len = 0;
while ((len = is.read(buffer)) != -1) {
data.append(buffer, 0, len);
}
//data will on this line contain all code received from the server