When working with Sockets in Java, how can you tell whether the client has finished sending all (binary) data, before you could start processing them. Consider for example:
istream = new BufferedInputStream (socket.getInputStream());
ostream = new BufferedOutputStream(socket.getOutputStream());
byte[] buffer = new byte[BUFFER_SIZE];
int count;
while(istream.available() > 0 && (count = istream.read(buffer)) != -1)
{
// do something..
}
// assuming all input has been read
ostream.write(getResponse());
ostream.flush();
I've read similar posts on SO such as this, but couldn't find a conclusive answer. While my solution above works, my understanding is that you can never really tell if the client has finished sending all data. If for instance the client socket sends a few chunks of data and then blocks waiting for data from another data source before it could send more data, the code above may very well assume that the client has finished sending all data since istream.available() will return 0 for the current stream of bytes.
Yes, you're right - using available() like this is unreliable. Personally I very rarely use available(). If you want to read until you reach the end of the stream (as per the question title), keep calling read() until it returns -1. That's the easy bit. The hard bit is if you don't want the end of the stream, but the end of "what the server wants to send you at the moment."
As the others have said, if you need to have a conversation over a socket, you must make the protocol explain where the data finishes. Personally I prefer the "length prefix" solution to the "end of message token" solution where it's possible - it generally makes the reading code a lot simpler. However, it can make the writing code harder, as you need to work out the length before you send anything. This is a pain if you could be sending a lot of data.
Of course, you can mix and match solutions - in particular, if your protocol deals with both text and binary data, I would strongly recommend length-prefixing strings rather than null-terminating them (or anything similar). Decoding string data tends to be a lot easier if you can pass the decoder a complete array of bytes and just get a string back - you don't need to worry about reading to half way through a character, for example. You could use this as part of your protocol but still have overall "records" (or whatever you're transmitting) with an "end of data" record to let the reader process the data and respond.
Of course, all of this protocol design stuff is moot if you're not in control of the protocol :(
I think this is the task more of a protocol, assuming that you are the man who writes both the transmitting and receiving sides of application.
For example you could implement some simple logic protocol and divide you data into packets. Then divide packets into two parts: the head and the body. And then to say that your head consists of a predefined starting sequence and contains number of bytes in the body. Of forget about starting sequence and simpy transfer number of bytes in the bofy as a first byte of the packet.
Then you've could solve you problem.
As some ppl already said you can't avoid some kind of protocol for communication.
It should look like this for example:
On the server side you have:
void sendMSG(PrintWriter out){
try {
//just for example..
Process p = Runtime.getRuntime().exec("cmd /c dir C:");
BufferedReader br = new BufferedReader(new InputStreamReader(
p.getInputStream()));
//and then send all this crap to the client
String s = "";
while ((s = br.readLine()) != null) {
out.println("MSG");
out.println(s);
}
} catch (Exception e) {
System.out.println("Command incorrect!");
}
out.println("END");
}
//You are not supposed to close the stream or the socket, because you might want to send smth else later..
On the client side you have:
void recieveMSG(BufferedReader in) {
try {
while (in.readLine().equals("MSG")) {
System.out.println(in.readLine());
}
} catch (IOException e) {
System.out.println("Connection closed!");
}
}
as Nikita said this is more of task of protocol. Either you can go by header and body approach or you can send a special character or symbol for end of stream to break processing loop. Something like if you send say '[[END]]' on socket to denote end of stream.
Related
I want to check if the InputStream buffer contains any data which it can read and output without having to initially call readLine() and waiting for data.
I have looked into available() but this didn't seem to work as it always output 0.
while (true)
{
fromServer = in.readLine(); //Causing a hang waiting for reply
System.out.println(fromServer);
if ((fromUser = stdIn.readLine()) != null)
{
out.println(fromUser);
fromServer = in.readLine();
System.out.println(fromServer);
}
}
available() might tell you the number of bytes available, if implemented, but nothing can tell you whether there is a complete line other than trying to read it.
You need to read in a separate thread.
The issue is readLine() causes the client to get stuck hanging for a server reply if access isn't permitted for the client.
So the issue is really that the server should send something 'if access isn't permitted for the client', i.e. a message that says so, rather than doing nothing. You can't use absence of a message as a message in a blocking I/O system.
You also need to check every readLine() result for null, and if you get it when reading a socket you need to close it.
Create a new Instance of BufferedInputStream and call available on that object:
InputStream is = ...;
BufferedInputStream bis = new BufferedInputStream(inputStream);
if (bis.available() == 0) {
// do sth if input is available
}
I tried it with a little server-client application, it worked for me.
EDIT: Type mismatch gone.
As the Java Documentation says, the InputStream.available() always returns zero. In comparison to that, the BufferedInputStream returns „the number of bytes remaining that can be read in the buffer“
I have a client/server application written in Java using non-blocking IO.
There are several message types which are transferred as Json encoding and a message delimiter appended at the end of each message.
The client reads bytes and merges the messages which are coming in chunks. On regular cases it is working but in heavy load cases i get a chunk which includes messages which are not in right order. I mean, lets say I have a message m1="AAABBBCCCDDD" and m2="EEEFFF" and delimiter is "||". When the message is received it is supposed to be "AAABBBCCCDDD||EEEFFF||". But it is received "AAABBBEEEFFF||CCCDDD||". As a result it fails to parse the message.
Actually, I would like to hear the ideas that should be considered while developing network applications using non-blocking IO. what can be the reason of being in the wrong order..?
Reader code is like this:
ByteBuffer buffer = ByteBuffer.allocate(20000);
count = 0;
while ((count = channel.read(buffer)) > 0) {
buffer.flip();
processSocketData(Charset.defaultCharset().decode(buffer));
}
processSocketData() method is like that:
socketData.append(newData);
delIndex = socketData.indexOf(cGlobals.delimiterSequence);
if (delIndex > -1) {
processRawMessage(socketData.substring(0, delIndex));
socketData.delete(0, delIndex + cGlobals.delimiterSize);
}
You need to flip() before processing, as you are doing, and you also need to either compact() or clear() the buffer after you process it.
I'm very new to networking in java and wanted to create a network chat program, I have found a few tutorials online and kind of drifted from that. I have the server of my program working and the only thing that is interfering is when I try to read the chat messages that the server sends over. The server does send the bytes of data over since the print message does work. So the problem is that the while loop never ends, what can this be a problem of?
public String[] updateChatDialog() throws IOException{
String returnString = "";
int accessed = -1;
while((accessed = in.read()) > 0){
System.out.println((char)accessed);
returnString = returnString + (char)accessed;
}
System.out.println(returnString);
return stringToTable(returnString);
}
Any tips on java networking would be helpful!
I do reset the BufferedInputStream every time the chats are rendered into a string, AKA the method above with the return in it.
The 'problem' is that your loop reads until end of stream, and the peer isn't closing the connection so EOS never arrives.
If you want to read messages you have to define them yourself. The easiest thing for you to do in this application is write and read lines, with e.g. PrintWriter.println() and BufferedReader.readLine().
I'm trying to write a server in Java. I know very little Java. I've found an example using Selector.
It looks good, but it behaves strangely. When I do my_socket_output_stream.writeBytes("hello world") in client code, the server reads this message one byte at a time. Shouldn't I be notified only when the complete message is sent? Now I'd have to check my buffer after getting every byte to know if I can already work with it. Seems terribly inefficient.
I wonder if that's due to Selector or is that just how sockets work (it's been a long time since I used them). Could I make them wait for the full message somehow? Also, can I associate some objects with a channel? Right now all sockets use the same buffer. I'm sure you see how that is a problem..
The reason I want to use a Selector is that my server is only going to do io with a HashTable. Multiple threads would just be constantly waiting. And I only have one core anyway. Though maybe a combination of ThreadPoolExecutor and ConcurrentHashMap would be a good choice? It would surely enable me to have a buffer per socket..
I'd appreciate suggestions.
I faced the same problem a long time ago. I solved by first sending the number of bytes of the message, then sending the message itself byte by byte. Then I expanded it to line by line.
At the sender's side:
// code at sender side
StreamConnectionNotifier service = (StreamConnectionNotifier) Connector.open( url );
//System.out.println("opened");
StreamConnection con = (StreamConnection) service.acceptAndOpen();
OutputStream outputStream = con.openOutputStream();
// file to send
Scanner in = new Scanner(inFile);
//just count lines
String s=null;
int countLines=0;
while(in.hasNext()) {
s=in.nextLine();
countLines++;
}
//send num of lines
outputStream.write(Integer.toHexString(countLines).getBytes());
try{Thread.sleep(100);} catch(InterruptedException e){}
//send lines
in = new Scanner(inFile);
for(int i=0; i<countLines; i++) {
s=in.nextLine()+"\n";
outputStream.write(s.getBytes());
Thread.sleep(100);
}
At the receiver's side:
// code at receiver side
byte buffer[] = new byte[80];
int bytes_read = inputStream.read( buffer );
String received = new String(buffer, 0, bytes_read);
try{Thread.sleep(100);} catch(InterruptedException e){}
int receiveLines = Integer.parseInt(received);
PrintWriter out = new PrintWriter(new FileOutputStream("received.txt"));
for(int i=0; i<receiveLines; i++) {
bytes_read = inputStream.read( buffer );
received = new String(buffer, 0, bytes_read);
out.println(received);
Thread.sleep(100);
}
I hope this helps :)
Unless you have gained some skill in understanding of the issues of multi-threading and synchronization, avoid NIO. It is good stuff, but you are (currently) not properly equipped to debug it, much less fully appreciate and understand its synchronization needs.
Write a Runnable class that wraps a ServerSocket in a while loop, allowing the loop to block on the accept method. Then grab that return socket and construct a "client handler" thread which will handle whatever data came in the NIC.
This resource will give you some pointers on writing this older, slightly slower, and much more understandable server listening loop. I linked to a "middle" page in the article as that's the code listing, but you might want to read the entire article.
This uses the older "one Thread to handle the request" model of network processing. It's not terribly bad, but it can encounter scalability issues.
The alternative is to take the deep dive and do it with non-blocking NIO. It's not terribly hard, but it does require you to completely structure your server code in a manner that's not straightforward. Effectively you get "pools" of worker Threads than can perform various tasks, and then you synchronize on passing the data from worker to worker.
Shouldn't I be notified only when the complete message is sent?
No. Without specifying how messages should be separated from each other, the API can only give you one byte at a time (or all available bytes). The easiest way to separate strings would be to use a java.io.PrintStream on the side that is sending the message and a java.io.BufferedReader on the side that is receiving, like so:
// code that sends strings
OutputStream out = ...; // get the output stream from the socket
PrintStream sender = new PrintStream(out);
sender.println("Hello, world.");
// code that receives strings
InputStream in = ...; // get the input stream from the socket
BufferedReader receiver = new BufferedReader(new InputStreamReader(in));
String message = receiver.readLine(); // reads "Hello, world."
I'm writing a protocol on top of sockets, so I've decided to implement headers then send the information. So there is one thread per connection on the server which sits there reading in headers, then delegates off to methods to read in the rest of the information when it arrives.
So essentially it looks like this:
while ((length = inStream.read(buffer)) != -1)
{
dispatch(buffer, length);
}
So the dispatch method then decrypts the headers and delegates the method depending what is found in the header. It looks similar to:
byte[] clearText = decrypt(message,length);
if (cleartext == foo) sendFooToSocket();
So then sendFooToSocket() would then sit there and read from the instream or send to the outstream.
This is where I seem to run into some problems, in the client I'm sending the header then flushing, then sending the rest of the data, but it appears it's all coming as one and not being split up into header then data. Also is there a best way to force out of the sendFooToSocket method?
public void sendFooToSocket()
{
byte[] buffer = new byte[1024];
int length = 0;
while ((length = inStream.read(buffer) >0)
{
message = decrypt(buffer, length);
}
}
I would assume flush would allow me to break out of this method as it closes then opens the stream?
So I have 2 problems, flush doesn't seem to be breaking up my messages and flush doesn't seem to be allowing to drop out of methods such as sendFooToSocket(), any suggestions?
For clarity sake, the client just does this:
byte[] header = "MESG".getBytes();
cipher = encrypt(header);
outStream.write(cipher,0,cipher.length);
outStream.flush();
byte[] message = "Hi server".getBytes();
cipher = encrypt(message);
outStream.write(cipher,0,cipher.length);
outStream.flush();
But this is received by the server as 1 message even though it's been flushed after every write. Sending just the header works, and we get stuck in the sendFooToSocket() method, but if I send the data after the flush it comes all at once.
The client uses OutputStream and InputStreams just from the socket.get. The client also uses OutputStream and InputStream. Not sure if this matters?
What you seem to want is "record boundaries". With streams in general there are no implicit record boundaries. If you want that kind of functionality you will need to implement it yourself, by buffering the input and looking for, say, newlines to indicate the end of a record.
Look at BufferedInputStream.
inStream.read() may not be returning on a message boundary. You can't assume that it'll return at any particular boundary (such as a blank line separating headers and content if that's how you're doing it.) You'll have to manually parse the content and ignore the fact that it could come from multiple read()s or maybe one read() contains both the headers and content.
Unless you actually want control at the level you have implemented, you could consider Object streams (see ObjectInputStream and ObjectOutputStream). Such streams will allow you to send Java Objects over sockets and read them at the other end with out having to deal with headers and boundaries etc. See ObjectOutputStream for more details, but it's pretty much:
Sender:
writeObject(objectX)
Receiver:
myCopyOfObjectx = readObject()
and you can send any objects you like (as long as they are Serializable).