I'm trying to write a server in Java. I know very little Java. I've found an example using Selector.
It looks good, but it behaves strangely. When I do my_socket_output_stream.writeBytes("hello world") in client code, the server reads this message one byte at a time. Shouldn't I be notified only when the complete message is sent? Now I'd have to check my buffer after getting every byte to know if I can already work with it. Seems terribly inefficient.
I wonder if that's due to Selector or is that just how sockets work (it's been a long time since I used them). Could I make them wait for the full message somehow? Also, can I associate some objects with a channel? Right now all sockets use the same buffer. I'm sure you see how that is a problem..
The reason I want to use a Selector is that my server is only going to do io with a HashTable. Multiple threads would just be constantly waiting. And I only have one core anyway. Though maybe a combination of ThreadPoolExecutor and ConcurrentHashMap would be a good choice? It would surely enable me to have a buffer per socket..
I'd appreciate suggestions.
I faced the same problem a long time ago. I solved by first sending the number of bytes of the message, then sending the message itself byte by byte. Then I expanded it to line by line.
At the sender's side:
// code at sender side
StreamConnectionNotifier service = (StreamConnectionNotifier) Connector.open( url );
//System.out.println("opened");
StreamConnection con = (StreamConnection) service.acceptAndOpen();
OutputStream outputStream = con.openOutputStream();
// file to send
Scanner in = new Scanner(inFile);
//just count lines
String s=null;
int countLines=0;
while(in.hasNext()) {
s=in.nextLine();
countLines++;
}
//send num of lines
outputStream.write(Integer.toHexString(countLines).getBytes());
try{Thread.sleep(100);} catch(InterruptedException e){}
//send lines
in = new Scanner(inFile);
for(int i=0; i<countLines; i++) {
s=in.nextLine()+"\n";
outputStream.write(s.getBytes());
Thread.sleep(100);
}
At the receiver's side:
// code at receiver side
byte buffer[] = new byte[80];
int bytes_read = inputStream.read( buffer );
String received = new String(buffer, 0, bytes_read);
try{Thread.sleep(100);} catch(InterruptedException e){}
int receiveLines = Integer.parseInt(received);
PrintWriter out = new PrintWriter(new FileOutputStream("received.txt"));
for(int i=0; i<receiveLines; i++) {
bytes_read = inputStream.read( buffer );
received = new String(buffer, 0, bytes_read);
out.println(received);
Thread.sleep(100);
}
I hope this helps :)
Unless you have gained some skill in understanding of the issues of multi-threading and synchronization, avoid NIO. It is good stuff, but you are (currently) not properly equipped to debug it, much less fully appreciate and understand its synchronization needs.
Write a Runnable class that wraps a ServerSocket in a while loop, allowing the loop to block on the accept method. Then grab that return socket and construct a "client handler" thread which will handle whatever data came in the NIC.
This resource will give you some pointers on writing this older, slightly slower, and much more understandable server listening loop. I linked to a "middle" page in the article as that's the code listing, but you might want to read the entire article.
This uses the older "one Thread to handle the request" model of network processing. It's not terribly bad, but it can encounter scalability issues.
The alternative is to take the deep dive and do it with non-blocking NIO. It's not terribly hard, but it does require you to completely structure your server code in a manner that's not straightforward. Effectively you get "pools" of worker Threads than can perform various tasks, and then you synchronize on passing the data from worker to worker.
Shouldn't I be notified only when the complete message is sent?
No. Without specifying how messages should be separated from each other, the API can only give you one byte at a time (or all available bytes). The easiest way to separate strings would be to use a java.io.PrintStream on the side that is sending the message and a java.io.BufferedReader on the side that is receiving, like so:
// code that sends strings
OutputStream out = ...; // get the output stream from the socket
PrintStream sender = new PrintStream(out);
sender.println("Hello, world.");
// code that receives strings
InputStream in = ...; // get the input stream from the socket
BufferedReader receiver = new BufferedReader(new InputStreamReader(in));
String message = receiver.readLine(); // reads "Hello, world."
Related
Hello stack overflow world, I've been struggling with the most straight forward and common problem within Java IO, for some time, and now need your help to tackle it.
Check out this piece of code I have in a try block, within a thread.run():
// connect to client socket, and setup own server socket
clientSocket = new Socket(serverHostname, CLIENT_PORT);
//send a test command to download a file
String downloadFileName = "sample.txt";
DataOutputStream dataOutputStream = new DataOutputStream(clientSocket.getOutputStream());
System.out.println("Sending a request to download file : " + downloadFileName + " from user: Arsa node"); //todo: replace with node user later
dataOutputStream.writeUTF("D/sample.txt");
//close socket if host isn't detected anymore, and if socket doesn't become null suddenly
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("****File has been sent****");
in = new DataInputStream(clientSocket.getInputStream());
byte[] retrievedFileData = new byte[8036];
if (in.readInt() > 0) {
System.out.println("Starting file download!");
in.read(retrievedFileData);
System.out.println("File data has been read, converting to file now");
//closing input stream will close socket also
in.close();
}
clientSocket.close();
2 Main questions that have been confusing me to death:
Why does dataOutputStream.close() need to be run for writeUTF to actually send my string to the server socket, I find that when I don't have dos.close(), data isn't retrieved on the other side, further because I close it, I no longer can read from the socket - as it seems the socket connection becomes closed when the Output Stream is previously closed...
What's a better way, following some sort of pattern to do this? For context, all I'm trying to do is write the filename I'm looking to download to my client, then read the response right away, which I expect to be bytes with the file, any error handling I will consider as a part of my development.
Overall, it shouldn't be complicated to write something to a socket, then read and ingest it's response...which doesn't seem to be the case here,
any help would be greatly appreciated! If the ServerSocket code snippet is needed I'm happy to share.
The observed behavior is just a side-effect of close(), as it calls flush() before closing to make sure any buffered data is sent. To solve your problem, you need to call the flush() method instead of closing.
This behavior is not unique to DataOutputStream: a lot of other OutputStream (or Writer) implementations apply buffering, and you will need to flush when you want to ensure the data is sent to the client, written to disk or otherwise processed.
BTW: The DataOutputStream and DataInputStream is for a very specific type of data serialization protocol that is particular to Java. You may want to consider carefully if this is the right protocol to use.
is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully? In my case I'm using it for sending a simple string through a java.net.Socket. In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock. What I would like to do is wait for the output to end, and then start reading the input.
Socket sk = new Socket("192.168.0.112", 3000);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write(message.getBytes());
bo.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
if (line.equals("ack")) {
System.out.println("ack");
}
sk.close();
Update
ServerSocket:
ServerSocket ss = new ServerSocket(3000);
System.out.println("server socket open");
while (true) {
Socket sk = ss.accept();
System.out.println("new connection");
BufferedReader br = new BufferedReader(new InputStreamReader(sk.getInputStream()));
String line = br.readLine();
System.out.println("received line: " + line);
BufferedOutputStream bo = new BufferedOutputStream(sk.getOutputStream());
bo.write("ack".getBytes()); bo.flush();
sk.close();
}
Update:
#Global Variable - the reason that read was blocking the socket is that it was waiting for the \n, indeed. Using
bo.write("ack\n".getBytes());
instead of
bo.write("ack".getBytes());
made it work.
Regarding the initial question, is there a way of knowing if flush() method has finished successfully, #Stephen C provided the answer:
there is no way to know that based on the Socket or OutputStream APIs.
The normal way to get that sort of assurance is to have the remote
application send an "reply" in response, and read it in the local
side.
This "reply" is implemented in the code sample and it works.
Is there a way of knowing when or whether the flush() method of a BufferedOutputStream thread has finished successfully?
It depends on what you mean by "finished successfully".
The flush() method ensures that all unsent data in the pipeline has been pushed as far as the operating system network stack. When that is done, then you could say that flush() has finished successfully. The way that you know that that has happened is that the flush() call returns.
On the other hand, if you want some assurance that the data has (all) been delivered to the remote machine, or that the remote application has read it (all) ... there is no way to know that based on the Socket or OutputStream APIs. The normal way to get that sort of assurance is to have the remote application send an "reply" in response, and read it in the local side.
In the following code, the flush() method is run in parallel with the BufferedReader.read() method and the socket output is immediately blocked by the input read resulting in something that resembles a deadlock.
The code that you are talking about is basically the correct approach. The way to wait for the response is to read it like that.
If it is not working, then you need to compare what the client and server side are doing:
Is the server waiting for the client to send something more? Maybe an end of line sequence?
Did the server sends the response?
Did it flush() the response?
A mismatch between what the client and server are doing can lead to a form or deadlock, but the solution is to fix the mismatch. Waiting for some kind of hypothetical confirmation of the flush() is not the answer.
UPDATE
The problem is indeed a mismatch. For example, the server writes "ack" but the client expects "ack\n". The same happens in the client -> server case ... unless message always ends with a newline.
Your code is reading reader.readLine() . Are your writing \n when writing? You may want to append \n to the string your are writing.
I tried to reproduce your problem. First, I ran in to some kind of blocking state too, until I realized, I was using readLine at Server-side, too. But the message I was sending did not have a concluding \n. Therefore, the serversocket was still waiting at its InputStream without sending the client the ACK through its OutputStream. I think, #Global Variable is right.
I'm very new to networking in java and wanted to create a network chat program, I have found a few tutorials online and kind of drifted from that. I have the server of my program working and the only thing that is interfering is when I try to read the chat messages that the server sends over. The server does send the bytes of data over since the print message does work. So the problem is that the while loop never ends, what can this be a problem of?
public String[] updateChatDialog() throws IOException{
String returnString = "";
int accessed = -1;
while((accessed = in.read()) > 0){
System.out.println((char)accessed);
returnString = returnString + (char)accessed;
}
System.out.println(returnString);
return stringToTable(returnString);
}
Any tips on java networking would be helpful!
I do reset the BufferedInputStream every time the chats are rendered into a string, AKA the method above with the return in it.
The 'problem' is that your loop reads until end of stream, and the peer isn't closing the connection so EOS never arrives.
If you want to read messages you have to define them yourself. The easiest thing for you to do in this application is write and read lines, with e.g. PrintWriter.println() and BufferedReader.readLine().
My question is: is there a way to perform a socket OutputStream shutdown or it is not right/fully implemented as it should be by nokia? (J2ME nokia implementation, tested at nokia c6-00 and not closing stream, tested on emulator and works fine)
The main problem is that J2SE server application does not get the end of stream info, the condition read(buffer) == -1 is never true, tries to read from an empty stream and hangs until client is force-killed. This works with a very, very, very ugly workaround on the server side application
Thread.sleep(10);//wait some time for data else you would get stuck........
while ((count = dataInputStream.read(buffer)) != -1) {
byteArrayOutputStream.write(buffer, 0, count);
if (count != BUFFER_SIZE_1024 || dataInputStream.available() == 0) { //the worlds worst condition ever written... but works
break;
}
Thread.sleep(10);//wait for data input to get some data for dataInputStream.available() to return != 0 if client still sends data else you would not read all data......
}
but this solution is absolutely not acceptable (i dont know something about nokia java coding, i'm missing something, or is it maybe similar to a some sort of nokia-J2ME coding standard and i should get used to it or change platform)
I can't close the client socket after sending data because server sends a response to the client after receiving and processing data.
It looks like this: J2ME client -> J2SE server (hangs on read because client does not perform a outputstream shutdown) -> J2ME
I've tried to:
close the dataOutputStream on the J2ME client - no effect
setSocketOptions (KEEPALIVE, SNDBUF and others) - no effect or errors
nothing seems to work on the target device
sorry but i'm a bit furious right now after this nonsense fight with little java.
I'have searched for the solution but non seems to work
Client code:
SocketConnection socketConnection = (SocketConnection) Connector.open("socket://" + ip + ":" + port);
int count;
byte[] buffer = new byte[BUFFER_SIZE_1024];
// client -> server
DataOutputStream dataOutputStream = new DataOutputStream(socketConnection.openDataOutputStream());
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
while ((count = byteArrayInputStream.read(buffer)) != -1) {
dataOutputStream.write(buffer, 0, count);
dataOutputStream.flush();
}
dataOutputStream.close();
byteArrayInputStream.close();
With J2SE, my advice would be to initialize Socket from the java.nio.channels.SocketChannel and just interrupt the blocked thread after reasonable timeout has expired.
I'm not sure which side you are trying to fix, but looks like with J2ME your only option would be to set socket timeout.
EDIT
Actually, now that you've posted client code, I see the problem. If the exception is thrown from the while loop for whatever reason, the output stream is not closed.
Here is my proposed fix for that:
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
try
{
DataOutputStream dataOutputStream = new DataOutputStream(
socketConnection.openDataOutputStream()
);
try
{
while ((count = byteArrayInputStream.read(buffer)) != -1) {
dataOutputStream.write(buffer, 0, count);
dataOutputStream.flush();
}
}
finally
{
dataOutputStream.close();
}
}
finally
{
byteArrayInputStream.close();
}
Note, that it is not strictly necessary to close ByteArrayInputStream, but the code has a habit to mutate, and some day that input stream may become something that needs explicit close.
I've tried the code with the same effect - on the emulator works like a charm, on the device hangs but i solved my problem as follows:
On the J2ME client before sending the 1024 byte packet I'm sending its length and its state (IsNext or IsLast) after this on the J2SE server side in a while(true) loop. I'm reading first the length with a readShort, then state with a readByte (I know it's better to combine it on a one short but I didn't knew if it will work and if the effort was worth it and now when it works I'm not touching this, besides it is easy to add a new state if necessarily and it works quite fast).
After this server goes in to a second nested loop [ while (dataInputStream.available() < length) {} - I'll have to put here a timeout but I'll worry about that later. Also note that on J2ME dataInputStream.available() always returns a 0 (!) so in the J2ME client read in this place is a for (int i = 0; i < length... loop reading a single byte]
When the while(dataInputStream.available() ... loop breaks I'm reading a block of data which length I have, and if the state is IsLast I break the while(true) loop. Works perfectly and stable.
Thanks for the advice and hope this info will help someone
When working with Sockets in Java, how can you tell whether the client has finished sending all (binary) data, before you could start processing them. Consider for example:
istream = new BufferedInputStream (socket.getInputStream());
ostream = new BufferedOutputStream(socket.getOutputStream());
byte[] buffer = new byte[BUFFER_SIZE];
int count;
while(istream.available() > 0 && (count = istream.read(buffer)) != -1)
{
// do something..
}
// assuming all input has been read
ostream.write(getResponse());
ostream.flush();
I've read similar posts on SO such as this, but couldn't find a conclusive answer. While my solution above works, my understanding is that you can never really tell if the client has finished sending all data. If for instance the client socket sends a few chunks of data and then blocks waiting for data from another data source before it could send more data, the code above may very well assume that the client has finished sending all data since istream.available() will return 0 for the current stream of bytes.
Yes, you're right - using available() like this is unreliable. Personally I very rarely use available(). If you want to read until you reach the end of the stream (as per the question title), keep calling read() until it returns -1. That's the easy bit. The hard bit is if you don't want the end of the stream, but the end of "what the server wants to send you at the moment."
As the others have said, if you need to have a conversation over a socket, you must make the protocol explain where the data finishes. Personally I prefer the "length prefix" solution to the "end of message token" solution where it's possible - it generally makes the reading code a lot simpler. However, it can make the writing code harder, as you need to work out the length before you send anything. This is a pain if you could be sending a lot of data.
Of course, you can mix and match solutions - in particular, if your protocol deals with both text and binary data, I would strongly recommend length-prefixing strings rather than null-terminating them (or anything similar). Decoding string data tends to be a lot easier if you can pass the decoder a complete array of bytes and just get a string back - you don't need to worry about reading to half way through a character, for example. You could use this as part of your protocol but still have overall "records" (or whatever you're transmitting) with an "end of data" record to let the reader process the data and respond.
Of course, all of this protocol design stuff is moot if you're not in control of the protocol :(
I think this is the task more of a protocol, assuming that you are the man who writes both the transmitting and receiving sides of application.
For example you could implement some simple logic protocol and divide you data into packets. Then divide packets into two parts: the head and the body. And then to say that your head consists of a predefined starting sequence and contains number of bytes in the body. Of forget about starting sequence and simpy transfer number of bytes in the bofy as a first byte of the packet.
Then you've could solve you problem.
As some ppl already said you can't avoid some kind of protocol for communication.
It should look like this for example:
On the server side you have:
void sendMSG(PrintWriter out){
try {
//just for example..
Process p = Runtime.getRuntime().exec("cmd /c dir C:");
BufferedReader br = new BufferedReader(new InputStreamReader(
p.getInputStream()));
//and then send all this crap to the client
String s = "";
while ((s = br.readLine()) != null) {
out.println("MSG");
out.println(s);
}
} catch (Exception e) {
System.out.println("Command incorrect!");
}
out.println("END");
}
//You are not supposed to close the stream or the socket, because you might want to send smth else later..
On the client side you have:
void recieveMSG(BufferedReader in) {
try {
while (in.readLine().equals("MSG")) {
System.out.println(in.readLine());
}
} catch (IOException e) {
System.out.println("Connection closed!");
}
}
as Nikita said this is more of task of protocol. Either you can go by header and body approach or you can send a special character or symbol for end of stream to break processing loop. Something like if you send say '[[END]]' on socket to denote end of stream.