I have a problem with a Java-Python Socket. My objective is to send a Json object from java application to python script via socket tcp and receive a response but the socket is blocked after Json sending. In the following there is my code:
try {
Socket socket = new Socket(dstAddress, dstPort);
is = new DataInputStream(socket.getInputStream());
os = new DataOutputStream(socket.getOutputStream());
PrintWriter pw = new PrintWriter(os, true);
pw.println(jsonObject.toString());
System.out.println("Send to the socket jsonObject.");
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String response = in.readLine();
System.out.println("Response: " + response);
is.close();
os.close();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
In the following lines the python code:
HOST = "192.168.1.101" #localhost
PORT = 7011
s = socket(AF_INET, SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
while (1):
print("\n\nAttending for client.....\n\n")
conn, addr = s.accept()
print("Connected by: " , addr)
data = ""
while 1:
temp = conn.recv(1024).decode()
if not temp:
break
data = data + temp
print("JSON Received!!!!!")
imageJson = {}
imageJson = json.loads(data)
# responding to the client
response = DbImages[elem[0]]
resp = "Prova"
conn.send(resp.encode())
If I terminate the java code (ctrl+C) the socket exit from block and json arrive to python. What is the problem? The problem seems to be in.readLine (). If I delete that statement then the socket works without blocks.
Your Python code is waiting for the Java side to finish and send EOF before responding (that’s what it means to recv until you get an empty bytes).
Your Java code is waiting for the Python side to respond before closing the socket.
So, they’re both waiting for each other.
Removing the readLine means the Java code is no longer waiting for anything, so it just hangs up on the Python code as soon as it’s done sending, which does make the problem go away—but it isn’t much of a solution if you actually needed a response.
So, what should they be doing? Well, there are a few different options.
Use a framed protocol, where the Java side either sends a “message-done” delimiter after each message or sends a header (with, e.g., the byte length of the message) before each one. So, the Python code can read until it has a complete message, instead of until EOF.
If you encode your JSON in compact format with everything but printable ASCII escaped, then the delimiter can just be a newline (at which point you’re using JSONlines as your protocol), and the Python code can use makefile on the socket and call readline instead of looping over recv.
Cheat and use JSON as if it were a framed protocol. It isn’t, but as long as the only top-level values you ever send are objects and arrays, it works. Then the Python code can use raw_decode (see the json module docs) after each receive until it succeeds.
If you’re only ever going to send a single message, you can just half-shutdown the socket (close the write end) from Java, and then Python will get its EOF and respond on the still-open other side of the socket. (This may sound hacky, but it’s actually perfectly common—it’s how web browsers traditionally work, although HTTP 1.1 made things a bit more complicated.)
Your response is not a line, as it doesn't seem to contain a line ending.
That means it readLine will read forever.
Try adding a newline to your response to make readLine happy:
resp = "Prova\n"
Related
Hello stack overflow world, I've been struggling with the most straight forward and common problem within Java IO, for some time, and now need your help to tackle it.
Check out this piece of code I have in a try block, within a thread.run():
// connect to client socket, and setup own server socket
clientSocket = new Socket(serverHostname, CLIENT_PORT);
//send a test command to download a file
String downloadFileName = "sample.txt";
DataOutputStream dataOutputStream = new DataOutputStream(clientSocket.getOutputStream());
System.out.println("Sending a request to download file : " + downloadFileName + " from user: Arsa node"); //todo: replace with node user later
dataOutputStream.writeUTF("D/sample.txt");
//close socket if host isn't detected anymore, and if socket doesn't become null suddenly
dataOutputStream.flush();
dataOutputStream.close();
System.out.println("****File has been sent****");
in = new DataInputStream(clientSocket.getInputStream());
byte[] retrievedFileData = new byte[8036];
if (in.readInt() > 0) {
System.out.println("Starting file download!");
in.read(retrievedFileData);
System.out.println("File data has been read, converting to file now");
//closing input stream will close socket also
in.close();
}
clientSocket.close();
2 Main questions that have been confusing me to death:
Why does dataOutputStream.close() need to be run for writeUTF to actually send my string to the server socket, I find that when I don't have dos.close(), data isn't retrieved on the other side, further because I close it, I no longer can read from the socket - as it seems the socket connection becomes closed when the Output Stream is previously closed...
What's a better way, following some sort of pattern to do this? For context, all I'm trying to do is write the filename I'm looking to download to my client, then read the response right away, which I expect to be bytes with the file, any error handling I will consider as a part of my development.
Overall, it shouldn't be complicated to write something to a socket, then read and ingest it's response...which doesn't seem to be the case here,
any help would be greatly appreciated! If the ServerSocket code snippet is needed I'm happy to share.
The observed behavior is just a side-effect of close(), as it calls flush() before closing to make sure any buffered data is sent. To solve your problem, you need to call the flush() method instead of closing.
This behavior is not unique to DataOutputStream: a lot of other OutputStream (or Writer) implementations apply buffering, and you will need to flush when you want to ensure the data is sent to the client, written to disk or otherwise processed.
BTW: The DataOutputStream and DataInputStream is for a very specific type of data serialization protocol that is particular to Java. You may want to consider carefully if this is the right protocol to use.
I have a multithreaded chat server that I have converted to work with Java SSL sockets. You can see the version without SSL sockets compared to the one I converted here, on Github. (Master branch has SSL, other branch has regular sockets)
This original model (without SSL) uses "ServerThreads" controlled by the Client to communicate with other Clients by sending messages to their "ClientThreads" on the server side, which then will echo their messages out to all other ServerThreads.
Here is the run method of ServerThread_w_SSL (client side)
#Override
public void run(){
System.out.println("Welcome :" + userName);
System.out.println("Local Port :" + socket.getLocalPort());
System.out.println("Server = " + socket.getRemoteSocketAddress() + ":" + socket.getPort());
//setup handshake
socket.setEnabledCipherSuites(socket.getSupportedCipherSuites());
try{
PrintWriter serverOut = new PrintWriter(socket.getOutputStream(), false);
InputStream serverInStream = socket.getInputStream();
Scanner serverIn = new Scanner(serverInStream);
// BufferedReader userBr = new BufferedReader(new InputStreamReader(userInStream));
// Scanner userIn = new Scanner(userInStream);
socket.startHandshake();
while(!socket.isClosed()){
if(serverInStream.available() > 0){
if(serverIn.hasNextLine()){
System.out.println(serverIn.nextLine());
}
}
if(hasMessages){
String nextSend = "";
synchronized(messagesToSend){
nextSend = messagesToSend.pop();
hasMessages = !messagesToSend.isEmpty();
}
serverOut.println(userName + " > " + nextSend);
serverOut.flush();
}
}
Here is the run method of ClientThread_w_SSL (server side)
#Override
public void run() {
try{
// setup
this.clientOut = new PrintWriter(socket.getOutputStream(), false);
Scanner in = new Scanner(socket.getInputStream());
//setup handshake
socket.setEnabledCipherSuites(socket.getSupportedCipherSuites());
socket.startHandshake();
// start communicating
while(!socket.isClosed()){
if(in.hasNextLine()){
String input = in.nextLine();
// NOTE: if you want to check server can read input, uncomment next line and check server file console.
System.out.println(input);
for(ClientThread_w_SSL thatClient : server.getClients()){
PrintWriter thatClientOut = thatClient.getWriter();
if(thatClientOut != null){
thatClientOut.write(input + "\r\n");
thatClientOut.flush();
}
}
}
}
The original program works with regular sockets, but after converting to SSL sockets, I encountered a problem: input is not being echoed back from the ClientThreads (server side) to the ServerThreads (client side).
In my first attempt at converting to SSL I used certificates, keystores and truststores. I encountered the same problem then as I do here without them, instead only using the default socket factory which relies on the cacerts file that comes with the JDK.
Note that before this bug was encountered, the first problem to address was the handshake failure occurring between the client and server. Because of the way SSL and the Java PrintWriter class work, the handshake gets initiated the first time PrintWriter.flush() is called, which happens as soon as the client sends a chat message to the server. This is only resolved by manually enabling supported ciphersuites in both the ClientThread (server) and ServerThread (client), then calling SSLSocket.StartHandshake() in at least the ClientThread, if not both.
Now the server is receiving messages from the client, but it is not echoing them out to the clients.
When I run it in a debugger and try stepping through the code I find that the ClientThread receives the client's message and sends it back by calling write() on the PrintWriter for each ClientThread, then flush(). The ServerThread is supposed to receive it by calling InputStream.available() to check for input without blocking, but available() always returns '0 bytes' and so it never hits Scanner.nextLine()
So either Printwriter.write() and .flush() aren't sending data or InputStream.available() is not reading data.
EDIT: After more debugging and testing, I can only narrow the problem down to output from the server side. I determined this by having the server immediately send its own message before waiting to receive messages, and had the client just grab the nextLine() instead of checking first with available(). Since this test failed it shows that data must be being blocked somehow coming from the server side only.
EDIT 2: I changed the code to use ObjectInputStreams and ObjectOuputStreams instead of using the Scanner and PrintWriters. Now I'm sending "Message" objects from a Serializable class I made to just hold Strings. This has fixed the output issue for messages coming from the server. If I make the client simply wait for input by calling readObject() it will receive messages from the server. However, if I use the availble() method of InputStream first, it still only returns 0 even when it shouldn't. Since the InputStream serverInStream is initialized by socket.getInputStream(), it gets an ssl.AppInputStream with an ssl.InputRecord, and I'm guessing one of the two does not implement available() correctly.
I figured it out: the problem was available(), It is useless with SSL in Java. I got the solution from this answer.
I'm using Erlang with SSL,
My server socket listens to incoming client connections and spawns a new thread for every incoming connection (assume the looping function called clientroutine())
This thread is designed based on this tutorial I found on web: http://erlycoder.com/89/erlang-ssl-sockets-example-ssl-echo-server-ssl-client-
so basically clientroutine() waits in receive, gets data from client, does some action based on received data and recursively calls itself again
Now, the problem is that when I issue ssl:send(Socket, Data), the client (Java-based) does not get anything from inputstream
Interestingly, this happens only when I recursively call clientroutine() after ssl:send like this (I skip socket close and default cases for simplicity):
clientroutine(Socket) ->
ssl:setopts(Socket, [{active, once}]),
receive
{ssl, Sock , Data} ->
ok = ssl:send(Sock, "~100 bytes list goes to client"),
clientroutine(Socket)
end.
The following works correctly (i.e. no recursion takes places and thread finishes) and my Java-client receives the string from inputstream:
clientroutine(Socket) ->
ssl:setopts(Socket, [{active, once}]),
receive
{ssl, Sock , Data} ->
ok = ssl:send(Sock, "~100 bytes list goes to client")
end.
Java-client launches inputstream listener in a separate thread like this (BufferedReader in has been declared above, among class fields):
new Thread(new Runnable(){
#Override
public void run() {
String msg;
try {
while((msg=in.readLine())!=null)
System.out.println("user received: " + msg);
} catch (IOException e) {
System.out.println("user: exception occured - inputstream reader");
}
}}).start();
I haven't yet checked if this works with Erlang client or not, I will update my post when I check it as well, but anyhow I need it to work with Java client
Any ideas why this happens?
Probably I should use some other BufferedReader routine instead of readLine(), or maybe BufferedReader requires some special character to be pushed into outputstream after the transferred message?
UPDATE. Erlang client receives everything correctly, with and without recursive call. Seems that this is somewhat related to Java inputstream
I found out that the newline character required for readLine() to fetch a line from inputstream, was (strangely) not included when I was sending my message in "recursive" version of the program, so everything goes well after I append \n to the transferred message
I'm writing a Java client/server application. It should allow clients to send text data to the server. This kind of communication should be repeatable many times using the same connection.
I write it like this:
// On a server:
ServerSocket serverSocket = new ServerSocket(port);
Socket socket = serverSocket.accept();
socket.setKeepAlive(true);
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
if (reader.ready()) {
for (String line = reader.readLine(); line != null; line = reader.readLine()) {
// do something with line
}
}
// On a client:
Socket socket = new Socket(host, port);
socket.setKeepAlive(true);
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
writer.write("Some data from client to server");
writer.flush();
The problem is: I can't read on a server before I close OutputStream on a client. Or I can't open OutputStream on a client again, if it was already closed. How can I do continuous sending and reading of data?
You need two threads at both ends, one for reading data and other one for writing data.
The problem is: I can't read on a server before I close OutputStream on a client.
Yes you can. You just can't get to the case where readLine() returns null. It isn't the same thing.
Or I can't open OutputStream on a client again, if it was already closed.
Of course not. You have to create a new Socket.
How can I do continuous sending and receiving of data?
I don't understand the question. The code you posted doesn't attempt that.
If your goal is to send many mesages over the same socket connection, these messages will have to be delimited by an application-level protocol. In other words, you won't be able to rely on any system calls like reader.ready() or reader.readLine() == null to detect the end of the message on te server.
One way to achieve this is to begin each message with its length in characters. The server will then read exactly that number of charecters, and then stop and wait for a new message. Another is to define a special character sequence which concludes each message. The server will react to reading that particular sequence by ending the reading of the current message and returning to the "wait for new message" state. You must ensure that this sequence never appears in the message itself.
I am using the below code to send data to a tcp server. I am assuming that I need to use socket.shutdownOutput() to properly indicate that the client is done sending the request. Is my assumption correct? If not please let me know the purpose of shutdownOutput(). Also appreciate any further optimizations I can make.
Client
def address = new InetSocketAddress(tcpIpAddress, tcpPort as Integer)
clientSocket = new Socket()
clientSocket.connect(address, FIVE_SECONDS)
clientSocket.setSoTimeout(FIVE_SECONDS)
// default to 4K when writing to the server
BufferedOutputStream outputStream = new BufferedOutputStream(clientSocket.getOutputStream(), 4096)
//encode the data
final byte[] bytes = reqFFF.getBytes("8859_1")
outputStream.write(bytes,0,bytes.length)
outputStream.flush()
clientSocket.shutdownOutput()
Server
ServerSocket welcomeSocket = new ServerSocket(6789)
while(true)
{
println "ready to accept connections"
Socket connectionSocket = welcomeSocket.accept()
println "accepted client req"
BufferedInputStream inFromClient = new BufferedInputStream(connectionSocket.getInputStream())
BufferedOutputStream outToClient = new BufferedOutputStream(connectionSocket.getOutputStream())
ByteArrayOutputStream bos=new ByteArrayOutputStream()
println "reading data byte by byte"
byte b=inFromClient.read()
while(b!=-1)
{
bos.write(b)
b=inFromClient.read()
}
String s=bos.toString()
println("Received request: [" + s +"]")
def resp = "InvalidInput"
if(s=="hit") { resp = "some data" }
println "Sending resp: ["+resp+"]"
outToClient.write(resp.getBytes());
outToClient.flush()
}
I am using the below code to send data to a tcp server. I am assuming
that I need to use socket.shutdownOutput() to properly indicate that
the client is done sending the request. Is my assumption correct?
YES Your assumption is correct. And this output ShutDown is known as half close . Using half close the TCP provides the ability for one end of the connection to terminate its output, while still receiving data from the other end. Let me walk you through the effects of socket.shutdownOutput() method :
Locally, the local socket and its input stream behave normally for reading
purposes, but for writing purposes the socket and its output stream behave
as though the socket had been closed by this end: subsequent writes to the
socket will throw an IOException
TCP’s normal connection-termination sequence (a - FIN acknowledged by
an ACK) is queued to be sent after any pending data has been sent and acknowledged.
Remotely, the remote socket behaves normally for writing purposes, but for
reading purposes the socket behaves as though it had been closed by this
end: further reads from the socket return an EOF condition, i.e. a read count
of -1 or an EOFException , depending on the method being called.
When the local socket is finally closed, the connection-termination sequence
has already been sent, and is not repeated; if the other end has already
done a half-close as well, all protocol exchanges on the socket are now
complete.
Hence we see that When the EOF is received, that end is assured that the other end has done the output shutdown. And this scenario is perfectly achieved by socket.shutDownOutput() on the other side.
Source: Fundamental Networking in Java, Esmond Pitt
Socket.shutdownOutput() means that the client is finished sending any data through the TCP connection. It will send the remaining data followed by a termination sequence which will completely close its OUTGOING connection. It is not possible to send any further data, which will also indicate to your program that the request is completely finished. So its recommended if you are sure you don't have to send any more data.
But it's not needed to indicate that the request is finished (you don't have to open/close the output all the time if you have multiple requests), there are other ways.