Python socket receives incomplete message from Java PrintWriter socket - java

I made a python "queue" (similar to a JMS protocol) that will receive questions from two Java clients. The python-server will receive the message from one of the Java clients and the second one will read the question and post an answer. The connection and messaging works, the problem comes when a Java client answers with a String of great length.
The response received by python is incomplete! What is worse, the message is cut at a certain number of characters and always at the same length, but, that number is different if someone else hosts the server. (i.e.: friend1 hosts the server, friend2 sends response, length received: 1380chars. Friend2 hosts the server, friend1 posts the answer, length received: 1431chars) This is the server-side python code:
s = socket.socket()
host = socket.gethostname()
# host = "192.168.0.20"
port = 12345
s.bind((host, port))
s.listen(5)
while True:
c, addr = s.accept()
# print 'Got connection from', addr
message = c.recv(8192) #Is this length a problem?
# print message
message = message.strip()
ipAddress = addr[0]
I read questions here on StackOverflow, that c.recv() should have no problem with a big number of bytes and our response is somewhere close to 1500 characters. This is the java client:
private void openConnection(){
try {
socket = new Socket(HOST, PORT);
out = new PrintWriter(socket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(
socketPregunta.getInputStream()));
stdIn = new BufferedReader(new InputStreamReader(System.in));
} catch (Exception e) {}
}
public void sendAnswer(String answer) throws IOException{
openConnection();
out.write("PUBLISH-" + answer); //This answer is send incomplete!
out.flush();
closeConnection();
}
Thanks in advance!

From the documentation:
recv(buffersize[, flags]) -> data
Receive up to buffersize bytes from the socket. For the optional
flags argument, see the Unix manual. When no data is available, block
until at least one byte is available or until the remote end is
closed. When the remote end is closed and all data is read, return
the empty string.
So recv() can return fewer bytes than you ask for, which is what's happening in your case. There is discussion of this in the socket howto.
Basically you need to keep calling recv() until you have received a complete message, or the remote peer has closed the connection (signalled by recv() returning an empty string). How you do that depends on your protocol. The options are:
use fixed sized messages
have some kind of delimiter or sentinel to detect end of message
have the client provide the message length as part of the message
have the client close the connection when it has finished sending a message. Obviously it will not be able to receive a response in this case.
Looking at your Java code, option 4 might work for you because it is sending a message and then closing the connection. This code should work:
s = socket.socket()
host = socket.gethostname()
# host = "192.168.0.20"
port = 12345
s.bind((host, port))
s.listen(5)
while True:
c, addr = s.accept()
# print 'Got connection from', addr
message = []
chars_remaining = 8192
recv_buf = c.recv(chars_remaining)
while recv_buf:
message.append(recv_buf)
chars_remaining -= len(recv_buf)
if chars_remaining = 0:
print("Exhausted buffer")
break
recv_buf = c.recv(chars_remaining)
# print message
message = ''.join(message).strip()
ipAddress = addr[0]

Related

Cant send minecraft login packet to server?

Minecraft is a game which also can be played on multiplayer servers. Each server has its own IP and the port is for every server "25565". In generell, for this problem you should be familiar with Minecaft protocol(https://wiki.vg/Protocol#Login_Start). Even if not, I created a hyperlink where you can look for this.
My goal is to create a Minecraft Chatbot, without even open Minecraftlauncher to join any server. I know, there are already a lot of these existing, but I want to create some new commands which the client should send.
In generell, there are two big steps of the joining process when you join a Minecraft server. First, you need a connection sending a handshake (state 1) and "ping-pong". After this, you are connected to the server. This first step works very well, so I think I needn´t to show you. But the second step is the authentification of every client. Herefor I send a handshake (state 2) and then there comes my problem : I always get "java.io.EOFException" as a error message on my second step, sending my username to the Server.
try {
private String host = "hypixel.net";
socket.connect(host, 25565); //I created already a Socket called "socket"
DataOutputStream output = new DataOutputStream(socket.getOutputStream());
DataInputStream input = new DataInputStream(socket.getInputStream()); //socket is for creating streams
byte[] handShakeMessage = createHandshakeMessageLogin(host, 25565);
writeVarInt(output, handShakeMessage.length);
output.write(handShakeMessage);
System.out.println("Send handShakeMessage!");
output.writeByte(0x01); //hopefully the right packet size
output.writeByte(0x00); //packetID
output.writeUTF("ExamplePlayer"); //sending username
}
public void writeVarInt(DataOutputStream out, int paramInt) throws IOException {
while (true) {
if ((paramInt & 0xFFFFFF80) == 0) {
out.writeByte(paramInt);
return;
}
out.writeByte(paramInt & 0x7F | 0x80);
paramInt >>>= 7;
}
}
public static byte [] createHandshakeMessageLogin(String host, int port) throws IOException {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
DataOutputStream handshake = new DataOutputStream(buffer);
handshake.writeByte(0x00); //packet id for handshake
writeVarInt(handshake, 4); //protocol version
writeString(handshake, host, StandardCharsets.UTF_8);
handshake.writeShort(port); //port
writeVarInt(handshake, 2); //state (2 for login)
return buffer.toByteArray();
}
public void writeString(DataOutputStream out, String string, Charset charset) throws IOException {
byte [] bytes = string.getBytes(charset);
writeVarInt(out, bytes.length);
out.write(bytes);
}
So, as you can see quite a complicated thing I guess. If somebody could answer me, why I´m getting this error message and how to fix it i would be very very happy! Thank you
EDIT: I found a post, which helped me a lot with the connection! Java sending handshake packets to minecraft server
You're writing a packet length prefix of 1, when your packet contains the entire string "ExamplePlayer".
Instead, construct the packet like is done in the createHandshakeMessageLogin function, then send the length as a varint before sending the content of that buffer.

Android Client not receiving from Python Server (SOCKET)

I am new to programming and has been looking through tutorials for my project. Please forgive me if I may seem clueless about most things.
I am currently working on a project that requires an Android client to send string data to a Python server where the data will be processed and sent back from the Python Server to the Android client. I am using Socket and managed to send the data from Client to the Server and also got the Server to send it back to the Client. However, my Client isnt receiving anything, would anyone be kind enough to help me with this problem..?
These are my codes.
Android Client Receive:
class recMsg extends Thread {
#Override
public void run()
{
try
{
Socket socket = new Socket("192.168.68.105", 8123);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String response = in.readLine();
showToast(response);
mTextView.setText((response));
socket.close();
}
catch(Exception e) {}
}
}
Python Server Receive and Send:
import socket
import sys
HOST = '192.168.68.105' # this is your localhost
PORT = 8123
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# socket.socket: must use to create a socket.
# socket.AF_INET: Address Format, Internet = IP Addresses.
# socket.SOCK_STREAM: two-way, connection-based byte streams.
print('socket created')
# Bind socket to Host and Port
try:
s.bind((HOST, PORT))
except socket.error as err:
print('Bind Failed, Error Code: ' + str(err[0]) + ', Message: ' + err[1])
sys.exit()
print('Socket Bind Success!')
# listen(): This method sets up and start TCP listener.
s.listen(10)
print('Socket is now listening')
while True:
conn, addr = s.accept()
print('Connect with ' + addr[0] + ':' + str(addr[1]))
if not conn:
break
buf = conn.recv(4096).decode()
print(buf)
# convert string to byte and send back to android
processedText = "Hello World"
conn.send(processedText.encode())
print("message sent")
s.close()

Socket Python(Server) to Java(Client) connection Java is not able to receive data from host

The Server receives String requests from the Client and replies with a String message to the Client.
I think the Server has to send the reply differently from how it is right now, so I think this line should be altered: c.sendall('Server: '+str.encode(reply))
Python Server:
# This Python file uses the following encoding: utf-8
import socket
def setupServer(port):
serv=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("socket ist da")
try:
# https://stackoverflow.com/questions/166506/finding-local-ip-addresses-using-pythons-stdlib
host=socket.gethostbyname(socket.gethostname())
print('Host ip: '+host)
serv.bind((host, port))
except socket.error as msg:
print(msg)
print("socket bind fertig")
return serv
def setupConnection(s):
# Erlaubt 1 Verbindung gleichzeitig
s.listen(1)
connection, address=s.accept()
print("Verbunden zu: "+address[0]+":"+str(address[1]))
return connection
def gET():
reply='GET ausgefuehrt'
return reply
def rEPEAT(te):
reply=te[1]
return reply
def dataTransfer(c,s):
# sendet und erhält Daten bis es stoppen soll
while True:
# Daten erhalten
data=c.recv(1024)#daten erhalten 1024 Buffer size
data=data.decode('utf-8')#nötig?
# Teilt die Datei in Befehl/Command und data auf
dataMessage=data.split(' ',1)
command=dataMessage[0]
if command=='GET'or command=='get':
reply=str(gET())
elif command=='REPEAT'or command=='repeat':
reply=str(rEPEAT(dataMessage))
elif command=='EXIT'or command=='exit':
print("Er ist geganngen")
break
elif command=='KILL'or command=='kill':
print("Server wird geschlossen!")
s.close()
break
else:
print("was?")
reply="Nicht Vorhandener Befehl"
# NAchricht senden
c.sendall('Server: '+str.encode(reply))
print(reply)
print('Klint: '+data)
print("Daten wurden geschickt")
c.close()
def main():
#print("hello")
#host='192.168.1.120'
#host='192.168.2.110'
#print('Host ip: '+str(host))
port=8999
print('Port: '+str(port))
s=setupServer(port)
while True:
try:
conn=setupConnection(s)
dataTransfer(conn,s)
except:
break
if __name__=="__main__":
main()
Java Client Thread:
public class SentThread extends Thread {
Socket socket;
Context cmain;
//boolean stop=false;
SentThread(Socket s,Context c) {
socket = s;
cmain=c;
}
#Override
public void run() {
Log.i("Roman", "run->");
socket=new Socket();
try{
socket.connect(new InetSocketAddress("192.168.2.110", 8999),5000);
}catch (Exception e) {
Log.e("Roman", e.toString());
Toast.makeText(cmain, e.toString(), Toast.LENGTH_SHORT).show();
}
try {
BufferedReader stdIn = new BufferedReader(new InputStreamReader(
socket.getInputStream()));
PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
while (true) {
out.print("Try");
out.flush();
System.out.println("Message sent");
System.out.println("Trying to read...");
String in = stdIn.readLine();
System.out.println(in);
}
} catch (Exception e) {
// TODO: handle exception
Log.e("Roman", e.toString());
}
}
}
The program gets stuck at String in = stdIn.readLine();
I can't figure out a way in which the java application is able to receive the Message from the server, even though the Java program is able to send messages to the server.
Thank you in advance
If you want to communicate with a server using a "line-oriented" protocol (ie, each line sent terminated by \n, reading lines using readline type functions, you should create file objects for the socket in python, and simply use these sock_in and sock_out objects instead of the standard IO channels sys.stdin and sys.stdout in your input/output statements.
socket.makefile() is used to create the sock_in and sock_out file objects. Using encoding='utf-8' specifies the conversion between Python characters and the socket's byte stream.
c.sendall is not used at all. Instead we use print(..., file=sock_out) to transmit the text. flush=True should only be used only on the last output from the server in response to any single command; it may be used on every output statement, but will result in more overhead and lower performance.
Here is a Minimal, Complete, Verifiable example of such as server, in Python3.6. It simply echo's back whatever is sent to it, except for the exit and kill commands. You can test it without involving a Java client using a command like telnet localhost 8889.
server.py:
import socket
class ClientKill(Exception):
pass
def _serviceClient(sock_in, sock_out):
for line in sock_in:
line = line.rstrip('\n')
if line == 'exit':
break
if line == 'kill':
raise ClientKill()
print(line, file=sock_out, flush=True) # Echo back
def serviceClient(client, addr):
with client, client.makefile('r', encoding='utf-8', newline='\n') as sock_in, \
client.makefile('w', encoding='utf-8', newline='\n') as sock_out:
_serviceClient(sock_in, sock_out)
port = 8889
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as listener:
listener.bind(('localhost', port))
listener.listen(0)
try:
while True:
client, addr = listener.accept()
serviceClient(client, addr)
except ClientKill:
print("Server killed by remote client")
On the Java side, change ...
out.print("Try");
out.flush();
... to ...
out.println("Try");
out.flush();
... or the server will keep waiting for the \n character which terminates the line sent from the client.
Also, since we explicitly use utf-8 as the encoding on the python server, you should add the corresponding character encoding in your InputStreamReader and wrap socket.getOutputStream() in a OutputStreamWriter with the same encoding.

Twistd - Telnet - Python server, Java Client, client never receives string until connection closed, when all strings concatenated and sent together

I've got a server written in python running twisted (twistd), and a client written in Java. The idea is that we'll be able to send Strings of encoded data between clients via this server. However, we're finding that clients never send a string to the server (it's never logged on the server as having been received). Does anyone have any ideas?
I've included the code for the client and server below.
Client:
Socket s = new Socket("localhost", 1025);
InputStream is = s.getInputStream();
OutputStream os = s.getOutputStream();
System.out.println("Before Read");
System.out.println(is.read());
System.out.println("After Read");
os.write("Hello from java!".getBytes());
Server:
class MyChat(basic.LineReceiver):
__opponent = None
def connectionMade(self):
print "SLOG"
self.factory.clients.append(self)
print "SLOG 1"
self.factory.notInGame.append(self)
print "SLOG 2"
while (len(self.factory.notInGame) >= 2):
x = self.factory.notInGame.pop(0)
y = self.factory.notInGame.pop(0)
x.__opponent = y
y.__opponent = x
print "SLOG FINISH YAY"
def connectionLost(self, reason):
print "SLOG Lost a client!"
self.factory.clients.remove(self)
if (self.__opponent == None):
self.factory.notInGame.remove(self)
else:
self.__opponent.__opponent = None
self.factory.notInGame.append(self.__opponent)
def lineReceived(self, data):
print "SLOG Sender data received"
if self.__opponent == None:
self.transport.write("E0") # not in game
print "SLOG E0"
return
self.__opponent.transport.write(data)
from twisted.internet import protocol
from twisted.application import service, internet
factory = protocol.ServerFactory()
factory.protocol = MyChat
factory.clients = []
factory.notInGame = []
application = service.Application("chatserver")
#internet.TCPServer(1025, factory).setServiceParent(application)
reactor.listenTCP(1025, factory)
Any help appreciated - thanks
Sam
LineReceiver accumulates data in a buffer and calls the lineReceived callback when it receives a full line. By default lines are terminated by the byte sequence "\r\n".
It looks like your Java application sends "Hello from java!". Since it does not send "\r\n" LineReceiver never decides to call lineReceived.

input.read() never returns -1

I am writing a proxy server in Java.
Initially, I do (simplified)
server = new ServerSocket(5568);
incoming = server.accept();
input = incoming.getInputStream();
...
outgoing = new Socket(host, 80);
output = outgoing.getOutputStream();
output.write(inputbuffer, 0, i);
where inputbuffer is some collection of bytes received so far (I read the incoming data up until the part where I know the host header, and then open a connection to the server and send what I have so far). So server is my welcome socket, input is the data coming to my proxy from the client, and output is the data to the serve from my proxy.
Next, I want the output from the server to be written to the client in parallel with the client still possibly writing stuff to the server. So I create a separate thread to read from the client:
final InputStream finalInput = input;
final OutputStream finalOutput = output;
Thread sendingStuff = new Thread(){
public void run(){
int c;
while ((c = finalInput.read()) != -1){
finalOutput.write((byte)c);
finalOutput.flush();
}
finalInput.close();
finalOutput.close();
}
}
sendingStuff.start();
Finally, I have a different section in the main thread to read from the server and write that to the client.
InputStream reverseInput = outgoing.getInputStream();
OutputStream reverseOutput = incoming.getOutputStream();
int c;
while ((c = reverseInput.read()) != -1){
reverseOutput.write((byte)c);
reverseOutput.flush();
}
reverseInput.close();
reverseOutput.close();
What happens is I get input, and send output, but the browser spins forever and the line in the thread that's reading from the client never gets a -1 signal.
A lot of the time I get errors that say things like "invalid header name" or "your browser sent a request that the server could not understand" and I think it has to do with this problem I'm having. One time I even got an IOException: Socket Closed on the line that reads from the client.
So why isn't the client sending an EOF? And is this the right way to go about doing this?
"I think it's because my HTTP request has Connection: keep-alive. How do I handle this?"
I think maybe you can just open your socket once for one connection.
Try to have flag like isNewConnection. set it to true at first and after the connection is initiated, set it to false.

Categories