On the one side of the socket I know the data is going into the socket correctly.
I set up a connection:
Connection sr = new Connection();
Server server = new Server("NAME", Interger.parseInt(port));
server.setIp(ip);
sr.setServer(server);
//I know my server connection code is correct because I can send and receive data in
//other areas of my program just fine
InputStream is = null;
try
{
is = sr.getChannel().socket().getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(is));
StringBuffer text = new StringBuffer();
int k =0;
while(k != -1)
{
k = br.read();
text.append((char) k);
}
}
catch(Exception e)
{
//no errors ever get thrown
}
And then I only get about half my data, 10989 bytes out of a total 21398 that I send. The amount of bytes it reads varies but the data always ends with ..., "values": [" which in the data I send over looks like , ..., "values": ["", ""].
Keep reading until you have all the data. This question has been showing up about once a week lately. There's no guarantee that the network is going to have all your data show up at once.
You need to keep reading until you have all your data. How do you know how much data was sent? You should probably build a little protocol between the client/server that defines how much data is going to be sent, the server reads that little header and continues to read until the full message has been received.
Don't know if this could help you :
int k =0;
while((k = br.read()) != -1){
text.append((char) k);
}
1) In your case it is making the check on the next iteration, which may lead to appending of non-representable character(char of -1) to the end of text.
2) Never leave catch block empty, may be there is some execption.
So because my sending side of the socket was in c++ I was accidentally passing in a null ASCII value into the socket. And it is undocumented on the java side of the socket that if the read encounters a null value it treats it as an end of file. So it was prematurely ending the stream because it hit the null.
Related
I want to send multiple packets separately with using tcp socket in Java. Here's my code.
try {
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
String[] array = new String[4];
array[0] = "stack";
array[1] = "over";
array[2] = "flow";
array[3] = "coding";
for (int i = 0; i < array.length; i++) {
out.write(array[i].getBytes()); //send packet
}
} catch (IOException e) {
throw new RuntimeException(e);
}
I take all data in one packet right now. Here is the received packet's terminal output:
Incoming Transmission => stackoverflowcoding
That's what i want:
Incoming Transmission => stack
Incoming Transmission => over
Incoming Transmission => flow
Incoming Transmission => coding
How can i receive data as 4 packets separately?
Why do i want this ? Because in my client there's an event that listens the coming tcp packages. This event must be triggered for each elements of array separately.
TCP logically represents a continuous stream, similar to a file. Think of it as one continuous sequence of bytes until at some point in time it's closed and the stream ends. But there's no definite clear boundary between any two of those bytes. Sometimes you might have to wait to get more bytes, but there's no intrinsic way to tell if that's because the other side stopped sending or if there's some kind of network issue between the two of you.
While packets are used as an underlying mechanism you shouldn't rely on their separation, because they could in theory be split and merged along the way (realistically they are mostly just split and rarely merged).
The usual solution is to use some kind of protocol on top of TCP to clearly mark the different chunks you're intrested in. The simplest such protocol would simply start by sending the amount of bytes the next chunk will be long, followed with whatever the data is.
Alternatively, you could switch to UDP, which is actually packet-based and guarantees that if you receive something it'll be a single packet from the other side (though it doesn't guarantee order of packets or even their delivery).
You might be able to make this work by strategically placing out.flush() in your code, but depending on that will make your code very fragile.
TCP is a stream of bytes, it has no concept of message boundaries (like UDP does, for instance). There is no 1-to-1 relationship between sends and reads. Sent data can be joined and fragmented in whatever packets the network deems necessary for efficient transmission. The only guarantees that TCP provides you are:
data will be delivery (unless the connection is closed or lost)
data will be received in the same order it was sent
So, to do what you are asking for, you have to explicitly mark where one data ends and the next begins. For example, by sending the data's length before the actual data, eg:
try {
DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
String[] array = new String[4];
array[0] = "stack";
array[1] = "over";
array[2] = "flow";
array[3] = "coding";
out.writeInt(array.length);
for (int i = 0; i < array.length; i++) {
byte[] bytes = array[i].getBytes(StandardCharsets.UTF_8);
out.writeInt(bytes.length);
out.write(bytes, 0, bytes.length);
}
out.flush();
} catch (IOException e) {
throw new RuntimeException(e);
}
This way, the receiver can first read an integer to know how many strings are being sent, then run a loop, where each iteration reads an integer for the string length then reads the bytes for that string, eg:
try {
DataInputStream in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
int count = in.readInt();
for (int i = 0; i < count; i++) {
int length = in.readInt();
byte[] bytes = new byte[length];
in.readFully(bytes);
String s = new String(bytes, StandardCharsets.UTF_8);
System.out.println("Incoming Transmission => " + s);
}
} catch (IOException e) {
throw new RuntimeException(e);
}
I wrote a piece of Java code to send PDF-turned postscript scripts to a network printer via Socket.
The files were printed in perfect shape but every job comes with one or 2 extra pages with texts like ps: stack underflow or error undefined offending command.
At beginning I thought something is wrong with the PDF2PS process so I tried 2 PS files from this PS Files. But the problem is still there.
I also verified the ps files with GhostView. Now I think there may be something wrong with the code. The code does not throw any exception.
The printer, Toshiba e-studion 5005AC, supports PS3 and PCL6.
File file = new File("/path/to/my.ps");
Socket socket = null;
DataOutputStream out = null;
FileInputStream inputStream = null;
try {
socket = new Socket(printerIP, printerPort);
out = new DataOutputStream(socket.getOutputStream());
DataInputStream input = new DataInputStream(socket.getInputStream());
inputStream = new FileInputStream(file);
byte[] buffer = new byte[8000];
while (inputStream.read(buffer) != -1) {
out.write(buffer);
}
out.flush();
} catch (IOException e) {
e.printStackTrace();
}
You are writing the whole buffer to the output stream regardless of how much actual content there is.
That means that when you write the buffer the last time it will most probably have a bunch of content from the previous iteration at the end of the buffer.
Example
e.g. imagine you have the following file and you use a buffer of size 10:
1234567890ABCDEF
After first inputStream.read() call it will return 10 and in the buffer you will have:
1234567890
After second inputStream.read() call it will return 6 and in the buffer you will have:
ABCDEF7890
After third inputStream.read() call it will return -1 and you will stop reading.
A printer socket will receive these data in the end:
1234567890ABCDEF7890
Here the last 7890 is an extra bit that the printer does not understand, but it can successfully interpret the first 1234567890ABCDEF.
Fix
You should consider the length returned by inputStream.read():
byte[] buffer = new byte[8000];
for (int length; (length = inputStream.read(buffer)) != -1; ){
out.write(buffer, 0, length);
}
Also consider using try-with-resources to avoid problems with unclosed streams.
Ive read many tutorials and posts about the java InputStream and reading data. Ive established a client and server implementation but i'm having weird issues where reading a variable length "payload" from the client is not consistent.
What im trying to do is to transfer up 100kB max in one single logical payload. Now i have verified that the TCP stack is not sending one mahousive 100kB packet from the client. I have played about with different read forms as per previous questions about the InputStream reading but ive nearly teared my hair out trying to get it to dump the correct data.
Lets for example say the client is sending a 70k payload.
Now the first observation i've noticed is that if I flow through the code line by line initiated from a break point, it will work fine, i get the exact same count in the outbound byte[]. When free running the byte[] will be different sizes every time i run the code with practically the same payload.
Timing problems?
second observation is that when the "inbuffer" size is set to 4096 for example this odd behaviour occurs. setting the "inbuffer" size to 1 presents the correct behaviour i.e. i get the correct payload size.
please understand i dont like the way ive had to get this to work and im not happy with the solution.
What experiences, problems have you had/seen respectively that might help me fix this code to be more reliable, easier to read.
public void listenForResponses() {
isActive = true;
try {
// apprently read() doesnt return -1 on socket based streams
// if big stuff comes through, TCP packets are segmented, but the inputstream
// does something odd and doesnt return the correct raw data.
// this is a work around to accept vari-length payloads into one byte[] buffer
byte[] inBuffer = new byte[1];
byte[] buffer = null;
int bytesRead = 0;
byte[] finalbuffer = new byte[0];
boolean isMultichunk = false;
InputStream istrm = currentSession.getInputStream();
while ((bytesRead = istrm.read(inBuffer)) > -1 && isActive) {
buffer = new byte[bytesRead];
buffer = Arrays.copyOfRange(inBuffer, 0, bytesRead);
int available = istrm.available();
if(available < 1) {
if(!isMultichunk) {
finalbuffer = buffer;
}
else {
finalbuffer = ConcatTools.ByteArrayConcat(finalbuffer,buffer);
}
notifyOfResponse(deserializePayload(finalbuffer));
finalbuffer = new byte[0];
isMultichunk = false;
}
else {
if(!isMultichunk) {
isMultichunk = true;
finalbuffer = new byte[0];
}
finalbuffer = ConcatTools.ByteArrayConcat(finalbuffer,buffer);
}
}
} catch (IOException e) {
Logger.consoleOut("PayloadReadThread: " + e.getMessage());
currentSession = null;
}
}
InputStream is working as designed.
if I flow through the code line by line initiated from a break point, it will work fine, i get the exact same count in the outbound byte[].
That's because stepping through the code is slower, so more data drives between reads, enough to fill your buffer.
When free running the byte[] will be different sizes every time i run the code with practically the same payload.
That's because InputStream.read() is contracted to block until at least one byte has been transferred, or EOS or an exception occurs. See the Javadoc. There's nothing in there about filling the buffer.
second observation is that when the "inbuffer" size is set to 4096 for example this odd behaviour occurs. setting the "inbuffer" size to 1 presents the correct behaviour i.e. i get the correct payload size.
That's the correct behaviour in the case of a 1 byte buffer for exactly the same reason given above. It's not the 'correct behaviour' for any other size.
NB Your copy loop is nonsense. available() has few correct uses, and this isn't one of them.
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
NB (2) read() does indeed return -1 on socket-based streams, but only when the peer has shutdown or closed the connection.
I have a Java application that consists of a client and a server. The client sends encrypted commands to the server, and the server executes them.
The problem that I am having right now is that, with my encryption algorithm, sometimes the encrypted command contains "\n" or "\r" characters, which mess up my server code. This is because I am using the readLine() method, which stops when it finds a line terminator. What I need is a way to read all the characters the client sends into one string.
Here is my code:
public void run(){
System.out.println("Accepted Client!");
try{
in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream(), "ISO8859_1"));
out = new PrintWriter(new OutputStreamWriter(clientSocket.getOutputStream(), "ISO8859_1"));
String clientCommand = null;
while(RunThread){
// read incoming stream
do{
clientCommand = in.readLine();
}while(clientCommand == null);
//decrypt the data
System.out.println("Client: " + clientCommand);
if(clientCommand.equalsIgnoreCase("quit")){
RunThread = false;
}else{
//do something
out.flush();
}
}
}catch(Exception e){
e.printStackTrace();
}
}
Everything I've tried (various forms of nested loops using the read() function) hasn't worked. I would welcome any help or suggestions. Thanks, everyone!
I don't see encryption in the code you posted, but usually it's not a good idea to rely on separator characters.
When sending binary data, you should prepend the length of the data to the stream, and then you know exactly how many bytes to expect (and when the message will end.) It's more efficient to do things that way too instead of looking for a newline character.
// read incoming stream
do{
clientCommand = in.readLine();
}while(clientCommand == null);
That == null seems wrong
Try
String line = null;
do {
line = in.readLine ();
clientCommand += line
} while (line != null);
One thing you must do, when working with TCP/IP, is to send the message length before the actual message. The application level cannot foresee the package size the TCP level is delivering to the destiny. So, before your message, you have to send a header with the message size and the destiny would read just these bytes.
About readLine(), I think it's better use another approaches like streams. Shortly, one suggestion:
Socket oSocket = new Socket(sAddress, iPort);
PrintWriter out = new PrintWriter(oSocket.getOutputStream(), true);
BufferedReader in = new BufferedReader(new InputStreamReader(oSocket.getInputStream()));
do{
clientCommand = in.readLine();
} while(clientCommand == null);
This makes no sense. readLine() only returns null at end of stream, so you are telling Java to loop infinitely at end of stream. I don't even understand why there is a loop at all. You don't want to ignore any input from the client, you want to process it all. You should read one line from the client, execute it, and read another line. Repeat until null, then close the socket.
For my homework assignment, I have a network of Nodes that are passing messages to each other. Each Node is connected to a set amount of other Nodes (I'm using 4 for testing). Each Link has a weight, and all the Nodes have computed the shortest path for how they want their messages sent. Every Packet that is sent is composed of the message protocol (a hard-coded int), an integer that tells how many messages have passed through the sending Node, and the routing path for the Packet.
Every Node has a Thread for each of its Links. There is an active Socket in each Link. The Packets are sent by adding a 4-byte int to the beginning of the message telling the message's length.
Everything works fine until I stress the network. For my test, there are 10 Nodes, and I get 5 of them to send 10000 packets in a simple while() loop with no Thread.sleep(). Without exception, there is always an error at some point during execution at the if(a!=len) statement.
Please let me know if I can clarify anything. Thanks in advance! Here is the code (from the Link Thread; send() and forward() are called from the Node itself):
protected void listen(){
byte[] b;
int len;
try{
DataInputStream in = new DataInputStream(sock.getInputStream());
while(true){
len = in.readInt();
b = new byte[len];
int a = in.read(b,0,len);
if(a!=len){
System.out.println("ERROR: " + a + "!=" + len);
throw new SocketException(); //may have to fix...this will happen when message is corrupt/incomplete
}
Message m = new Message(b);
int p = m.getProtocol();
switch (p){
case CDNP.PACKET:
owner.incrementTracker();
System.out.print("\n# INCOMMING TRACKER: " + m.getTracker() + "\n>>> ");
owner.forward(m);
}
}
}catch (IOException e){
e.printStackTrace();
}
}
public void send(int tracker){
String[] message = { Conv.is(CDNP.PACKET), Conv.is(tracker), owner.getMST().toString() };
Message m = new Message(message);
forward(m);
}
public synchronized void forward(Message m){
try{
OutputStream out = sock.getOutputStream();
//convert length to byte array of length 4
ByteBuffer bb = ByteBuffer.allocate(4+m.getLength());
bb.putInt(m.getLength());
bb.put(m.getBytes());
out.write(bb.array());
out.flush();
}catch (UnknownHostException e){
System.out.println("ERROR: Could not send to Router at " + sock.getRemoteSocketAddress().toString());
return;
}catch (IOException e1){
}
}
int a = in.read(b,0,len);
if(a!=len){
That won't work. The InputStream may not read all the bytes you want, it may read only what is available right now, and return that much without blocking.
To quote the Javadocs (emphasis mine):
Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read, possibly zero. The number of bytes actually read is returned as an integer.
You need to continue reading in a loop until you have all the data you want (or the stream is finished).
Or, since you are using a DataInputStream, you can also use
in.readFully(b, 0, len);
which always reads exactly len bytes (blocking until those have arrived, throwing an exception when there is not enough data).