We Keep Coding

Android Socket Stream

In my app I'm using socket connection to communicate with a server. Everything works fine, I'm able to send/receive from/to the server with no issues. None, but one. There's a certain option/response from the server than leaves the connection open for around 30 seconds. Server sends the reply right away, but it keeps the connection open and as such the app hangs there showing the 'waiting' message, because I can't seem to figure out how to close the connection/inputStream without it waiting for the server to close it.
Is there a way to read each character received by the buffer and as soon as a character is found ('*' for example) the buffer should close and also the connection to the server.
Any help?

you need to manage it between client and server. One approach is HTTP chunked. HTTP chunked send first length of message, then message data. Or, if as it is your case, if you know a "magic" byte notifies client that connection can close, you can read data one by one, and when you reach the finalizer byte, you can complete your reading and close connection. Otherwise, application hangs in input.read() until connection reaches timeout or new byte arrives
InputStream input = ....;
ByteArrayOutputStream bo = new ByteArrayOutputStream();
while (true){
int singlebyte = input.read();
if (singlebyte == -1) break; //stream ends
bo.write(singlebyte);
if (singlebyte == '*'){
//the byte you are wating. at this point, you can break loop. or continue to read
bo.write(singlebyte);
byte data[] = bo.toByteArray();
}
}

You do 'read each character as it is found'. Your problem here is not reading characters, it is the server not closing the connection, so you're blocked waiting for end of stream. Such a protocol is either broken, in which case it needs redesigning, or else it includes a length indicator or some other means of knowing when the message is complete,mwhich you're not taking proper notice of.

How to continue downloading the file Java (socket)

I'm doing a program that sends large files through sockets client-server, that piece of code
while ((bytesRead = in.read(mybytearray, 0, mybytearray.length)) != -1) {
bos.write(mybytearray, 0, bytesRead);
}
All code can be found here
I have already downloaded data as where to pass this number was spooled file? For example there is a file size 35000 bytes transmitted 20000 is broken and how to make the program to start downloading from 20000 bytes to continue?
P.S. I`m very sorry, my English is bad

When you open the connection to the destination file just keep reading it until you reach 20000 bytes, discarding the data.
After you reach your "resume point" start appending the new data to the existing file.
Unfortunately you cannot "seek" a socket the same way you would in something like local file I/O.

Java SSLSocket what to do after read returns -1?

I'm using a java server to connect to a browser with secure websockets. All works fine with the connect, but many times i get an unexpected -1 result from socket.in.read(buffer,off,len), this happens also in the middle of a frame. Normally i close a socket directly upon reception of -1, since it is end of stream. However i noted that it can also happen on a connection reset. I have come over many cases where in my tests the socket whould return valuable data after read returned -1. I even have the feeling this is more often than not. My problem arrises when sometimes i just get some scrambled data out of the socket after such a case. Another problem is that the other side is not notified when a frame cannot be delivered... So what good is TCP/SSL than? if you need to consider it an unreliable connection for transporting websocket frames in java?
I have some schemes to use that are used to deal with unreliable connections for making shure a packet arrives. But i hope that somebody knows what to do after read returns -1.
Sorry for the somewhat vague description in this one... i'm getting tired with solving this issue.
Just an example of some rubbish comming in (only text frames are submitted containing JSON data):
16-06-13 22:43:13.918;WebSocket;7: Read frame from websocket: 377, opcode:UNKNOWN
data: null
16-06-13 22:43:13.918;WebSocket;7: Read frame from websocket: 377, opcode:PONG_FRAME
data: null
16-06-13 22:43:13.918;WebSocket;7: Read frame from websocket: 377, opcode:TEXT_FRAME
data: =,6GiGGV7C6_TfPHg\~\c
Here another example of a received frame that is just a bit malformed!? how is this possible with a TCP/TLS connection???:
17-06-13 09:42:37.510;WebSocket;7: Read frame from websocket: 15, opcode:TEXT_FRAME
data: "kep-aiveY:"d613Nb2-N24eV463K-808-fJb30I9e3M02
It is supposed to read {"keep-alive":"[UUID]"}
Meanwhilst i have done some more testing and found that 9 out of 10 times it works if you continue reading after reception of -1. So even if you are reading halfway the frame and receive a -1 then you should test somehow if the socket is closed or not, i now use: socket.isInputShutdown(). if this is not the case then just continue filling up the buffer. To do so i now use the following code where socket is the SSLSocket:
public static int readFully(Socket socket, InputStream is, byte[] buffer, int off, int len) throws IOException
{
int read = 0;
while(read < len)
{
int b = is.read();
if(b < 0)
{
Logger.log(TAG, "readFully read returned: " + b + " testing if connection is reset or closed.", Logger.WARNING);
if(socket.isInputShutdown())
{
throw new IOException("InputStream closed before data could be fully read! (readFully read returned -1 and socket.isInputShutdown() is true");
}
}
else
{
buffer[off + (read++)] = (byte) b;
}
}
return read;
}
It is still not a hundred % correct but at leas i get more reliable results then before.

i get an unexpected -1 result from socket.in.read(buffer,off,len)
You have already reached EOS (end of stream) before you called this method.
this happens also in the middle of a frame.
There is no such thing as a 'frame' in TCP. If you mean it happens in the middle of an application message, you have an application protocol error.
Normally i close a socket directly upon reception of -1, since it is end of stream.
Correct.
However i noted that it can also happen on a connection reset
No it doesn't. If it did, you could not possibly have detected the reset. The statement is self-contradictory.
I have come over many cases where in my tests the socket whould return valuable data after read returned -1.
No you haven't. A socket can't return anything but -1 after it first does so. You can't be getting any data at all, let alone 'valuable' data, unless you are ignoring the -1 somewhere.
My problem arrises when sometimes i just get some scrambled data out of the socket after such a case.
Only if you ignore the -1, as you are doing.
Another problem is that the other side is not notified when a frame cannot be delivered.
Of course it isn't. If you could deliver a notification to the other side, you could deliver the packet. This doesn't make sense either. If you mean that the other side doesn't get notified when it couldn't deliver the packet, you are up against the fact that TCP sends are asyncrhonous, so you won't normally get a send error on the send that caused it. You will get it on a later send. If you need per-send acknowledgements, you need to build them into your application protocol.
So what good is TCP/SSL then?
TCP is a reliable data-stream protocol, and SSL is a secure reliable data-stream protocol. That's what use they are.
if you need to consider it an unreliable connection for transporting websocket frames in java?
Neither of them is unreliable.
I hope that somebody knows what to do after read returns -1.
Close the socket.
Meanwhilst i have done some more testing and found that 9 out of 10 times it works if you continue reading after reception of -1.
No it doesn't. 1000 times of 1000 it continues to return -1. All you are seeing here is the effect of other bugs in your code.
So even if you are reading halfway the frame and receive a -1 then you should test somehow if the socket is closed or not
You can't. The socket isn't closed. Proof: you just read from it without getting an exception. You can't test whether the connection is closed either, other than by read() returning -1.
I now use: socket.isInputShutdown().
Pointless. That tells you whether you have called Socket.shutdownInput() on your own socket. It doesn't tell you diddly-squat about the state of the connection. There is no TCP API that can do that, other than reading or writing.
if this is not the case then just continue filling up the buffer.
I.e. reading gargabe by ignoring the -1 that read() is returning.
To do so i now use the following code where socket is the SSLSocket:
Why? DataInputStream.readFully() already exists. Re-implementing it won't help.
if(b < 0)
{
Logger.log(TAG, "readFully read returned: " + b + " testing if connection is reset or closed.", Logger.WARNING);
if(socket.isInputShutdown())
At this point it is 100% irrelevant whether your Socket is shutdown for input. read() has returned -1, which means the peer has closed the connection. Period.
{
throw new IOException("InputStream closed before data could be fully read! (readFully read returned -1 and socket.isInputShutdown() is true");
}
This is all nonsense.
}
else
{
buffer[off + (read++)] = (byte) b;
}
Here you are adding the low byte of -1, which is 0xff, to the buffer. This is also nonsense.

Who is tampering with my data stream?

The piece of code below downloads a file from some URL and saves it to a local file. Piece of cake. What could possible be wrong here?
protected long download(ProgressMonitor montitor) throws Exception{
long size = 0;
DataInputStream dis = new DataInputStream(is);
int read = 0;
byte[] chunk = new byte[chunkSize];
while( (read = dis.read(chunk)) != -1){
os.write(chunk, 0, read);
size += read;
if(montitor != null)
montitor.worked(read);
}
chunk = null;
dis.close();
os.flush();
os.close();
return size;
}
The reason I am posting a question here is because it works in 99.999% of the time and doesn't work as expected whenever there is an antivirus or some other protection software installed on a computer running this code. I am blindly pointing a finger that way because whenever I stop (or disable) it, the code works perfect again. The end result of such interference is that the MD5 of downloaded file don't match the expected, and a whole new saga begins.
So, the question is - is it really possible that some smart "protection" software would alter the actual stream coming from the URL without me knowing about it? And if yes - how do you deal with this? (verified with Kasperksy and Norton products).
EDIT-1:
Apparently I've got a hold on the problem and it's got nothing to do with antiviruses. The download takes place from the FTP server (FileZilla in particular) and we use apache commons ftp on client side . What I did is went to the FTP server and terminated the connection (kicked it out) in a middle of the download. I expected that is.read(..) would throw an IOException on client side, but this never happened. Instead, the is.read(..) returns -1 meaning that there is no more data coming from the stream. This is definitely unexpected and explains why sometimes I get partial files. This doesn't explain however why sometimes the data gets altered as well.

Yeah this happens to me all the time. In my case it's caused by transparent HTTP proxying by Websense on my corporate network. The worst problem are caused by the block page being returned with 200 OK.
Do you get the same or similar corruption every time? E.g., do you get some HTML explaining why the request was blocked? The best you can probably do is compare the first few bytes of the downloaded data to some text in the block page, and throw an exception in this case.
Edit: based on your update, have you got the FTP client set to image/binary mode?

Java Server - TCP Socket detect EOF without closing the socket connection

Is there a way to detect the EOF when reading from a TCP Socket whilst the socket connection stays open?
Most of the examples I have seen are something along the lines of:
int n=0;
While((n = read.inStream(data)) != -1){
destType.write(data, 0, n);
}
However this means that you are forced to create a new connection every time you want to receive a new piece of data.
In my case this is a constant stream of images that are sent across the socket as bytes and I would like to process each image without having to close the connection between so that I can have all the images associated with a single user, aswell as it is just more efficient to not constantly open and close connections at a high frequency.
So is there a way to do this or some information on a possible alternative?

No - if the connection stays open, the stream hasn't reached its end. The idea of a stream reporting EOF and then having more data later goes against the principle of a stream.
If you want to send multiple messages across a TCP stream, the simplest way is to prefix each message with its length:
HEADER BODY
HEADER BODY
HEADER BODY
Then the client will read the header, find out how long the message is, read it, then read the next header, etc.