I am coding a Web Framework for me with an WebServer and an WebSocket Server.
My Current Problem since days is, that the Response Content of my WebSocket Client is very funny...
It sends me not the Content as bytes, every time the value is another.
Web Response for normal HTTP and the Socket request works perfectly.
My current Code:
poolIO.execute(new Thread() {
#Override
public void run() {
try {
InputStream inputIO = clientIO.getInputStream();
StringBuilder builderIO = new StringBuilder();
while (clientIO.isConnected()) {
int cacheIO = 0;
int countIO = 0;
byte[] binaryIO = new byte[0];
while ((inputIO.available() != 0 && (cacheIO = inputIO.read()) != 0)) {
binaryIO = new byte[binaryIO.length + 1];
binaryIO[countIO] = (byte) cacheIO;
builderIO.append((char) cacheIO);
countIO++;
}
if (builderIO.length() > 0) {
string(clientIO, builderIO.toString());
binary(clientIO, binaryIO);
binaryIO = new byte[0];
builderIO.delete(0, builderIO.length());
}
}
inputIO.close();
disconnect(clientIO);
this.stop();
} catch (IOException errorIO) {
if (errorIO.getMessage().equalsIgnoreCase("Stream closed.")) {
logIO.debug(GuardianLog.Type.INFO, "Client with IP " + clientIO.getInetAddress().getHostAddress() + " disconnected from Server with Port " + networkIO.getPort() + ".");
} else logIO.append(GuardianLog.Type.ERROR, "Socket throw an Error ", errorIO);
}
}
});
Regards
Jan
Fixed getting raw bytes with following Code:
byte[] bufferIO = new byte[inputIO.available()];
inputIO.read(bufferIO);
if (bufferIO.length != 0) {
binary(clientIO, bufferIO);
string(clientIO, new String(bufferIO));
}
Related
I'm trying to download files in java in a multi-segment way (i.e., dividing it to several parts and downloading each part in a separate thread parallelly) but when I use the code below, it seems each thread is downloading the whole file instead of just a part of it but when it finishes, file is downloaded correctly.
note that "downloadedSizeCombined" is sum of all bytes which are downloaded by all the threads and ArrayList "downloadedSize" keeps track of bytes which are downloaded by a single thread.
this method is in class Download which extends SwingWorker.
public Void doInBackground() {
ExecutorService es = Executors.newCachedThreadPool();
for (int i = 0; i < MAX_NUMBER_OF_PARTS; i++) {
int numOfThePart = i;
es.execute(new Runnable() {
#Override
public void run() {
RandomAccessFile file = null;
InputStream stream = null;
try {
while (Download.this.getStatus() == WAITINGLIST) {
Thread.sleep(1);
}
// Open connection to URL.
HttpURLConnection connection =
(HttpURLConnection) url.openConnection();
// Specify what portion of file to download.
int startByte = numOfThePart * sizeOfFile / MAX_NUMBER_OF_PARTS;
int endByte = ((numOfThePart + 1) * sizeOfFile / MAX_NUMBER_OF_PARTS) - 1;
if (numOfThePart == MAX_NUMBER_OF_PARTS)
endByte = ((numOfThePart + 1) * sizeOfFile / MAX_NUMBER_OF_PARTS);
connection.setRequestProperty("Range",
"bytes=" + ((startByte + downloadedSize.get(numOfThePart))) + "-" + endByte);
// Connect to server.
connection.connect();
// Check for valid content length.
int contentLength = connection.getContentLength();
if (contentLength < 1) {
System.out.println("1");
}
/* Set the size for this download if it
hasn't been already set. */
if (sizeOfFile == -1) {
sizeOfFile = contentLength;
}
file = new RandomAccessFile(new File(s.getCurrentDirectory(), getFileName(url)),
"rw");
file.seek(startByte + downloadedSize.get(numOfThePart));
fileLocation = new File(s.getCurrentDirectory(), getFileName(url));
stream = connection.getInputStream();
while (status == CURRENT) {
file.seek(startByte + downloadedSize.get(numOfThePart));
byte buffer[];
buffer = new byte[MAX_BUFFER_SIZE];
// Read from server into buffer.
int read = stream.read(buffer);
if (read == -1)
break;
// Write buffer to file.
file.write(buffer, 0, read);
downloadedSizeCombined += read;
downloadedSize.set(numOfThePart, downloadedSize.get(numOfThePart) + read);
publish(numOfThePart);
while (status == PAUSED) {
Thread.sleep(1);
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
// Close file.
if (file != null) {
try {
file.close();
} catch (Exception e) {
e.printStackTrace();
}
}
// Close connection to server.
if (stream != null) {
try {
stream.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
});
}
return null;
}
Thanks in advance.
Can't we use UDP connection? So if we use DatagramSocket class, it will anyways send the data in packets. Try this.
Will get back on this soon..
I am trying to serve an mp3 using HttpHandler and getting a broken pipe. It works with Google Chrome on my mac and my iPad but Android cause the HttpHander to just hang after getting the IOException and I have to restart. Using very simple code and works fine with images and html.
try {
String requestURI = t.getRequestURI().toString().substring(1);
if(requestURI.equals("") || requestURI.equals("/"))
requestURI = "index.htm";
requestURI = requestURI.replaceAll("%20", " ");
if(requestURI.contains("mp3")) {
urlToResource = new File(System.getProperty("user.home") + "/test/" +
requestURI).toURI().toURL();
}
System.out.println("Modified requestURI:" + requestURI);
if(requestURI.contains("mp3")) {
sContentType = "audio/mpeg";
} else if(requestURI.contains("png")) {
sContentType = "image/png";
} else if(requestURI.contains("jpg")) {
sContentType = "image/jpg";
} else if(requestURI.contains("favicon.ico")) {
sContentType = "content/unknown";
} else if(requestURI.contains("css")) {
sContentType = "text/css";
} else {
sContentType = "text/html";
}
if(!requestURI.contains("mp3")) {
urlToResource = new File("src/com/daford/web/" + requestURI).toURI().toURL();
}
if(urlToResource != null) {
conn = urlToResource.openConnection();
int size = conn.getContentLength();
System.out.println("file " + requestURI + " size is:" + size);
inConnectionReader = conn.getInputStream();
headers = t.getResponseHeaders();
headers.add("Content-Type", sContentType);
t.sendResponseHeaders(200, size);
os = t.getResponseBody();
int iReadByte = inConnectionReader.read();
while (iReadByte != -1) {
os.write(iReadByte);
iReadByte = inConnectionReader.read();
}
} else {
headers = t.getResponseHeaders();
headers.add("Content-Type", "text/html");
String sErrorMessage = "Error getting webpage.";
t.sendResponseHeaders(404, sErrorMessage.length());
os = t.getResponseBody();
os.write(sErrorMessage.getBytes());
}
if(os != null) {
os.close();
}
} catch (Exception e) {
e.printStackTrace();
}
I want to read files on Google Drive partially (Range request or Partial download). Moreover, I want to do it in series - to read next interval just only after previous interval has been read (with CountDownLatch) and in a separate thread (Android prohibits HTTP-requests in main thread). For such purpose I want to implement ByteChannel method:
#Override
public int read(final ByteBuffer dst) {
final com.google.api.services.drive.model.File googleFile = mFile.getImpliedFile();
final int bufferLength = dst.capacity();
final int[] bytesReceived = {-1};
final CountDownLatch latch = new CountDownLatch(1);
new Thread(new Runnable(){
public void run() {
byte[] receivedByteArray = getByteArrayGoogle(mService, googleFile, position, bufferLength);
// byte[] receivedByteArray = getByteArrayApache(googleFile, position, bufferLength);
dst.put(receivedByteArray, 0, receivedByteArray.length);
bytesReceived[0] = receivedByteArray.length;
position += receivedByteArray.length;
latch.countDown();
}
}).start();
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
return bytesReceived[0];
}
If I use libraries google-api-services-drive-v2-rev151-1.18.0-rc.jar and google-api-client-assembly-1.18.0-rc-1.18.0-rc.zip recommended by Google :
private byte[] getByteArrayGoogle(Drive drive, File file, long position, int byteCount) {
String downloadUrl = file.getDownloadUrl();
byte[] receivedByteArray = null;
if (downloadUrl != null && downloadUrl.length() > 0) {
try {
com.google.api.client.http.HttpRequest httpRequestGet = drive.getRequestFactory().buildGetRequest(new GenericUrl(downloadUrl));
httpRequestGet.getHeaders().setRange("bytes=" + position + "-" + (position + byteCount - 1));
com.google.api.client.http.HttpResponse response = httpRequestGet.execute();
InputStream is = response.getContent();
receivedByteArray = IOUtils.toByteArray(is);
response.disconnect();
System.out.println("google-http-client-1.18.0-rc response: [" + position + ", " + (position + receivedByteArray.length - 1) + "]");
} catch (IOException e) {
e.printStackTrace();
}
}
return receivedByteArray;
}
I can never read more than six intervals! However, if I use Apache HTTP client:
private byte[] getByteArrayApache(File file, long position, int byteCount) {
String downloadUrl = file.getDownloadUrl();
byte[] receivedByteArray = null;
if (downloadUrl != null && downloadUrl.length() > 0) {
try {
org.apache.http.client.methods.HttpGet httpRequestGet = new HttpGet(downloadUrl);
httpRequestGet.addHeader("Authorization", "Bearer " + GoogleDriveActivity.getAccessToken(mContext));
httpRequestGet.addHeader("Range", "bytes=" + position + "-" + (position + byteCount - 1));
org.apache.http.HttpResponse response = new DefaultHttpClient().execute(httpRequestGet);
InputStream is = response.getEntity().getContent();
receivedByteArray = IOUtils.toByteArray(is);
System.out.println("apache-http-client response: [" + position + ", " + (position + receivedByteArray.length - 1) + "]");
} catch (IOException e) {
e.printStackTrace();
}
}
return receivedByteArray;
}
I can read all needed intervals.
The problem appears to be in com.google.api.client.googleapis.extensions.android.gms.auth.GoogleAccountCredential.RequestHandler
public void intercept(HttpRequest request) throws IOException {
try {
token = getToken();
request.getHeaders().setAuthorization("Bearer " + token);
} catch (GooglePlayServicesAvailabilityException e) {
throw new GooglePlayServicesAvailabilityIOException(e);
} catch (UserRecoverableAuthException e) {
throw new UserRecoverableAuthIOException(e);
} catch (GoogleAuthException e) {
throw new GoogleAuthIOException(e);
}
}
because I can't get token for seventh request and application always freezes in this point under debugging.
token = getToken();
1. What is the reason of that strange behaviour and how to make more than six requests with Google API?
2. Are there other ways to do partial download with Google API?
P.S. By the way, class RequestHandler has annotation #Beta...
COMMENT:
It seems the problem is in Google Play Service proprietary class com.google.android.gms.auth.GoogleAccountCredential used for getting a token:
GoogleAuthUtil.getToken(context, accountName, scope)
As I suspect GoogleAuthUtil.getToken has to use main thread to validate authentication, but main thread is blocked by CountDownLatch in a waiting of request result. Therefore, I have deadlock at this point. First six requests are executed from AsyncTask and they does not block main thread.
I have to make an abstaction in my software - replace direct unblockable NIO sockets ( client/server ) to software abstraction.
For example, instead of connecting via tcp client would exec openssl s_client -connect xxx.xxx.xxx.xxx . I have written a little demo, and it even works. Sometimes :(
The first trouble is that Process's streams can't be used with Selector, so I can't replace socketchannel with any other type of channel, so I have to read/write without any chance to avoid blocking.
The second one is that a protocol is a duplex binary file-transfer protocol ( binkp ), so process's buffered streams are unusabe. I've tried to avoid that converting in/out data to base64 and it works, but also sometimes.
I can't understant why it works or not sometimes. I put a piece of test code below. The first word is frame's length, but first bit is ignored. Please, tell me your guesses. Thanks.
public class BufferedSocketBase64 {
static class InToOut implements Runnable {
InputStream is;
OutputStream os;
boolean direction; //
public InToOut(InputStream is, OutputStream os, boolean direction) {
super();
this.is = is;
this.os = os;
this.direction = direction;
}
#Override
public void run() {
System.out.println(Thread.currentThread().getId() + " start "
+ ((direction) ? "encode from to" : "decode from to"));
boolean eof = false;
while (true) {
if (direction) {
// encode to base64 data
try {
int[] head = new int[2];
for (int i = 0; i < 2; i++) {
head[i] = is.read();
}
int len = (head[0] & 0xff << 8 | head[1] & 0xff) & 0x7FFF;
byte[] buf = new byte[len + 2];
buf[0] = (byte) (head[0] & 0xff);
buf[1] = (byte) (head[1] & 0xff);
for (int i = 2; i < len; i++) {
buf[i] = (byte) (is.read() & 0xff);
}
System.out.println(Thread.currentThread()
.getId() + " << " + new String(buf));
if (len > 0) {
String send = Base64Util.encode(buf, len);
send += "\n";
os.write(send.getBytes());
os.flush();
}
} catch (IOException e) {
eof = true;
}
} else { // decode from base64
try {
StringBuilder sb = new StringBuilder(1024);
byte c = 0x0a;
do {
c = (byte) is.read();
if (c >= 0 && c != 0x0a) {
sb.append(new String(new byte[] { c }));
}
} while (c != 0x0a && c >= 0);
if (sb.length() != 0) {
try {
byte[] buf = Base64Util.decode(sb.toString());
System.out.println(Thread.currentThread()
.getId() + " >> " + buf.length);
os.write(buf);
os.flush();
} catch (StringIndexOutOfBoundsException e) {
System.out
.println(Thread.currentThread().getId()
+ " error on " + sb.toString());
}
}
} catch (IOException e) {
eof = true;
}
}
if (eof) {
System.out.println(Thread.currentThread().getId() + " EOF");
break;
}
}
try {
is.close();
os.close();
} catch (IOException e) {
}
}
}
public static void main(String[] args) throws Exception {
Process proc2 = Runtime.getRuntime().exec("nc -l -p 2020");
Process proc1 = Runtime.getRuntime().exec("nc 127.0.0.1 2020");
Socket sock1 = new Socket();
sock1.connect(new InetSocketAddress("127.0.0.1", 24554), 30);
Socket sock2 = new Socket();
sock2.connect(new InetSocketAddress("127.0.0.1", 24557), 30);
new Thread(new InToOut(sock1.getInputStream(), proc1.getOutputStream(),
true)).start();
new Thread(new InToOut(proc1.getInputStream(), sock1.getOutputStream(),
false)).start();
new Thread(new InToOut(sock2.getInputStream(), proc2.getOutputStream(),
true)).start();
new Thread(new InToOut(proc2.getInputStream(), sock2.getOutputStream(),
false)).start();
}
UPDATED:
I've found right way. I uses syncchronized queries for each stream and synchronized threads to fill or erase that queries. All threads mutually blocks themselves. And it works! :)
Sorry for bother.
I've found right way. I uses syncchronized queries for each stream and synchronized threads to fill or erase that queries. All threads mutually blocks themselves. And it works! :) Sorry for bother.
I'm making a simple FTP server in java. I have everything working when I test it locally (running both the server and client on my own machine). When I run the server and client on two different remote machines, however, the client hangs somewhere shortly after it receives the "150 File status okay" message from the server. I can't understand why it works fine in one location but not the other. Here is the relevant code:
Server (sending the file):
FileInputStream input = null;
try {
input = new FileInputStream(filePath);
} catch (FileNotFoundException e) {
out.writeBytes("550 File not found or access denied.\r\n");
}
out.writeBytes("150 File status okay.\r\n");
// TCP CONNECT
DataOutputStream outToClient_d = null;
Socket clientSocket1 = null;
try {
ipAddress = ipAddress.substring(0,
ipAddress.length() - 1);
clientSocket1 = new Socket(ipAddress,
portNumber);
outToClient_d = new DataOutputStream(
clientSocket1.getOutputStream());
}
catch (UnknownHostException e) {
out.writeBytes("425 Can not open data connection.\r\n");
}
byte[] buf = new byte[2048];
int len;
while ((len = input.read(buf)) > 0) {
outToClient_d.write(buf, 0, len);
}
input.close();
out.writeBytes("250 Requested file action completed.\r\n");
clientSocket1.close();
outToClient_d.close();
Client (saving file into /retr_files):
InputStream inFromServer_d = null;
if (welcomeSocket != null) {
if (!welcomeSocket.isClosed()) {
welcomeSocket.close();
}
}
try {
welcomeSocket = new ServerSocket(port);
System.out.print("PORT " + myIP + "," + num1 + "," + num2 + "\r\n");
out.writeBytes("PORT " + myIP + "," + num1 + "," + num2 + "\r\n");
System.out.print(parseReply(getResponse()));
System.out.print("RETR " + pathname + "\r\n");
out.writeBytes("RETR " + pathname + "\r\n");
String reply = parseReply(getResponse());
if (reply.charAt(10) == '1') {
System.out.print(reply);
System.out.print(parseReply(getResponse()));
try {
clientSocket_d = welcomeSocket.accept();
} catch (IOException e) {
System.out
.print("GET failed, FTP-data port not allocated.\r\n");
System.exit(-1);
}
inFromServer_d = clientSocket_d.getInputStream();
// READ
InputStream input = inFromServer_d;
OutputStream output = new FileOutputStream("retr_files/file"
+ retrCnt);
byte[] buf = new byte[2048];
int len;
while ((len = input.read(buf)) > 0) {
output.write(buf, 0, len);
}
input.close();
output.close();
clientSocket_d.close();
} else {
System.out.print(reply);
}
} catch (IOException e) {
System.out.print("GET failed, FTP-data port not allocated.\r\n");
System.exit(-1);
}
Any help is appreciated!
I would guess there is a firewall between the client and server blocking the reverse connection from the server to the client. this problem is the reason that people typically use "passive" transfers instead of "active" transfers these days.