We Keep Coding

Remote file transfer using java nio

I am trying to use JAVA NIO to transfer a file from host A to client B without having to download the file locally and then giving the client B a link to download the file.
I am running a spark Apache framework and using maven project.
I mapped the request http://localhost:8080/download/hello in Spark using :
get("/download/:id",RequestHandler::downloadHandler);
Inside of the function is the code that downloads the file from :
"https://download.springsource.com/release/STS/3.8.1.RELEASE/dist/e4.6/spring-tool-suite-3.8.1.RELEASE-e4.6-linux-gtk-x86_64.tar.gz"
try {
URL url = new URL("https://download.springsource.com/release/STS/3.8.1.RELEASE/dist/e4.6/spring-tool-suite-3.8.1.RELEASE-e4.6-linux-gtk-x86_64.tar.gz");
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
int respCode = +httpURLConnection.getResponseCode();
System.out.println("response code : "+respCode);
if (respCode == HttpURLConnection.HTTP_OK){
String fileName = "";
String disposition = httpURLConnection.getHeaderField("Content-Disposition");
String contentType = httpURLConnection.getContentType();
int contentLength = httpURLConnection.getContentLength();
if (disposition != null) {
// extracts file name from header field
int index = disposition.indexOf("filename=");
if (index > 0) {
fileName = disposition.substring(index + 10,
disposition.length() - 1);
}
} else {
// extracts file name from URL
fileName = url.toString().substring(url.toString().lastIndexOf("/") + 1,
url.toString().length());
}
System.out.println("Content-Type = " + contentType);
System.out.println("Content-Disposition = " + disposition);
System.out.println("Content-Length = " + contentLength);
System.out.println("fileName = " + fileName);
httpURLConnection.disconnect();
System.out.println("other stuff : ");
System.out.println(url.getHost());
ReadableByteChannel readableByteChannel = Channels.newChannel(url.openStream());
FileOutputStream fileOutputStream = new FileOutputStream(fileName);
FileChannel fileChannel = fileOutputStream.getChannel();
fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
fileOutputStream.close();
readableByteChannel.close();
}
} catch (IOException e) {
e.printStackTrace();
}
I fetch the filename and file size using httpURLConnection and then processed to download the file. what I am trying to do is, instead of downloading the file locally using fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE) transfer the file directly to the client.
I did some research and I think it is possible with using Socketchannels but I didn't understand how it is supposed to work.
I also read this article
https://examples.javacodegeeks.com/core-java/nio/java-nio-large-file-transfer-tutorial/
and tried to understand the class Reciever, but it is still not clear to me how.
I would appreciate some guidance. Thank you

Server sends html but no images to the browser HTTP

This code loads my html but it will not load my jpgs in the html, when I run the code it seems like it is working and all the bytes are being sent for each image but they do not show up in my browser. Im supposed to use the DataOutputStream to send the bytes but maybe thats the problem? im really lost and brand new to network programming.
public void run(){
System.out.println("CLIENT THREAD STARTING");
String req = "";
if(clientIn.hasNextLine()){
req = clientIn.nextLine();
System.out.println("Header: " + request);
}
PrintWriter toClient = null;
try {
toClient = new PrintWriter(sock.getOutputStream());
} catch (IOException e2) {
e2.printStackTrace();
}
//only sends files if client requests them
if(request.contains("GET")){
req = req.substring(request.indexOf("/") + 1);
String name = req.substring(0, req.indexOf(" "));
String type = "";
if(name.contains(".jpg")){
type = "text/html";
}
else if(name.contains(".html")){
type = "image/jpeg";
}
File theFile = new File(fileName);
FileInputStream fileIn = null;
try{
fileIn = new FileInputStream(fileName);
}catch(FileNotFoundException e){
toClientText.println("404");
toClientText.flush();
System.exit(-1);
}
System.out.println("File name " + name);
toClientText.println("HTTP/1.1 200 OK");
toClientText.println("Connection: closed");
toClientText.println("Content-Length: " + theFile.length());
toClientText.println("Content-Type: " + type);
toClientText.println("\n");
toClientText.flush();
DataOutputStream dataStream = null;
try {
dataStream = new DataOutputStream(sock.getOutputStream());
} catch (IOException e1) {
e1.printStackTrace();
}
byte[] send = new byte[1024] ;
try {
while ((fin.read(send)) != -1 ) {
toClientData.write(send);
toClientData.flush();
}
toClientData.close();
fin.close();
} catch (IOException e) {
e.printStackTrace();
}
}
else{
toClientText.println("Bad Request 400");
toClientText.flush();
}
}

toClientText.println("Content-Type: " + fileType);
toClientText.println("\n");
This will create 3 line breaks between your header and the file data, so your browser will think the third line break is part of the image, which will break the image.
Also, even though all browsers can handle \n linebreaks, be aware that the standard demands \r\n linebreaks, not \n.
This should work:
toClientText.println("Content-Type: " + fileType);
toClientText.println();

InputStream doens't read the whole answer of an URL connection

I'm struggling against the uncompleted download of a file.
For example, I upload some data on github :https://gist.githubusercontent.com/rdanniau/3b7f26bb1101b28400bf24f2f9664828/raw/980d6ff511404bf14d3efc56be3dfb081541991f/LEDirium.hex
and on pasteBin : http://pastebin.com/raw/FcVfLf5b
I want to retrieve them and save them into a file "filename".
I've watch a lot of example on internet and it must be working.
Here is the code :
private void download(final URL myUrl){
new Thread(new Runnable() {
//InputStream is = null;
//FileOutputStream fos = null;
public void run() {
try {
URLConnection connection = myURLL.openConnection();
//HttpURLConnection connection = (HttpURLConnection) myUrl.openConnection();
//connection.setRequestMethod("GET");
connection.setReadTimeout(5000);
connection.setConnectTimeout(10000);
connection.connect();
//is = myUrl.openStream();
is = connection.getInputStream();
File file = new File(context.getFilesDir(),fileName);
file.delete();
file = new File(context.getFilesDir(),fileName);
fos = new FileOutputStream(file);
byte data[] = new byte[1024];
String str ="";
int count = 0;
while ((count = is.read(data)) != -1) {
fos.write(data, 0, count);
}
is.close();
fos.close();
}
catch (Exception e) {
downloadedFileCallback.onError(e);
Log.e("DownloadedFile", "Unable to download : " + e.getMessage() + " cause :" + e.getCause());
return;
}
downloadedFileCallback.onDownloadedFinished();
readFile(context);
}
}).start();
}
public void readFile(Context context){
// read
try {
BufferedReader br = new BufferedReader(new FileReader(new File(context.getFilesDir(),fileName)));
String line;
while ((line = br.readLine()) != null) {
Log.v("DL", line);
}
br.close();
}
catch (Exception e) {
Log.e("DownloadedFile", "Unable to read : " + e.getMessage() + " cause :" + e.getCause());
}
//Log.v("DownloadedFile", text.toString());
}
where myURL are called like
URL myURL = new URL("http://pastebin.com/raw/FcVfLf5b");
In the Log.v, I can see that I downloaded only a part of the file which is never the same (it could be the entire file, the half, the quarter, we don' know...)
It's probably the inputStream connection which is closed too fast. But why ?
Last question, instead of using Log.v to check if the file is correctly downloaded. Where can I found it on my phone ? I searched in many folders but I never seen my File.
Thanks a lot for any advice
EDIT : It seems to be the same here InputStream returns -1 before end of file but no one answered.

Java socket server, PHP socket client; communication works, however PHP 'echoes' data only after the socket connection is closed

I am trying to create a simple socket based server/client communication between a Java-based socket server and PHP-based socket client. I am getting all the responses as expected but not immediately. Here is what I am trying to do. PHP client sends a command to the Java server, it responds back with an answer. This answer should be displayed on PHP page immediately. Then a second command is sent and again, the server sends a response which should be displayed immediately. However in my case, the responses are 'echoed' only after entire socket based communication is terminated. That is after I close the socket connection from PHP client.
Following is my Java Socket Server code:
public class MultiThreadServer implements Runnable {
Socket csocket;
MultiThreadServer(final Socket csocket) {
this.csocket = csocket;
}
private static void download(final String url) {
try {
final URL downloadURL = new URL(url);
URLConnection conn = null;
try {
conn = downloadURL.openConnection();
final InputStream inputStream = conn.getInputStream();
final long filesize = conn.getContentLength();
System.out.println("Size of file : " + (filesize / 1024)
+ " (kb)");
final FileOutputStream outputStream = new FileOutputStream(
System.getProperty("user.dir") + "/test.exe");
final long startTime = System.currentTimeMillis();
final byte[] buffer = new byte[1024];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
final long endTime = System.currentTimeMillis();
System.out.println("File downloaded");
System.out.println("Download time in sec. is : "
+ ((endTime - startTime) / 1000));
outputStream.close();
inputStream.close();
} catch (final IOException e) {
e.printStackTrace();
}
} catch (final MalformedURLException e) {
e.printStackTrace();
}
}
public static void main(final String args[]) throws Exception {
final ServerSocket ssock = new ServerSocket(3333);
System.out.println("Listening on " + ssock.toString());
while (true) {
final Socket sock = ssock.accept();
System.out.println("Connected to " + sock.getRemoteSocketAddress());
new Thread(new MultiThreadServer(sock)).start();
}
}
#Override
public void run() {
try {
final BufferedReader br = new BufferedReader(new InputStreamReader(
this.csocket.getInputStream()));
final BufferedWriter bw = new BufferedWriter(
new OutputStreamWriter(this.csocket.getOutputStream()));
bw.write(this.csocket.getRemoteSocketAddress().toString()
.replace("/", "")
+ "\n");
bw.flush();
String line;
while ((line = br.readLine()) != null) {
if (line.equalsIgnoreCase("getMacName")) {
final String macName = InetAddress.getLocalHost()
.getHostName();
bw.write(macName + "\n");
bw.flush();
System.out.println("Machine Name : " + macName);
} else if (line.equalsIgnoreCase("getMacIP")) {
final String macIP = InetAddress.getLocalHost()
.getHostAddress();
bw.write(macIP + "\n");
bw.flush();
System.out.println("Machine IP : " + macIP);
} else if (line.equalsIgnoreCase("getCurrentVersion")) {
final String currVersion = "0.1a";
bw.write(currVersion + "\n");
bw.flush();
System.out.println("Current Version : " + currVersion);
} else if (line
.equalsIgnoreCase("downUrl:http://webserver/webapp/test.exe")) {
final String url = line.substring(8);
bw.write("Downloading : " + url + "\n");
bw.flush();
MultiThreadServer.download(url);
System.out.println("URL : " + url);
} else if (line.equalsIgnoreCase("exit")) {
bw.write("Closing\n");
bw.flush();
bw.close();
br.close();
System.out.println("Exiting!");
this.csocket.close();
break;
}
}
} catch (final IOException e) {
e.printStackTrace();
}
}
}
Following is my sample PHP client source code:
<html>
<head>
<title>Socket Connection Test</title>
</head>
<body>
<h3>Testing Socket Connection</h3>
<br>
<?php
$host="127.0.0.1";
$port = 3333;
$fp;
$macName;
$macIP;
$message;
$result;
// open a client connection
$fp = fsockopen ($host, $port, $errno, $errstr);
if (!$fp)
{
$result = "Error: Could not open socket connection! Error No: " . $errno . ". Error String: " . $errstr;
die($result);
}
else
{
$message = fgets ($fp, 1024);
$message = trim($message);
echo "Connected to remote server on current port : " . $message;
echo "<br>";
sleep(5);
// get machine name
fwrite ($fp, "getMacName\n");
$macName = fgets ($fp, 1024);
$macName = trim($macName);
echo "Machine Name : " . $macName;
echo "<br>";
sleep(5);
// get IP address
fwrite ($fp, "getMacIP\n");
$macIP = fgets ($fp, 1024);
$macIP = trim($macIP);
echo "Machine IP : " . $macIP;
echo "<br>";
sleep(5);
fwrite ($fp, "getCurrentVersion\n");
$currVersion = fgets ($fp, 1024);
$currVersion = trim($currVersion);
echo "Current Version : " . $currVersion;
echo "<br>";
sleep(5);
fwrite ($fp, "downUrl:http://webserver/webapp/text.exe\n");
$downResponse = fgets ($fp, 1024);
$downResponse = trim($downResponse);
echo "Download Response : " . $downResponse;
echo "<br>";
sleep(5);
fwrite ($fp, "exit\n");
$exitStatus = fgets ($fp, 1024);
fclose ($fp);
echo "Connection Closed.";
}
?>
</body>
</html>
In this case, the entire PHP output is displayed at the end. I even tried putting sleeps at various places, however it seems to be waiting for socket connection to be closed before writing the output. How can I can I get the output "LIVE". Like a normal chat communication? What am I doing wrong here?

Your script works okay, and has no big flaws for what you're looking for. Since you obtain the desired result when you run the script from command line rather than a web browser, we can pinpoint the cause of the issue: the HTTP protocol
HTTP isn't made for sustained connections (like you're used to with Java sockets) but it's simplified workflow is based along the lines of Request/Elaborate/Response/Forget. Therefore, you can't have a "live chat" with a pure HTML/PHP over HTTP solution.
Alas, not all your hopes are lost! To implement a "live" communication you can use Ajax, which isn't too hard to get used to. I've said "live" because it's still an HTTP based solution, but at least you can have requests and receive responses from within the same webpage, which is the closest you can get with the HTML/PHP/AJAX triad.

It seems you dont understand php with http well .
You will get the HTML response only after your php client code completes execution. (i.e) all your echo's will be put in place where you specified and returned as a whole.
Putting sleep will only delay the execution.

ready() in Java testing url too fast

When debugging I can confirm 6 out of 9 url connections work. One is dead, and others are sending a exceptions. Question 1:I want it to show either alive or dead, nothing else. Question 2 : when running the test, it tests too fast and it shows all are "dead" links when running (even though I know 6 work). Any suggestions?
try{
//Read File
FileReader file = new FileReader("c:\\blocked\\domains1.txt");
BufferedReader buff = new BufferedReader(file);
BufferedReader rd;
OutputStreamWriter wr;
while (true)
{
String line = buff.readLine();
if (line == null)
break;
else
try
{// Test URL Connection
URL url = new URL("http://www."+line);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
wr = new OutputStreamWriter(conn.getOutputStream());
wr.flush();
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
if( rd.ready())
{
//write to output file
System.out.println("Good URL: " + line);
urlAlive ++;
//Write to file
Path filePath = Paths.get("c:\\RevisedUrls\\revised.txt");
try
{
BufferedWriter myWriter = new BufferedWriter(new FileWriter(filePath.toFile()));
// write the user supplied data to the file plus a line separator.
myWriter.write(line + System.getProperty("line.separator"));
myWriter.close(); // immediately close the file when finished with it.
}
catch(Exception e)
{
System.out.println(e);
}
}
else// increment dead url
urlDead++;
}
catch (Exception e)
{
System.out.println(e.toString());
}
}
buff.close();
}
finally
{
int totalSites = urlAlive + urlDead;
JOptionPane.showMessageDialog(null,"Total Number of Alive Sites: " + urlAlive);
JOptionPane.showMessageDialog(null,"Total Number of Dead Sites: " + urlDead);
JOptionPane.showMessageDialog(null,"Total Number of Sites: " + totalSites);
}
}
}