We Keep Coding

Get list of files from server and Call download function on that list to download content

I have a web server that stores the files at http://user.mysite.com/content
Now all I want to achieve in my android application is to download every files that user can upload on this server, I have created function in android that can download files and stores it into sdcard which is something like this:
public void doDownload(){
try {
int count;
URL url = new URL("http://user.mysite.com/content");
URLConnection connection = url.openConnection();
connection.connect();
int lengthOfFile = connection.getContentLength();
long total = 0;
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(f);
byte data[] = new byte[1024];
while ((count = input.read(data)) != -1) {
total += count;
publishProgress((int)(total/1024),lengthOfFile/1024);
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
}
catch (Exception e) {
Log.e("Download Error: ", e.toString());
}
}
How can I retrive the list of files on server and URL for those files + name of files and download Each one of them on to app using loop?
To get the list of file I have some thing list this:
public List clientServerFileList(){
URL url;
List serverDir = null;
try {
url = new URL("http://user.mysite.com/content/");
ApacheURLLister lister = new ApacheURLLister();
serverDir = lister.listAll(url);
}
catch (Exception e) {
e.printStackTrace();
Log.e("ERROR ON GETTING FILE","Error is " +e);
}
System.out.println(serverDir);
return serverDir;
}
My server is: Apache/2.2.24 (Unix) mod_ssl/2.2.24 OpenSSL/1.0.0-fips mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 Server at user.mysite.com Port 80

Send a POST or GET request to your server. When your server recive this request, response the JSON or XML to the client.
Parse the XML or JSON that server response to you, get the filename and ..., you can download the file in the file list.

File transfer in Java is not working correctly

I was trying to write simple "FTP" program, but then suddenly an error occured. So this is a network with client and server and a server storages files uploaded from client, there is also a possibility to download files from server. But when I upload file it is saved in Server directory as an empty file, will someone help me find an error in code?
Here is Client
String nameOfFileToUp = fileFromFileChooser.getName();
System.out.println("fileChooserfile name= " + fileFromFileChooser.getName());
System.out.println("File path= " + fileFromFileChooser.getPath());
pw.println(nameOfFileToUp);
File sendFile = new File(fileFromFileChooser.getPath());
FileInputStream fis = new FileInputStream(sendFile);
int size =(int) fileFromFileChooser.length();
byte[] buffer = new byte[size+1];
int bytes = 0;
while((bytes = fis.read(buffer)) != -1)
{
out.write(buffer,0,bytes);
}
fis.close();
Where pw is PrintWriter,
And Server
FileOutputStream fos = new FileOutputStream(f);
DataOutputStream dops = new DataOutputStream(fos);
while(done)
{
fc = in.readLine();
if(fc == null)
{
done = false;
}
else
{
dops.writeChars(fc);
}
}
fos.close();
Can anyone help? Please

You need to flush/close the output stream.
Also, your server should not be reading by "line", it should be reading bytes (just like your client code).

File in java not downloading correctly

I want to download a file from a Server into a client machine. But i want the file to be downloaded from a browser : I want the file to be saved at the Downloads Folder.
Im using the following code to download files.
public void descarga(String address, String localFileName) {
OutputStream out = null;
URLConnection conn = null;
InputStream in = null;
try {
// Get the URL
URL url = new URL(address);
// Open an output stream to the destination file on our local filesystem
out = new BufferedOutputStream(new FileOutputStream(localFileName));
conn = url.openConnection();
in = conn.getInputStream();
// Get the data
byte[] buffer = new byte[1024];
int numRead;
while ((numRead = in.read(buffer)) != -1) {
out.write(buffer, 0, numRead);
}
// Done! Just clean up and get out
} catch (Exception exception) {
exception.printStackTrace();
} finally {
try {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
} catch (IOException ioe) {
// Shouldn't happen, maybe add some logging here if you are not
// fooling around ;)
}
}
It works but unless i specify the absolute path it does not download the file, therefore is useless to use from different clients with different browsers, because the webpage does not even prompts the message that lets the user know that a file is being downloaded. What can i add to get that to work?
Thanks

Resumable upload from Java client to Grails web application?

After almost 2 workdays of Googling and trying several different possibilities I found throughout the web, I'm asking this question here, hoping that I might finally get an answer.
First of all, here's what I want to do:
I'm developing a client and a server application with the purpose of exchanging a lot of large files between multiple clients on a single server. The client is developed in pure Java (JDK 1.6), while the web application is done in Grails (2.0.0).
As the purpose of the client is to allow users to exchange a lot of large files (usually about 2GB each), I have to implement it in a way, so that the uploads are resumable, i.e. the users are able to stop and resume uploads at any time.
Here's what I did so far:
I actually managed to do what I wanted to do and stream large files to the server while still being able to pause and resume uploads using raw sockets. I would send a regular request to the server (using Apache's HttpClient library) to get the server to send me a port that was free for me to use, then open a ServerSocket on the server and connect to that particular socket from the client.
Here's the problem with that:
Actually, there are at least two problems with that:
I open those ports myself, so I have to manage open and used ports myself. This is quite error-prone.
I actually circumvent Grails' ability to manage a huge amount of (concurrent) connections.
Finally, here's what I'm supposed to do now and the problem:
As the problems I mentioned above are unacceptable, I am now supposed to use Java's URLConnection/HttpURLConnection classes, while still sticking to Grails.
Connecting to the server and sending simple requests is no problem at all, everything worked fine. The problems started when I tried to use the streams (the connection's OutputStream in the client and the request's InputStream in the server). Opening the client's OutputStream and writing data to it is as easy as it gets. But reading from the request's InputStream seems impossible to me, as that stream is always empty, as it seems.
Example Code
Here's an example of the server side (Groovy controller):
def test() {
InputStream inStream = request.inputStream
if(inStream != null) {
int read = 0;
byte[] buffer = new byte[4096];
long total = 0;
println "Start reading"
while((read = inStream.read(buffer)) != -1) {
println "Read " + read + " bytes from input stream buffer" //<-- this is NEVER called
}
println "Reading finished"
println "Read a total of " + total + " bytes" // <-- 'total' will always be 0 (zero)
} else {
println "Input Stream is null" // <-- This is NEVER called
}
}
This is what I did on the client side (Java class):
public void connect() {
final URL url = new URL("myserveraddress");
final byte[] message = "someMessage".getBytes(); // Any byte[] - will be a file one day
HttpURLConnection connection = url.openConnection();
connection.setRequestMethod("GET"); // other methods - same result
// Write message
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
out.writeBytes(message);
out.flush();
out.close();
// Actually connect
connection.connect(); // is this placed correctly?
// Get response
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String line = null;
while((line = in.readLine()) != null) {
System.out.println(line); // Prints the whole server response as expected
}
in.close();
}
As I mentioned, the problem is that request.inputStream always yields an empty InputStream, so I am never able to read anything from it (of course). But as that is exactly what I'm trying to do (so I can stream the file to be uploaded to the server, read from the InputStream and save it to a file), this is rather disappointing.
I tried different HTTP methods, different data payloads, and also rearranged the code over and over again, but did not seem to be able to solve the problem.
What I hope to find
I hope to find a solution to my problem, of course. Anything is highly appreciated: hints, code snippets, library suggestions and so on. Maybe I'm even having it all wrong and need to go in a totally different direction.
So, how can I implement resumable file uploads for rather large (binary) files from a Java client to a Grails web application without manually opening ports on the server side?

HTTP GET method have special headers for range retrieval: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35 It's used by most downloaders to do resumable download from server.
As I understand, there are no standard practice for using this headers for POST/PUT request, but it's up to you, right? You can make pretty standard Grails controller, that will accept standard http upload, with header like Range: bytes=500-999. And controller should put this 500 uploaded bytes from client into file, starting at position 500
At this case you don't need to open any socket, and make own protocols, etc.
P.S. 500 bytes is just a example, probably you're using much bigger parts.

Client Side Java Programming:
public class NonFormFileUploader {
static final String UPLOAD_URL= "http://localhost:8080/v2/mobileApp/fileUploadForEOL";
static final int BUFFER_SIZE = 4096;
public static void main(String[] args) throws IOException {
// takes file path from first program's argument
String filePath = "G:/study/GettingStartedwithGrailsFinalInfoQ.pdf";
File uploadFile = new File(filePath);
System.out.println("File to upload: " + filePath);
// creates a HTTP connection
URL url = new URL(UPLOAD_URL);
HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
httpConn.setDoOutput(true);
httpConn.setRequestMethod("POST");
// sets file name as a HTTP header
httpConn.setRequestProperty("fileName", uploadFile.getName());
// opens output stream of the HTTP connection for writing data
OutputStream outputStream = httpConn.getOutputStream();
// Opens input stream of the file for reading data
FileInputStream inputStream = new FileInputStream(uploadFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer)) != -1) {
System.out.println("bytesRead:"+bytesRead);
outputStream.write(buffer, 0, bytesRead);
outputStream.flush();
}
System.out.println("Data was written.");
outputStream.flush();
outputStream.close();
inputStream.close();
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
// reads server's response
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String response = reader.readLine();
System.out.println("Server's response: " + response);
} else {
System.out.println("Server returned non-OK code: " + responseCode);
}
}
}
Server Side Grails Programme:
Inside the controller:
def fileUploadForEOL(){
def result
try{
result = mobileAppService.fileUploadForEOL(request);
}catch(Exception e){
log.error "Exception in fileUploadForEOL service",e
}
render result as JSON
}
Inside the Service Class:
def fileUploadForEOL(request){
def status = false;
int code = 500
def map = [:]
try{
String fileName = request.getHeader("fileName");
File saveFile = new File(SAVE_DIR + fileName);
System.out.println("===== Begin headers =====");
Enumeration<String> names = request.getHeaderNames();
while (names.hasMoreElements()) {
String headerName = names.nextElement();
System.out.println(headerName + " = " + request.getHeader(headerName));
}
System.out.println("===== End headers =====\n");
// opens input stream of the request for reading data
InputStream inputStream = request.getInputStream();
// opens an output stream for writing file
FileOutputStream outputStream = new FileOutputStream(saveFile);
byte[] buffer = new byte[BUFFER_SIZE];
int bytesRead = inputStream.read(buffer);
long count = bytesRead
while(bytesRead != -1) {
outputStream.write(buffer, 0, bytesRead);
bytesRead = inputStream.read(buffer);
count += bytesRead
}
println "count:"+count
System.out.println("Data received.");
outputStream.close();
inputStream.close();
System.out.println("File written to: " + saveFile.getAbsolutePath());
code = 200
}catch(Exception e){
mLogger.log(java.util.logging.Level.SEVERE,"Exception in fileUploadForEOL",e);
}finally{
map <<["code":code]
}
return map
}
I have tried with above code it is worked for me(only for file size 3 to 4MB, but for small size files some bytes of code missing or not even coming but in request header content-length is coming, not sure why it is happening.)

FTP Client issue

I'm having a tough time figuring something out. (I'm pretty new to all this.)
I wrote this java pgm to ftp a large file to a destination server.
Here's the code (codes been modified a bit for display):
public static void ftpUpload(String path, String upfileName, String dirName) throws Exception
{
FTPClient client = new FTPClient();
client.addProtocolCommandListener((ProtocolCommandListener) new PrintCommandListener(new PrintWriter(System.out)));
client.enterLocalPassiveMode();
FileInputStream fis = null;
int reply;
try {
client.connect(ftpserver);
client.login(ftpuserid, ftppasswd);
reply = client.getReplyCode();
if(FTPReply.isPositiveCompletion(reply)){
client.changeWorkingDirectory(ftpdirectoryName + "/" + dirName);
boolean mkDir = client.makeDirectory(getCurrentMMMYY().toLowerCase());
client.changeWorkingDirectory(getCurrentMMMYY().toLowerCase());
//Create an InputStream of the file to be uploaded
fis = new FileInputStream(path + upfileName);
//Store file to server
client.storeFile(upfileName, fis);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (fis != null) {
fis.close();
}
client.logout();
//client.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Something weird is happening on files I'm sending...
One of my files on the origination server is 82575786 in size, and when I ftp this file it almost sends the entire file. It actually sends 82574867. (missing 919)
Another file on the origination server is 717885, and when I ftp this file it almost sends the entire file. It actually sends 717522. (missing 363)
I pulled the log to see if something crashed, but it didn't show anything wrong with the transfer. Here are the 2 log entries showing the transfer.
[08/09/11 20:21:13:618 EDT] 00000043 SystemOut O 221-You have transferred 717522 bytes in 1 files.
221-You have transferred 82574867 bytes in 1 files.
Anyone's help would greatly be appreciated.
Thanks
Dan.

Are you transferring in ASCII mode instead of binary? ASCII mode converts CR/LF to LF and vice-versa depending on server and client settings.
Are you using Apache's FTP client? It says the default is ASCII, you could try setting BINARY_FILE_TYPE with setFileType:
client.setFileType(FTPClient.BINARY_FILE_TYPE);

To upload a binary File you have to use the FTP.BINARY_FILE_TYPE but is not enough.
You are using only an INPUT stream, and you need to use an outputstream too
I hope that this example will help you:
FTPClient client = new FTPClient();
client.connect("192.168.30.20");
client.login("pwd", "pwd");
client.setFileType(FTP.BINARY_FILE_TYPE);
String path_base = "/myPath/";
InputStream fis = new FileInputStream("A.pdf");
OutputStream os = client.storeFileStream(path_base+ "B.pdf");
byte buf[] = new byte[8192];
int bytesRead = fis.read(buf);
while (bytesRead != -1) {
os.write(buf, 0, bytesRead);
bytesRead = fis.read(buf);}
fis.close();
os.close();
client.completePendingCommand();
client.logout();
client.disconnect();