I am trying to create a program that will download resources from a webpage into a file. I created an mkir function that creates a directory whose name is a hexadecimal version of the hash of a given String. I then, created a saveResource function that saves a resource in a file, as well as in a byte array. However, when I try saving the resource into the file I get an error message stating: java.io.FileNotFoundException: 648451a1 (Is a directory)
Here are the functions:
public static File mkdir(String s) throws IOException
{
String dirname = s;
s = Integer.toHexString(dirname.hashCode());
File directory = new File(s);
if (!directory.exists() && !directory.mkdir())
throw new IOException("can't make directory for " + s);
return directory;
}
public static byte[] saveResource(File dir, String urlString,
String argURLString)
throws IOException, URISyntaxException
{
URL u = new URL(urlString);
URLConnection uc = u.openConnection();
urlString = uc.getContentType();
int contentLength = uc.getContentLength();
try (InputStream raw = uc.getInputStream()) {
InputStream in = new BufferedInputStream(raw);
byte[] data = new byte[contentLength];
int offset = 0;
while (offset < contentLength) {
int bytesRead = in.read(data, offset, data.length - offset);
if (bytesRead == -1) break;
offset += bytesRead;
}
if (offset != contentLength) {
throw new IOException("Only read " + offset
+ " bytes; Expected " + contentLength + " bytes");
}
urlString = u.getFile();
urlString = urlString.substring(urlString.lastIndexOf('/') + 1);
try (FileOutputStream fout = new FileOutputStream(dir)) {
fout.write(data);
fout.flush();
}
return data;
}
}
The error is simple and coming because you can't write data in directory.
Try to print dir.isDirectory() to confirm if it's directory. Since its part of argument, check the caller method.
Related
I have a task that
read a zip file from local into binary message
transfer binary message through EMS as String (done by java API)
receive transferred binary message as String (done by java API)
decompress the binary message and then print it out
The problem I am facing is DataFormatException while decompress the message.
I have no idea which part went wrong.
I use this to read file into binary message:
static String readFile_Stream(String fileName) throws IOException {
File file = new File(fileName);
byte[] fileData = new byte[(int) file.length()];
FileInputStream in = new FileInputStream(file);
in.read(fileData);
String content = "";
System.out.print("Sent message: ");
for(byte b : fileData)
{
System.out.print(getBits(b));
content += getBits(b);
}
in.close();
return content;
}
static String getBits(byte b)
{
String result = "";
for(int i = 0; i < 8; i++)
result = ((b & (1 << i)) == 0 ? "0" : "1") + result;
return result;
}
I use this to decompress message:
private static byte[] toByteArray(String input)
{
byte[] byteArray = new byte[input.length()/8];
for (int i=0;i<input.length()/8;i++)
{
String read_data = input.substring(i*8, i*8+8);
short a = Short.parseShort(read_data, 2);
byteArray[i] = (byte) a;
}
return byteArray;
}
public static byte[] unzipByteArray(byte[] file) throws IOException {
byte[] byReturn = null;
Inflater oInflate = new Inflater(false);
oInflate.setInput(file);
ByteArrayOutputStream oZipStream = new ByteArrayOutputStream();
try {
while (! oInflate.finished() ){
byte[] byRead = new byte[4 * 1024];
int iBytesRead = oInflate.inflate(byRead);
if (iBytesRead == byRead.length){
oZipStream.write(byRead);
}
else {
oZipStream.write(byRead, 0, iBytesRead);
}
}
byReturn = oZipStream.toByteArray();
}
catch (DataFormatException ex){
throw new IOException("Attempting to unzip file that is not zipped.");
}
finally {
oZipStream.close();
}
return byReturn;
}
The message I got is
java.io.IOException: Attempting to unzip file that is not zipped.
at com.sourcefreak.example.test.TibcoEMSQueueReceiver.unzipByteArray(TibcoEMSQueueReceiver.java:144)
at com.sourcefreak.example.test.TibcoEMSQueueReceiver.main(TibcoEMSQueueReceiver.java:54)
After check, the binary message does not corrupted after transmission.
Please help to figure out the problem.
Have you tried using InflaterInputStream? Based on my experience, using Inflater directly is rather tricky. You can use this to get started:
public static byte[] unzipByteArray(byte[] file) throws IOException {
InflaterInputStream iis = new InflaterInputStream(new ByteArrayInputStream(file));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[512];
int length = 0;
while ((length = iis.read(buffer, 0, buffer.length) != 0) {
baos.write(buffer, 0, length);
}
iis.close();
baos.close();
return baos.toByteArray();
}
I finally figure out the problem.
The problem is the original file is a .zip file, so I should use zipInputStream to unzip the file before further processing.
public static byte[] unzipByteArray(byte[] file) throws IOException {
// create a buffer to improve copy performance later.
byte[] buffer = new byte[2048];
byte[] content ;
// open the zip file stream
InputStream theFile = new ByteArrayInputStream(file);
ZipInputStream stream = new ZipInputStream(theFile);
ByteArrayOutputStream output = new ByteArrayOutputStream();
try
{
ZipEntry entry;
while((entry = stream.getNextEntry())!=null)
{
//String s = String.format("Entry: %s len %d added %TD", entry.getName(), entry.getSize(), new Date(entry.getTime()));
//System.out.println(s);
// Once we get the entry from the stream, the stream is
// positioned read to read the raw data, and we keep
// reading until read returns 0 or less.
//String outpath = outdir + "/" + entry.getName();
try
{
//output = new FileOutputStream(outpath);
int len = 0;
while ((len = stream.read(buffer)) > 0)
{
output.write(buffer, 0, len);
}
}
finally
{
// we must always close the output file
if(output!=null) output.close();
}
}
}
finally
{
// we must always close the zip file.
stream.close();
}
content = output.toByteArray();
return content;
}
This code work for zip file containing single file inside.
I have a problem regarding on uploading a file. When I upload the files myself(localhost) it actually works but when I let other people in the same network upload their file it gives me an error:
(The system cannot find the path specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at sources.UploadAIP.actions.uploadAction.processRequest(uploadAction.java:49)
Here is my actual code:
public class uploadAction extends AbstractAppAction
{
public boolean processRequest(HttpServlet servlet, HttpServletRequest request, HttpServletResponse response)
{
try{
Properties appProperties = getAppProperties();
String dbMap = appProperties.getProperty("dbMap");
DB db = getDBConnection(dbMap);
String myDirectory = "C:\\tmp";
String uploadedFile = request.getParameter("filename");
System.out.println("srcfile: " +myDirectory);
System.out.println("file: " +uploadedFile);
String errorMessage = "";
ServletContext sc = servlet.getServletContext();
String fileName = StringUtil.stringReplace(uploadedFile,"\\","\\");
int i = fileName.lastIndexOf("\\");
if (i > 0) {
fileName = fileName.substring(i+1);
}
File srcFile = new File(uploadedFile);
File targetDirectory = new File(myDirectory);
String dirname = StringUtil.stringReplace(targetDirectory.toString() ,"\\","\\");
System.out.println("directory name:" +dirname);
File destFile = new File(dirname+"\\"+fileName);
System.out.println(destFile);
System.out.println("here is the parent directory: " +targetDirectory);
if(!targetDirectory.exists()){
targetDirectory.mkdirs();
}
InputStream inStream;
OutputStream outStream;
try{
inStream = new FileInputStream(srcFile);
outStream = new FileOutputStream(destFile);
byte[] buffer = new byte[4096];
int length;
//copy the file content in bytes
while ((length = inStream.read(buffer)) > 0){
outStream.write(buffer, 0, length);
}
outStream.close();
}catch(Exception e){
e.printStackTrace();
}
fileName = StringUtil.stringReplace(uploadedFile, "\\", "\\");
int u = fileName.lastIndexOf("\\");
if (u > 0)
{
fileName = fileName.substring(i + 1);
}
if (!dirname.endsWith("\\"))
{
dirname = dirname + "\\";
}
File f = new File(dirname);
String uploadDir = dirname;
System.out.println("uploadDirectory" +uploadDir);
} catch (Exception ex) {
request.setAttribute("message", ex.toString());
ex.printStackTrace();
}
return (true);
}
}
Your code assumes that the file being uploaded is present on the same local machine, which isnt true, since you are receiving uploads from a local network.
This is why it works on your local machine, but not across a network.
To upload a file, you need a multipart form and a servlet that correctly handles a multipart request.
This tutorial should help you:
http://www.avajava.com/tutorials/lessons/how-do-i-upload-a-file-to-a-servlet.html
When I try to write to the file specified it comes up with the error below. I have tried closing the FileInputStream but I still come up with the same problem.
Here is the relevant code:
Error log:
Error: C:\Path\Hours Log.csv (The requested operation cannot be performed on a file with a user-mapped section open)
Code:
Creating the log:
private void writeLog() throws IOException{
//set up vars and write directories
File yearStatDir = new File("C:\\Path);
File yearStatPath = new File(yearStatDir + "\\" + "Hours Log.csv");
String yearStatString = yearStatPath.toString();
//read the files
String existingYearLog = readLogFile(yearStatString, yearStatPath);
//write the updated file
String hoursString = "1";
String dataYear = existingYearLog + hoursString;
String folderYear = "Satistics\\Yearly data\\" + yearString;
writeFile(dataYear, ".csv", folderYear, "Hours Log");
}
Writing the file:
private void writeFile(String data, String fileType, String folder, String fileName){
try{
File fileDir = new File("C:\\Path\\" + folder);
File filePath = new File(fileDir + "\\"+ fileName + fileType);
writeDir(fileDir);
// Create file
FileWriter fstream = new FileWriter(filePath);
try (BufferedWriter out = new BufferedWriter(fstream)) {
out.write(data);
}
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
}
Reading the file:
private static String readLogFile(String path, File f) throws IOException {
if (f.exists()){
try (FileInputStream stream = new FileInputStream(new File(path))) {
FileChannel fc = stream.getChannel();
MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
/* Instead of using default, pass in a decoder. */
fc.close();
return Charset.defaultCharset().decode(bb).toString();
}
}
else {
return "";
}
}
For anyone that comes across this, here is the alternative code that I am using now:
private static String readLogFile(String path) throws IOException {
File f = new File(path);
if(f.exists()) {
FileInputStream fis = new FileInputStream(f);
Integer fileLength = (int) (long) f.length();
byte[] b = new byte[fileLength];
int read = 0;
while (read < b.length) {
read += fis.read(b, read, b.length - read);
}
String text = new String(b);
return text;
} else {
String text = "";
return text;
}
}
I'm trying to upload a file to my Spring server running on Tomcat7. It's a simple POST request, the code is below:
#RequestMapping(method = RequestMethod.POST)
public void saveFile(HttpServletRequest request, #RequestParam("file_name") String fileName) {
Logger.getLogger(FileRestAction.class).info("saving file with name " + fileName);
try {
byte[] buf = readFromRequest(request);
String filePath = writeToFile(buf, fileName);
File_ file = new File_(filePath, request.getContentType());
Logger.getLogger(FileRestAction.class).info(request.getContentType() + " " + request.getContentLength());
fService.save(file);
} catch (IOException e) {
Logger.getLogger(FileRestAction.class).error("Failed to upload file. " +
"Exception is: " + e.getMessage());
}
}
private String writeToFile(byte[] buf, String fileName) throws IOException {
String fileBasePath = ConfigurationProvider.getConfig().getString(Const.FILE_SAVE_PATH);
File file = new File(fileBasePath + fileName);
FileOutputStream fos = new FileOutputStream(file);
fos.write(buf);
fos.close();
Logger.getLogger(FileRestAction.class).info("filepath: " + file.getAbsolutePath());
return file.getAbsolutePath();
}
private byte[] readFromRequest(HttpServletRequest request) throws IOException {
InputStream is = request.getInputStream();
byte[] buf = new byte[request.getContentLength()];
is.read(buf);
is.close();
return buf;
}
Now the problem is that the file on the server is only "half-done", it's as if all the bytes aren't there. For example, if I send a 256x256 .png-file with a size of 54kB, the file written on the server is also 54kB and 256x256 in size, but the actual picture cuts off near the beginning (the rest is blank). No exceptions are thrown.
After a bit of testing I've found out that the cutoff is around 15-20Kb (images below that are fully uploaded).
Any ideas as to what might cause this?
EDIT: I changed the readFromRequest method according to what GreyBeardedGeek suggested. It's now as follows:
private byte[] readFromRequest(HttpServletRequest request) throws IOException {
InputStream is = request.getInputStream();
int fileLength = (int) request.getContentLength();
byte[] buf = new byte[fileLength];
int bytesRead = 0;
while (true) {
bytesRead += is.read(buf, bytesRead, fileLength - bytesRead);
Logger.getLogger(FileRestAction.class).info("reading file: " + bytesRead + " bytes read");
if (bytesRead == fileLength) break;
}
is.close();
return buf;
}
InputStream.read is not guaranteed to read the amount of data that you ask for.
The size that you ask for is the maximum number of bytes that it will read.
That's why the read() method returns the number of bytes actually read.
See http://docs.oracle.com/javase/6/docs/api/java/io/InputStream.html#read(byte[])
So, the answer is to read multiple times, until read() returns -1
URL url = new URL("http://download.thinkbroadband.com/20MB.zip");
URLConnection connection = url.openConnection();
File fileThatExists = new File(path);
OutputStream output = new FileOutputStream(path, true);
connection.setRequestProperty("Range", "bytes=" + fileThatExists.length() + "-");
connection.connect();
int lenghtOfFile = connection.getContentLength();
InputStream input = new BufferedInputStream(url.openStream());
byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1) {
total += count;
output.write(data, 0 , count);
}
in this code I try to resume download. Target file is 20MB. But when I stop download on 10mb, then contunue, I get file with filesize 30MB. It seems that it continue writing to file, but cant partly download from server. Wget -c works great with this file. How can I resume file download?
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if(ISSUE_DOWNLOAD_STATUS.intValue()==ECMConstant.ECM_DOWNLOADING){
File file=new File(DESTINATION_PATH);
if(file.exists()){
downloaded = (int) file.length();
connection.setRequestProperty("Range", "bytes="+(file.length())+"-");
}
}else{
connection.setRequestProperty("Range", "bytes=" + downloaded + "-");
}
connection.setDoInput(true);
connection.setDoOutput(true);
progressBar.setMax(connection.getContentLength());
in = new BufferedInputStream(connection.getInputStream());
fos=(downloaded==0)? new FileOutputStream(DESTINATION_PATH): new FileOutputStream(DESTINATION_PATH,true);
bout = new BufferedOutputStream(fos, 1024);
byte[] data = new byte[1024];
int x = 0;
while ((x = in.read(data, 0, 1024)) >= 0) {
bout.write(data, 0, x);
downloaded += x;
progressBar.setProgress(downloaded);
}
This is not my code, but it works.
I guess the problem you are facing is calling url.openStream() after url.openConnection().
url.openStream() is equivalent to url.openConnection().getInputStream(). Hence, you are requesting the url twice. Particularly the second time, it is not specifying the range property. Therefore download always starts at the beginning.
You should replace url.openStream() with connection.getInputStream().
This is what I am using to download the file in chunk Updating the UI with progress.
/*
* #param callback = To update the UI with appropriate action
* #param fileName = Name of the file by which downloaded file will be saved.
* #param downloadURL = File downloading URL
* #param filePath = Path where file will be saved
* #param object = Any object you want in return after download is completed to do certain operations like insert in DB or show toast
*/
public void startDownload(final IDownloadCallback callback, String fileName, String downloadURL, String filePath, Object object) {
callback.onPreExecute(); // Callback to tell that the downloading is going to start
int count = 0;
File outputFile = null; // Path where file will be downloaded
try {
File file = new File(filePath);
file.mkdirs();
long range = 0;
outputFile = new File(file, fileName);
/**
* Check whether the file exists or not
* If file doesn't exists then create the new file and range will be zero.
* But if file exists then get the length of file which will be the starting range,
* from where the file will be downloaded
*/
if (!outputFile.exists()) {
outputFile.createNewFile();
range = 0;
} else {
range = outputFile.length();
}
//Open the Connection
URL url = new URL(downloadURL);
URLConnection con = url.openConnection();
// Set the range parameter in header and give the range from where you want to start the downloading
con.setRequestProperty("Range", "bytes=" + range + "-");
/**
* The total length of file will be the total content length given by the server + range.
* Example: Suppose you have a file whose size is 1MB and you had already downloaded 500KB of it.
* Then you will pass in Header as "Range":"bytes=500000".
* Now the con.getContentLength() will be 500KB and range will be 500KB.
* So by adding the two you will get the total length of file which will be 1 MB
*/
final long lenghtOfFile = (int) con.getContentLength() + range;
FileOutputStream fileOutputStream = new FileOutputStream(outputFile, true);
InputStream inputStream = con.getInputStream();
byte[] buffer = new byte[1024];
long total = range;
/**
* Download the save the content into file
*/
while ((count = inputStream.read(buffer)) != -1) {
total += count;
int progress = (int) (total * 100 / lenghtOfFile);
EntityDownloadProgress entityDownloadProgress = new EntityDownloadProgress();
entityDownloadProgress.setProgress(progress);
entityDownloadProgress.setDownloadedSize(total);
entityDownloadProgress.setFileSize(lenghtOfFile);
callback.showProgress(entityDownloadProgress);
fileOutputStream.write(buffer, 0, count);
}
//Close the outputstream
fileOutputStream.close();
// Disconnect the Connection
if (con instanceof HttpsURLConnection) {
((HttpsURLConnection) con).disconnect();
} else if (con instanceof HttpURLConnection) {
((HttpURLConnection) con).disconnect();
}
inputStream.close();
/**
* If file size is equal then return callback as success with downlaoded filepath and the object
* else return failure
*/
if (lenghtOfFile == outputFile.length()) {
callback.onSuccess(outputFile.getAbsolutePath(), object);
} else {
callback.onFailure(object);
}
} catch (Exception e) {
e.printStackTrace();
callback.onFailure(object);
}
}
interface IDownloadCallback {
void onPreExecute(); // Callback to tell that the downloading is going to start
void onFailure(Object o); // Failed to download file
void onSuccess(String path, Object o); // Downloaded file successfully with downloaded path
void showProgress(EntityDownloadProgress entityDownloadProgress); // Show progress
}
public class EntityDownloadProgress {
int progress; // range from 1-100
long fileSize;// Total size of file to be downlaoded
long downloadedSize; // Size of the downlaoded file
public void setProgress(int progress) {this.progress = progress;}
public void setFileSize(long fileSize) {this.fileSize = fileSize;}
public void setDownloadedSize(long downloadedSize) {this.downloadedSize = downloadedSize;}
}
Check out this thread which has a problem similar to yours. If wget is working, then the server clearly supports resuming downloads. It looks like you're not setting the If-Range header as mentioned in the accepted answer of the above link. ie. add:
// Initial download.
String lastModified = connection.getHeaderField("Last-Modified");
// ...
// Resume download.
connection.setRequestProperty("If-Range", lastModified);
How about this?
public static void download(DownloadObject object) throws IOException{
String downloadUrl = object.getDownloadUrl();
String downloadPath = object.getDownloadPath();
long downloadedLength = 0;
File file = new File(downloadPath);
URL url = new URL(downloadUrl);
BufferedInputStream inputStream = null;
BufferedOutputStream outputStream = null;
URLConnection connection = url.openConnection();
if(file.exists()){
downloadedLength = file.length();
connection.setRequestProperty("Range", "bytes=" + downloadedLength + "-");
outputStream = new BufferedOutputStream(new FileOutputStream(file, true));
}else{
outputStream = new BufferedOutputStream(new FileOutputStream(file));
}
connection.connect();
inputStream = new BufferedInputStream(connection.getInputStream());
byte[] buffer = new byte[1024*8];
int byteCount;
while ((byteCount = inputStream.read(buffer)) != -1){
outputStream.write(buffer, 0, byteCount);
break;
}
inputStream.close();
outputStream.flush();
outputStream.close();
}
Used break; to test the code.. ;)
I have a way for your code to work.
First, check if the file exits or not
If the file exists, set the connection:
connection.setRequestProperty("Range", "bytes=" + bytedownloaded + "-");
If file does not exist, do the same download in a new file.
Since the question is tagged with Android:
Have you tried using DownloadManager.
It handles all this stuff nicely for you.