Attempting to overwrite files results in blank files - java

In the app I am working on right now, part of the functionality is to write data saved on the device to a flash drive connected via a USB-OTG adapter. Specifically, the device is a rooted Motorola Xoom running 4.2.2. I can successfully write files to the drive and read them on my computer. That part works fine. However, when I try to replace existing files with new information, the resulting files come out empty. I even delete the existing files before writing new data. What's weird is that after copying the contents of my internal file to the flash drive, I log the length of the resulting file. It always matches the input file and is always a non-0 number, yet the file still shows up as blank on my computer. Can anyone help with this problem? Relevant code from the AsyncTask that I have doing this work is below.
#Override
protected Void doInBackground(Void... params) {
File[] files = context.getFilesDir().listFiles();
for (File file : files) {
if (file.isFile()) {
List<String> nameSegments = Arrays.asList(file.getName().split(
"_"));
Log.d("source file", "size: " + file.length());
String destinationPath = "/storage/usbdisk0/"
+ nameSegments.get(0) + "/" + nameSegments.get(1) + "/";
File destinationPathFile = new File(destinationPath);
if (!destinationPathFile.mkdirs()) {
destinationPathFile.mkdirs();
}
File destinationFile = new File(destinationPathFile,
nameSegments.get(2));
FileReader fr = null;
FileWriter fw = null;
try {
fr = new FileReader(file);
fw = new FileWriter(destinationFile, false);
int c = fr.read();
while (c != -1) {
fw.write(c);
c = fr.read();
}
fw.flush();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
fr.close();
fw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Log.d("destination file", "size: " + new File(destinationFile.getPath()).length());
}
}
return null;
}
EDIT:
Per #Simon's suggestion, I added output.flush() to my code. This does not change the result.
EDIT #2:
I did some further testing with this and found something interesting. If I go to Settings->Storage->Unmount USB Storage after writing to the flash drive but before removing it from the OTG adapter, everything works perfectly. However, failing to eject the drive after writing results in the data not being written. What's strange is that the folder structure and file itself are created on the drive, but the file is always empty. One more thing: if I go to a file manager application and open up the file prior to removing the drive, the files all exist as they should. However, even removing the device, plugging it straight back in to the tablet and opening any of the files results in the file looking empty. I can't make heads or tails of this, and this is incredibly frustrating. Can anyone help with this?
EDIT #3:
I also changed to using FileReaders and FileWriters just to wee what would happen. I don't care about efficiency at this point, I simply want file writing to work reliably. This change did not affect the issue. Updated code is posted above.

Try using FileReader.ready() method before your FileReader.read() call,
and ensure if your FileReader really has some bytes in it.

Try this , Used buffered reader for writing
try
{
fw = new FileWriter(destinationFile);
BufferedWriter writer=new BufferedWriter(fw);
writer.append(yourText); // Append can be changed to write or something if you want to overwrite
writer.close();
}
catch (Exception e) {
throw new RuntimeException(e);
}
finally {
if (fw != null) {
try {
fw.flush();
fw.close();
}
catch (IOException e) {
}

I found the solution to my problem. It appears that the Android system buffers some files off of the SD card/flash drive, and then writes them to the flash drive upon eject. The following code after my file operations synchronizes the buffer with the filesystem and allows the flash drive to be immediately removed from the device without data loss. It's worth noting that this DOES require root access; it will not work on a non-rooted device.
try {
Process p = Runtime.getRuntime().exec("su");
DataOutputStream os = new DataOutputStream(p.getOutputStream());
os.writeBytes("sync; sync\n");
os.writeBytes("exit\n");
os.flush();
} catch (Exception e) {
e.printStackTrace();
}
Source of my solution: Android 2.1 programatically unmount SDCard

It sounds like the filesystem is caching your changes, but not actually writing them to the flash drive until you eject it. I don't think there's a way to flush the filesystem cache, so the best solution seems to be just to unmount and then remount the flash drive.

Related

File.delete() & File.renameTo() Not Working in Project Environment

I am trying to create an authentication system of sorts that uses a file called Users.dat to store user data. Currently, I am developing a method to remove users by rewriting the Users.dat file, omitting the user specified. The code below works in a basic environment with an all-encompassing directory containing the .java files and the Users.dat file in the same spot. The old Users.dat file is deleted and Users.dat.tmp is renamed to User.dat. (No problems here, everything works as intended).
public static boolean RemoveUser(String userName) {
// TODO remove username from Users.dat
try {
File originalFile = new File("Users.dat");
System.out.println(originalFile.getAbsolutePath());
BufferedReader read = new BufferedReader(new FileReader("Users.dat"));
String line = null;
while ((line = read.readLine()) != null) {
if (line.indexOf(userName) != -1) {
break;
}
}
String[] userInfo = line.split(", ");
if (!userName.equals(userInfo[2])) {
System.out.println("Username not found. No users removed.");
read.close();
return false;
}
File tempFile = new File(originalFile.getAbsolutePath() + ".tmp");
PrintWriter print = new PrintWriter(new FileWriter(tempFile));
String lineToRemove = line;
BufferedReader read2 = new BufferedReader(new FileReader("Users.dat"));
while ((line = read2.readLine()) != null) {
if (!line.trim().equals(lineToRemove)) {
print.println(line);
print.flush();
}
}
print.close();
read.close();
read2.close();
System.out.println(originalFile.getAbsolutePath());
originalFile.delete(); //This line is not executing correctly
tempFile.renameTo(originalFile); //Nor is this line
} catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
return true;
}
Users.dat file format:
Joe, Last, jlast, 58c536ed8facc2c2a293a18a48e3e120, true
Sam, sone, samsone, 2c2a293a18a48e3e12058c536ed8facc, false
Jane, Best, jbest, 293a18a48e3e12052058c536ed8facc2c, false
Andrew, Estes, Aestes, 63a490d69aa544fd1272a976014ad570, true
Test, User, tuser, 63a490d69aa544fd1272a976014ad570, true
I have two System.out.println(originalFile.getAbsolutePath()) statements, one at the beginning, one at the end to make sure the path isn't getting screwed up in the process of everything somehow.
Like I said, the code works, however, when I try to implement it in my project, it creates the Users.dat.tmp and it writes the correct data to it, but it does not delete the old Users.dat file, nor does it rename the Users.dat.tmp file to replace Users.dat. I'm certain the directory is correct, as I am literally displaying it as the code executes. I can't figure out any other reason why originalFile.delete() and tempFile.renameTo(originalFile) aren't functioning properly.
EDIT:
Using java.nio.file, I was able to produce an error message. it reads:
java.nio.file.FileSystemException: C:\Path\Users.dat: The process cannot access the file because it is being used by another process.
I don't have the file open when this error message is shown, and I don't get this error using java.nio in my testing environment mentioned at the beginning. I'm not sure what other process the message is referring to.
EDIT 2:
I tried running the code on other machines, one a Mac, the other a Windows laptop, and the code functioned on the Mac just fine, but I was still seeing the same issue on the Windows laptop.
I had the similar issue. My problem was not closing all the streams I read and written to the file. Thanks for your Edit #1, that was helpful
When you wrap
BufferedReader read = new BufferedReader(new FileReader("Users.dat"));
don't you need to close the inner readers too?
If not for the author, but for those who stambled upon this question (like me), hope this suggestion will be useful
I had an earlier function that I was calling in main that was accessing Users.dat, but I never closed the BufferredReader in that function.

Android Write file to a different application folder

My device is rooted
The target audience (if app is shared) will be rooted devices only
I'm trying to customize another app slightly by modifying some image files. So objective is to simply overwrite tiny jpg files (one at a time as required)
/data/data/com.otherapp/files/somefile.jpg for example needs to be overwritten
p = Runtime.getRuntime().exec("su"); is already run before I tried the write method.
I also added Runtime.getRuntime().exec("chmod 077 " +myFile.getAbsolutePath()); before the write code as suggested on a similar question
I'm getting permission denied.
java.io.FileNotFoundException: /data/data/com......jpg: open failed: EACCES (Permission denied)
The write code I have is:
//myDir is /data/data/com.......
File myFile = new File (myDir.getAbsolutePath(), fname);
try {
//myFile.createNewFile();//tried with and without this
FileOutputStream out = new FileOutputStream(myFile);
btbmp.compress(Bitmap.CompressFormat.JPEG, 60, out);
out.flush();
out.close();
} catch (Exception e) {
e.printStackTrace();
}
How can I do this?
Okay it's taken all day but I've achieved the desired result thus:
Create file on SD card
Then copy file to root destination
src_file = "somefile/on/sdcard.jpg"
dest_file = "/data/data/com.anotherapp/files/abcde.jpg" got path using context
try {
Process process = Runtime.getRuntime().exec("su");
DataOutputStream os = new DataOutputStream(process.getOutputStream());
os.writeBytes("rm "+dest_file + "\n"); //removes destination file -might not be required - haven't tested
os.flush();
os.writeBytes("cat "+src_file+" > "+dest_file + "\n");
os.flush();
os.writeBytes("exit\n");
os.flush();
// Waits for the command to finish.
process.waitFor();
}
catch (IOException e) {
e.printStackTrace();
}
catch (InterruptedException e) {
e.printStackTrace();
}
Note In some cases I had to chmod the root folder 777 for it to work. I'm sure this isn't good practice and doing so will mean the app that uses that folder might not be able to access it. In my case that's what happened until I rectified it using ES Explorer

Read/Write File in blackberry

I am writing to a file using this code.
protected void writeFile(String text) {
DataOutputStream os = null;
FileConnection fconn = null;
try {
fconn = (FileConnection) Connector.open("file:///store/home/user/documents/file.txt", Connector.READ_WRITE);
if (!fconn.exists())
fconn.create();
os = fconn.openDataOutputStream();
os.write(text.getBytes());
} catch (IOException e) {
System.out.println(e.getMessage());
} finally {
try {
if (null != os)
os.close();
if (null != fconn)
fconn.close();
} catch (IOException e) {
System.out.println(e.getMessage());
}
}}
the code is working fine.
My problem is if I write first time "Banglore" and when I read it, I get "Banglore".
But, second time when I write "India" and when I read it, I get, "Indialore".
so, basically its content is not changing according the text , I am giving.
Please tell me how to fix this.
writing in a file doesn't remove the content but it just replaces the content, so writing 'india' over 'Bangalore' will replace the 'Banga' with 'India' and the rest would remain the same. If you want to completely remove old content with newer one, you need to truncate()
the file from where the newer data ends. truncate(text.getBytes().length)

Convenient place to put a change log

I am an old school C programmer and I have the habbit of putting my change log in comments at the top of my main source file.
I would like to put my change log in a more convenient place so that I could pull it up at run time and display it to the user on request.
Any suggestions ?
Most projects keep the change log in a file called Changelog in the project root.
Often this file is created manually: Developers are often ... "creative" when committing things and/or comments like "Reduced global variables", "Organized imports", etc. don't make much sense to the user.
In your case, I suggest to put the file next to the class which displays it or in the root of your (re-)source folder. That way, you can easily load it using About.class.getResourceAsStream("Changelog") (relative to About) or getClass().getClassLoader().getResourceAsStream("Changelog") (relative to source root folder)
My suggestion would be to put it in an html file on the filesystem under assets. Then in the Activity in which you want to display the change log, just use the following web view code:
WebView changeLogWebView = (WebView) findViewById(R.id.ChangeLogWebView);
InputStream fin = null;
try {
fin = getAssets().open("changelog.html");
byte[] buffer = new byte[fin.available()];
fin.read(buffer);
String contents = new String(buffer, "UTF-8");
changeLogWebView.loadData(contents, "text/html", "UTF-8");
} catch (IOException e) {
throw new RuntimeException(e);
} finally {
if (fin != null) {
try {
fin.close();
} catch (IOException e) {
}
}
}
Check this out: http://code.google.com/p/android-change-log/
You can simply add a folder "raw" in the resource folder and store your own files there - also text files. A file "res/raw/changelog.txt" gets an identifier R.raw.changelog in your Java code, you then can open and read the file using getResources().openRawResource(). :-)
http://developer.android.com/reference/android/content/res/Resources.html#openRawResource%28int%29
Depending on how your application is structured, you could put an "About" menu tab or something similar and have a changelog button that brings up your change log when it is clicked. Then you could just read the local changelog and copy over it whenever you make changes to keep it updated. Maybe something like that?
Here is a piece of code that combines mreichelt's and Micah's answers. Thanks
Text in a raw resource dispayed in a WebView.
WebView computerWebView = (WebView) findViewById(R.id.changelog_webview);
InputStream istream = null;
String strLine;
StringBuilder strbText = new StringBuilder();
try
{
istream = getResources().openRawResource(R.raw.changelog_html);
InputStreamReader isreader = new InputStreamReader(istream);
BufferedReader myReader = new BufferedReader(isreader);
while ((strLine = myReader.readLine()) != null)
{
strbText.append(strLine);
}
computerWebView.loadData(strbText.toString(), "text/html", "UTF-8");
}
catch (Exception e)
{
computerWebView
.loadData("No data to display", "text/html", "UTF-8");
}
finally
{
if (istream != null)
try
{
istream.close();
}
catch (IOException e)
{
}
}

file.delete() returns false even though file.exists(), file.canRead(), file.canWrite(), file.canExecute() all return true

I'm trying to delete a file, after writing something in it, with FileOutputStream. This is the code I use for writing:
private void writeContent(File file, String fileContent) {
FileOutputStream to;
try {
to = new FileOutputStream(file);
to.write(fileContent.getBytes());
to.flush();
to.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
As it is seen, I flush and close the stream, but when I try to delete, file.delete() returns false.
I checked before deletion to see if the file exists, and: file.exists(), file.canRead(), file.canWrite(), file.canExecute() all return true. Just after calling these methods I try file.delete() and returns false.
Is there anything I've done wrong?
Another bug in Java. I seldom find them, only my second in my 10 year career. This is my solution, as others have mentioned. I have nether used System.gc(). But here, in my case, it is absolutely crucial. Weird? YES!
finally
{
try
{
in.close();
in = null;
out.flush();
out.close();
out = null;
System.gc();
}
catch (IOException e)
{
logger.error(e.getMessage());
e.printStackTrace();
}
}
It was pretty odd the trick that worked. The thing is when I have previously read the content of the file, I used BufferedReader. After reading, I closed the buffer.
Meanwhile I switched and now I'm reading the content using FileInputStream. Also after finishing reading I close the stream. And now it's working.
The problem is I don't have the explanation for this.
I don't know BufferedReader and FileOutputStream to be incompatible.
I tried this simple thing and it seems to be working.
file.setWritable(true);
file.delete();
It works for me.
If this does not work try to run your Java application with sudo if on linux and as administrator when on windows. Just to make sure Java has rights to change the file properties.
Before trying to delete/rename any file, you must ensure that all the readers or writers (for ex: BufferedReader/InputStreamReader/BufferedWriter) are properly closed.
When you try to read/write your data from/to a file, the file is held by the process and not released until the program execution completes. If you want to perform the delete/rename operations before the program ends, then you must use the close() method that comes with the java.io.* classes.
As Jon Skeet commented, you should close your file in the finally {...} block, to ensure that it's always closed. And, instead of swallowing the exceptions with the e.printStackTrace, simply don't catch and add the exception to the method signature. If you can't for any reason, at least do this:
catch(IOException ex) {
throw new RuntimeException("Error processing file XYZ", ex);
}
Now, question number #2:
What if you do this:
...
to.close();
System.out.println("Please delete the file and press <enter> afterwards!");
System.in.read();
...
Would you be able to delete the file?
Also, files are flushed when they're closed. I use IOUtils.closeQuietly(...), so I use the flush method to ensure that the contents of the file are there before I try to close it (IOUtils.closeQuietly doesn't throw exceptions). Something like this:
...
try {
...
to.flush();
} catch(IOException ex) {
throw new CannotProcessFileException("whatever", ex);
} finally {
IOUtils.closeQuietly(to);
}
So I know that the contents of the file are in there. As it usually matters to me that the contents of the file are written and not if the file could be closed or not, it really doesn't matter if the file was closed or not. In your case, as it matters, I would recommend closing the file yourself and treating any exceptions according.
There is no reason you should not be able to delete this file. I would look to see who has a hold on this file. In unix/linux, you can use the lsof utility to check which process has a lock on the file. In windows, you can use process explorer.
for lsof, it's as simple as saying:
lsof /path/and/name/of/the/file
for process explorer you can use the find menu and enter the file name to show you the handle which will point you to the process locking the file.
here is some code that does what I think you need to do:
FileOutputStream to;
try {
String file = "/tmp/will_delete.txt";
to = new FileOutputStream(file );
to.write(new String("blah blah").getBytes());
to.flush();
to.close();
File f = new File(file);
System.out.print(f.delete());
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
It works fine on OS X. I haven't tested it on windows but I suspect it should work on Windows too. I will also admit seeing some unexpected behavior on Windows w.r.t. file handling.
If you are working in Eclipse IDE, that could mean that you haven't close the file in the previous launch of the application. When I had the same error message at trying to delete a file, that was the reason. It seems, Eclipse IDE doesn't close all files after termination of an application.
Hopefully this will help. I came across similar problem where i couldn't delete my file after my java code made a copy of the content to the other folder. After extensive googling, i explicitly declared every single file operation related variables and called the close() method of each file operation object, and set them to NULL. Then, there is a function called System.gc(), which will clear up the file i/o mapping (i'm not sure, i just tell what is given on the web sites).
Here is my example code:
public void start() {
File f = new File(this.archivePath + "\\" + this.currentFile.getName());
this.Copy(this.currentFile, f);
if(!this.currentFile.canWrite()){
System.out.println("Write protected file " +
this.currentFile.getAbsolutePath());
return;
}
boolean ok = this.currentFile.delete();
if(ok == false){
System.out.println("Failed to remove " + this.currentFile.getAbsolutePath());
return;
}
}
private void Copy(File source, File dest) throws IOException {
FileInputStream fin;
FileOutputStream fout;
FileChannel cin = null, cout = null;
try {
fin = new FileInputStream(source);
cin = fin.getChannel();
fout = new FileOutputStream(dest);
cout = fout.getChannel();
long size = cin.size();
MappedByteBuffer buf = cin.map(FileChannel.MapMode.READ_ONLY, 0, size);
cout.write(buf);
buf.clear();
buf = null;
cin.close();
cin = null;
fin.close();
fin = null;
cout.close();
cout = null;
fout.close();
fout = null;
System.gc();
} catch (Exception e){
this.message = e.getMessage();
e.printStackTrace();
}
}
the answer is when you load the file, you need apply the "close" method, in any line of code, works to me
There was a problem once in ruby where files in windows needed an "fsync" to actually be able to turn around and re-read the file after writing it and closing it. Maybe this is a similar manifestation (and if so, I think a windows bug, really).
None of the solutions listed here worked in my situation. My solution was to use a while loop, attempting to delete the file, with a 5 second (configurable) limit for safety.
File f = new File("/path/to/file");
int limit = 20; //Only try for 5 seconds, for safety
while(!f.delete() && limit > 0){
synchronized(this){
try {
this.wait(250); //Wait for 250 milliseconds
} catch (InterruptedException e) {
e.printStackTrace();
}
}
limit--;
}
Using the above loop worked without having to do any manual garbage collecting or setting the stream to null, etc.
The problem could be that the file is still seen as opened and locked by a program; or maybe it is a component from your program that it had been opened in, so you have to ensure you use the dispose() method to solve that problem.
i.e. JFrame frame;
....
frame.dispose();
You have to close all of the streams or use try-with-resource block
static public String head(File file) throws FileNotFoundException, UnsupportedEncodingException, IOException
{
final String readLine;
try (FileInputStream fis = new FileInputStream(file);
InputStreamReader isr = new InputStreamReader(fis, "UTF-8");
LineNumberReader lnr = new LineNumberReader(isr))
{
readLine = lnr.readLine();
}
return readLine;
}
if file.delete() is sending false then in most of the cases your Bufferedreader handle will not be closed. Just close and it seems to work for me normally.
I had the same problem on Windows. I used to read the file in scala line by line with
Source.fromFile(path).getLines()
Now I read it as a whole with
import org.apache.commons.io.FileUtils._
// encoding is null for platform default
val content=readFileToString(new File(path),null.asInstanceOf[String])
which closes the file properly after reading and now
new File(path).delete
works.
FOR Eclipse/NetBeans
Restart your IDE and run your code again this is only trick work for me after one hour long struggle.
Here is my code:
File file = new File("file-path");
if(file.exists()){
if(file.delete()){
System.out.println("Delete");
}
else{
System.out.println("not delete");
}
}
Output:
Delete
Another corner case that this could happen: if you read/write a JAR file through a URL and later try to delete the same file within the same JVM session.
File f = new File("/tmp/foo.jar");
URL j = f.toURI().toURL();
URL u = new URL("jar:" + j + "!/META-INF/MANIFEST.MF");
URLConnection c = u.openConnection();
// open a Jar entry in auto-closing manner
try (InputStream i = c.getInputStream()) {
// just read some stuff; for demonstration purposes only
byte[] first16 = new byte[16];
i.read(first16);
System.out.println(new String(first16));
}
// ...
// i is now closed, so we should be good to delete the jar; but...
System.out.println(f.delete()); // says false!
Reason is that the internal JAR file handling logic of Java, tends to cache JarFile entries:
// inner class of `JarURLConnection` that wraps the actual stream returned by `getInputStream()`
class JarURLInputStream extends FilterInputStream {
JarURLInputStream(InputStream var2) {
super(var2);
}
public void close() throws IOException {
try {
super.close();
} finally {
// if `getUseCaches()` is set, `jarFile` won't get closed!
if (!JarURLConnection.this.getUseCaches()) {
JarURLConnection.this.jarFile.close();
}
}
}
}
And each JarFile (rather, the underlying ZipFile structure) would hold a handle to the file, right from the time of construction up until close() is invoked:
public ZipFile(File file, int mode, Charset charset) throws IOException {
// ...
jzfile = open(name, mode, file.lastModified(), usemmap);
// ...
}
// ...
private static native long open(String name, int mode, long lastModified,
boolean usemmap) throws IOException;
There's a good explanation on this NetBeans issue.
Apparently there are two ways to "fix" this:
You can disable the JAR file caching - for the current URLConnection, or for all future URLConnections (globally) in the current JVM session:
URL u = new URL("jar:" + j + "!/META-INF/MANIFEST.MF");
URLConnection c = u.openConnection();
// for only c
c.setUseCaches(false);
// globally; for some reason this method is not static,
// so we still need to access it through a URLConnection instance :(
c.setDefaultUseCaches(false);
[HACK WARNING!] You can manually purge the JarFile from the cache when you are done with it. The cache manager sun.net.www.protocol.jar.JarFileFactory is package-private, but some reflection magic can get the job done for you:
class JarBridge {
static void closeJar(URL url) throws Exception {
// JarFileFactory jarFactory = JarFileFactory.getInstance();
Class<?> jarFactoryClazz = Class.forName("sun.net.www.protocol.jar.JarFileFactory");
Method getInstance = jarFactoryClazz.getMethod("getInstance");
getInstance.setAccessible(true);
Object jarFactory = getInstance.invoke(jarFactoryClazz);
// JarFile jarFile = jarFactory.get(url);
Method get = jarFactoryClazz.getMethod("get", URL.class);
get.setAccessible(true);
Object jarFile = get.invoke(jarFactory, url);
// jarFactory.close(jarFile);
Method close = jarFactoryClazz.getMethod("close", JarFile.class);
close.setAccessible(true);
//noinspection JavaReflectionInvocation
close.invoke(jarFactory, jarFile);
// jarFile.close();
((JarFile) jarFile).close();
}
}
// and in your code:
// i is now closed, so we should be good to delete the jar
JarBridge.closeJar(j);
System.out.println(f.delete()); // says true, phew.
Please note: All this is based on Java 8 codebase (1.8.0_144); they may not work with other / later versions.

Categories