My java code receives stream data like twitter. I need to store the data e.g. 10000 records for each file. So, I need to recreate the file writer and buffered writer to create a new file then write data on it.
// global variables
String stat;
long counter = 0;
boolean first = true;
Date date;
SimpleDateFormat format;
String currentTime;
String fileName;
BufferedWriter bw = null;
FileWriter fw = null;
public static void main(String[] args) {
String dirToSave = args[0];
String fileIdentifier = args[1];
createFile(dirToSave, fileIdentifier);
StatusListener listener = new StatusListener() {
#Override
public void onStatus(Status status) {
stat = TwitterObjectFactory.getRawJSON(status);
try {
if(bw!=null){
bw.write(stat + "\n");
}
} catch (IOException ex) {
System.out.println(ex.getMessage());
}
counter++;
if (counter == 10000) {
createFile(dirToSave, fileIdentifier);
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException ex) {
System.out.println(ex.getMessage());
}
counter = 0;
}
}
};
TwitterStream twitterStream = new TwitterStreamFactory(confBuild.build()).getInstance();
twitterStream.addListener(listener);
// twitterStream.filter(filQuery);
}
public static void createFile(String path, String fileIdentifier) {
date = new Date();
format = new SimpleDateFormat("yyyyMMddHHmm");
currentTime = format.format(date);
fileName = path + "/" + fileIdentifier + currentTime + ".json";
// if there was buffer before, flush & close it first before creating new file
if (!first) {
try {
bw.flush();
bw.close();
fw.close();
} catch (IOException ex) {
Logger.getLogger(LocalFile_All_en.class
.getName()).log(Level.SEVERE, null, ex);
}
} else {
first = false;
}
// create a new file
try {
fw = new FileWriter(fileName);
bw = new BufferedWriter(fw);
} catch (IOException ex) {
Logger.getLogger(Stack.class
.getName()).log(Level.SEVERE, null, ex);
}
}
However, i always get error after some hours.
SEVERE: null
java.io.IOException: Stream closed
EDIT: The error message says that, these codes throw the error
if (counter == 10000) {
createFile(dirToSave, fileIdentifier);
...
and
bw.flush();
What is the problem of my code? or is there a better way to write stream data like this?
If this error comes every now and then and writing after this error is ok again i think it can happen that bw is closed and not yet reopened while onStatus() tries to write of flush it.
So bw can be be not null but closed. You need to synchronize the closing/opening somehow.
For example make this stuff in onStatus() like so that you do not just write directly to bw but with some callbacks that handle the close/reopen new file.
Update: assuming here that this twitterStream can call onStatus() without waiting previous call finished. The first call has just closed the stream and the second is right after that writing to. Rare, but will happen in a long period of time.
Update2: this applies also to the flush() part.
I added this also as a short comment already but people often tell to get rid of static and especially global statics in java argumenting that it will cause big problems later which are hard to resolve/debug. This might be good case of it.
Read also:
Why are static variables considered evil?
Volatile Vs Static in java
Latter has an example how to sychronize concurrent requests.
I am making a simple application in Java (Animal shelter), it allows you to add animals to the list, edit free places, send email when the free space ends and all the changes are saved when the program closes and loaded after it is turned on.
However, the list of animals (ArrayList) is saved in one file, a list of free places in the second, email address (string) in the third, etc. ...
I have tried to place objects that need to be saved in one collection (ArrayList ), but this file does not load correctly.
Any ideas how to save and read different types in a txt file?
Example of my import/export class:
void autoSave() {
try {
FileOutputStream fout = new FileOutputStream(path);
ObjectOutputStream oos = new ObjectOutputStream(fout);
oos.writeObject(AddAnimalScene.listOfAnimals);
oos.close();
FileOutputStream fout1 = new FileOutputStream(path1);
ObjectOutputStream oos1 = new ObjectOutputStream(fout1);
oos1.writeObject(EditFreeSpaceScene.places);
oos1.close();
} catch (IOException e) {
e.printStackTrace();
}
}
void autoLoad() {
try {
ObjectInputStream in = new ObjectInputStream(new FileInputStream(path));
ArrayList<Animal> LoadedAnimalList = (ArrayList<Animal>) in.readObject();
in.close();
AddAnimalScene.listOfAnimals = LoadedAnimalList;
ObjectInputStream in1 = new ObjectInputStream(new FileInputStream(path1));
Integer LoadedPlaces = (Integer) in1.readObject();
in1.close();
EditFreeSpaceScene.places = LoadedPlaces;
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
}
A tip: Always read the Javadoc, especially as a beginner. Might be challenging at first, but it will pay off later. In my opinion, consulting the official documentation is always the right way to clear any issues.
From the ObjectOutputStream Javadoc:
Only objects that support the java.io.Serializable interface can be written to streams.
Your Animal class should implement Serializable interface.
On the other hand, if you want to explicitly specify a field should not be serialized, you could use the transient modifier.
After a very bad attempt at doing my homework, I decided it would be faster to abandon everything and start from scratch. Well not everything ... I copied this part since it worked perfectly so I saw no need to modify it. While maybe not perfect, it worked.
However now, when I compile just to test it out, I get an unexpected error:
Input error: java.io.EOFException.
Notice that "Input error" is from my catch(IOException ioe).
The file (fileName) is completely empty. Nothing in it. What could cause this. Is there a way to tell the ObjectInputStream to do nothing if the file is empty ?
Also I tested this out with an empty file on my other "iteration", didn't have this problem. I even named my file the same.
public Repository (String fileName) throws FileNotFoundException,
IOException,
SecurityException,
ClassNotFoundException {
this.fileName = fileName;
this.clients = new ArrayList<Client> ();
FileInputStream fileIn = null;
ObjectInputStream in = null;
try {
fileIn = new FileInputStream(this.fileName);
in = new ObjectInputStream(fileIn);
this.clients = (ArrayList<Client>) in.readObject();
} catch (FileNotFoundException fnfe) {
System.out.println("File not found, error: " + fnfe);
} catch (IOException ioe) {
System.out.println("Input error: " + ioe);
} catch (ClassNotFoundException cnfe) {
System.out.println("Class not found, error: " + cnfe);
} catch (SecurityException se) {
System.out.println(
"You do not have permission to access this file, error: "
+ se);
} finally {
if (fileIn != null)
fileIn.close();
if (in != null)
in.close();
}
Surely before
in = new ObjectInputStream(fileIn);
this.clients = (ArrayList<Client>) in.readObject();
you want to check the file size via File.length().
I assume if it's empty then you'd want to return an empty array list. You can't do that via deserialising an empty file. After all, even an empty array list has a non-zero size (and would need to identify itself as an array list via serialised attributes)
The file (fileName) is completely empty. Nothing in it.
That's exactly the problem. You cannot read an object (or an array) from an empty file. It will not find any data and throw an End-of-file-Exception (EOFException).
Even an empty array - when serialized to a file - will produce some data because the object stream will write the type (ArrayList) and the size of the array (0) to the file. When you try to read it, it will expect to find this data.
I'm currently building an application where the user will generate data over time and, should he/she has an internet connection, transmit it to the web. However, if he doesn't have web access, I need to store this data in the phone until the user recovers his access, when I'll need to recover this data to be transmitted. However, I'm facing lots of troubles to do this, as per below.
Note: before anything, I'm using a local java-created file because I know no other way to save/restore this data on the device. If you happen to know any other way to store/access this data from within the device please feel free to comment here.
Just for reference,
phantoms is an ArrayList containing objects with the data I need to
store,
Arquivador is the class that I'm using to make my data persistent and to recover it,
Funcionario is the class with the data generated by the program (just a few strings and numbers)
I am able to write a file to the file system through the code below, on my Activity:
try {
arq = new Arquivador();
arq.addFirstObjectInFile(
openFileOutput("dados.jlog", MODE_WORLD_WRITEABLE),
phantoms.get(0));
phantoms.remove(phantoms.get(0));
for (Funcionario func : phantoms) {
arq.addObjectInFile(openFileOutput("dados.jlog", MODE_APPEND),
func);
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
}
Here is the code inside Arquivador that adds the data to a file:
public void addObjectInFile(FileOutputStream arquivo,
Object objetoAAdicionar) {
try {
ObjectOutputStream aoos = new ObjectOutputStream(arquivo);
aoos.writeObject(objetoAAdicionar);
aoos.close();
} catch (IOException ioe) {
Log.d(TAG_NAME, "Erro no Appendable OOS.");
}
}
public void addFirstObjectInFile(FileOutputStream arquivo,
Object objetoAAdicionar) {
try {
AppendableObjectOutputStream aoos = new AppendableObjectOutputStream(
arquivo);
aoos.writeObject(objetoAAdicionar);
aoos.close();
} catch (IOException ioe) {
Log.d(TAG_NAME, "Erro no Appendable OOS.");
}
}
You will notice that I'm adding data to persistence in 2 steps, the first Object and the rest of them. This was an idea I saw on this post, here in StackOverflow, to allow appending data to a Java generated file. I have no problem with this code, it works perfectly.
Later on, back on my Activity, the internet connection is detected and I try to recover the file saved on the disk:
phantoms = new ArrayList<Funcionario>();
Object obj = arq.readObjectFromFile(openFileInput("dados.jlog"));
Funcionario func = null;
if (obj instanceof Funcionario) {
func = (Funcionario) obj;
}
while (func != null) {
phantoms.add(func);
arq.removeObjectFromFile(openFileInput("dados.jlog"), func,
getApplicationContext());
func = (Funcionario) arq
.readObjectFromFile(openFileInput("dados.jlog"));
}
The original idea was to read 1 object at a time, then attempt to transmit it and, if successful, erase the object from the file (so it didn't get retransmitted). However, I was having too many error messages with this. Instead, I decided to load all the objects at once, one by one, to see where my problem was more clearly.
Back to the Arquivador class:
public Object readObjectFromFile(FileInputStream arquivo) {
Object retorno = null;
if (arquivo.equals(null)) {
Log.e(TAG_NAME, "FIS is null!");
}
ObjectInputStream ois = null;
try {
ois = new ObjectInputStream(arquivo);
retorno = ois.readObject();
} catch (IOException ioex) {
} catch (ClassNotFoundException e) {
} finally {
try {
if (ois != null) ois.close();
} catch (IOException e) {
}
}
return retorno;
}
public void removeObjectFromFile(FileInputStream arqPrincipal,
Object objetoARemover, Context contexto) {
try {
// Construct the new file that will later be renamed to the original
// filename.
ObjectOutputStream oos = new ObjectOutputStream(
contexto.openFileOutput("dados.jlog.temp",
contexto.MODE_APPEND));
ObjectInputStream ois = new ObjectInputStream(arqPrincipal);
Object obj = null;
// Read from the original file and write to the new
// unless content matches data to be removed.
try {
while ((obj = ois.readObject()) != null) {
if (!(objetoARemover.equals(obj))) {
oos.writeObject(obj);
oos.flush();
}
}
} catch (EOFException eof) {
} finally {
oos.close();
ois.close();
// Delete the original file
File aDeletar = contexto.getFileStreamPath("dados.jlog");
File aRenomear = contexto.getFileStreamPath("dados.jlog.tmp");
if (!aDeletar.delete()) {
return;
} else {
// Rename the new file to the filename the original file
// had.
if (!aRenomear.renameTo(aDeletar)) Log.d(TAG_NAME,
"Error renaming file");
else Log.d(TAG_NAME, "Renaming successful");
}
}
} catch (FileNotFoundException ex) {
ex.printStackTrace();
Log.d(TAG_NAME, "Arquivo não encontrado");
} catch (IOException ex) {
ex.printStackTrace();
Log.d(TAG_NAME, "Erro de entrada/saída");
} catch (ClassNotFoundException e) {
Log.d(TAG_NAME, "Classe Não Encontrada.");
}
}
The method readObjectFromFile() seems to work just fine. I can even convert the read Object to Funcionario class and read its data.
My problems appear when I use removeObjectFromFile(). The idea is to create a temporary file to store objects from "dados.jlog" file other than the one that has been already loaded in the main program, then once this temp file is created the file "dados.jlog" should be deleted and the temporary file should be renamed to replace it.
The first thing I found out to be strange here is that the ois.readobject() keeps throwing an EOFException. While this makes sense, the tutorial I read on the internet doesn't mention this error. In fact, their code indicates that when the readObject() method reaches the EOF, it would return a reference to null, but instead this class throws this EOFException. I handled this exception in the code - though I'm not sure if this would be the right way to do it.
Another thing I find strange is the fact that this code fails to recognize the object that it should NOT copy. When I compare the object read from the file to the one received as argument, no matter what I try ( == , equals(), etc) they seem different objects to the compiler. Funcionario class is serializable has a serialversionUID, so the object read from the file should be identical to the one I stored. Worse than this, these 2 Objects being compared are read from the same file. They should be identical, right?
After creating the temporary file, I try to delete the original file and rename the temporary file. Though this seems to be working, once the removeObjectFromFile() ends the first time, the program is unable to read the data from the file "dados.jlog" again. I can't read the remaining data from the file and the program enters on an endless loop - since the 1st object is never removed from the list in the file.
Please enlighten me with this matter.
Personally I'd use an SQLLite database. Store each object in a row in the database. Once you've successfully transmitted you can remove the row from the database.
You can even reuse most of your code that you've already done. The easiest way to get there from where you are is to use a separate file for each object and store only the filename of the object in the database. You can then iterate over the rows in the database. Each time you transmit an object to your server simply delete that row from the database (and remove the file from the filesystem!). No rows in the database means no objects remain to be transmitted.
I'm trying to delete a file, after writing something in it, with FileOutputStream. This is the code I use for writing:
private void writeContent(File file, String fileContent) {
FileOutputStream to;
try {
to = new FileOutputStream(file);
to.write(fileContent.getBytes());
to.flush();
to.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
As it is seen, I flush and close the stream, but when I try to delete, file.delete() returns false.
I checked before deletion to see if the file exists, and: file.exists(), file.canRead(), file.canWrite(), file.canExecute() all return true. Just after calling these methods I try file.delete() and returns false.
Is there anything I've done wrong?
Another bug in Java. I seldom find them, only my second in my 10 year career. This is my solution, as others have mentioned. I have nether used System.gc(). But here, in my case, it is absolutely crucial. Weird? YES!
finally
{
try
{
in.close();
in = null;
out.flush();
out.close();
out = null;
System.gc();
}
catch (IOException e)
{
logger.error(e.getMessage());
e.printStackTrace();
}
}
It was pretty odd the trick that worked. The thing is when I have previously read the content of the file, I used BufferedReader. After reading, I closed the buffer.
Meanwhile I switched and now I'm reading the content using FileInputStream. Also after finishing reading I close the stream. And now it's working.
The problem is I don't have the explanation for this.
I don't know BufferedReader and FileOutputStream to be incompatible.
I tried this simple thing and it seems to be working.
file.setWritable(true);
file.delete();
It works for me.
If this does not work try to run your Java application with sudo if on linux and as administrator when on windows. Just to make sure Java has rights to change the file properties.
Before trying to delete/rename any file, you must ensure that all the readers or writers (for ex: BufferedReader/InputStreamReader/BufferedWriter) are properly closed.
When you try to read/write your data from/to a file, the file is held by the process and not released until the program execution completes. If you want to perform the delete/rename operations before the program ends, then you must use the close() method that comes with the java.io.* classes.
As Jon Skeet commented, you should close your file in the finally {...} block, to ensure that it's always closed. And, instead of swallowing the exceptions with the e.printStackTrace, simply don't catch and add the exception to the method signature. If you can't for any reason, at least do this:
catch(IOException ex) {
throw new RuntimeException("Error processing file XYZ", ex);
}
Now, question number #2:
What if you do this:
...
to.close();
System.out.println("Please delete the file and press <enter> afterwards!");
System.in.read();
...
Would you be able to delete the file?
Also, files are flushed when they're closed. I use IOUtils.closeQuietly(...), so I use the flush method to ensure that the contents of the file are there before I try to close it (IOUtils.closeQuietly doesn't throw exceptions). Something like this:
...
try {
...
to.flush();
} catch(IOException ex) {
throw new CannotProcessFileException("whatever", ex);
} finally {
IOUtils.closeQuietly(to);
}
So I know that the contents of the file are in there. As it usually matters to me that the contents of the file are written and not if the file could be closed or not, it really doesn't matter if the file was closed or not. In your case, as it matters, I would recommend closing the file yourself and treating any exceptions according.
There is no reason you should not be able to delete this file. I would look to see who has a hold on this file. In unix/linux, you can use the lsof utility to check which process has a lock on the file. In windows, you can use process explorer.
for lsof, it's as simple as saying:
lsof /path/and/name/of/the/file
for process explorer you can use the find menu and enter the file name to show you the handle which will point you to the process locking the file.
here is some code that does what I think you need to do:
FileOutputStream to;
try {
String file = "/tmp/will_delete.txt";
to = new FileOutputStream(file );
to.write(new String("blah blah").getBytes());
to.flush();
to.close();
File f = new File(file);
System.out.print(f.delete());
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
It works fine on OS X. I haven't tested it on windows but I suspect it should work on Windows too. I will also admit seeing some unexpected behavior on Windows w.r.t. file handling.
If you are working in Eclipse IDE, that could mean that you haven't close the file in the previous launch of the application. When I had the same error message at trying to delete a file, that was the reason. It seems, Eclipse IDE doesn't close all files after termination of an application.
Hopefully this will help. I came across similar problem where i couldn't delete my file after my java code made a copy of the content to the other folder. After extensive googling, i explicitly declared every single file operation related variables and called the close() method of each file operation object, and set them to NULL. Then, there is a function called System.gc(), which will clear up the file i/o mapping (i'm not sure, i just tell what is given on the web sites).
Here is my example code:
public void start() {
File f = new File(this.archivePath + "\\" + this.currentFile.getName());
this.Copy(this.currentFile, f);
if(!this.currentFile.canWrite()){
System.out.println("Write protected file " +
this.currentFile.getAbsolutePath());
return;
}
boolean ok = this.currentFile.delete();
if(ok == false){
System.out.println("Failed to remove " + this.currentFile.getAbsolutePath());
return;
}
}
private void Copy(File source, File dest) throws IOException {
FileInputStream fin;
FileOutputStream fout;
FileChannel cin = null, cout = null;
try {
fin = new FileInputStream(source);
cin = fin.getChannel();
fout = new FileOutputStream(dest);
cout = fout.getChannel();
long size = cin.size();
MappedByteBuffer buf = cin.map(FileChannel.MapMode.READ_ONLY, 0, size);
cout.write(buf);
buf.clear();
buf = null;
cin.close();
cin = null;
fin.close();
fin = null;
cout.close();
cout = null;
fout.close();
fout = null;
System.gc();
} catch (Exception e){
this.message = e.getMessage();
e.printStackTrace();
}
}
the answer is when you load the file, you need apply the "close" method, in any line of code, works to me
There was a problem once in ruby where files in windows needed an "fsync" to actually be able to turn around and re-read the file after writing it and closing it. Maybe this is a similar manifestation (and if so, I think a windows bug, really).
None of the solutions listed here worked in my situation. My solution was to use a while loop, attempting to delete the file, with a 5 second (configurable) limit for safety.
File f = new File("/path/to/file");
int limit = 20; //Only try for 5 seconds, for safety
while(!f.delete() && limit > 0){
synchronized(this){
try {
this.wait(250); //Wait for 250 milliseconds
} catch (InterruptedException e) {
e.printStackTrace();
}
}
limit--;
}
Using the above loop worked without having to do any manual garbage collecting or setting the stream to null, etc.
The problem could be that the file is still seen as opened and locked by a program; or maybe it is a component from your program that it had been opened in, so you have to ensure you use the dispose() method to solve that problem.
i.e. JFrame frame;
....
frame.dispose();
You have to close all of the streams or use try-with-resource block
static public String head(File file) throws FileNotFoundException, UnsupportedEncodingException, IOException
{
final String readLine;
try (FileInputStream fis = new FileInputStream(file);
InputStreamReader isr = new InputStreamReader(fis, "UTF-8");
LineNumberReader lnr = new LineNumberReader(isr))
{
readLine = lnr.readLine();
}
return readLine;
}
if file.delete() is sending false then in most of the cases your Bufferedreader handle will not be closed. Just close and it seems to work for me normally.
I had the same problem on Windows. I used to read the file in scala line by line with
Source.fromFile(path).getLines()
Now I read it as a whole with
import org.apache.commons.io.FileUtils._
// encoding is null for platform default
val content=readFileToString(new File(path),null.asInstanceOf[String])
which closes the file properly after reading and now
new File(path).delete
works.
FOR Eclipse/NetBeans
Restart your IDE and run your code again this is only trick work for me after one hour long struggle.
Here is my code:
File file = new File("file-path");
if(file.exists()){
if(file.delete()){
System.out.println("Delete");
}
else{
System.out.println("not delete");
}
}
Output:
Delete
Another corner case that this could happen: if you read/write a JAR file through a URL and later try to delete the same file within the same JVM session.
File f = new File("/tmp/foo.jar");
URL j = f.toURI().toURL();
URL u = new URL("jar:" + j + "!/META-INF/MANIFEST.MF");
URLConnection c = u.openConnection();
// open a Jar entry in auto-closing manner
try (InputStream i = c.getInputStream()) {
// just read some stuff; for demonstration purposes only
byte[] first16 = new byte[16];
i.read(first16);
System.out.println(new String(first16));
}
// ...
// i is now closed, so we should be good to delete the jar; but...
System.out.println(f.delete()); // says false!
Reason is that the internal JAR file handling logic of Java, tends to cache JarFile entries:
// inner class of `JarURLConnection` that wraps the actual stream returned by `getInputStream()`
class JarURLInputStream extends FilterInputStream {
JarURLInputStream(InputStream var2) {
super(var2);
}
public void close() throws IOException {
try {
super.close();
} finally {
// if `getUseCaches()` is set, `jarFile` won't get closed!
if (!JarURLConnection.this.getUseCaches()) {
JarURLConnection.this.jarFile.close();
}
}
}
}
And each JarFile (rather, the underlying ZipFile structure) would hold a handle to the file, right from the time of construction up until close() is invoked:
public ZipFile(File file, int mode, Charset charset) throws IOException {
// ...
jzfile = open(name, mode, file.lastModified(), usemmap);
// ...
}
// ...
private static native long open(String name, int mode, long lastModified,
boolean usemmap) throws IOException;
There's a good explanation on this NetBeans issue.
Apparently there are two ways to "fix" this:
You can disable the JAR file caching - for the current URLConnection, or for all future URLConnections (globally) in the current JVM session:
URL u = new URL("jar:" + j + "!/META-INF/MANIFEST.MF");
URLConnection c = u.openConnection();
// for only c
c.setUseCaches(false);
// globally; for some reason this method is not static,
// so we still need to access it through a URLConnection instance :(
c.setDefaultUseCaches(false);
[HACK WARNING!] You can manually purge the JarFile from the cache when you are done with it. The cache manager sun.net.www.protocol.jar.JarFileFactory is package-private, but some reflection magic can get the job done for you:
class JarBridge {
static void closeJar(URL url) throws Exception {
// JarFileFactory jarFactory = JarFileFactory.getInstance();
Class<?> jarFactoryClazz = Class.forName("sun.net.www.protocol.jar.JarFileFactory");
Method getInstance = jarFactoryClazz.getMethod("getInstance");
getInstance.setAccessible(true);
Object jarFactory = getInstance.invoke(jarFactoryClazz);
// JarFile jarFile = jarFactory.get(url);
Method get = jarFactoryClazz.getMethod("get", URL.class);
get.setAccessible(true);
Object jarFile = get.invoke(jarFactory, url);
// jarFactory.close(jarFile);
Method close = jarFactoryClazz.getMethod("close", JarFile.class);
close.setAccessible(true);
//noinspection JavaReflectionInvocation
close.invoke(jarFactory, jarFile);
// jarFile.close();
((JarFile) jarFile).close();
}
}
// and in your code:
// i is now closed, so we should be good to delete the jar
JarBridge.closeJar(j);
System.out.println(f.delete()); // says true, phew.
Please note: All this is based on Java 8 codebase (1.8.0_144); they may not work with other / later versions.