I'm wondering the best method for my program to determine if a file has been appended or completely changed.
What I am currently doing is using a LogIOThread I wrote that is using a FileInputStream.avail to check if a file has been appended. If so, I am appending to a non-editable JTextArea (actually a modified JTextArea with variable scroll features). My end-game was to have a autoscrolling JTextArea that is basically scrolling a log4j outputted logfile. This is working great for me right now. I'd like to adapt this to a non-log files.
What I would like to add to the LogIOThread is a monitor that will check if the file contents changed rather than just have new appended text. The first use case I am trying to solve is the file gets rewritten mid run. At the most basic level I figured I could use this function to reload my JTextArea close and reopen the FileInputStream, and start over if the file get overwritten.
I figured while I was doing that I might want to make it more robust to handle the second use case of mid-file insertions, like a properties file change. I figured I could get the textArea to do a line replace if I can figure out if a line changed.
I have a requirement (not set by me) to use Java6, so some of the new nio FileWatcher's are no good for me. Notifiers in general seem counter productive as I'm catching appends via FileInputStream. This also led me to discount other libs such as Commons and jnotify. I could be wrong though, and maybe FileInputStream.avail is not the best way for me anymore.
My first thought was that I could store the file size, and check if it drastically changed (to a value less than stored). Since a new created log file would need a fresh textArea, this doesn't seem to bad, and solves my first use case. This would not work for the second use case of a specific value change in the file. It also might be problematic for a file that gets recreated with a consistent size while my IOThread is sleeping, although I doubt it's likely.
My second thought was that I could continually check the file modified time, and if there is no appendable text, I would reread the file and do a line comparison? I would need to do this anyway if I'm going to change a line in the textArea unless I reload it every time. This seems horribly inefficient though.
Any suggestions? I'm not opposed to changing how the LogIOThread works if there is a suggestion to get new text and changes with something better than an avail + file modification check combo.
If the file size decreases it has certainly been overwritten. However it may also be overwritten with something as large or larger, so the converse does not hold.
Don't use available() as a surrogate for File.length(). See the Javadoc. That's not what it's for.
How about creating an intermediate OutputStream that reads the log data, makes it available to the JTextField and then routes that data to the file?
Related
I have a task to implement a logging method that will take all program content up to logging method call and save it to file. The method should not overwrite already present info in file, but rather add new content to it.
What I need to write:
all program messages and output;
all user inputs in the way they appear on console (the program inquires it several times);
My thought on it is to create a StringBuilder object and start appending everything to it. Once logging method is invoked, ask for a file to save log to and save contents of StringBuilder to it. Then flush StringBuilder and continue to gather information. On second invocation, if the same filename is provided, just append new info gathered by StringBuilder.
However, this means that I will need to place gathering invocations all over the place where program output and user input are. Seems like not very optimal to me.
Are there any ideas on how to reach my goal differently and more optimally?
Thank you in advance.
Best regards,
Vadim
UPDATE: I actually was able to redirect system.out to gather everything to ByteArrayStream and then write to file on demand. But I still don't understand how to do it for inputstream. I don't need to redirect it, I still have to input eveyrthing from keyboard, it's just values that have to make it to logs in correct places. Am still searching for a solution.
Do not reinvent the wheel. Go for a logging framework. There is one integrated into java anyway.
Alternatively you can use log4j, and there are other such frameworks out there.
I have used PrintWriter to write to a text file with java, the process takes time, bcz there is an iteration process, at end of each iteration, a string is appended to the file, if I wait till the process ends, I can open and see the content of the file without problem, but how to see the updating process to the file while it is still writing to, e.g. I refresh inside the folder then I see the size becomes larger and larger with each refresh click?
You can't see the progress of your stuff being written into the file, since it might not actually be.
First: Java Streams usually optimize access to the harddrive, to access it as little as possible since doing so is slow. To force Java to do what it thinks is wrinting to disk, call PrintWriter.flush() after you have appended something.
Second: even that might not do the trick, since most operating systems do the same optimisation with the same result, and you can't simply force you os to flush().
As already mentioned, the best you can do is Printwriter.flush().
As #Elliott mentioned you should do pw.flush() after each append to see the size change immediately.
I am using RandomAccessFile but writeBytes() overwrites the
existing content Mostly i want to implement this without using any new
temporary file at least methods name few clues or techniques will do.
To insert without the use of any temporary file, you'll have to find a way to read in and shift all subsequent lines down by one when you insert a new line.
The sequential storage of lines of text poses the same kind of issues that the use of a standard array (vs. a LinkedList) does: You can't just unhook a pointer, plop in some text, and then hook up the pointer to next item to point to a subsequent line in the file. You have to perform the entire shift.
So I'd guess that you'd want to go to end of file, shift it down by a line, then move up each line and perform the same shift until you hit the position at which you want to insert the new line, at which point, you'll have cleared a space for it.
This seems very performance inefficient.
Hopefully someone knows of a better way, but this would be my approach, were a temporary file not an option.
(Alternately, you could also always just read the whole thing into a StringBuffer if it were small enough, peform the insert within that, and then write the file back out from scratch, but I imagine that you've considered that option already.)
So say you have a file that is written in XML or soe other coding language of that sort. Is it possible to just rewrite one line rather than getting the entire file into a string, then changing then line, then having to rewrite the whole string back to the file?
In general, no. File systems don't usually support the idea of inserting or modifying data in the middle of a file.
If your data file is in a fixed-size record format then you can edit a record without overwriting the rest of the file. For something like XML, you could in theory overwrite one value with a shorter one by inserting semantically-irrelevant whitespace, but you wouldn't be able to write a larger value.
In most cases it's simpler to just rewrite the whole file - either by reading and writing in a streaming fashion if you can (and if the file is too large to read into memory in one go) or just by loading the whole file into some in-memory data structure (e.g. XDocument), making the changes, and then saving the file again. (You may want to consider saving to a different file then moving the files around to avoid losing data if the save operation fails for some reason.)
If all of this ends up being too expensive, you should consider using multiple files (so each one is smaller) or a database.
If the line you want to replace is larger than the new line that you want to replace it with, then it is possible as long as it is acceptable to have some kind of padding (for example white-space characters ' ') which will not effect your application.
If on the other hand the new content are larger than the content to be replaced you will need to shift all the data downwards, so you need to rewrite the file, or at least from the replaced line onwards.
Since you mention XML, it might be you are approaching your problem in the wrong way. Could it be that what you need is to replace a specific XML node? In which case you might consider using DOM to read the XML into a hierarchy of nodes and adding/updating/removing in there before writing the XML tree back to the file.
I need some way to store a configuration/status file that needs to be changed rapidly. The status of each key value pair (key-value) is stored in that file. The status needs to be changed rather too rapidly as per the status of a communication (Digital multimedia broadcasting) hardware.
What is the best way to go about creating such a file? ini? XML? Any off the shelf filewriter in Java? I can't use databases.
It sounds like you need random access to update parts of the file frequently without re-writing the entire file. Design binary file format and use RandomAccessFile API to read/write it. You are going to want to use fixed number of bytes for key and for value, such that you can index into the middle of the file and update the value without having to re-write all of the following records. Basically, you would be re-implementing how a database stores a table.
Another alternative is to only store a single key-value pair per file such that the cost of re-writing the file is minor. Maybe you can think of a way to use file name as the key and only store value in the file content.
I'd be inclined to try the second option unless you are dealing with more than a few thousand records.
The obvious solution would be to put the "configuration" information into a Properties object, and then use Properties.store(...) or Properties.storeToXML(...) to save to a file output stream or writer.
You also need to do something to ensure that whatever is reading the file will see a consistent snapshot. For instance, you could write to a new file each time and do a delete / rename dance to replace the the old with the new.
But if the update rate for the file is too high, you are going to create a lot of disc traffic, and you are bound slow down your application. This is going to apply (eventually) no matter what file format / API you use. So, you may want to consider not writing to a file at all.
At some point, configuration that changes too rapidly becomes "program state" and not configuration. If it is changing so rapidly, why do you have confidence that you can meaningfully write it to, and then read it from, a filesystem?
Say more about what the status is an who the consumer of the data is...