Within a Java program i've got a bunch of text files which the program reads and writes to (i know this is a really bad way to implement an app) but I need some way to ensure the integrity of the text files every time the program loads.
If the text file is deleted the program will be able to re-create it as it was last. Is there any way of doing something like this where I can store data between program executions? - But the important thing is that i'm able to change the data stored.
(Usually would use a database but it's not an option atm).
edit: (Clarify what I'm looking for)
There exists a text file full of data.
User deletes the text file.
Program detects wrong or missing file and re-creates it from a backup which the user can't get his hands on.
This is the kind of process i'm trying to implement.
You can't save data locally in a safe way. Everything that is stored on the users machine is under the users control. You can make them jump through hoops, like with using encryption or storing files in obscure formats in strange places, but you will just make it less convenient to change the files, not impossible for a determined user.
The only way to get around this is to store the data online.
Related
I have a Java app that uses a .txt file as a database. Every time I run the program it reads the file and copies its contents into an array. After I'm done the data in the array will have changed, and since I don't know any way of deleting lines in a file I update the file by deleting it, creating a new one with the same name and writing back the content of the array to the file.
This works if I always run the program from the same user (my OS is Ubuntu by the way). But if I run the program from user A, it will create a file that only has read/write permission for user A. When I attempt to run the program from user B, it will be able to read the file, but it won't be able to delete it in order to update it because it does not have permission to delete the file.
My question is: is there any way to make my Java program create a file that has read/write permission for everyone? or is there any better way to update the contents of the file without having to delete it? maybe there is a better alternative to using files altogether to store data once the program is closed?
P.D. This is my first time asking in StackOverflow, so please bear with me if I didn't write the question properly or I missed some details you might need. Also I am still pretty noob with programming, so I'm sorry in advance if I asked any stupid questions. Thank you very much for your time and for your answers.
If this is a Posix-conforming system, set the Posix file permissions to whatever you want.
See Files.setPosixFilePermissions.
By the way, you don't have to explicitly delete the file, you can just overwrite it.
I am working on DARPA dataset for network intrusion detection system. The DARPA dataset contains tcpdump files for training and testing purpose. Now when I open the file in text editor like wordpad, notepad++, I can't read the output file.
How can we read tcpdumfile so that I can save the records in database ?
Well, one way to read it is with, well, tcpdump; that's why they're called tcpdump files, after all.
Another possibility would be to use it with the TShark program that comes with Wireshark; it can be told to write the values of particular protocol fields to the standard output, and you could have a program that reads those values and puts them in a database.
If you want to do this in a Java program, some possibilities are:
jpcap;
jNetPcap;
the jNetWORKS SDK, if I understand what their page for it is saying - that's a commercial product;
possibly other packet-parsing Java libraries.
One thing that is most definitely NOT a possibility is trying to process the files as text - for example, trying to read them in a text editor - because they're not text files! They're binary files, and the packet data for each packet is also binary, and you'd need code that understands the protocols in order to parse that binary data and extract whatever fields you want to put into the database.
I've had a look around and haven't been able to come up with an answer to my issue.
I'm creating a fitness app and it allows users to save different workouts for future use. All the information is saved in txt file on internal storage. I'm trying to now implement a feature to be able to edit these workouts, so my question is:
What is the best way to edit a .txt file in android?
Should I just delete the old entry and save the new one in its place or is there a better way?
You can only append to a text file (add to the end); any other edit requires that you load the full file in working memory, modify it, and save it a new file (possibly overwriting the old one).
If this sounds like a bad idea (because the files are large and complex) then perhaps you should be looking at using SQLLite facilities which are standard android libraries and designed relatively simple record keeping tasks of this nature.
Unless your data is extremely unusual the SQL path will make for easier, clearer code in the long term.
Using SQLite database maybe the best way of saving data for different users. And if you want to edit a text file, you can load the full file in memory and rewrite the file after modify the content.
I'm adding autosave functionality to a graphics application in Java. The application periodically autosaves the current document and also autosaves on exit. When the user starts the application, the autosave file is reloaded.
If the autosave file is corrupted in any way (I assume a power cut when the file is in the middle of being saved would do this?), the user will lose their work. How can I prevent such situations and do all I can to guarantee that the autosave document is in a consistent state?
To further complicate matters, to autosave the document I need to save one .xml file and several .png files. Also, the .png saving occurs in C code over JNI.
My current strategy is to write each .png with the extension .png.tmp, write the .xml file with the extension .xml.tmp, and then rename each file to remove the .tmp part leaving the .xml until last. On startup, I only load the autosave document if I can find a .xml file and ignore .xml.tmp files. I also don't delete the previous autosave document until the .xml.tmp file for the new document is renamed.
I guess my knowledge of what happens when you write to disk is poor. I know you can have software read/write buffers when using files, as well as OS and hardware buffers and that all of these need to be flushed. I'm confused how I can know for sure when something really has been written to disk and what I can do to protect myself. Does the renaming operation do anything to make sure buffers are flushed?
If the autosave file is corrupted in any way (I assume a power cut when the file is in the middle of being saved would do this?), the user will lose their work. How can I prevent such situations and do all I can to guarantee that the autosave document is in a consistent state?
To prevent loss of data due to partially written autosave file, don't overwrite the autosave file. Instead, write to a new file each time, and then rename it once the file has been safely written.
To guard against not noticing that an autosave file has not been correctly written:
Pay attention to the exceptions thrown as the autosave file is written and closed in case a disc error, file system full, etc.
Keep a running checksum of the file as it is written and write it at the end of the file. Then when you load the autosave file, check that the checksum is there and is correct.
If the checkpointed state involves multiple files, make sure that you write the files in a well known order (without overwriting!), and write the checksum on the autosave file after all of the other files have been safely closed. You might want to create a directory for each checkpoint.
FOLLOW UP
No. I'm not saying that rename always succeeds. However, it is atomic - it either succeeds (and completes) or the file system is not changed. So, if you do this:
write "file.new" and close,
delete "file",
rename "file.new" to "file"
then provided the first step succeeds you are guaranteed to have the latest "file" safely on disc. And it is simple to add a couple of steps so that you have a backup of "file" at all times. (If the 3rd step fails, you are left with "file.new" and no "file". This can be recovered manually, or automatically by the application next time you run it.)
Also, I'm not saying that writes always succeed, or that applications don't crash, or that the power never goes off. And the point of the checksum is to allow you to detect the cases where these things have happened and the autosave file is incomplete.
Finally, it is a good idea to have two autosaves in case your application gets itself into a state where its data structures are messed up and the last autosave is nonsensical as a result. (The checksum won't protect against this.) Be cautious about autosaving when the application crashes for the same reason.
As an aside, since you have several different files as part of this one document, consider using either a project directory to hold them all together, or using some encapsulation format (like .zip) to put them all inside one file.
What you want to do is atomically replace the old backup files with new ones. Unfortunately, I don't believe that Java gives you enough control do this directly. You also need to reason about what operations are atomic in the underlying operating system. I know Linux file systems, so my answer will be biased towards a Java program running on that system. I would be shocked if Windows didn't do the same thing, but I can't say for certain.
Most Linux file systems (e.g. the meta-data journaled ones) let you rename files atomically. If the system crashes half-way through a rename, when you restart, it will be as if you never renamed a file in the first place. For this reason, a common way to atomically update an existing file F is to write your new data to a temporary file T and then rename T to F. Any system or application crash up to that rename will not affect F, so it will always be consistent.
Of course, before you rename, you need to make sure that your temporary file is consistent. Make sure that all streaming buffers for the file are flushed to the OS (Channel.force() or OutputStream.flush()) and the OS buffers are flushed to the disk (FileOutputStream.getFD.sync()). Of course, unless your OS disables the write cache on the hard disk itself (it probably hasn't), there's still a chance that your data can be corrupted. Add a checksum to the XML if you really want to be really sure. If you're truly paranoid, you should flush the OS and hard disk buffer caches and re-read the file to verify that it is consistent. This is beyond any reasonable expectation for normal consumer applications.
But that's just to atomically write write a single file. Your propblem is more complex: you have many files to update atomically. For example, I'll say that you have two files, img.png and main.xml. I'd do one of these:
The easy solution is to make a per-savefile directory. You wouldn't need to worry about renaming each individual file, and you could still atomically rename the new backup dir over the old backup dir you're replacing. That is, if your old backup is bak/img.png and bak/main.xml, write bak.tmp/img.png and bak.tmp/main.xml and rename bak.tmp to bak.
Name the new auxiliary files something else and let them coexist with the old ones for a little while. That is, write img.2.png and main.xml.tmp (which should refer to img.2.png, not img.png) and only rename main.xml.tmp to main.xml. Then delete img.png.
addition: If you don't have atomic renames, the next best thing extends on #2. Whenever you save the project, give it a new name (e.g. ver342.xml). When you load, just find the most recent XML that is consistent (i.e. its checksum verifies). Keep around 2 or 3 to be safe. Only delete an auto-save if you have successfully restored from a more-recent copy.
I have a web application in GWT and a complementary desktop client also written in Java (so the same solution basically applies to both). In my program users can attach files, then download them later or do whatever. These files are stored as blobs and can be in just about any format. Many of the users that use Excel and Word want to be able to open the file, make changes, then have those changes stored back in the attached file. In other words, need an inline editing of attachments.
Any ideas on how to make this happen? Should I have an 'edit' mode that keeps a file handler while the file is open, and then store that File handler? Some way keeping track of whether the file is changing, or not?
Sorry about the late response. Amol >> I have that going. I want to save directly back to a blob as if it were a filehandle. Thought that was clear in my question.
I have decided that this is almost impossible with a web application without writing some kind of client interface for each and every potential file type - word, excel, pdf, graphics, etc...