Lets say I have a database with a few hundred blobs stored in separate files I want to move to the SD card, what is the best way of programatically doing this ensuring nothing gets left behind?
I realize I can only copy files across different mounts.
So can I copy all the files to a subdirectory in "cache" and then move the subdirectory to the correct spot atomically? Or can I write all to the files to a directory with a "temp" prefix, and rename it in place when verified?
The possibility to move the folder (I mean, relocate it just by renaming) depends on the file system and probably it is better not to rely on in. Also, even when it works, it only works inside the boundaries on the single mount. I would propose simply to rename the folder after all files have been successfully copied.
If the folder with "temporary" prefix would be discovered due previous crash, it can be simply removed ("rollback").
I would recommend finding a library with an algorithm that is designed for problems such as this. You might want to start by reviewing this question Any good rsync library for Java?
Related
I want to move two files to a different directory in same filesystem.
Concrete example, I want to move /var/bigFile to /var/path/bigFile, and /var/smallFile to /var/path/smallFile.
Currently I use Files.move(source, target), without any options, moving the small file first and big file second. I need this order since there is another process waiting for this files to arrive, and the order is important.
Problem is that, sometimes I see the creation date for small file being greater than the creation date for the big file, like the moving order is not followed.
Initially I thought I have to do a sync, but it does not make sense.
Given the fact that the move will actually be a simple rename, there is no system buffers included, to force them to be flushed to disk.
Timestamp for the files was checked using ls -alrt command.
Does anyone have any idea what could be wrong?
So I have made a simple game using Java. I have my Jar file and inside it I am keeping my pictures. It's like a lot of pictures going on there. Like 140+ pictures at around 9 MB (mostly icons) (Not worried about efficiency, game already runs pretty smoothly). Now I wrapped my jar file into an .exe and then used Inno Setup to make a setup file. The thing is I am creating a folder in Program Files that only has a the application.exe file and the uninstallation file. This looks a bit vague to me and unprofessional. Though I've seen great games having all of their icons outside of the executable of course, just wondering would it be a good practice to keep the images inside or out ?
Suggestions are welcomed :D
I prefer to keep images and other resources inside the JAR file. You don't want the user to change any of them, same as for the program classes. So I don't see why you should have a different storage for images compared to classes.
The most robust and simple (and that should be the meaning of "professional") setup is to have the whole program with all its prerequisites in just one file - either it's there and works, or it's missing, and you'll see that immediately. No need to check for existence and accessibility of dozens of subdirectories and files.
And as you're asking the question, you surely managed to read the images from the classpath, thus allowing to store them together with the class files, e.g. in a JAR.
Regarding wrapping in an EXE, that shouldn't make a difference: if the ClassLoader finds the classes, it'll surely find the images the same way. And even Ahead-Of-Time-Compilers support getResourceAsStream() (we've done that with Excelsior Jet).
I am storing lots of data into hdfs. And I need to move the files from one folder to another.
May I ask generally how much is the cost of filesystem's rename method?
Say I have to move terabytes of data.
Thank you very much.
Moving files in HDFS or any file system if implemented properly involves changes to the name space and not moving of the actual data. Going through the code only changes in the name space (memory and edit log) in the Name node are done.
From the NameNode.java class
The NameNode controls two critical tables:
1) filename->blocksequence (namespace)
2) block->machinelist ("inodes")
Only the first part needs to be modified, block to machine list need not be. I haven't tried it out, but I guess it should be OK.
Rename is a metadata-only operation in HDFS. Therefore it is be very cheap like it is in a normal POSIX filesystem, too. No data is moved. The only server involved is the namenode.
The source code for the rename can be found here. Pretty straight forward.
My Android application downloads data only the first lunch. the data is ~50 mb with ~2500 files.
1. Is it a good idea to store if the files got downloaded in SharedSettings? The problem is that if a user clears the data application (maybe by mistake), he has to redownload everything. I manually copy a prepacked database to /data/data/../databases/, is it a good idea to check if the db exists, and if no then download everything?:
if(new File(/data/data/../databases/myDB.db).exists){//dont download}
2.Is getting the folder size and checking if its the same a good way to see if the folder+data are good? or is there a better way to check if 2 folders are the same?
Thanks.
No, do not put 50MB of data into SharedSettings. That will fall over and die. A set of SharedSettings is stored in XML on disk and entirely loaded into RAM when opened. This also won't keep the user from clearing this data.
For determining whether the data has been downloaded, I would suggest just having a file you make once the download is complete indicating it is done. The user can't selectively remove files. They can clear your data, but that will also clear the sentinel file and you will know you need to re-download. (Also keep in mind you will need to deal with restarting the download if it gets interrupting in the middle.)
Also be sure you correctly handle filesystem operations as described here: http://android-developers.blogspot.com/2010/12/saving-data-safely.html
An alternate idea if you're worried about missing data files... If at any point your app looks for a file and it doesn't exist, throw an exception, pass it to a handler that shows a dialog and 'verifies' your data. You can keep a list of all needed data files, and then only download ones that don't exist. Something like a system check, if you will.
That way they don't end up downloading 50MB if they were only missing a couple files they accidentally deleted in root explorer ;-)
I am doing a project in java and in that i need to add and modify my
text file at runtime,which is grouped in the jar.
I am using class.getResourceAsStream(filename) this method we
can read that file from class path.
i want to write into the same textfile.
What is the possible solution for this.
If i can't update the text file in jar what other solution is there?
Appreciate any help.
The easiest solution here is to not put the file in the jar. It sounds like you are putting files in your jar so that your user only needs to worry about one file that contains everything related to that program. This is an artificial constraint and just add headaches.
There is a simple solution that still allows you to distribute just the jar file. At start up, attempt to read the file from the file system. If you don't find it, use default values that are encoded in you program. Then when changes are made, you can write it to the file system.
In general, you can't update a file that you located using getResourceAsStream. It might be a file in a JAR/ZIP file ... and writing it would entail rewriting the entire JAR file. It might be a remote file served up by a Url classloader.
For your sanity (and good practice), you should not attempt to update files that you access via the classpath. If you need to, read the file out of the JAR file (or whatever), copy it into the regular file system, and then update the copy.
I'm not saying that it is impossible to do this in all cases. Indeed, in most normal cases you can do it with some effort. However, this is not supported, and there are no standard APIs for doing this.
Furthermore, attempts to update resources are liable to cause anomalies in the classloader. For example, I'd expect resources in JAR files to not update (from the perspective of the application) until the application restarted. But resources in exploded JAR files probably would update ... though new resources might not show up.
Finally, there are cases where updating a resource is impossible:
When the user doesn't have write access to the application's installation directory. This is typical for a properly administered UNIX / Linux machine.
When the JAR file is fetched from a remote server, you are likely not to be able to write the updates back.
When you are using an arbitrary custom classloader, you've got no way of knowing where the actual bytes of an updated resource should be stored, and no way of storing them.
All JAR rewriting techniques in Java look similar. Open the Jar file, read all of it's contents, and write a new Jar file containing the unmodified contents (and the modifications you whished to make). Such techniques are not advisable for a Jar file on the class path, much less a Jar file you're running from.
If you decide you must do it this way, Java World has a few articles:
Modifying Archives, Part 1
Modifying Archives, Part 2
A good solution that avoids the need to put your items into a Jar file is to read (if present) a properties file out of a hidden subdirectory in the user's home directory. The logic looks a bit like this:
if (the hidden directory named after my application doesn't exist) {
makeTheHiddenDirectory();
writeTheDefaultPropertiesFile();
}
Properties appProps = new Properties();
appProps.load(new FileInputStream(fileInHiddenDir));
...
... After the appProps have changed ...
...
appProps.store(new FileOutputStream(fileInHiddenDir), "Do not modify this file");
Look to java.util.Properties, and keep in mind that they have two different load and store formats (key = value based and XML based). Pick the one that suits you best.
If i can't update the text file in jar what other solution is there?
Store the information in any of:
Cookies
The server
Deploy the applet using 1.6.0_10+, launch it using JWS and use the PersistenceService to store the information. Here is my demo. of the PersistenceService.
Also, if your users will agree to a trusted applet (which seems overkill for this), you might write the information to a sub-directory of user.home.