We Keep Coding

Java framework to manage BLOB data outside of database

I want to store my blobs outside of the database in files, however they are just random blobs of data and aren't directly linked to a file.
So for example I have a table called Data with the following columns:
id
name
comments
...
I can't just include a column called fileLink or something like that because the blob is just raw data. I do however want to store it outside of the database. I would love to create a file called 3.dat where 3 is the id number for that row entry. The only thing with this setup is that the main folder will quickly start to have a large number of files as the id is a flat folder structure and there will be OS file issues. And no the data is not grouped or structured, it's one massive list.
Is there a Java framework or library that will allow me to store and manage the blobs so that I can just do something like MyBlobAPI.saveBlob(id, data); and then do MyBlobAPI.getBlob(id) and so on? In other words something where all the File IO is handled for me?

Simply use an appropriate database which implements blobs as you described, and use JDBC. You really are not looking for another API but a specific implementation. It's up to the DB to take care of effective storing of blobs.

I think a home rolled solution will include something like a fileLink column in your table and your api will create files on the first save and then write that file on update.
I don't know of any code base that will do this for you. There are a bunch that provide an in memory file system for java. But it's only a few lines of code to write something that writes and reads java objects to a file.
You'll have to handle any file system limitations yourself. Though I doubt you'll ever burn through the limitations of modern file systems like btrfs or zfs. FAT32 is limited to 65K files per directory. But even last generation file systems support something on the order of 4 billion files per directory.
So by all means, write a class with two functions. One to serialize an object to a file; given it a unique key as a name. And another to deserialize the object by that key. If you are using a modern file system, you'll never run out of resources.

As far as I can tell there is no framework for this. The closest I could find was Hadoop's HDFS.
That being said the advice of just putting the BLOB's into the database as per the answers below is not always advisable. Sometimes it's good and sometimes it's not, it really depends on your situation. Here are a few links to such discussions:
Storing Images in DB - Yea or Nay?
https://softwareengineering.stackexchange.com/questions/150669/is-it-a-bad-practice-to-store-large-files-10-mb-in-a-database
I did find some addition really good links but I can't remember them offhand. There was one in particular on StackOverFlow but I can't find it. If you believe you know the link please add it in the comments so that I can confirm it's the right one.

How to manage the creation and deletion of temporary files

I'm adding code to a large JSP web application, integrating functionality to convert CGM files to PDFs (or PDFs to CGMs) to display to the user.
It looks like I can create the converted files and store them in the directory designated by System.getProperty("java.io.tmpdir"). How do I manage their deletion, though? The program resides on a Linux-based server. Will the OS automatically delete from /tmp or will I need to come up with functionality myself? If it's the latter scenario, what are good ways to go about doing it?
EDIT: I see I can use deleteOnExit() (relevant answer elsewhere), but I think the JVM runs more or less continuously in the background so I'm not sure if the exits would be frequent enough.
I don't think I need to cache any converted files--just convert a file anew every time it's needed.

You can do this
File file = File.createTempFile("base_name", ".tmp", new File(temporaryFolderPath));
file.deleteOnExit();
the file will be deleted when the virtual machine terminates
Edit:
If you want to delete it after the job is done, just do it:
File file = null;
try{
file = File.createTempFile("webdav", ".tmp", new File(temporaryFolderPath));
// do sth with the file
}finally{
file.delete();
}

There are ways to have the JVM delete files when the JVM exits using deleteOnExit() but I think there are known memory leaks using that method. Here is a blog explaining the leak: http://www.pongasoft.com/blog/yan/java/2011/05/17/file-dot-deleteOnExit-is-evil/
A better solution would either be to delete old files using a cron or if you know you aren't going to use the file again, why not just delete it after processing?

From your comment :
Also, could I just create something that checks to see if the size of my files exceeds a certain amount, and then deletes the oldest ones if that's true? Or am I overthinking it?
You could create a class that keeps track of the created files with a size limit. When the size of the created files, after creating a new one, goes over the limit, it deletes the oldest one. Beware that this may delete a file that still needs to exist even if it is the oldest one. You might need a way to know which files still need to be kept and delete only those that are not needed anymore.
You could have a timer in the class to check periodically instead of after each creation. This solution is tied to your application while using a cron isn't.

Rapidly changing Configuration/Status File? JAVA

I need some way to store a configuration/status file that needs to be changed rapidly. The status of each key value pair (key-value) is stored in that file. The status needs to be changed rather too rapidly as per the status of a communication (Digital multimedia broadcasting) hardware.
What is the best way to go about creating such a file? ini? XML? Any off the shelf filewriter in Java? I can't use databases.

It sounds like you need random access to update parts of the file frequently without re-writing the entire file. Design binary file format and use RandomAccessFile API to read/write it. You are going to want to use fixed number of bytes for key and for value, such that you can index into the middle of the file and update the value without having to re-write all of the following records. Basically, you would be re-implementing how a database stores a table.
Another alternative is to only store a single key-value pair per file such that the cost of re-writing the file is minor. Maybe you can think of a way to use file name as the key and only store value in the file content.
I'd be inclined to try the second option unless you are dealing with more than a few thousand records.

The obvious solution would be to put the "configuration" information into a Properties object, and then use Properties.store(...) or Properties.storeToXML(...) to save to a file output stream or writer.
You also need to do something to ensure that whatever is reading the file will see a consistent snapshot. For instance, you could write to a new file each time and do a delete / rename dance to replace the the old with the new.
But if the update rate for the file is too high, you are going to create a lot of disc traffic, and you are bound slow down your application. This is going to apply (eventually) no matter what file format / API you use. So, you may want to consider not writing to a file at all.

At some point, configuration that changes too rapidly becomes "program state" and not configuration. If it is changing so rapidly, why do you have confidence that you can meaningfully write it to, and then read it from, a filesystem?
Say more about what the status is an who the consumer of the data is...

Saving Multiple Images in a Single File

In my program i want the user to be able to take some images from a directory, and save them under a single file, that can be transferred to another computer possibly, and actually read and displayed(using the same program).
How would i go about doing this, especially if i want to save other data along with it, perhaps objects and such. I know you can use the ObjectOutputStream class, but im not sure how to integrate it with images.
So overall, i want the program to be able to read/write data, objects, and images to/from a single file.
Thanks in Advance.
[EDIT - From Responses + Comment regarding Zip Files]
A zip might be able to get the job done.
But i want it to be read only be the program. ( You think making it a zip, changing the file extension would work, then when reading it just chaing it back and reading as a zip?? ) I dont want users to be able to see the contents directly.
Ill elaborate a bit more saying its a game, and users can create their own content using xml files, images and such. But when a user creates something i dont want other users to be able to see exactly how they created it, or what they used, only the end result.

You can programatically create a zip file, and read a zip file from Java, no need to expose it as a regular .zip file.
See: java.io.zip pacakge for more information, and these others for code samples on how to read/write zip using java.
Now if you want to prevent the users from unzipping this file, but you don't want to complicate your life by encrypting the content, or creating a complex format, you can emulate a simple internet message format, similar to the one used for e-mails to attach files.
You can read more about the internet message format here
This would be a custom file format only used by your application so you can do it as simple as you want. You just have to define your format.
It could be:
Header with the names ( and number ) of files in that bundle.
Followed by a list of separators ( for instance limit.a.txt=yadayada some identifier to know you have finished with that content )
Actual content
So, you create the bundle with something like the following:
public void createBundle() {
ZipOutputStream out = ....
writeHeader( out );
writeLimits( out yourFiles );
for( File f : youFiles ) {
writeFileTo( f, out );
}
out.close();
}
Sort of...
And the result would be a zipped file with something like:
filenames =a.jpg, b.xml, c.ser, d.properties, e.txt
limits.a.jpg =poiurqpoiurqpoeiruqeoiruqproi
limits.b.xml =faklsdjfñaljsdfñalksjdfa
limit.s.ser =sdf09asdf0as9dfasd09fasdfasdflkajsdfñlk
limit.d.properties =adfa0sd98fasdf90asdfaposdifasdfklasdfkñm
limit.e.txt =asdf9asdfaoisdfapsdfñlj
attachments=
<include binary data from a.jpg here>
--poiurqpoiurqpoeiruqeoiruqproi
<include binary data from b.xml here>
--faklsdjfñaljsdfñalksjdfa
etc
Since is your file format you can keep it as simple as possible or complicate your life at infinitum.
If you manage to include a MIME library in your app, that could save you a lot of time.
Finally if you want to add extra security, you have to encrypt the file, which is not that hard after all, the problems is, if you ship the encrypting code too, your users could get curious about it and decompile them to find out. But a good encrypting mechanism would prevent this.
So, depending on your needs you can go from a simple zip, a zip with a custom format, a zip with a complicated customformat or a zip with a custom complicated encrypted format.
Since that's to broad you may ask about specific parts here: https://stackoverflow.com/questions/ask

In your case I would use a ZIP library to package all the images in a ZIP file. For the metadata you want to save along with these, use XML files. XML and ZIP are quite a de-facto standard today, simple to handle and though flexible if you want to add new files or metadata. There are also serializing tools to serialize your objects into XML. (I don't know them exactly in Java, but I'm sure there are.)

Yep, just pack/unpack them with java.util.zip.* which is pretty straightforward to go. Every Windows Version since XP has built in zip support, so your good to go. There are many good (and faster) free zip libraries for java/c#, too.

I know you can use the ObjectOutputStream class, but im not sure how to integrate it with images.
Images are binary data, so reading it into a byte[] and writing the byte[] to ObjectOutputStream should work. It's however only memory hogging since every byte eats at least one byte of JVM's memory. You'll need to take this into account.

How to rename an existing file

I have an existing file in my HDD and I want rename this file.

Like this:
new File(path).renameTo(new File(newPath));

You may look at commons-io FileUtils.moveFile. That method tries File.renameTo, and if that fails, tries to copy&delete the file (e.g. when moving from one drive to another).
If you use File.renameTo directly (which should suffice if you only want to rename a file in the same directory), make sure you evaluate the returned boolean value!