We Keep Coding

Need a head start in reading AFP files and extracting the content and metadata

I was assigned to work on this specific project, where we will be getting AFP(advanced function presentation) files and we need to get the documents, i.e.the content and the corresponding meta data. I have been looking into AFP(advanced function presentation) file format and haven't actually got any useful resource about how I should proceed with the task.
I have almost got no information up until now and don't know where to proceed. I looked into some open source projects and found this: https://github.com/yan74/afplib
I tried running it.. But it does not work on the sample AFP file which I have.
Really need some insight upon what resources should I go through to be able this project.
I need to write the code in Java and have gone through some licensed softwares which do the same,like PROARCHIVER and PAPYRUS.
Thanks in advance

AFP is an easy format, it's composed of structured fields, your first step is decoding them, download this: "Mixed Object Document Content Architecture Reference" read first 50 pages and write code to split afp into structured fields, in order to create an easy dump of your file.
After that if you want to extract images AFP world calls them IOCA, so you need: Image Object Content Architecture reference
If you want to extract text (called PTX) you need: Presentation Text Object Content Architecture Reference
good job

Java framework to manage BLOB data outside of database

I want to store my blobs outside of the database in files, however they are just random blobs of data and aren't directly linked to a file.
So for example I have a table called Data with the following columns:
id
name
comments
...
I can't just include a column called fileLink or something like that because the blob is just raw data. I do however want to store it outside of the database. I would love to create a file called 3.dat where 3 is the id number for that row entry. The only thing with this setup is that the main folder will quickly start to have a large number of files as the id is a flat folder structure and there will be OS file issues. And no the data is not grouped or structured, it's one massive list.
Is there a Java framework or library that will allow me to store and manage the blobs so that I can just do something like MyBlobAPI.saveBlob(id, data); and then do MyBlobAPI.getBlob(id) and so on? In other words something where all the File IO is handled for me?

Simply use an appropriate database which implements blobs as you described, and use JDBC. You really are not looking for another API but a specific implementation. It's up to the DB to take care of effective storing of blobs.

I think a home rolled solution will include something like a fileLink column in your table and your api will create files on the first save and then write that file on update.
I don't know of any code base that will do this for you. There are a bunch that provide an in memory file system for java. But it's only a few lines of code to write something that writes and reads java objects to a file.
You'll have to handle any file system limitations yourself. Though I doubt you'll ever burn through the limitations of modern file systems like btrfs or zfs. FAT32 is limited to 65K files per directory. But even last generation file systems support something on the order of 4 billion files per directory.
So by all means, write a class with two functions. One to serialize an object to a file; given it a unique key as a name. And another to deserialize the object by that key. If you are using a modern file system, you'll never run out of resources.

As far as I can tell there is no framework for this. The closest I could find was Hadoop's HDFS.
That being said the advice of just putting the BLOB's into the database as per the answers below is not always advisable. Sometimes it's good and sometimes it's not, it really depends on your situation. Here are a few links to such discussions:
Storing Images in DB - Yea or Nay?
https://softwareengineering.stackexchange.com/questions/150669/is-it-a-bad-practice-to-store-large-files-10-mb-in-a-database
I did find some addition really good links but I can't remember them offhand. There was one in particular on StackOverFlow but I can't find it. If you believe you know the link please add it in the comments so that I can confirm it's the right one.

Is there a way to associate data with a file in a folder hierarchy?

Using Java, I am creating a program that indexes a folder structure and allows a user to search for files and also tag a file with keywords and then search for files based off of those tags.
I have been traversing through the folder hierarchy using the FileUtils listFiles method at the moment which is essentially this question: Recursively list files in Java
I haven't yet begun to code the tagging functionality, but thinking ahead I'm fearing that if a file is renamed or moved after I associate it with a tag then it will lose the tag. This defeats the purpose of my program, so can anybody offer suggestions as to how to store each file located in the folder hierarchy or associate the tag so that if a file is renamed or moved it will still have the tag associated with it.

If you want to keep track of a file, even when its name and/or location changes, you should use its unique identifier, which in most file systems is called its inode. (I think NTFS/Windows calls it a "file ID.") You can read a file's inode using its BasicFileAttributes.fileKey:
Object key = Files.getAttribute(file.toPath(), "fileKey");
That key is suitable for use as a HashMap key.

If the OS doesn't support file tagging, you could:
maintain a mapping of file path to tags
maintain a mapping of file hash to tags
Using option #2, your tags would be preserved even if a file was moved. But if someone moved AND modified the file, then the tags would be lost.

I don't think there's a way to do this without updating your tag relationship to the newly create file since the rename/mv operation is at the disk level and there is actually a delete and 'create' file compound action happening in the background. Because of that, there's no guarantee that a file will even be in the same place on the disk. If you know for sure that the file will have the same contents, you could take an MD5 signature of the file's contents in a String object and then always compare those when a tag is queried, of course this has its down falls too when the file's content changes.
Your best bet is to use a hash map w/ the files' paths and tags and then use directory watcher to update the hash map when a file name changes. Thats the best I can think of!

Update objects written to a text files in java

Writing Java objects or a List into a text file is ok. But I want to know how I can update or rewrite a object which was written previously without writing objects again. For example, let s assume there is a java.util.List has a set of Objects. and then that list is written to a text file. Then later that file will be read again and get all objects from list and then change one object's value at run time by a java application. Then I don't need to write entire list back to the text file. Instead only the updated object in the list is required to be rewritten or updated in the text file without rewriting the whole list again. Any suggestion, or helpful source with sample codes please.

Take a look at RandomAccessFile. This will let you seek to the place in the file you want, and only update the part that you want to update.
Also take a look at this question on stackoverflow.

Without some fairly complex logic, you won't usually be able to update an object without rewriting the entire file. For example, if one of the objects on your list contains a string "shortstring", and you need to update it with string "muchmuchlongerstring", there will be no space in the file for the longer string without rewriting all the following content in the file.
If you want to persist large object trees to a file and still have the ability to update them, your code will be less buggy and life will be simplified by using one of the many file-based DBs out there, like:
SQLite (see Java and SQLite)
Derby
H2 (disk-based tables)

File-based Document Storage in android

I'm in the early stages of a note-taking application for android and I'm hoping that somebody can point me to a nice solution for storing the note data.
Ideally, I'm looking to have a solution where:
Each note document is a separate file (for dropbox syncing)
A note can be composed of multiple pages
Note pages can have binary data (such as images)
A single page can be loaded without having to parse the entire document into memory
Thread-safety: Multiple reads/writes can occur at the same time.
XML is out (at least for the entire file), since I don't have a good way to extract a single page at a time. I considered using zip files, but (especially when compressed) I think they'd be stuck loading the entire file as well.
It seems like there should be a Java library out there that does this, but my google-fu is failing me. The only other alternative I can think of is to make a separate sqlite database for every note.
Does anybody know of a good solution to this problem? Thanks!

Seems like a relational database would work here. You just need to play around with the schema a little.
Maybe make a Pages table with each page including, say, a field for the document it belongs to and a field for its order in the document. Pages could also have a field for binary data, which might be contained in another table. If the document itself has additional data, maybe you have a table for documents too.
I haven't used SQLite transactions on an Android device, but it seems like that would be a good way to address thread safety.

I would recommend using SQLite to store the documents. Ultimately, it'll be easier than trying to deal with file I/O every time you access the note. Then, when somebody wants to upload to dropbox, you generate the file on the fly and upload it. It would make sense to have a Notes table and a pages table, at least. That way you can load each page individually and a note is just a collection of pages anyway. Additionally, you can store images as BLOBS in the database for a particular page. Basically, if you only want one type of content per page, then you would have, in the pages table, something like an id column and a content column. Alternatively, if you wanted to support something that is more complex such as multiple types of content then you would need to make your pages a collection of something else, like "entities."
IMO, a relational database is going to be the easiest way to accomplish your requirement of reading from particular pages without having to load the entire file.