I have a method that creates PDFs and a servlet that uploads files to my database. Is there anyway I could directly upload the created PDFs to my database? I am using FileOutputStream to create my PDFs.
Sounds like you need something like Java Caching System or Ehcache. Putting PDF's in a database is generally not a good idea, though it has certainly been done.
I suppose it depends on how long you expect the user to want to retrieve the file. If a long time, perhaps a database could have advantages because it is administered, backed up, etc. However, all the same things can be said about a file cache and if you have a lot of the files then you should have much happier system admins using a simple file store.
Related
I'm trying to make an executable .jar file from a program that uses both an SQLite database and a serialized read/write file system for storage inside the program. We are able to make an executable .jar file, but it doesn't read from any database nor file system that we have written within the program. Does anybody know how to do this?
"To make a .jar file read from a database" when you have a Java or Kotlin (or Scala) program which does that you are almost done, you just have to compile it and which provides you a .jar file which does exactly that.
I think your question is what does it take to write a program that does it.
For that you have to think about what your database should be able to handle.
Read from filesystem
A text file on a disc is very rudimentary but might suffice if the application is just to read/write some persisted state. For that the Java API has all you need, take a look at Baeldung - read from file via NIO.
Read from database
If you need transactions or support for multiple applications working with the persisted storage, a database would be the more sensible thing to use.
Although the Java API again provides you everything you need for that (e.g.Processing SQL Statements with JDBC) you would need to reinvent the wheel in a sense. The easier thing would be to rely on an OR-mapper like Hibernate or JOOQ.
For writing enterprise Java software you could overengineer it with Spring-Boot JPA.
In my web application which I use primefaces+SpringWebflow. I need to add a picture which is scanned or uploaded to every customer registered. I can not think of any good solution to store the files. Criteria is that.
I want only application have access to files and it shouldnt be possible to access the images directly.
I tried to store the files in database but thats not good idea. i store the path in database.
I would like to have relative paths to files. and efficient wy to access the file.
Just store the files in the database. If you try to put them somewhere else, it will be difficult to migrate your application from one server to another, because its data will be spread across multiple sources. Upgrades will be more difficult.
If performance / database size becomes an issue, revisit this decision, but make sure you always measure rather than guess the performance.
Why is it not a good idea to store the files in DB ?
What will happen when you deploy your application on a clustered server environment ? In clustered server environment your solution of keeping images on file-system will not work.
You need to think more on keeping storage of images in DB in mind. Just investigate what performance hit you are getting in storage/retrieval of images in DB. try different kind of storage frameworks like nosql DBs. Essentially you can not get away with storing data at a central location I think.
I need an indexed file format that can hold a few hundred large variable sized binary blobs.
Blobs are around 1-5MB and the file could be as large as 1 GB. I need to be able to quickly find, read, add and remove blobs without recreating the the entire file. I have no need to compress the blobs, however if blobs were removed, I'd like to reclaim or reuse the space.
Ideally there would be a Java API.
I'm currently doing this with a ZIP format, but there's no known way to update a ZIP file without recreating it and performance is bad.
I've looked into SQLite but its blob performance was slow, and its overkill for my needs.
Any thoughts, or should I roll my own?
And if I do roll my own, any book or web page suggestions?
Berkeley DB Java Edition does what you need. It's free.
You need some virtual file system. Our SolFS is the one of the options yet we have only JNI layer, as the engine is written in C. There exists one more option, CodeBase, but as they don't provide an evaluation version of their file system, I know a few about it.
SolFS is ideally suitable for your task, because it lets you have alternative streams for files and associate searchable metadata with each file or even alternative stream.
I am creating a few JAX-WS endpoints, for which I want to save the received and sent messages for later inspection. To do this, I am planning to save the messages (XML files) into filesystem, in some sensible hierarchy. There will be hundreds, even thousands of files per day. I also need to store metadata for each file.
I am considering to put the metadata (just a couple of fields) into database table, but the XML file content itself into files in a filesystem in order not to bloat the database with content data (that is seldomly read).
Is there some simple library that helps me in saving, loading, deleting etc. the files? It's not that tricky to implement it myself, but I wonder if there are existing solutions? Just a simple library that already provides easy access to filesystem (preferrably over different operating systems).
Or do I even need that, should I just go with raw/custom Java?
Is there some simple library that
helps me in saving, loading, deleting
etc. the files? It's not that tricky
to implement it myself, but I wonder
if there are existing solutions? Just
a simple library that already provides
easy access to filesystem (preferrably
over different operating systems).
Java API
Well, if what you need to do is really simple, you should be able to achieve your goal with java.io.File (delete, check existence, read, write, etc.) and a few stream manipulations with FileInputStream and FileOutputStream.
You can also throw in Apache commons-io and its handy FileUtils for a few more utility functions.
Java is independent of the OS. You just need to make sure you use File.pathSeparator, or use the constructor File(File parent, String child) so that you don't need to explicitly mention the separator.
The Java file API is relatively high-level to abstract the differences of the many OS. Most of the time it's sufficient. It has some shortcomings only if you need some relatively OS-specific feature which is not in the API, e.g. check the physical size of a file on the disk (not the the logical size), security rights on *nix, free space/quota of the hard drive, etc.
Most OS have an internal buffer for file writing/reading. Using FileOutputStream.write and FileOutputStream.flush ensure the data have been sent to the OS, but not necessary written on the disk. The Java API support also this low-level integration to manage these buffering issue (example here) for system such as database.
Also both file and directory are abstracted with File and you need to check with isDirectory. This can be confusing, for instance if you have one file x, and one directory /x (I don't remember exactly how to handle this issue, but there is a way).
Web service
The web service can use either xs:base64Binary to pass the data, or use MTOM (Message Transmission Optimization Mechanism) if files are large.
Transactions
Note that the database is transactional and the file system not. So you might have to add a few checks if operations fails and are re-tried.
You could go with a complicated design involving some form of distributed transaction (see this answer), or try to go with a simpler design that provides the level of robustness that you need. A possible design could be:
Update. If the user wants to overwrite a file, you actually create a new one. The level of indirection between the logical file name and the physical file is stored in database. This way you never overwrite a physical file once written, to ensure rollback is consistent.
Create. Same story when user want to create a file
Delete. If the user want to delete a file, you do it only in database first. A periodic job polls the file system to identify files which are not listed in database, and removes them. This two-phase deletes ensures that the delete operation can be rolled back.
This is not as robust as writting BLOB in real transactional database, but provide some robustness. You could otherwise have a look at commons-transaction, but I feel like the project is dead (2007).
There is DataNucleus, a Java persistence provider. It is little too heavy for this case, but it supports JPA and JDO java standards with different datastores (RDBMS, object storage, XML, JSON, Excel, etc.). If the product is already using JPA or JDO, it might be worth considering using NataNucleus, as saving data into different datastores should be transparent. I suppose DataNucleus supports splitting the data into several files, creating the sensible directory/file structure I wanted (in my question), but this is just a guess.
Support for XML and JSON seems to be experimental.
I am using Hibernate in a Java application to access my Database and it works pretty well with MS-SQL and MySQL. But some of the data I have to show on some forms has to come from Text files, and by Text files I mean Human-Readable files, they can be CSV, Tab-Delimited, or even a key, value pair, per line since my data is as simple as this, but my preference of course is XML files.
My question is: Can I use hibernate to read those files using HQL, Query , EntityManager and all those resources Hibernate provides me to access files. Which file format should I use and How I configure My persistence.xml file to recognize files as Tables?
Hibernate is written against the JDBC API. So, you need a JDBC driver that works with the file format you are interested in. Obviously, even for read-only access, this isn't going to perform well, but it might still be useful if that's not a high priority. On a Windows system, you can set up ODBC datasources for delimited text files, Excel files, etc. Then you can set up the JdbcOdbcDriver in your Java application to use this data source.
For most of the applications I work on, I would not consider this approach; I would use an import/export mechanism to convert from a real database (even if it's an in-process database like Berkeley DB or Derby) to the text files. Yes, it's an extra step, but it could be automated, and the performance isn't likely to be much worse than trying to use the text files directly (it will likely be much better, overall), and it will be more robust and easy to develop.
A quick google came up with
JDBC driver for csv files
JDBC driver for XML files
Hope this might provide some inspiration?
Like erickson said, your only hope is in finding a JDBC driver for that task. There is maybe xlsql (CSV, XML and Excel driver) which could fit the task. After that, you just have to either find or write the most simple Hibernate Dialect which fits your driver.