Can I use H2, HSQLDB, or any other embedded database, with a database from an InputStream instead of a file?
I'm planning to use AssetManager.open() on Android, which can return an InputStream in random access mode.
H2 supports a pluggable file system that allows you to access read-only databases in a zip or jar files. However, there is currently no file system implementation for the AssetManager. It should be relatively easy to implement it. The best starting point is probably FileSystemZip and FileObjectZip.
Most databases need random access to underlying files so an InputStream will not do.
AFAIK, H2 and HSQLDB provide only file-based and in-memory connection URIs when used in "standalone" mode. So a file, or some heap memory space, shall be needed; but you don't give it a File/InputStream, everything is in the connection URI, eg jdbc:h2:~/test.
If you're prepared to do a bit of development, it can be done with HSQLDB, especially version 1.8.1.x which is smaller in size.
Check the org.hsqldb.lib.ResourceStreamProvider class and modify it to use the streams that you provide for file names db.properties and db.script when the calls are made to its static getResourceAsStream method.
Pretty simple, especially because the db.script and db.properties consist of text, which you can easily generate in the rest of your program.
Related
I have a method that creates PDFs and a servlet that uploads files to my database. Is there anyway I could directly upload the created PDFs to my database? I am using FileOutputStream to create my PDFs.
Sounds like you need something like Java Caching System or Ehcache. Putting PDF's in a database is generally not a good idea, though it has certainly been done.
I suppose it depends on how long you expect the user to want to retrieve the file. If a long time, perhaps a database could have advantages because it is administered, backed up, etc. However, all the same things can be said about a file cache and if you have a lot of the files then you should have much happier system admins using a simple file store.
I'd like to save persistent objects to the file system using Hibernate without the need for a SQL database.
Is this possible?
Hibernate works on top of JDBC, so all you need is a JDBC driver and a matching Hibernate dialect.
However, JDBC is basically an abstraction of SQL, so whatever you use is going to look, walk and quack like an SQL database - you might as well use one and spare yourself a lot of headaches. Besides, any such solution is going to be comparable in size and complexity to lighweight Java DBs like Derby.
Of course if you don't insist absolutely on using Hibernate, there are many other options.
It appears that it might technically be possible if you use a JDBC plaintext driver; however I haven't seen any opensource ones which provide write access; the one I found on sourceforge is read-only.
You already have an entity model, I suppose you do not want to lose this nor the relationships contained within it. An entity model is directed to be translated to a relational database.
Hibernate and any other JPA provider (EclipseLink) translate this entity model to SQL. They use a JDBC driver to provide a connection to an SQL database. This, you need to keep as well.
The correct question to ask is: does anybody know an embedded Java SQL database, one that you can start from within Java? There are plenty of those, mentioned in this topic:
HyperSQL: stores the result in an SQL clear-text file, readily imported into any other database
H2: uses binary files, low JAR file size
Derby: uses binary files
Ashpool: stores data in an XML-structured file
I have used HyperSQL on one project for small data, and Apache Derby for a project with huge databases (2Gb and more). Apache Derby performs better on these huge databases.
I don't know exactaly your need, but maybe it's one of below:
1 - If your need is just run away from SQL, you can use a NoSQL database.
Hibernate suports it through Hibernate OGM ( http://www.hibernate.org/subprojects/ogm ).
There are some DBs like Cassandra, MongoDB, CouchDB, Hadoop... You have some suggestions Here
.
2 - Now, if you want not to use a database server (with a service process running always), you can use Apache Derby. It's a DB just like any other SQL, but no need of a server. It uses a singular file to keep data. You can easily transport all database with your program.
Take a look: http://db.apache.org/derby/
3 - If you really want some text plain file, you can do like Michael Borgwardt said. But I don't know if Hibernate would be a good idea in this case.
Both H2 and HyperSQL support embedded mode (running inside your JVM instead of in a separate server) and saving to local file(s); these are still SQL databases, but with Hibernate there's not many other options.
Well, since the question is still opened and the OP said he's opened to new approaches/suggestions, here's mine (a little late but ok).
Do you know Prevayler? It's a Java Prevalence implementation which keep all of your business objects in RAM and mantain Snapshots/Changelogs in the File System, this way it's extremely fast and reliable, since if there's any crash, it'll restore it's last state and reapply every change to it.
Also, it's really easy to setup and run in your app.
Ofcourse this is possible, You can simply use file io features of Java, following steps are required:-
Create a File Object
2.Create an object of FileInputStream (though there are ways which use other Classes)
Wrap this object in a Buffer object or simply inside a java.util.Scanner.
use specific write functions of the object created in previous step.
Note that your object must implement Serializable interface. See following link,
I am creating a few JAX-WS endpoints, for which I want to save the received and sent messages for later inspection. To do this, I am planning to save the messages (XML files) into filesystem, in some sensible hierarchy. There will be hundreds, even thousands of files per day. I also need to store metadata for each file.
I am considering to put the metadata (just a couple of fields) into database table, but the XML file content itself into files in a filesystem in order not to bloat the database with content data (that is seldomly read).
Is there some simple library that helps me in saving, loading, deleting etc. the files? It's not that tricky to implement it myself, but I wonder if there are existing solutions? Just a simple library that already provides easy access to filesystem (preferrably over different operating systems).
Or do I even need that, should I just go with raw/custom Java?
Is there some simple library that
helps me in saving, loading, deleting
etc. the files? It's not that tricky
to implement it myself, but I wonder
if there are existing solutions? Just
a simple library that already provides
easy access to filesystem (preferrably
over different operating systems).
Java API
Well, if what you need to do is really simple, you should be able to achieve your goal with java.io.File (delete, check existence, read, write, etc.) and a few stream manipulations with FileInputStream and FileOutputStream.
You can also throw in Apache commons-io and its handy FileUtils for a few more utility functions.
Java is independent of the OS. You just need to make sure you use File.pathSeparator, or use the constructor File(File parent, String child) so that you don't need to explicitly mention the separator.
The Java file API is relatively high-level to abstract the differences of the many OS. Most of the time it's sufficient. It has some shortcomings only if you need some relatively OS-specific feature which is not in the API, e.g. check the physical size of a file on the disk (not the the logical size), security rights on *nix, free space/quota of the hard drive, etc.
Most OS have an internal buffer for file writing/reading. Using FileOutputStream.write and FileOutputStream.flush ensure the data have been sent to the OS, but not necessary written on the disk. The Java API support also this low-level integration to manage these buffering issue (example here) for system such as database.
Also both file and directory are abstracted with File and you need to check with isDirectory. This can be confusing, for instance if you have one file x, and one directory /x (I don't remember exactly how to handle this issue, but there is a way).
Web service
The web service can use either xs:base64Binary to pass the data, or use MTOM (Message Transmission Optimization Mechanism) if files are large.
Transactions
Note that the database is transactional and the file system not. So you might have to add a few checks if operations fails and are re-tried.
You could go with a complicated design involving some form of distributed transaction (see this answer), or try to go with a simpler design that provides the level of robustness that you need. A possible design could be:
Update. If the user wants to overwrite a file, you actually create a new one. The level of indirection between the logical file name and the physical file is stored in database. This way you never overwrite a physical file once written, to ensure rollback is consistent.
Create. Same story when user want to create a file
Delete. If the user want to delete a file, you do it only in database first. A periodic job polls the file system to identify files which are not listed in database, and removes them. This two-phase deletes ensures that the delete operation can be rolled back.
This is not as robust as writting BLOB in real transactional database, but provide some robustness. You could otherwise have a look at commons-transaction, but I feel like the project is dead (2007).
There is DataNucleus, a Java persistence provider. It is little too heavy for this case, but it supports JPA and JDO java standards with different datastores (RDBMS, object storage, XML, JSON, Excel, etc.). If the product is already using JPA or JDO, it might be worth considering using NataNucleus, as saving data into different datastores should be transparent. I suppose DataNucleus supports splitting the data into several files, creating the sensible directory/file structure I wanted (in my question), but this is just a guess.
Support for XML and JSON seems to be experimental.
I am making a desktop application in java and using MSAccess in data base.
I want that if i run the setup of the application the database should be created on client machine because there can be different client using the application and how can i create the setup?
is there any tools available for this free of cost?
please explain me in detail..
thanks
Java 6 (enhanced for desktop application work) comes with a built-in database called JavaDb (formerly IBM's Derby). That will do what you want.
Here's a set of guides and tutorials on how to use it.
I would suggest that when your application first starts, it checks for the presence of the created database, and if it doesn't exist, it builds the database (via the appropriate SQL). I've used this approach before and it works quite well.
I prefer nullsoft. Take a look at Open Source Installers Generators in Java
#pratap: database should be created on client machine..
Add an empty access database to your setup.
Have a look at SQLite, which is used by Mozilla (Firefox stores all bookmarks and history in a database) and several other major applications.
When you say
access in database
do you mean Microsoft Access or access the data in a database.
I would advise against MS Access if that is the case. If not, you could either use the JavaDB or HSQLDB and the use SQL scripts to create the database. As a summary
Package the application in one of the installers (InnoSetup or NSIS are good ones)
When installing, extract all the files in proper folders
Execute the SQL scripts before first running the application to ensure the database is setup, you can do other housekeeping tasks along with this step (refer to installer documentation for after-install steps)
Your application is good to go
In the last distribution of NetBeans I used, there was a wizard to create such application. The application used the Java Persistence API to store the Data.
My option is HSQLDB since it's fast, reliable and easy to use.
In the documentation it's explained how to use the standalone database mode, this is primarily used for unit testing but it fits your case too. The good thing with this is that you just connect to the file based database without any special set up and if the files doesn't exist, they're created.
I would second the posters who recommend JavaDB.
It is absurdly easy to adminster from inside your application. Whats more because everything is native Java you dont get the char->unicode little-endian->big-endien and all the other conversion malarky you normally get when reading SQL into java.
The one tip is that with JavaDB is prepare your SQL statements. Prepared statements get cached and the resulting access program (similar to an access plan but actually a jvm program) is reused, the programs for "executed' statements are not cached.
If you are really set on MSAccess then I would suggest you package an "default.mdb" file with all your required tables defined and your classifcation tables populated. If the user's table does not exist then simply copy over the default .mdb file and open that.
I recommend the H2 database because it is simple, fast, pure Java, and small. See this page for how H2 compares to other Java databases, including those mentioned here in other answers. It has many features Derby/JavaDB and HSQLDB do not.
I am using Hibernate in a Java application to access my Database and it works pretty well with MS-SQL and MySQL. But some of the data I have to show on some forms has to come from Text files, and by Text files I mean Human-Readable files, they can be CSV, Tab-Delimited, or even a key, value pair, per line since my data is as simple as this, but my preference of course is XML files.
My question is: Can I use hibernate to read those files using HQL, Query , EntityManager and all those resources Hibernate provides me to access files. Which file format should I use and How I configure My persistence.xml file to recognize files as Tables?
Hibernate is written against the JDBC API. So, you need a JDBC driver that works with the file format you are interested in. Obviously, even for read-only access, this isn't going to perform well, but it might still be useful if that's not a high priority. On a Windows system, you can set up ODBC datasources for delimited text files, Excel files, etc. Then you can set up the JdbcOdbcDriver in your Java application to use this data source.
For most of the applications I work on, I would not consider this approach; I would use an import/export mechanism to convert from a real database (even if it's an in-process database like Berkeley DB or Derby) to the text files. Yes, it's an extra step, but it could be automated, and the performance isn't likely to be much worse than trying to use the text files directly (it will likely be much better, overall), and it will be more robust and easy to develop.
A quick google came up with
JDBC driver for csv files
JDBC driver for XML files
Hope this might provide some inspiration?
Like erickson said, your only hope is in finding a JDBC driver for that task. There is maybe xlsql (CSV, XML and Excel driver) which could fit the task. After that, you just have to either find or write the most simple Hibernate Dialect which fits your driver.