I want to use HSQLDB in a production environment for stroring some data in memory and for data export using files. Does anybody have experience with using hsqldb in production environments? Is hsqldb treating server resources gracefully and does it cleanup unused resources properly?
I've seen a critical post on those issues from red hat and I'm wondering whether this still holds for hsqldb:
http://kbase.redhat.com/faq/docs/DOC-15194
I've used hsql on numerous occasions in production (mainly as quick filebased storage for complex preferences) and never encountered any problems.
Don't know about HSQLDB but we've been using H2 on same purposes with no problems at all.
I can confirm some of the issues listed on the Red Hat page.
We have had issues using HSQLDB as a standalone instance in a Tomcat container. The application would not shutdown properly and hang at 100% cpu. There was a code fix, though.
We have also had issues were some data was lost after the server was forcibly killed. I could not reliably reproduce the situations.
I also have some strangeness that I cannot start multiple instances of the same application using HSQLDB at the same time.
You need to evaluate the if a standalone, in-memory DB is the right choice. If consistency and integrity is essential, HSQLDB may not be the right choice.
We experienced database corruption (entire database was lost) a few times times over a year using HSQLDB when it was not shut down cleanly.
Related
We have installed 2 instance of same application in a same datacenter. Both the app is using same oracle DB. But we are observing performance issue in one application. In AppDynamics we can see the response time of one application is much higher that other.
Is it possible to intentionally prioritise/configure the DB such a way. If yes, where should I look into the database.
Any Idea why this is happening? I am totally clueless here.
In theory, yes: if Resource Manager has been enabled it could be the case that different Resource Manager plans have such an impact but experience shows that this feature is seldom used.
In practive this kind of difference can have many cause:-
different SQL statements run
data is different
database statistics differences
different database configuration
different hardware
etc.
The first thing to look at database level is something similar to Statspack report (or AWR if licensing allows) to compare database configuration and activity.
And don't forget that application performance is not only database performance it depends also on application server, network and front-end.
We are having an application i.e. exposing RESTful web services and we are targeting this application to be deployed in cloud. We need to one time setup a database schema for the application on some database instance in the cloud.
Can someone tell me if it is a good approach to use migrations with liquibase for the one time database schema setup. We will be using alter scripts in case some DDL modification needed in future releases.
Someone stop me if I'm wrong, but the fact you application will be deplyed on the cloud only mean it will be on a virtual server hosted by an extern compagnie, wich in the case of your question don't chance anything.
So the question is "is the database versionning system Liquibase on a database with an aimed stable shema worth it".
In absolute it could be considered overkill, and a lot of big companies still manage database schema evolution with bare sql scripts. You could simply export the final built script of your developpement database and go with it.
But since you know Liquibase, the overhead is pretty cheap, and the comfort of using if you happen to have to modify tour shema later is important.
So yes, I think it's a pretty good pratice (safer than hand applying script under the stress of a production server problem) which cost one or two hours(given you know how to use the tool) and can save dozens when having to handle hotfixing of a production database.
I assume that you will be deploying this application in more than one place - not just production in the cloud, but also development servers, test servers, staging, etc. If that is true, then it seems to me that you definitely want to have a process around how you make changes to the database schema.
For me, over the course of my 20+ years in software development, I have seen several things that I use now that were not in common use when I started but that have now become 'baseline' practices on any project I work on. Yeah, I used to work without source control, but that is an absolute must now. I used to write software without tests, but not any more. I used to work without continuous integration, but that is yet another practice that I consider a must-have. The most recent addition to my must-have list is some sort of automated database migration process.
Also, since Liquibase is built-in to Dropwizard, I don't see any reason not to use it.
I want to make a java application with a rather small database. The pc on which I want to install this has nothing of database stuff on his pc (no wamp server, no oracle, nothing...). I'm rather new in this kind of stuff, and i don't know if it's already been asked but this is what i want to accomplish
Now I have a couple of questions:
Is this doable?
What should I use? Mysql, Oracle,...
How can i do this?
I hope this is enough to get a decent answer.
Yes, it is doable.
For use with Java, I strongly recommend Apache Derby because
you have the huge flexibility of being able to choose between embedded and client-server db, with no code refactoring needed to change data access mode
over H2 or HSQLDB: according to my experience I've found Apache Derby
to be much more reliable/resilient (other embedded DBMSs tend to break more than derby when power fails)
to eat up less RAM
to have better performance on bigger deployments (lots of rows, lots of data [in microbenchmarks with little real-world data H2 and HSQLDB can actually score better]).
to be particularly fast with select queries in heavily multithreaded environments
over MySql and PostgreSql
it's actually faster, when you are not CPU/network -bound, because I've seen it perform better than them in many cases (especially with bigger DBs -- say 10GB) when it comes to filesystem access (MySql and PostgreSql, however, are more efficient in terms of CPU/network utilization, when these are a constraint)
over MySql, PostgreSql, Oracle db, etc.
it's surprisingly fast (often faster), with very big DBs (say, 30 GB) -- something one wouldn't expect from a DBMS you can embed in any application with no deployment/configuration
To get started, see
Apache Derby Getting Started guide
Apache Derby tutorial
Apache Derby FAQs
WorkingWithDerby wiki
If you don't need clients from the network to remotely connect to your database, an "embedded database" is what you want to implement.
Flame-preventing disclaimer: all the statements above are according to my very own personal experience, with the projects I've worked on and/or articles/benchmarks that I read and trusted as reliable. Unless otherwise stated (and in fact I'm not stating otherwise anywhere :) ), I'm referring to fresh out-of-the-box un-fine-tuned installations.
You should probably use an embedded database like H2 or HSQLDB. They are just a simple libraries that you drop in your application, but they provide exactly the same JDBC interface.
You can use the full power of SQL database without any external dependencies. H2, my personal favourite, allows you to create in-memory as well as persistent databases, you can optionally connect to it using socket, it can expose web interface over default 8082 port, so on and so on. On my developer machine I don't even have "normal" database installed, I always use H2.
HSQL or use one of the SQLite JDBC adapters.
I recommend using Derby database. It is very simple to embed in java application.
How is this computer's hardware? What CPU memory and hard-disk?
What is the OS? Do you have the administrator/root access?
If you have a typical PC with windows OS, and enough CPU, memory and hard-disk.
I recommend you to install mysql. Just download the mysql for your OS and install it.
Download link:
http://www.mysql.com/downloads/mysql/
Here is install documents:
http://dev.mysql.com/doc/refman/5.5/en/installing.html
Good luck.
I have an established software product that uses HSQLDB as its internal settings database. Customer projects are stored in this database. Over the years, HSQLDB has served us reasonably well, but it has some stability/corruption issues that we've had to code circles around, and even then, we can't seem to protect ourselves from them completely.
I'm considering changing internal databases. Doing this would be fairly painful from a development perspective, but corrupted databases (and lost data) are not fun to explain to customers.
So my question is: Does anyone have enough experience to weigh in on the long-term stability of Apache Derby? I found a post via Google complaining that Derby was unstable, but it was from 2006 so I'd entertain the idea that it has been improved in the last 4 years. Or, is there another pure Java embedded (in-process) database that I could use (commercial or open-source). Performance isn't very important to me. Stability is king. Data integrity across power loss, good BLOB support, and hot-backups are all a must.
Please don't suggest something that isn't a SQL-based relational database. I'm trying to retrofit an existing product, not start from scratch, thanks.
For each database engine there is a certain risk of corruption. I am the main author of the H2 database, and I also got reports about broken databases. Testing can reduce the probability of bugs, but unfortunately it's almost impossible to guarantee some software is 'bug free'.
As for the three Java database HSQLDB, Apache Derby, and H2, I can't really say which one is the most stable. I can only speak about H2. I think for most operations, H2 is now stable. There are many test cases that specially test for databases getting corrupt. This includes automated tests on power loss (using a christmas light timer). With power failure tests I found out stability also depends on the file system: sometimes I got 'CRC error' messages meaning the operating system can't read the file (it was Windows). In that case, there is not much you can do.
For mission critical data, in any case I wouldn't rely on the software being stable. It's very important to create backups regularly, and test them. Some databases have multiple way to create backups. H2 for example has an online backup feature, and a feature to write a SQL script file. An alternative is to use replication or clustering. H2 supports a simple cluster mode, I believe Derby supports replication.
I ran Derby 24/7 as the internal database supporting a build automation and test management system for 4 years. It was used by a worldwide team, and never crashed, lost data, or corrupted my records. The only reason we stopped using it is because our company was bought by another and a higher-level decision was handed down. Derby is solid, reliable, and well worth your consideration.
This search shows 215 posts in HSQLDB Users mailing list containing the string "corrupt".
http://search.gmane.org/?query=corrupt&author=&group=gmane.comp.java.hsqldb.user&sort=date&DEFAULTOP=and&xP=Zcorrupt&xFILTERS=Gcomp.java.hsqldb.user---A
This search shows 264 posts in Derby Users mailing list containing the same string.
http://search.gmane.org/?query=corrupt&author=&group=gmane.comp.apache.db.derby.user&sort=date&DEFAULTOP=and&xP=Zcorrupt&xFILTERS=Gcomp.apache.db.derby.user---A
This one shows 1003 posts in Derby Dev mailing list with the same string
http://search.gmane.org/?query=corrupt&author=&group=gmane.comp.apache.db.derby.devel&sort=date&DEFAULTOP=and&xP=Zcorrupt&xFILTERS=Gcomp.apache.db.derby.devel---A
A look at some of the posts shows possible or real cases of database corruption happen despite all the best efforts of database developers.
HSQLDB has had its own share of database corruption issues but has improved over the years. In the latest versions precautions and fixes have been introduced to prevent all the issues that were reported in the last few years.
The new lob storage feature however, turned out to have a logic bug that results in the lobs being "forgotten" after an update. This is being fixed right now, with more extensive tests to support the fix.
Users like CarlG have helped a lot over the years in the bug fixing efforts of both Derby and HSQLDB.
Fred Toussi, HSQLDB Project
Does anyone have enough experience to weigh in on the long-term stability of Apache Derby? (...)
Derby, ex IBM Cloudscape (and now also distributed by Sun as JavaDB) is an ACID-compliant database that can stand a lot of concurrent users, running embedded or in server mode, and is know to be robust and production ready. It is not as fast as HSQLDB (Derby uses durable operations), but it's robust. Still, you should run your own tests against it.
See also
François Orsini's blog
I have been using Apache Derby since 2009 in many of my projects, some of them with 24/7 operation and many millions of rows.
Never ever had a single event of data corruption. Rock solid and fast.
I keep choosing it as my RDBMS of choice, unless a good reason not to pops out.
Try looking into H2. It was created by the guy who originally made HSQLDB but built from scratch so doesn't use any HSQLDB code. Not sure how its stability compares to HSQL since I haven't used HSQL in ages and I'm only using H2 for short-lived databases currently. I personally found H2 to be easier to get going than Derby but maybe that's because H2 has a cheat sheet web page.
It might be possible to re-code to use an abstraction layer and then run tests to compare H2 and Derby with the issues you have found.
On the project management side of the fence, does your roadmap have a major version coming up? That might be a rather appropriate time to rip out the guts this way and I wouldn't say you were crazy cause it could potentially remove lots of hard to manage work arounds. If you wanted to make the change where it could affect live systems without plenty of warning and backups in place then you may be crazy.
With regard to HSQLDB, one thing that it doesn't have as a project that SQLite has is the documentation of a robust testing suite and online documentation of assiduous ACID compliance.
I don't mean to take anything away from HSQLDB. It's meant to serve as an alternative to MySQL not to fopen() as SQLite is intended. One can say that the scope of HSQLDB (all the Java RDBMS's really) is much more ambiitious. Fredt and his group have accomplished an extraordinary achievement with HSQLDB. Even so, doing the Google search "Is HSQLDB ACID compliant" doesn't leave an early adopter feeling as confident as one feels after reading about the testing harnesses on the SQLite website.
At http://sqlite.org/transactional.html
"SQLite is Transactional
A transactional database is one in which all changes and queries appear to be Atomic, Consistent, Isolated, and Durable (ACID). SQLite implements serializable transactions that are atomic, consistent, isolated, and durable, even if the transaction is interrupted by a program crash, an operating system crash, or a power failure to the computer.
We here restate and amplify the previous sentence for emphasis: All changes within a single transaction in SQLite either occur completely or not at all, even if the act of writing the change out to the disk is interrupted by
a program crash,
an operating system crash, or
a power failure.
The claim of the previous paragraph is extensively checked in the SQLite regression test suite using a special test harness that simulates the effects on a database file of operating system crashes and power failures."
At http://sqlite.org/testing.html
"1.0 Introduction
The reliability and robustness of SQLite is achieved in part by thorough and careful testing.
As of version 3.7.14, the SQLite library consists of approximately 81.3 KSLOC of C code. (KSLOC means thousands of "Source Lines Of Code" or, in other words, lines of code excluding blank lines and comments.) By comparison, the project has 1124 times as much test code and test scripts - 91421.1 KSLOC.
1.1 Executive Summary
Three independently developed test harnesses
100% branch test coverage in an as-deployed configuration
Millions and millions of test cases
Out-of-memory tests
I/O error tests
Crash and power loss tests
Fuzz tests
Boundary value tests
Disabled optimization tests
Regression tests
Malformed database tests
Extensive use of assert() and run-time checks
Valgrind analysis
Signed-integer overflow checks"
Give SQLite a try if you're looking for something self contained (no server involved). This is what backs android's db api, and is highly stable.
I'm debeloping a Java Swing application, which persists the information through Hibernate, currently using PostgreSQL. Now, I would like to simplify the database setup on the client and I'm thinking about single file databases. The database I'm using is rather small and simple, so there are no special requirements. I'm only asking for one that is stable and reliable.
What solutions are there available which are compatible with Hibernate? The final application shouldn't have any installation steps such as installing a database server or so. The perfect scenario would be a .zip file that is unzipped and everything is ready to go, like Eclipse does.
Thanks!
Java database such as JavaDB (aka Derby), HSQLDB or the more recent H2 (by the original author of HSQLDB) in embedded mode are all possible candidates. Between them JavaDB tend to be considered as the most robust (full ACID, robustness against failures, unlimited db size etc) and would be my choice.
Here is a quick summary of the reasons:
H2 is very interesting and I use it for testing (its compatibility mode feature is really great) but I'm not sure of its maturity for production use.
H2 and HSQLDB are faster than Derby because they do not sync data to disk on commit - while Derby does. So performance comes from the lack of Durability. If you change this behavior, they all hit the some bottleneck: disk IO.
HSQLDB only supports dirty read, transactions see uncommitted values from other transactions.
HSQLDB is not fully Atomic, a transaction can be partially(WTF?) committed
Derby is more powerful, has specs compliant drivers, scales well.
Don't misinterpret me, I'm not saying HSQLDB is bad but you need to know what you're dealing with and in which context/application. Sometimes speed is more important (e.g. for unit testing), and sometimes it is data integrity. If you are in the latter case, Java DB is IMO a better choice.
Some of the references below are 2/3 years old but I'm not aware of any revolution. If I made some mistake, let me know, I'll be happy to update my answer to make it accurate.
References
Database Comparison Table : H2, HSQLDB, DERBY, PostgreSQL, MySQL
HSQLDB or JavaDB ???
H2 Performance... "Notes"
Derby vs. Hsqldb for Embedding in a Swing App?
What is the best embedded database for Java?
hsqldb - who needs acid!!
I have used HSLQDB in conjunction with hibernate for long. It is really fast, easy to use application for basic small db requirements.