I am looking for a database which I can use to store data about certain stock over a number of years. There will probably be a few thousand records. I am writing an application in Java and Clojure which will pull out data from this local database when required to display the data.
I was wondering if anyone knew of a good database to work with for this purpose? I only have experience with MySQL running on the server side.
Which database would be easiest to work with in Clojure and Java for local storage?
Thanks,
Adam
JDK 6 and greater comes bundled with Java DB which good enough for your use case.
For this kind of small-scale application it will almost certainly be easiest if you pick one of the many good embedded Java databases.
My personal top choices would probably be:
H2 - probably the best performance pure Java database overall, and if you believe their benchmarks then it is considerably faster than MySQL and indeed most other databases when run in a single machine environment.
Apache Derby - good all rounder, mature and well supported (Oracle have included a version branded as Java DB in recent JDKs)
After that, you should be able to use them pretty easily using the standard JDBC toolset, so not much different from MySQL.
If you're after a really nice DSL for interfacing with SQL databases with Clojure, you should definitely also take a look at Korma.
I have used Apache Derby for a similar application (although written mostly in Java). They have been running it for almost four years now, and performed more than 60,000 transactions with it with no major problems. Only the occasional bug on my part.
Derby is the same database as JavaDB, however with Derby its easier to keep up on the releases as you can just include it as a dependency, rather than wait on the whim of when the next JDK rev is coming out.
Also, IIRC, JavaDB is only included with JDK, not the JRE.
Depending on the nature of your data and application and your willingness and/or constraints in working with a new database modality, you might also want to consider one of the document-oriented databases, MongoDB or CouchDB. If your data and application are SQL oriented, use one of the databases suggested.
Related
I currently have a web application that runs with all of the data in Oracle. At the high level, the application consists of a java applet, some java servlets, some Ajax, and the oracle database. I was wondering what converting the whole suite to Hadoop instead would cost in terms of work? Below are some questions that can help me get a grasp on it.
Is there any software that can take SQL database schema creation scripts and queries and convert them to appropriate calls in Hadoop?
How different are the Java APIs for communicating with Hadoop to that of oracle SQL?
Theres a bit of Ajax in there too, how different is that from SQL to Hadoop?
Please consider me a beginner when explaining anything having to do with Hadoop. I don't need to drill down into specifics (unless you want to), just high level talks.
Thanks!
Hadoop is not suitable for usecases which needs real time querying and processing. Hadoop is best when used for offline batch processing and data analysis. You can refer to following link - Common Questions for getting some of you questions answered. You dont have a schema concept in HDFS, which is the filesystem in Hadoop. Data is stored in blocks on disk as a regular file.
I would suggest you visit apache hadoop to learn what is hadoop and in which use case it fits best.
If you are looking for SQL on Hadoop solution that is performant, then check out InfiniDB.
http://infinidb.co
We are a 4th generation columnar MPP engine behind MySQL. We can sit on top of HDFS, GlusterFS or on your local system, so we can be on Hadoop or not, your choice. We are fully open source, GPLv2, there is no difference between the open source version and the enterprise version, use it as you want, scale as you need.
We operate in the interactive SQL area, many people use us for analytical queries against their data. Hadoop MapReduce is great at batch work and transformations, but falls short on the interactive side of things, and that is where solutions like InfiniDB come in.
While you are on Oracle and using Oracle SQL, there may not be much difference between that and the MySQL syntax we support, depends on all the features of Oracle you are using. Many people use us a drop in replacement with their existing MySQL database to start to get the performance of having a cluster MPP database. Also transitioning to Hadoop as you mentioned is another use case, as we can provide the SQL interface for your applications to not even realize they are working on top of a Hadoop cluster.
Feel free to contact me if you have any questions / comments.
I want to make a java application with a rather small database. The pc on which I want to install this has nothing of database stuff on his pc (no wamp server, no oracle, nothing...). I'm rather new in this kind of stuff, and i don't know if it's already been asked but this is what i want to accomplish
Now I have a couple of questions:
Is this doable?
What should I use? Mysql, Oracle,...
How can i do this?
I hope this is enough to get a decent answer.
Yes, it is doable.
For use with Java, I strongly recommend Apache Derby because
you have the huge flexibility of being able to choose between embedded and client-server db, with no code refactoring needed to change data access mode
over H2 or HSQLDB: according to my experience I've found Apache Derby
to be much more reliable/resilient (other embedded DBMSs tend to break more than derby when power fails)
to eat up less RAM
to have better performance on bigger deployments (lots of rows, lots of data [in microbenchmarks with little real-world data H2 and HSQLDB can actually score better]).
to be particularly fast with select queries in heavily multithreaded environments
over MySql and PostgreSql
it's actually faster, when you are not CPU/network -bound, because I've seen it perform better than them in many cases (especially with bigger DBs -- say 10GB) when it comes to filesystem access (MySql and PostgreSql, however, are more efficient in terms of CPU/network utilization, when these are a constraint)
over MySql, PostgreSql, Oracle db, etc.
it's surprisingly fast (often faster), with very big DBs (say, 30 GB) -- something one wouldn't expect from a DBMS you can embed in any application with no deployment/configuration
To get started, see
Apache Derby Getting Started guide
Apache Derby tutorial
Apache Derby FAQs
WorkingWithDerby wiki
If you don't need clients from the network to remotely connect to your database, an "embedded database" is what you want to implement.
Flame-preventing disclaimer: all the statements above are according to my very own personal experience, with the projects I've worked on and/or articles/benchmarks that I read and trusted as reliable. Unless otherwise stated (and in fact I'm not stating otherwise anywhere :) ), I'm referring to fresh out-of-the-box un-fine-tuned installations.
You should probably use an embedded database like H2 or HSQLDB. They are just a simple libraries that you drop in your application, but they provide exactly the same JDBC interface.
You can use the full power of SQL database without any external dependencies. H2, my personal favourite, allows you to create in-memory as well as persistent databases, you can optionally connect to it using socket, it can expose web interface over default 8082 port, so on and so on. On my developer machine I don't even have "normal" database installed, I always use H2.
HSQL or use one of the SQLite JDBC adapters.
I recommend using Derby database. It is very simple to embed in java application.
How is this computer's hardware? What CPU memory and hard-disk?
What is the OS? Do you have the administrator/root access?
If you have a typical PC with windows OS, and enough CPU, memory and hard-disk.
I recommend you to install mysql. Just download the mysql for your OS and install it.
Download link:
http://www.mysql.com/downloads/mysql/
Here is install documents:
http://dev.mysql.com/doc/refman/5.5/en/installing.html
Good luck.
I'm looking for the best database software for a new open source application. The primary criteria is it has to be lightning fast for searching among tens of thousands of entries. Ideally it would be entirely Java based but simply having a Java API is OK. I'm looking to license under GPL so the project would have to be compatible with that. So far SQLite seems to be the most ubiquitous solution but I don't want to overlook something else if it could turn out to be better.
When I search the general internet, most results seems to be for object databases. I don't care if the database is object-based or relational, and I don't think I care if it's "NoSQL" . I have lots of experience with MySQL but I'm not terribly afraid of learning a new query language or interface if it's faster that way. The main kind of data this will be managing is filenames with at least 20 metadata fields attached; I'd want to have multiple datasets with the same fields, and it would be nice to also store some application preferences in the database.
I see from some responses that there may be confusion about my (former) use of "embedded" in the title. I want to clarify that I mean "embedded in the application and redistributed" and not "in use on an embedded device." The application is currently targeting full scale computers, although one reason for "ideally it would be entirely java based" is a dreamy aspiration of creating an Android version.
Ultimately it really depends on your application. SQLite is not designed to be as robust as standard client\server databases like Oracle and MySQL. From the FAQ for SQLite they say the following on the subject:
However, client/server database engines (such as PostgreSQL, MySQL, or Oracle) usually support a higher level of concurrency and allow multiple processes to be writing to the same database at the same time. This is possible in a client/server database because there is always a single well-controlled server process available to coordinate access. If your application has a need for a lot of concurrency, then you should consider using a client/server database. But experience suggests that most applications need much less concurrency than their designers imagine.
That being said SQLite is very fast but then again this depends on how you'll be using it and on what platforms. If you are running on an embedded device you may see significant performance differences than when running on a regular desktop\server which is why its hard to give a exact answer. SQlite does see significant performance gains from not abiding to the standard client\server model.
Your best bet is to pick a few, like SQLite, PostgreSQL, MySQL, and see the performance implications of each by running some tests which simulate common scenarios you will encounter in you application.
Take a look at http://www.polepos.org/ there is a benchmark which clains thathttp://www.db4o.com/
is one of the fastest embedded dbs.
I personally worked with db4o and its very nice and its licensed under GPL so it should possibly fit your needs
I'm debeloping a Java Swing application, which persists the information through Hibernate, currently using PostgreSQL. Now, I would like to simplify the database setup on the client and I'm thinking about single file databases. The database I'm using is rather small and simple, so there are no special requirements. I'm only asking for one that is stable and reliable.
What solutions are there available which are compatible with Hibernate? The final application shouldn't have any installation steps such as installing a database server or so. The perfect scenario would be a .zip file that is unzipped and everything is ready to go, like Eclipse does.
Thanks!
Java database such as JavaDB (aka Derby), HSQLDB or the more recent H2 (by the original author of HSQLDB) in embedded mode are all possible candidates. Between them JavaDB tend to be considered as the most robust (full ACID, robustness against failures, unlimited db size etc) and would be my choice.
Here is a quick summary of the reasons:
H2 is very interesting and I use it for testing (its compatibility mode feature is really great) but I'm not sure of its maturity for production use.
H2 and HSQLDB are faster than Derby because they do not sync data to disk on commit - while Derby does. So performance comes from the lack of Durability. If you change this behavior, they all hit the some bottleneck: disk IO.
HSQLDB only supports dirty read, transactions see uncommitted values from other transactions.
HSQLDB is not fully Atomic, a transaction can be partially(WTF?) committed
Derby is more powerful, has specs compliant drivers, scales well.
Don't misinterpret me, I'm not saying HSQLDB is bad but you need to know what you're dealing with and in which context/application. Sometimes speed is more important (e.g. for unit testing), and sometimes it is data integrity. If you are in the latter case, Java DB is IMO a better choice.
Some of the references below are 2/3 years old but I'm not aware of any revolution. If I made some mistake, let me know, I'll be happy to update my answer to make it accurate.
References
Database Comparison Table : H2, HSQLDB, DERBY, PostgreSQL, MySQL
HSQLDB or JavaDB ???
H2 Performance... "Notes"
Derby vs. Hsqldb for Embedding in a Swing App?
What is the best embedded database for Java?
hsqldb - who needs acid!!
I have used HSLQDB in conjunction with hibernate for long. It is really fast, easy to use application for basic small db requirements.
My workmate and I are trainees and we got an exercise to realize a project. We have decided us to create a customer management in Java. Now we have to choose a database. We are able to use Oracle, MySQL, PostgreSQL, HSQLDB and of course other Open Source databases.
So, what database is recommend for us?
I thought Oracle is too complex for our small project, isn´t it?
Thank you in advance!
finsterr
Why not just use the database bundled with Java 6 ? JavaDb is originally a production-quality database from IBM called Derby, and works well.
Does your company already use Oracle and have people who are expert in it to go to when you get stuck? Then I would use it (if you can learn Oracle, all the other dbs are easy in comparison, take advatage of the resources if you have them). If not then use one of the others.
More important than which database you use is to get up to speed on relational database design before you start to put this together. Here's a starting place:
http://www.deeptraining.com/litwin/dbdesign/FundamentalsOfRelationalDatabaseDesign.aspx
another good read
Database development mistakes made by application developers
You will save yourself quite a bit of tedious setup and platform dependencies if you use an embeddable database written in Java. Apache Derby (in any of its incarnations) would be a good start.
It is strongly recommended to use a database abstraction layer like Hibernate to avoid having raw SQL in your code. This will allow you to chose a database at deployment time, allowing you to scale effortlessly.
I would agree that Oracle is probably a bit much, however since you gave so little details I cannot really say. I would recommend MySQL and it would probably work well and be nice to the budget however you need to give more information.
Do you need the ability to distribute your software? If so, the license will be a part of the decision-making process. Of the ones you listed, Postgresql is the easiest to work with in terms of licensing, essentially having a "do whatever you want to with it" license.
Depending on the size/scope of your project, SQLite may be a good fit.
Firebird can be a good database for project like you :
small footprint
easy to administrate
free and opensource
good driver for Java (Jaybird)