Does anyone know of a sqlite alternative for Java that would support multiple write(s) at the same time?
I'm aware of the option of checking to see if the database is available and pause the attempt to write until it can actually write. But still I'm looking for alternatives where I can really do concurrent writes.
Update (further explanation):
I'm setting up a system where there will be multiple users keying in a single database so I would naturally like the idea of being able to work non-stop instead of having to pause to wait for the database to be available for writing.
Take a look at H2 - it's feature rich embeddable file-based DB.
Related
I am a Java guy, I can work with Oracle Database, I know PLSQL, SQL. But I am not good at managing database servers. I think it is a completely different area.
My question is related to database replication. I googled it, found millions of answers but I am still confused.
I could see many times in my professional carrier that developers create complete (complicated) applications to keep sync the source database schema to a target one. It takes time to develop sync apps and very hard to maintain them, especially in case of any data structure modification for example in tables.
I could see that apps built with JPA, JDBC, Spring, myBatis, and PLSQL as well. Usually, they sync DBs during the night, scheduled by Cron, Quartz, Spring, etc. During the sync process usually, the source DB is only available for querying data, not for inserting and DB constraints and triggers are disabled.
These kinds of custom applications always scare me. I do not believe that there is no general, easy, and official way to keep sync two databases without developing a new application.
Now, I got a similar task and honestly, I would like to write zero lines of code related to this task. I believe that there are recommended and existing solutions, cover this topic offered by the database vendors.
That would be great if you can push me in the right direction. I feel that writing another new DB sync application is not the right way.
I need to focus on Oracle Database sync, but I would be happy to know a general, database vendor-independent way.
There are many ways to perform replication in a Oracle Database. Oracle has two replication techniques in the database "Advanced Replication" and "GoldenGate". GoldenGate us the new perferred method of replication which uses the redo logs files from the database. Both methods are geared for a Oracle DBA.
Often application developers will create a "interface" that will move data from one database to other. A interface is a program ( pl/sql, bash, c, etc ) that runs on a cron (database or system) that wakes on a event to move data. Interfaces are useful when data is needed to be process during replication.
I'm looking for the best database software for a new open source application. The primary criteria is it has to be lightning fast for searching among tens of thousands of entries. Ideally it would be entirely Java based but simply having a Java API is OK. I'm looking to license under GPL so the project would have to be compatible with that. So far SQLite seems to be the most ubiquitous solution but I don't want to overlook something else if it could turn out to be better.
When I search the general internet, most results seems to be for object databases. I don't care if the database is object-based or relational, and I don't think I care if it's "NoSQL" . I have lots of experience with MySQL but I'm not terribly afraid of learning a new query language or interface if it's faster that way. The main kind of data this will be managing is filenames with at least 20 metadata fields attached; I'd want to have multiple datasets with the same fields, and it would be nice to also store some application preferences in the database.
I see from some responses that there may be confusion about my (former) use of "embedded" in the title. I want to clarify that I mean "embedded in the application and redistributed" and not "in use on an embedded device." The application is currently targeting full scale computers, although one reason for "ideally it would be entirely java based" is a dreamy aspiration of creating an Android version.
Ultimately it really depends on your application. SQLite is not designed to be as robust as standard client\server databases like Oracle and MySQL. From the FAQ for SQLite they say the following on the subject:
However, client/server database engines (such as PostgreSQL, MySQL, or Oracle) usually support a higher level of concurrency and allow multiple processes to be writing to the same database at the same time. This is possible in a client/server database because there is always a single well-controlled server process available to coordinate access. If your application has a need for a lot of concurrency, then you should consider using a client/server database. But experience suggests that most applications need much less concurrency than their designers imagine.
That being said SQLite is very fast but then again this depends on how you'll be using it and on what platforms. If you are running on an embedded device you may see significant performance differences than when running on a regular desktop\server which is why its hard to give a exact answer. SQlite does see significant performance gains from not abiding to the standard client\server model.
Your best bet is to pick a few, like SQLite, PostgreSQL, MySQL, and see the performance implications of each by running some tests which simulate common scenarios you will encounter in you application.
Take a look at http://www.polepos.org/ there is a benchmark which clains thathttp://www.db4o.com/
is one of the fastest embedded dbs.
I personally worked with db4o and its very nice and its licensed under GPL so it should possibly fit your needs
I'm developing a Java Desktop Application using JDBC, and I wanted to manage the concurrent access to the Database. Someone told me to use Sessions but after some research it turned out that Sessions are not possible in Desktop app.
This is why I'm asking for some help. You have any ideas on how to manage this thing.
Thanks
From what you described, I recommend you to check for SQL exceptions while trying to insert or update some row that may be already be changed by someone else. In that case maybe you should reload what your app shows to the user so they have up-to-date data. Another option is to show a user-friendly error.
If your app executes several queries (insert, update) in a row, I suggest using transactions. I think the easiest way to set them in a Desktop app is to use the Spring framework, if you are familiar with it.
It is not clear exactly what you mean by manage concurrent access - do you want to avoid multiple select queries to the DB? In that case using SELECT for UPDATE might be an option. If you are looking for more general method in limiting only single user to access the DB at any time, you will have to roll your own locking mechanism in the code I suppose.
So long as each Thread is using a different Connection, there should not be any concurrency issues in the JDBC. There are any number of ways to achieve this. e.g. ThreadLocal or a connection pool.
I don't see how a single desktop app can be accessed by many users. You can have many copies of a desktop app and each user has their own connections. This shouldn't cause an issue. You need to clarify what your concern is.
I'm looking to add a pretty simple SQLite database to an existing Java EE application. I'm very used to using EJBs, ORMs, EntityManager, etc. to work with SQL databases. I've never not used some sort of ORM to work with relational DBs in Java.
I've been "recommended" to use a Java wrapper for SQLite, rather than a JDBC driver - so I'm kind of naked and ORM-less (right?). I'd like to get this up and running quickly.
Context
I've been using an in-memory cache, implemented as a Map, which gets filled with entries linearly over time. At some point, like when the app runs overnight, the cache will use all available heap space. Hence, storing the cache on disk (as a SQLite database) rather than in memory (as a Java Map).
Questions
How should I manage resources like SQLiteConnection? Normally I would let the container worry about all this, but since I'm not using JDBC, I have to do all this !##$%ing, non-value-added stuff manually - right?
Is there a way to implement this cleanly and transparently? I'd like to be able to just swap out an implementing class - e.g. replace FooMapCacheImpl with FooSQLiteCacheImpl.
"[Most] methods are confined to the thread that was used to open the connection". Is there a simple, straightforward way to ensure that I don't try to access a SQLiteConnection from threads other than the one that opened it?
...and the flip side of that question: can I avoid creating a new connection every time I want to read from/write to the database? It seems a bona fide PITA to have to manage connections per-thread rather than, say, per instance, which is how I've been thinking about communicating with databases in the past.
Basically
I'm rather lost when it comes to working with databases in Java/Java EE, without using an ORM.
What's the right way to do it?
I don't think It is too hard to make a front end that would implements Map and save everything to a database using JDBC, but before doing it, think twice about it. The performance of the whole system might be affected badly.
However, if the root cause of your problem is the lack of Heap space, you should take a look at Terracotta's BigMemory. However, it is a commercial (non-free) product.
Terracotta has a pretty good cache framework as well (ehcache) which is opensource. Look at the cookbook, it might be inspiring.
If you want to do everything by hand, and you don't mind using Spring, try spring-jdbc. It is very easy to integrate with any JDBC driver. Take a look at SimpleJdbcTemplate. It does all the boiler plate code for you. You should probably use a connection pool as well, such as commons-dbcp
The easiest SQLite JDBC driver to use is this one. Since it doesn't rely on JNI. It might not be as fast, but for quick testing it is perfect.
If you aren't binded to SQLite, you can take a look at other available JDBC solutions such as hsqldb or derby
I hope this will help you out.
You may also want to look at Berkeley DB Java Edition. It allows you to persist and manage Java objects directly in the library, without requiring an ORM (and the associated overhead). It runs on Android, it's an Java library and can manage data sets ranging in size from very small to very large. It was designed with Java application developers in mind and should be both faster and simpler to use than an ORM+RDBMS solution. You can find more out more about it on our web site at Oracle Berkeley DB Java Edition.
Regards,
Dave
The sqlite4java wrapper is basically a JNI wrapper, it is nowhere near what you want.
An ORM like eclipseLink would anyway be a layer on top of JDBC and the Entity manager would always end up using JDBC accesses.
Instead, sqlite4java allows you to call SQLite in java instead of having to do all the JNI wrapping yourself.
If you want to use an ORM and your preferred entity manager then you should use a JDBC driver and the sqlite4java wiki references a few of them.
Hope this helps.
I'm trying to run several instances of a program accessing a sqlite database in java (always the same file) and actually I don't know whether it's possible or not that several jobs access the same database....
SQLite will, in fact, take care of the locking, and you shouldn't expect concurrency issues. Not any that originate in SQLite, in any case.
However, do note that this solution is totally not scalable. If that is an issue that concerns your application you should check out other DB solutions.
Trying to access a single SQLite database from different processes is perfectly fine (whatever language you are using) as SQLite will take care to ensure proper locking. However, please note that SQLite doesn't handle lock contention particularly well - so if you have multiple processes constantly accessing the database at the same time, you might want to consider a different database or using a single server for accessing the database.