We are developing an application which uses JDBC API. We deal with lot of queries
for various use cases. Any idea on how to isolate these SQL queries out of the
code? Is there any pattern or configuration API would help to manage these queries?
I'm not quite sure what you're asking, but perhaps setting up some stored procedures would provide the layer of abstraction you desire...
You could set up stored procedures for each usage case, and then the queries in your code would consist simply of calling the various procedures.
You can use the MyBatis SQL Mapper for this. You segregate your SQL statements into XML files. It's very clean and very intuitive if you are familiar with SQL.
Use a config/resource file and read the query string from it? A HashTable would be another option as well.
You can try to place your queries into separate files as is and then put files' names into any configuration file.
Use http://www.mybatis.org/ to externalise the queries into XML files
Ensure that classes that access the database only do this, and not other logic
Related
I have an existing process I'm using to track SQL changes - essentially it's just a lot of SQL changes that I track and move around to different environments. I'm wanting to start tracking these changes through Liquibase. Everything looks good from what I've seen, but I'm not seeing any way to generate ChangeSets from Java. I would think there's a way using the Liquibase jar that I could leverage to write out ChangeSets without handling the formatting myself. Essentially my first implementation is just going to be writing the raw SQL changes out into a Liquibase format - I have the raw SQL, and an initial ChangeLog file, but I'm not sure how to write this out without doing it manually.
Does anybody know if Liquibase has any existing logic that does this sort of thing? Or does anybody have any other options to help me do this without managing formatting myself? The format I'm going to be using is JSON.
Changessets can not be generated from java. But there are is a way to generate changesets from an existing database.
In our applications we are using hibernate to create the table structure. After the database is up to date we use the generateChangelog method of liquibase to generate the changesets. There is also a way to generate a change log by different two tables.
I want to or need to (without the use of other databases) setup Entities(database tables) in memory that have relationships, like one-to-many or many-to-many etc.
I saw something related here on this forum:
Map SQL (not JPQL) to a collection of simple Java objects?
I need to query these Entities that have relationships and get the resultsets from this,
in order to push the resulting data into an Access database, I am using Jackcess and its not a JDBC driver.
So far I have looked at MetaModel and jOOQ.
Is there anything else out there. I have a little bit of exposure to ORM's, do they query the in-memory collections or only just pass the sql query to the database.
Any help or suggestions is greatly appreciated.
Apparently, you're looking for something like .NET's LINQ-to-Objects in the Java ecosystem. There's nothing as sophisticated as LINQ-to-Objects, but there are a couple of ways to "query" collections in Java as well. You might be interested in any of these libraries:
Quaere: http://quaere.codehaus.org
Coolection: https://github.com/wagnerandrade/coollection
Lambdaj: https://code.google.com/p/lambdaj
JXPath: http://commons.apache.org/proper/commons-jxpath
JoSQL: http://josql.sourceforge.net
All of the above projects are open source and may not be so actively maintained anymore, as Java 8 will introduce a much better collections API along with language-supported lambda expressions, which renders these non-SQL focused LINQesque Java APIs obsolete.
Note, you were asking specifically about MetaModel and jOOQ. These provide you with a querying API for querying databases. I think that will not help you much for your use-cases.
Hibernate will query the object cache, but only if you query using Criteria or HQL. If you query straight SQL, it'll get run directly against the database.
Your problem description sounds like it's more than Jackcess can handle natively, but what if at program startup you read the full Access DB into an in-memory database (one that has a JDBC driver), run Hibernate queries against that in-memory database, and then at program exit just flush all Hibernate changes to the in-memory database and then write the in-memory database's contents into the Access database? You get all the complicated querying capability of Hibernate, and all you have to do is write Jackcess-to-JDBC code to load the Access DB into an equivalent schema in the in-memory database and then the inverse code to copy it back, which is way easier than writing the full JDBC driver for Jackcess.
I'd like to save persistent objects to the file system using Hibernate without the need for a SQL database.
Is this possible?
Hibernate works on top of JDBC, so all you need is a JDBC driver and a matching Hibernate dialect.
However, JDBC is basically an abstraction of SQL, so whatever you use is going to look, walk and quack like an SQL database - you might as well use one and spare yourself a lot of headaches. Besides, any such solution is going to be comparable in size and complexity to lighweight Java DBs like Derby.
Of course if you don't insist absolutely on using Hibernate, there are many other options.
It appears that it might technically be possible if you use a JDBC plaintext driver; however I haven't seen any opensource ones which provide write access; the one I found on sourceforge is read-only.
You already have an entity model, I suppose you do not want to lose this nor the relationships contained within it. An entity model is directed to be translated to a relational database.
Hibernate and any other JPA provider (EclipseLink) translate this entity model to SQL. They use a JDBC driver to provide a connection to an SQL database. This, you need to keep as well.
The correct question to ask is: does anybody know an embedded Java SQL database, one that you can start from within Java? There are plenty of those, mentioned in this topic:
HyperSQL: stores the result in an SQL clear-text file, readily imported into any other database
H2: uses binary files, low JAR file size
Derby: uses binary files
Ashpool: stores data in an XML-structured file
I have used HyperSQL on one project for small data, and Apache Derby for a project with huge databases (2Gb and more). Apache Derby performs better on these huge databases.
I don't know exactaly your need, but maybe it's one of below:
1 - If your need is just run away from SQL, you can use a NoSQL database.
Hibernate suports it through Hibernate OGM ( http://www.hibernate.org/subprojects/ogm ).
There are some DBs like Cassandra, MongoDB, CouchDB, Hadoop... You have some suggestions Here
.
2 - Now, if you want not to use a database server (with a service process running always), you can use Apache Derby. It's a DB just like any other SQL, but no need of a server. It uses a singular file to keep data. You can easily transport all database with your program.
Take a look: http://db.apache.org/derby/
3 - If you really want some text plain file, you can do like Michael Borgwardt said. But I don't know if Hibernate would be a good idea in this case.
Both H2 and HyperSQL support embedded mode (running inside your JVM instead of in a separate server) and saving to local file(s); these are still SQL databases, but with Hibernate there's not many other options.
Well, since the question is still opened and the OP said he's opened to new approaches/suggestions, here's mine (a little late but ok).
Do you know Prevayler? It's a Java Prevalence implementation which keep all of your business objects in RAM and mantain Snapshots/Changelogs in the File System, this way it's extremely fast and reliable, since if there's any crash, it'll restore it's last state and reapply every change to it.
Also, it's really easy to setup and run in your app.
Ofcourse this is possible, You can simply use file io features of Java, following steps are required:-
Create a File Object
2.Create an object of FileInputStream (though there are ways which use other Classes)
Wrap this object in a Buffer object or simply inside a java.util.Scanner.
use specific write functions of the object created in previous step.
Note that your object must implement Serializable interface. See following link,
Is there any way for us to query the db to suggest index creation/index deletion that would improve the performance of the db system?
We understand that a dba can manually view the trace files to create/drop indices but can i write a java program that queries the db engine to suggest the same automatically.
Or some open source tools that i can check out to perform the same automatically.
Thx.
Well there's no standard JDBC way to do this. There may be specific driver implementations for specific DBS that would allow you to EXPLAIN your query (trace the use of indexes), etc. But there's no one-size fits all answer here.
in general I would lean to saying NO.
Your index performance is dependent on the queries you would fire on them...So No !
With MySQL specifically, you can flag slow queries, as well as queries not using indexes.
Ultimately, the database will do its best (within what you've created) to optimize your query.
http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html
This data would be logged to a file, and not necessarily available via an API.
I've found numerous posts about reading CSV with Java and the APIs they were pointing at all had a line-oriented approach when it came to reading a CSV file. Something like "while you get a line, get the values of every column".
I'd appreciate a higher-level API, like in Perl where DBI allows you to use SQL on CSV like if it where a DB table. Otherwise I'll have to implement lots of access logic by myself.
Is there such an API? Am I missing something? There are some references about JDBC drivers but most are projects that haven't been updated the last 5 years.
You can use HSQL in order to do it, see the following links from the docs and a blog post describing exactly that.
You could give H2Database a go - it is rather heavy weight, but at least it is maintained.
Are you trying to avoid using a regular database for accessing CSV? MySQL supports CSV as one of it's table types if you are open to using a database system.
yes, JDBC with a CSV driver. You can try implement yourself or just try something like HXTT