On mixing Java and SQL queries, how to do best? [duplicate] - java

This question already has answers here:
Java Programming - Where should SQL statements be stored? [closed]
(15 answers)
Closed 9 years ago.
As part of my Java program, I need to do a run a lot of queries against (Oracle) database.
Currently, we create a mix SQL and Java, which (i know) is a bad bad thing.
What is a right way to handle something like this? If possible, include examples.
Thank you.
EDIT:
A bit more information about the application. It is a web application that derives content mainly from the database (it takes user input and paints content to be seen next based on what database believes to be true).
The biggest concern I have with how it's done today is that mixing Java code and a SQL queries look "out-of-place" when coupled as tightly as it is (Queries hardcoded as part of source code)
I am looking for a cleaner way to handle this situation, which would improve maintainability and clarity of the project at hand


For what you've described, incorporating an object relational mapper (ORM) or rewriting as stored procedures is probably more work than you want to embrace. Both have non-trivial learning curves.
Instead a good practice is consolidating SQL in a class per table or purpose. Take a look at the table data gateway object and the data access object design patterns to see how this is done in practice.
The upshot of this approach is myriad. You are better positioned for reuse because queries are in one spot. Client code becomes more readable as you replace several lines of JDBC and SQL with a method call (e.g. userTableDataGateway.getContentToShow(pageId)). Finally, this will help you see the problem more clearly an ORM helps solve.


Well, one thing you could consider is an Object Relational Mapper (for example, Hibernate). This would allow you to map your database schema to Java objects, which would generally clean up your Java code.
However, if performance and speed is of the essence, you might be better off using a plain JDBC driver.
This would of course also be dependent upon the task your application is trying to accomplish. If, for example, you need to do batch updates based on a CSV file, I migh go with a pure JDBC solution. If you're designing a web application, I would definitely go with an ORM solution.
Also, note that a pure JDBC solution would involve having SQL in your Java code. Actually, for that matter, you would have to have some form of SQL, be it HQL, JPQL, or plain SQL, in any ORM solution as well. Point being, there's nothing wrong with some SQL in your Java application.
Edit in response the OP's edits
If I were writing a web application from scratch, I would use an ORM. However, since you already have a working application, making the transition from a pure JDBC solution to an ORM would be pretty painful. It would clean up your code, but there is a significant learning curve involved and it takes quite a bit of set-up. Some of the pain from setting-up would be alleviated if you are working with some sort of bean-management system, like Spring, but it would still be pretty significant.
It would also depend on where you want to go with your application. If you plan on maintaining and adding to this code for a significant period, a refactor may be in order. I would not, however, recommend a re-write of your system just because you don't like having SQL hard-coded in your application.


Based on your updates, I concur with Tim Pote's edits re: the learning curve to integrate ORM. However, instead of integrating ORM, you could do things like using prepared statements, which you in turn store in a properties file. Or even store your queries in the DB so that you can make subtle updates to them that can then be read in immediately without restarting your app server. Both of these strategies would declutter your Java code of hard-coded SQL.
Ultimately though, I don't think there's a clear answer to your question, because there's nothing inherently wrong with what you're doing. It's just a bit inflexible, but perhaps acceptably so for your circumstances.
That said, I'm posting this as an answer!


I'm not sure of the state of the project but you may also be able to find an 'alternate' object relational mapper called MyBatis. It has a lower learning curve than the popular hibernate or eclipselink and let's you actually write the queries so you know what the code is doing. That is if ORM is your thing.
I'm working with JPA right now (mainly because it is the current trend and it needs to be learned). JPA is the Java standard for ORM. If you are going to learn what is currently a typical ORM way of doing things, JPA is probably the best way to go. Frameworks like Hibernate and Eclipselink drive it. Depending on what framework you choose to underpin your JPA app, you can use proprietary features but that will tie you to that framework pretty much for good. JPA is not hard to start using, but can be very cryptic when it doesn't work since it obfuscates the interaction with the database quite a bit (mind you, it does allow the option using native SQL queries, but that kind of negates the reason why people say JPA style DB access is good).
And yes, there are still people using JDBC with prepared statements. And normally there are practices/patterns that you will use when programming with plain old JDBC that act like a very, very minimalist ORM... or really, closer to MyBatis. Again, if you go this route, use prepared statements. They negate a number of dangers.
This is a religious kind of question, so you will hear a lot of proselytizing the way you wrote the question. In fact someone might shoot down your question for this. I think the only thing you could ask that might be worse is whether emacs or vi is better to a crowd of unix geeks.


Your question seems too generic, however if you have a mix of Direct SQL on Oracle and Java SQL, it would be better to invest some time in an ORM like Hibernate or Apache Cayenne. The ORM is a separate design approach to segregate Database operations from the Java side. All the db interactions and DB design is implemented on the ORM and all the access and business logic will reside in Java, this is a suggestion. Still unclear about your actual problem though.


The biggest concern I have with how it's done today is that mixing
Java code and a SQL queries look "out-of-place" when coupled as
tightly as it is (Queries hardcoded as part of source code)
This assumption of yours is not really "correct" in a way that there is going to be a true / false answer to your question. This question here explains that there are several ways of dealing with mixing Java and SQL:
Java Programming - Where should SQL statements be stored?
It essentially distinguishes between SQL being:
Hardcoded in business objects
Embedded in SQLJ clauses
Encapsulated in separate classes e.g. Data Access Objects
Metadata driven (decouple the object schema from the data schema - describe the mappings between them in metadata)
Put into external files (e.g. Properties or Resource files)
Put into stored procedures
I'll add to that:
Embedded in CriteriaQuery statements
Embedded in jOOQ statements.


Apache Cayenne, is one of the easiest ORM to use. It comes with a Cayenne Modeller to Model data objects and does mappings. I would recommend Cayenne for a beginner in ORM. It can create mapping classes and DB sync through the modeller.

Related

Java H2 Database Framework

I know this is a newb question, but that's what I am so here goes.
I am writing an application in java that has a lot of H2 database queries so far I have written methods that pull the data I need from the database with queries, because this is the only way I know how.
My question is, is there an easier way to go about getting data from my database that would be more efficient and make things less work. In my research Spring does something like this, but if it does I have been unable to find good information on how to implement it.
Thanks,

I would say there is even better approach called Java Persistence API. It will make your code ORM agnostic and provide some flexibility.
JPA 2.0 is quite rich and will satisfy all your needs. So I do not think you should use Hibernate directly, instead you should try to use JPA where you can. Please note, Hibernate is JPA 2.0 provider.
Please see the following example Creating Good DAOs with Hibernate 3.5 and JPA 2.0 Annotations

There are many options. As ShyJ wrote, Spring Data JPA is one. Many people use Hibernate. There are other libraries you could use, for example SimpleORM.
But I wonder if "which one is better" is the right type of question for StackOverflow. There are many ways to do it "right", and many things to consider.

I am also using H2 rather heavily in a large environment. My advice is to use JPA and particularly Hibernate as it is one of the most popular implementation.
What you want to avoid is writing native sql as if you are going to change a database (if you are) you will run into numerous problems with native sql. JPA solves it by defining JPQL which is like SQL, but will work on any database.
Another great benefit from hibernate is the possibility of using L2 cache which can speed up your application drasticaly.
The last benefit is perhaps most relevant to you- it may take you slightly longer to set up, but once its there, it is much easier to work with the database from pure java.

Java - JDBC alternatives [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
this is just theoretical question.
I use JDBC with my Java applications for using database (select, insert, update, delete or whatever).
I make "manually" Java classes which will contain data from DB tables (attribute = db column). Then I make queries (ResultSet) and fill those classes with data. I am not sure, if this is the right way.
But I've read lot of about JDO and another persistence solutions.
Can someone please recommend the best used JDBC alternatives, based on their experience?
I would also like to know the advantages of JDO over JDBC (in simple words).
I've been able to google lot of this stuff, but opinions from the "first hand" are always best.
Thanks

The story of database persistence in Java is already long and full of twists and turns:
JDBC is the low level API that everybody uses at the end to talk to a database. But without using a higher level API, you have to do all the grunt work yourself (writing SQL queries, mapping results to objects, etc).
EJB 1.0 CMP Entity Beans was a first try for a higher level API and has been successfully adopted by the big Java EE providers (BEA, IBM) but not by users. Entity Beans were too complex and had too much overhead (understand, poor performance). FAIL!
EJB 2.0 CMP tried to reduce some of the complexity of Entity Beans with the introduction of local interfaces, but the majority of the complexity remained. EJB 2.0 also lacked portability (because the object-relational mapping were not part of the spec and the deployment descriptor were thus proprietary). FAIL!
Then came JDO which is a datastore agnostic standard for object persistence (can be used with RDBMS, OODBMS, XML, Excel, LDAP). But, while there are several open-source implementations and while JDO has been adopted by small independent vendors (mostly OODBMS vendors hoping that JDO users would later switch from their RDBMS datastore to an OODBMS - but this obviously never happened), it failed at being adopted by big Java EE players and users (because of weaving which was a pain at development time and scaring some customers, of a weird query API, of being actually too abstract). So, while the standard itself is not dead, I consider it as a failure. FAIL!
And indeed, despite the existence of two standards, proprietary APIs like Toplink, an old player, or Hibernate have been preferred by users over EJB CMP and JDO for object to relational database persistence (competition between standards, unclear positioning of JDO, earlier failure of CMP and bad marketing have a part of responsibility in this I believe) and Hibernate actually became the de facto standard in this field (it's a great open source framework). SUCCESS!
Then Sun realized they had to simplify things (and more generally the whole Java EE) and they did it in Java EE 5 with JPA, the Java Persistence API, which is part of EJB 3.0 and is the new standard for object to relational database persistence. JPA unifies EJB 2 CMP, JDO, Hibernate, and TopLink APIs / products and seems to succeed where EJB CMP and JDO failed (ease of use and adoption). SUCCESS!
To summarize, Java's standard for database persistence is JPA and should be preferred over others proprietary APIs (using Hibernate's implementation of JPA is fine but use JPA API) unless an ORM is not what you need. It provides a higher level API than JDBC and is meant to save you a lot of manual work (this is simplified but that's the idea).

If you want to write SQL yourself, and don't want an ORM, you can still benefit from some frameworks which hides all the tedious connection handling (try-catch-finally). Eventually you will forget to close a connection...
One such framework that is quite easy to use is Spring JdbcTemplate.

I can recommend Hibernate. It is widely used (and for good reasons), and the fact that the Java Persistence API specification was lead by the main designer of Hibernate guarantees that it will be around for the foreseeable future :-) If portability and vendor neutrality is important to you, you may use it via JPA, so in the future you can easily switch to another JPA implementation.
Lacking personal experience with JDO, I can't really compare the two. However, the benefits of Hibernate (or ORM in general) at first sight seem to be pretty much the same as what is listed on the JDO page. To me the most important points are:
DB neutrality: Hibernate supports several SQL dialects in the background, switching between DBs is as easy as changing a single line in your configuration
performance: lazy fetching by default, and a lot more optimizations going on under the hood, which you woulds need to handle manually with JDBC
you can focus on your domain model and OO design instead of lower level DB issues (but you can of course fine-tune DML and DDL if you wish so)
One potential drawback (of ORM tools in general) is that it is not that suitable for batch processing. If you need to update 1 million rows in your table, ORM by default will never perform as well as a JDBC batch update or a stored procedure. Hibernate can incorporate stored procedures though, and it supports batch processing to some extent (I am not familiar with that yet, so I can't really say whether it is up to the task in this respect compared to JDBC - but judging from what I know so far, probably yes). So if your app requires some batch processing but mostly deals with individual entities, Hibernate can still work. If it is predominantly doing batch processing, maybe JDBC is a better choice.

Hibernate requires that you have an object model to map your schema to. If you're still thinking only in terms of relational schemas and SQL, perhaps Hibernate is not for you.
You have to be willing to accept the SQL that Hibernate will generate for you. If you think you can do better with hand-coded SQL, perhaps Hibernate is not for you.
Another alternative is iBatis. If JDBC is raw SQL, and Hibernate is ORM, iBatis can be thought of as something between the two. It gives you more control over the SQL that's executed.

JDO builds off JDBC technology. Similarly, Hibernate still requires JDBC as well. JDBC is Java's fundamental specification on database connectivity.
This means JDBC will give you greater control but it requires more plumbing code.
JDO provide higher abstractions and less plumbing code, because a lot of the complexity is hidden.
If you are asking this question, I am guessing you are not familiar with JDBC. I think a basic understanding of JDBC is required in order to use JDO effectively, or Hibernate, or any other higher abstraction tool. Otherwise, you may encounter scenario where ORM tools exhibit behavior you may not understand.
Sun's Java tutorial on their website provide a decent introductory material which walks you through JDBC. http://java.sun.com/docs/books/tutorial/jdbc/.

Have a look at MyBatis. Often being overlooked, but great for read-only complex queries using proprietary features of your DBMS.
http://www.mybatis.org

Here is how it goes with java persistence. You have just learnt java, now you want to persist some records, you get to learn JDBC. You are happy that you can now save your data to a database. Then you decide to write a bit bigger application. You realize that it has become tedious to try, catch , open connection, close connection , transfer data from resultset to your bean .... So you think there must be an easier way. In java there is always an alternative. So you do some googling and in a short while you discover ORM, and most likely, hibernate. You are so exited that you now dont have to think about connections. Your tables are being created automatically. You are able to move very fast. Then you decide to undertake a really big project, initially you move very fast and you have all the crud operations in place. The requirements keep comming, then one day you are cornered. You try to save but its not cascading to the objects children. Somethings done work as explained in the books that you have read. You dont know what to do because you didnt write the hibernate libraries. You wish you had written the SQL yourself. Its now time to rethink again... As you mature , you realize that the best way to interact with the Database is through SQL. You also realize that some tools get you started very fast but they cant keep you going for long. This is my story. I am now a very happy ibatis/User.

Ebean ORM is another alternative http://ebean-orm.github.io/
Ebean uses JPA Annotations for Mapping but it is architected to be sessionless. This means that you don't have the attached/detached concepts and you don't persist/merge/flush - you just simply save() your beans.
I'd expect Ebean to be much simplier to use than Hibernate, JPA or JDO
So if you are looking for a powerful alternative approach to JDO or JPA you could have a look at Ebean.

JPA/Hibernate is a popular choice for ORM. It can provide you with just about every ORM feature that you need. The learning curve can be steep for those with basic ORM needs.
There are lots of alternatives to JPA that provide ORM with less complexity for developers with basic ORM requirements. Query sourceforge for example:
http://sourceforge.net/directory/language:java/?q=ORM
I am partial to my solution, Sormula: sourceforge or bitbucket. Sormula was designed to minimize complexity while providing basic ORM.

Hibernate, surely. It's popular, there is even a .NET version.
Also, hibernate can be easily integrated with Spring framework.
And, it will mostly fit any developer needs.

A new and exciting alternative is GORM, which is the ORM implementation from Grails. Can now be used stand alone.
Under the hood it uses Hibernate, but gives you a nice layer on top with cool dynamic finders etc.

All these different abstraction layers eventually use JDBC. The whole idea is to automate some of the tedious and error prone work much in the same way that compilers automate a lot of the tedious work in writing programs (resizing a data structure - no problem, just recompile).
Note, however, that in order for these to work there are assumptions that you will need to adhere to. These are usually reasonable and quite easy to work with, especially if you start with the Java side as opposed to have to work with existing database tables.
JDO is the convergence of the various projects in a single Sun standard and the one I would suggest you learn. For implementation, choose the one your favorite IDE suggests in its various wizards.

There is also torque (http://db.apache.org/torque/) which I personally prefer because it's simpler, and does exactly what I need.
With torque I can define a database with mysql(Well I use Postgresql, but Mysql is supported too) and Torque can then query the database and then generate java classes for each table in the database. With Torque you can then query the database and get back Java objects of the correct type.
It supports where clauses (Either with a Criteria object or you can write the sql yourself) and joins.
It also support foreign keys, so if you got a User table and a House table, where a user can own 0 or more houses, there will be a getHouses() method on the user object which will give you the list of House objects the user own.
To get a first look at the kind of code you can write, take a look at http://db.apache.org/torque/releases/torque-3.3/tutorial/step5.html which contains examples which show how to load/save/query data with torque. (All the classes used in this example are auto-generated based on the database definition).

I recommend to use the Hibernate, its really fantastic way of connecting to the database, earlier there were few issues, but later it is more stable.
It uses the ORM based mapping, it reduces your time on writing the queries to an extent and it allows to change the databases at a minimum effort.
If you require any video based tutorials please let me know I can uplaod in my server and send you the link.

Use hibernate as a stand alone JAR file then distribute it to your different web apps. This far is the best solution out there. You have to design your Classes, Interfaces, Enums to do an abstract DAO pattern. As long as you have correct entities and mappings. You will only need to work with Objects(Entities) and not HSQL.

Suggestion for annotation only ORM framework (Java)

I'm working on a medium-sized project in Java (GWT to be precise), and I'm still in the process of deciding what ORM to use.
I just refuse to write SQL queries unless utterly and completely necessary (not the case :D)
I want to use ONLY annotations, no XML configuring [except database location, username, etc], and I DON'T want to create any tables or define them. I want this to be done by the framework completely.
Call me lazy, but I like Java/GWT programming, not creating tables and coping with that sort of things, and it's a plus in my assignment if I actually use an ORM :D
I've considered so far:
Hibernate with annotations: I've found little documentation to get started from ground using this. I've found little examples and alike. It's as if they didn't actually want you to use 100% annotations.
DataNucleus
JDO: It seems interesting, I'd never heard of DataNucleus up to until this week, but it seems extremely mature, and I actually discovered it because Google uses it in GWT, so that's a good sign. I also like the fact that they mentioned I don't need to define any tables or columns, though I think hibernate can achieve this as well. I actually enjoyed reading though their documentation (though I haven't finished yet), something quite opposite to hibernate.
JPA I'm not totally sure if DataNucleus/JPA can work with annotation-only configuration, though I might need to take a deeper look into the documentation.
As you might guess, I'm quite inclined to JDO... but it'd be nice to hear what people who've used it have to say vs the other alternatives, and if i'm missing some very important point here.
Edit 1: I know I'll need to XML the database location/usr/pwd, I meant I don't want to use an XML to configure the mapping or database schema.

JPA (1 and 2) is pretty much XML free, depending on how it's packaged. You most certainly don't need it for the schema. It also supports annotations for details when the tables are generated.
The only issue with these is that while they can create a database, they're a DB MAPPING tool, not a DB DEFINITION tool. Specifically, most won't allow you to create the arbitrary indexes that you may well need to get the DB tuned properly to your queries.
But other than that, JPA should fill your needs, and it has a lot of implementations (Hibernate is just one implementation).

This is a self publicizing but I'm been working for a while on a simple Java ORM package called ORMLite. I wanted something much less complicated than hibernate but without writing SQL directly. It's completely annotation based and currently supports MySQL, Postgres, Derby, and H2. Adding other database would be simple if I have access to a server. It is completely annotation based and can create (and destroy) tables.
http://ormlite.com/
It has pretty flexible QueryBuilder and table paging. Joining is, however, not supported.

"Integration" between Rails' ActiveRecord and Java's Hibernate

Hi everybody: let me do a bit of "concept mining" here: I am involved in mantaining/extending an application whose functionality is distributed across several servers. For example, we have a machine running the ApplicationServer, another running the DataServer and so on.
This application has a Web Interface. The current UI is totally implemented in Java, and in a way that makes adding new functionality hard. One of my goals is extending this interface, and we're considering shifting the whole thing to another platform, like Rails, for example.
Problem being, the database that is manipulated by the UI (possibly Rails in the future) is also manipulated by ApplicationServer (Java).
So, my main question is: both Rails and Java can access databases through their own ORM (ActiveRecord for Rails and Hibernate or similar for Java). Is there any way to guarantee that the mappings are consistent?*
Even if the answer is a hard "no", I'd also like to hear your thoughts on how you'd approach this scenario.
I hope the question is clear enough, but warn me if it isn't and I'll edit accordingly. =D
*Edit: per request, I'm extending this explanation: what I mean is, how to make sure things don't break when someone needs to add a new field to the database and edits the Hibernate mapping because of it? I know that Rails "guesses" the entity attributes pretty much by itself (making things easier), but I was wondering if there was some "magical way" to "connect" the ActiveRecord directly to the Hibernate mapping.

Depends on your case and how important it is to actually ensure that things won't break. I would probably code the Rails app to do its best, and then write a good set of db integration test cases for Rails to test against breakage.
Because Hibernate needs a mapping conf whereas Rails uses the database layout directly, it's best to do the db changes on Hibernate/mapped Java class side and then run the test suite on Rails side after changes.

this might be coming too late to the party, but ActiveJDBC is an ActiveRecord- like implementation in Java which reads metadata and configures self pretty much the same as ActiveRecord: http://code.google.com/p/activejdbc/

You should look at using DataMapper instead of ActiveRecord. DataMapper and Hibernate following roughly the same pattern so the mappings would be similar. Also, DataMapper defines the mapping in the class itself rather than figuring it out from the model. This is much closer to Hibernate and you could probably write a simple hbm to dm converter and just eval the output at the top of your model classes. If you didn't design your original data model with Rails in mind, none of the convention over configuration standards are likely to be there; with DataMapper, the default seems to be to map properties and relationships like Hibernate.
Another idea: if you use the Hibernate annotations instead of xml mapping, maybe you could JRuby as the bridge to build the Ruby model from the Java one.
But either way, if you have good tests, it should be obvious when a data model change break something.

What's the best way to persist data in a Java Desktop Application?

I have a large tree of Java Objects in my Desktop Application and am trying to decide on the best way of persisting them as a file to the file system.
Some thoughts I've had were:
Roll my own serializer using DataOutputStream: This would give me the greatest control of what was in the file, but at the cost of micromanaging it.
Straight old Serialization using ObjectOutputStream and its various related classes: I'm not sold on it though since I find the data brittle. Changing any object's structure breaks the serialized instances of it. So I'm locked in to what seems to be a horrible versioning nightmare.
XML Serialization: It's not as brittle, but it's significantly slower that straight out serialization. It can be transformed outside of my program.
JavaDB: I'd considered this since I'm comfortable writing JDBC applications. The difference here is that the database instance would only persist while the file was being opened or saved. It's not pretty but... it does lend itself to migrating to a central server architecture if the need arises later and it introduces the possibility of quering the datamodel in a simpler way.
I'm curious to see what other people think. And I'm hoping that I've missed some obvious, and simpler approach than the ones above.
Here are some more options culled from the answers below:
An Object Database - Has significantly less infrastructure than ORM approaches and performs faster than an XML approach. thanks aku

I would go for the your final option JavaDB (Sun's distribution of Derby) and use an object relational layer like Hibernate or iBatis. Using the first three aproaches means you are going to spend more time building a database engine than developing application features.

Have a look at Hibernate as a simpler way to interface to a database.

In my experience, you're probably better off using an embedded database. SQL, while less than perfect, is usually much easier than designing a file format that performs well and is reliable.
I haven't used JavaDB, but I've had good luck with H2 and SQLite. SQLite is a C library which means a little more work in terms of deployment. However, it has the benefit of storing the entire database in a single, cross-platform library. Basically, it is a pre-packaged, generic file format. SQLite has been so useful that I've even started using it instead of text files in scripts.
Be careful using Hibernate if you're working with a small persistence problem. It adds a lot of complexity and library overhead. Hibernate is really nice if you're working with a large number of tables, but it will probably be cumbersome if you only need a few tables.

db4objects might be the best choice

XStream from codehaus.org
XML serialization/deserialization largely without coding.
You can use annotations to tweak it.
Working well in two projects where I work.
See my users group presentation at http://cjugaustralia.org/?p=61

I think it depends on what you need. Let's see the options:
1) Descarded imediatelly! I'll not even justify. :)
2) If you need a simple, quick, one-method persistence, stick with it. It will persist the complete data graph as it is! Beware of how long you'll be maintaning the persisted objects. As yourself pointed out, versioning can be a problem.
3) Slower than (2), need extra code and can be edited by the user. I would only use it the data is supposed to be used by a client in another language.
4) If you need to query your data in anyway, stick with the DB solution.
Well, I think you had already answered your question :)

Categories