Here I'm referring to http://en.wikipedia.org/wiki/Merge_%28SQL%29
https://stackoverflow.com/a/8553030/304690 offers even a wider list of databases supporting MERGE SQL statement.
http://hibernate.org/docs do not seem to offer references to MERGE.
Is there some way to write entities or configure Hibernate for MERGE statements?
there is session.merge() which does basicly the same using standard sql selects to determine if it is there or not. It has therefor 2 roundtrips but is supported across all sqldatabases.
Related
I have to design a web application to retrieve data from a huge single table with 40 columns and several thousands of rows for select query and few rows/columns for updation.
Can you please suggest me that for faster performance, use of Hibernate is feasible or not as i only have single table and do not have any joins ?
Or should i use jdbc dao ?
database : sql server 2008
java 7
If you use Hibernate right, there's no problem in fetching an arbitrarily large result set. Just avoid from queries (use select ... from ... queries) and use ScrollableResults. If you use plain JDBC, you'll be able to get started quicker because Hibernate needs to be configured first, you need to write the mapping file, etc. but later on it might pay off since the code you write will be much simpler. Hibernate is very good at taking the boilerplate out of client code.
If you want to retrieve several thousand records and pagination is not possible then It might be a performance issue. Because hibernate will create an object against everyone and store it in its persistence context. If you create too many objects, it uses up a lot of memory. For these type of operations JDBC is better. For similar discussion see Hibernate performance issues using huge databases
I've just tested my application under the profiler and found out that sql strings use about 30% of my memory! This is bizarre.
There are a lot of strings like this stored in app memory. This is SQL queries generated by hibernate, note the different numbers and trailing underscores:
select avatardata0_.Id as Id4305_0_,...... where avatardata0_.Id=? for update
select avatardata0_.Id as Id4347_0_,...... where avatardata0_.Id=? for update
Here is the part I can't understand. Why does hibernate have to generate different sql strings with different identifiers like "Id4305_0_" for each query? Why can't it use one query string for all identical queries? Is this some kind of trick to bypass query caching?
I would greatly appreciate if someone would describe me why it happening and how to avoid such resource wasting.
UPDATE
Ok. I found it. I was wrong assuming memory leak, It was my fault. Hibernate is working as intended.
My app created 121(!) SessionFactories in 10 threads, they produced about 2300 instances of SingleTableEntityPersisters. And each SingleTableEntityPersister generates about 15 SQL queries with different identifiers. Hibernate was forced to generate about 345.000 different SQL queries. Everything is fine, nothing weird :)
There is a logic behind the query string that hibernate generates. Its primary aim is to get unique aliases for tables and columns names.
From your query,
select avatardata0_.Id as Id4305_0_,...... where avatardata0_.Id=?
avatardata0_ ==> avatardata is the alias of the table and 0_ is appended to indicate it is the first table in the query. So if it were the second table(or Entity) in the query it should have been shown as avatardata1_. It uses the same logic for the column aliases.
So, this way all the possible conflicts are avoided.
You are seeing theses queries because you have turns on the show_sql flag the configuration. This is intended for the debugging of queries. Once you application started working you are supposed turn it off.
Read more on the API docs here.
I am not much aware of the memory consumption part, but you repeat your tests with the above flag turned off and see if there is any improvement.
Assuming you are using sql server, you might want to check the parameter type declaration for '?', making sure the declaration results in the same, fixed length declaration every time.
Dynamic length parameters would result in separate execution plans for each query. This could possibly comsume a lot of resources. What we see as the same procedure, get's interpreted by sql server as a different query, rendering a separate execution plan.
Thus,
exec myprocedure #p1 varchar(3)='foo'
and
exec myprocedure #p1 varchar(6)='foobar'
would result in different plans. Simply by the fact that the declarations of #p1, differ in size.
There is a lot to know about this behaviour. If the above applies to you, I would recommend you read up on 'parameter sniffing'.
No... you can generate you common query inside the hibernate. The logic behind is to mapping with table and fetch the record from there. It is used common query for all the database. Please create a common query like that :
Example :
select t.Id as Id4305_0_,...... from t where t.Id=?
I need to use an Entity framework with my application, and I have used table - partitions in Oracle database. With simple JDBC, I am able to select data from a specific partition. But I don't know whether I can do the same with hibernate or Eclipse link (JPA). If someone knows how to do that, please do let me know.
usually the select statement in JDBC - SQL is,
select * from TABLE_NAME partiton(PARTITON_NAME) where FIELD_NAME='PARAMETER_VALUE';
How can I do the same with Hibernates or JPA?
Please share at least a link for learning sources.
Thanks!!!
JPA or any other ORM framework does not support Oracle partition tables natively (atleast in my knowledge).
There are different possible solutions though, depending on the nature of your problem:
Refactor your classes so that data that needs to be treated differently in real-life, belongs in a separate class. Sometimes this is called vertical partitioning (partitions are not obtained across rows, rather across columns).
Use Oracle partition tables underneath and use native SQL queries or stored procedures from JPA. This is just a possibile solution (I haven't attempted this).
Use Hibernate Shards. Although the typical use case for Hibernate Shards is not for a single database, it presents a singular view of distributed databases to an application developer.
Related:
JPA Performance, Don't Ignore the Database
EclipseLink supports partitioning or sharding with different options.
You can find more about this and examples here:
http://wiki.eclipse.org/EclipseLink/UserGuide/JPA/Advanced_JPA_Development/Data_Partitioning
Table partitioning is data organization on physical level. In a word, partitioning is a poor man index. Like the later, it is supposed to be entirely transparent to the user. A SQL query is allowed to refer to the entire table, but not partition. Then, it is query optimizer job to decide if it can leverage a certain partition, or index.
We use Oracle on a project and would like to also support MySQL. How close are their SQL dialects?
Is it perhaps even possible to use the same SQL source for both without too many gymnastics?
Details:
We're using iBatis, a persistence manager that cleanly segregates the SQL statements into resource files. But we work at the SQL level, which has its advantages (and disadvantages).
We'd prefer not to move to an object-relational mapper like Hibernate, which would fully shield us from dialect differences.
We've tried hard to keep to a generic subset of Oracle SQL.
There's no PL/SQL.
We don't use stored procedures or triggers (yet, anyway).
We use check constraints, unique constraints, and foreign key constraints.
We use ON DELETE CASCADEs.
We use transactions (done at the iBatis API level).
We call a few Oracle timestamp functions in the queries.
We would use the InnoDB storage engine with MySQL (it supports transactions and constraints).
So what are your thoughts? Would we need to maintain two different sets of iBatis SQL resource files, one for each dialect, or is it possible to have a single set of SQL supporting both MySQL and Oracle?
Final Update: Thanks for all the answers, and especially the pointers to Troels Arvin's page on differences. It's really regrettable that the standard isn't more, well, standard. For us the issues turn out to be the MySQL auto-increment vs. the Oracle sequence, the MySQL LIMIT vs. the Oracle Rowumber(), and perhaps the odd function or two. Most everything else ought to transfer pretty easily, modulo a few edits to make sure we're using SQL-92 as #mjv points out. The larger issue is that some queries may need to be hand-optimized differently in each DBMS.
Expect a few minor bumps on the road, but on whole should be relatively easy.
From the list of features you currently use, there should only be a few synctactic or semantic differences, in general easy to fix or account for. The fact that you do not use PL/SQL and/or Stored Procedures is a plus. A good rule of thumb is to try and stick to SQL-92 which most DBMSes support, in particular both Oracle and MySQL. (Note this is not the current SQL standard which is SQL-2008).
A few of the differences:
"LIMIT" is a famous one: to limit the number of rows to retrieve in the results list, MySQL uses LIMIT n, at the end of the query, Oracle uses RowNumber() in the WHERE clause (which is pain, for you also need to reference it in the SELECT list...)
Some datatypes are different. I think mostly BOOLEAN (but who uses this ;-) ) Also some I think subtle differences with the DATETIME type/format.
Some function names are different (SUBSTRING vs. SUBSTR and such...)
Just found what seems to be a good resource about differences between SQL implementations.
Reading the responses from others, yeah, DDL, could be a problem. I discounted that probably because many applications do not require DDL, you just need to set the data schema etc. at once, and then just use SQL for querying, adding or updating the data.
I believe that maintaining a single set of SQL resource files with MySQL and Oracle, has several disadvantages as being caught between backward compatibility and solve a particular problem. it is best to have a sql for each SQL engine and thus maximize the capabilities of each.
Features that look identical in a brochure may be implemented very differently.
see these examples
Limiting result sets
MYSQL
SELECT columns
FROM tablename
ORDER BY key ASC
LIMIT n
ORACLE
SELECT * FROM (
SELECT
ROW_NUMBER() OVER (ORDER BY key ASC) AS rownumber,
columns
FROM tablename
)
WHERE rownumber <= n
Limit—with offset
MYSQL
SELECT columns
FROM tablename
ORDER BY key ASC
LIMIT n OFFSET skip
ORACLE
SELECT * FROM (
SELECT
ROW_NUMBER() OVER (ORDER BY key ASC) AS rn,
columns
FROM tablename
)
WHERE rn > skip AND rn <= (n+skip)
You can check this Comparison of different SQL implementations
In addition to the stuff others have mentioned, oracle and mysql handle outer joins quite differently. Actually, Oracle offers a syntax that mySql won't cope with, but Oracle will cope with the standard syntax.
Oracle only:
SELECT a.foo, b.bar
FROM a, b
WHERE a.foo = b.foo(+)
mySql and Oracle:
SELECT a.foo, b.bar
FROM a
LEFT JOIN b
ON (a.foo=b.foo)
So you may have to convert some outer joins.
You definitely won't be able to keep your DDL the same. As far as DML goes, there are many similarities (there's a core subset of ANSI SQL standard supported by every database) but there are some differences as well.
To start, MySQL uses auto increment values and Oracle uses sequences. It's possible to work around this (sequence + trigger on Oracle side to simulate auto increment), but it's there. Built-in functions are quite different.
Basically, depending on what exactly you intend to use it may or may not be possible to keep one set of statements for both. Incidentally, even with Hibernate dialects it's not always possible to have the same set of queries - HQL is great but not always enough.
Oracle treats empty strings as nulls. MySQL treats empty strings as empty strings and null strings as null strings.
How can I handle an array in a prepared statement? i.e, I want to do a query and one of the parameters I get is an array of strings which I want to use in the query (Don't select rows that have a field that's in the array)?
Some JDBC drivers may already (before JDBC 4) contain proprietary extensions that support array-type parameters in prepared statements - you would need to consult with API for this. This would mean that you have to use and manipulate array-like type in SQL.
One work around would be using temporary tables. These are meta-steps for such solution:
Begin transaction (this is automatic if you are inside transactional
method - EJB or Spring);
Using JDBC batch insert with prepared statement create and populate a temporary table with arrary elements (temporary table must have transactional scope - this is also proprietary to databases but supported by Oracle at least);
Construct your desired SQL that includes a join to temporary table to use array values (it could be explicit inner or outer JOIN or implicit join, e.g. using EXISTS, etc.);
Commit (or rollback if application exception) transaction (this should destroy temporary table; concurrent transactions should have no conflict for the same name of temporary table).
Example: IN expression gets replaced with JOIN to temporary table.
This probably won't help you now, but I read that JDBC 4 will support array types as defined in the 2003 version of SQL.
That pretty much depends upon RDBMS being used. Often such functionality can be accomplished using vendor's jdbc driver extensions.
2 variants I found are (for Oracle):
http://blogs.itemis.de/kloss/2009/03/05/arrays-preparedstatements-jdbc-and-oracle/
http://www.angelfire.com/home/jasonvogel/java_jdbc_arrays.html
Try to look if that would help you.