I am trying to use below mentioned query in my spring jdbc template, but getting bad sql grammar exception. Is there any other way to fix this problem except stored procedure.
SET #row_number:=0;
SELECT * FROM(SELECT *,#row_number:=#row_number+1 AS row_number FROM COURSE
ORDER BY C_ID) As a where a.row_number BETWEEN 1 AND 1000 limit 15;
why you need #row_number in select query of COURSE? If pagination is your concern then use default mysql limit query.
SELECT * FROM COURSE limit ?,15;
pass the parameter from where user need to view 15 result set, please note it will start from 0.
Related
I am doing a Java project connected to Documentum and I need to retrieve data from object table. The thing is when I retrieve from 1 table I can get the answeres in max 2 seconds for each of the following tables with the following DQLs:
SELECT * FROM cosec_general
and
SELECT * FROM dm_dbo.cosec_general_view
however once I want to join those two tables together in retrieve from the result it takes 5 min to do so.
Is there any way that I can make it faster?
Here is the DQL that I use to join them I get teh columns that I need:
SELECT dm_dbo.cosec_general_view.name, dm_dbo.cosec_general_view.comp_id,
dm_dbo.cosec_general_view.bg_name, dm_dbo.cosec_general_view.incorporation_date,
dm_dbo.cosec_general_view.status, dm_dbo.cosec_general_view.country_name,
cosec_general.acl_domain, cosec_general.acl_name
FROM dm_dbo.cosec_general_view, cosec_general
There is no condition on which fields you are trying to join,
Add WHERE clause containing condition for join, like
WHERE dm_dbo.cosec_general_view.field_1=cosec_general.field_2
You are using wrong approach. In query
SELECT * FROM cosec_general
the asterisk * means return me everything. Once you loaded information to the memory object manipulation with it should be measured in milliseconds.
I want to select the Top 10 records for a given query. So, I can use one of the following options:
Using the JDBC Statement.setMaxRows() method
Using LIMIT and OFFSET in the SQL query
What are the advantages and disadvantages of these two options?
SQL-level LIMIT
To restrict the SQL query result set size, you can use the SQL:008 syntax:
SELECT title
FROM post
ORDER BY created_on DESC
OFFSET 50 ROWS
FETCH NEXT 50 ROWS ONLY
which works on Oracle 12, SQL Server 2012, or PostgreSQL 8.4 or newer versions.
For MySQL, you can use the LIMIT and OFFSET clauses:
SELECT title
FROM post
ORDER BY created_on DESC
LIMIT 50
OFFSET 50
The advantage of using the SQL-level pagination is that the database execution plan can use this information.
So, if we have an index on the created_on column:
CREATE INDEX idx_post_created_on ON post (created_on DESC)
And we execute the following query that uses the LIMIT clause:
EXPLAIN ANALYZE
SELECT title
FROM post
ORDER BY created_on DESC
LIMIT 50
We can see that the database engine uses the index since the optimizer knows that only 50 records are to be fetched:
Execution plan:
Limit (cost=0.28..25.35 rows=50 width=564)
(actual time=0.038..0.051 rows=50 loops=1)
-> Index Scan using idx_post_created_on on post p
(cost=0.28..260.04 rows=518 width=564)
(actual time=0.037..0.049 rows=50 loops=1)
Planning time: 1.511 ms
Execution time: 0.148 ms
JDBC Statement maxRows
According to the setMaxRows Javadoc:
If the limit is exceeded, the excess rows are silently dropped.
That's not very reassuring!
So, if we execute the following query on PostgreSQL:
try (PreparedStatement statement = connection
.prepareStatement("""
SELECT title
FROM post
ORDER BY created_on DESC
""")
) {
statement.setMaxRows(50);
ResultSet resultSet = statement.executeQuery();
int count = 0;
while (resultSet.next()) {
String title = resultSet.getString(1);
count++;
}
}
We get the following execution plan in the PostgreSQL log:
Execution plan:
Sort (cost=65.53..66.83 rows=518 width=564)
(actual time=4.339..5.473 rows=5000 loops=1)
Sort Key: created_on DESC
Sort Method: quicksort Memory: 896kB
-> Seq Scan on post p (cost=0.00..42.18 rows=518 width=564)
(actual time=0.041..1.833 rows=5000 loops=1)
Planning time: 1.840 ms
Execution time: 6.611 ms
Because the database optimizer has no idea that we need to fetch only 50 records, it assumes that all 5000 rows need to be scanned. If a query needs to fetch a large number of records, the cost of a full-table scan is actually lower than if an index is used, hence the execution plan will not use the index at all.
I ran this test on Oracle, SQL Server, PostgreSQL, and MySQL, and it looks like the Oracle and PostgreSQL optimizers don't use the maxRows setting when generating the execution plan.
However, on SQL Server and MySQL, the maxRows JDBC setting is taken into consideration, and the execution plan is equivalent to an SQL query that uses TOP or LIMIT. You can run the tests for yourself, as they are available in my High-Performance Java Persistence GitHub repository.
Conclusion
Although it looks like the setMaxRows is a portable solution to limit the size of the ResultSet, the SQL-level pagination is much more efficient if the database server optimizer doesn't use the JDBC maxRows property.
For most cases, you want to use the LIMIT clause, but at the end of the day both will achieve what you want. This answer is targeted at JDBC and PostgreSQL, but is applicable to other languages and databases that use a similar model.
The JDBC documentation for Statement.setMaxRows says
If the limit is exceeded, the excess rows are silently dropped.
i.e. The database server may return more rows but the client will just ignore them. The PostgreSQL JDBC driver limits on both the client and server side. For the client side, have a look at the usage of maxRows in the AbstractJdbc2ResultSet. For the server side, have a look of maxRows in QueryExecutorImpl.
Server side, the PostgreSQL LIMIT documentation says:
The query optimizer takes LIMIT into account when generating a query
plan
So as long as the query is sensible, it will load only the data it needs to fulfill the query.
setFetchSize Gives the JDBC driver a hint as to the number of rows that should be fetched from the database when more rows are needed for ResultSet objects generated by this Statement.
setMaxRows Sets the limit for the maximum number of rows that any ResultSet object generated by this Statement object can contain to the given number.
I guess using above 2 JDBC API you can try by using setFetchSize you can try if it works for 100K records. Else you can fetch in batch and form ArrayList and return it to your Jasper report.
not sure if i am right, but i remember in the past i was involved in big project to change all queries that were expected to return one row into 'TOP 1' or numrows=1. Reason was that the DB would stop searching for 'next possible matches' when this 'hint' was used. And in high volume environments this really made a difference. The remark that you can 'ignore' superfluous records in the client or in the resultset is not enough. You should avoid unnecessary reads as early as possible. But i have no idea whether the JDBC methods add those db specific hints to the query y/n. I may need to test however to see and use it ... i am not db specialist and can imagine i am not right, but "Speedwise it seems like no difference" can be a wrong assumption ... E.g. if you are asked to search in box for red balls and you only need one, it does not add value to keep searching for all where for you one is enough ... Then it matters to specify 'TOP 1' ...
I usually use SELECT 1 as my preferred validation-query from tomcat jdbc pools, because it returns only one row with the result 1 that is very faster, but today I've found one terrible mistake:
My database only has one table with its primary key and is not nullable. This table sometimes is dropped, and then appears again by application circumstances. And that's the problem, SELECT 1 validate connection to database because it's already up but the table is missing so I get a terrible exception.
So, the solutions passes by finding one validation-query against the only table that exists in the database. And also, I need the query to be as fast as possible, because the performance in the application is one of the main objetives.
You can answer than an obvius query may be SELECT 1 FROM THE_TABLE but this query returns 1 for each row that the table has, and that's not very quick.
So, what could be the faster validation-query to this table??
EDIT
If I need to return at least one result, How should be the validation-query?
I ask this because some pool implementations, like commons-dbcp doesn't accept a query without results as a validated query.
How about this, which should validate the table exists without actually loading any data by pulling 0 rows and no actual columns.
SELECT TOP 0 1 FROM THE_TABLE
Demo: http://www.sqlfiddle.com/#!3/c670b/3
There are also built-in ways to check for the existance of objects in SQL SERVER. Here are two examples that do effectively the same thing.
select count(1) from information_schema.tables where table_name = 'THE_TABLE'
select OBJECT_ID('THE_TABLE') is not null
I'm using an "upsert", and I need to know whether it has created a new row or not.
The SQL is of the form:
update mytable set col1='value1' where (col2='value2');
insert into mytable select 'value1', 'value2' where not exists (select 1 from mytable where (col2='value2'));
If I run this in Postgres, it returns the count of affected rows for the 2nd statement. I.e. "1 row affected" if it does an insert, "0 rows affected" if it does an update. Which is what I'd expect.
Howevere if I run this over JDBC+JPA using a javax.persistence.Query object, I get the opposite behaviour: it returns 1 if it does an update, and 0 for an insert.
Is the Java behaviour reliable?
I.e. can I code against that, or is it liable to change with future releases of the driver jar?
Is there a better way of writing the upsert so that I also get the information back about which operation it did?
Version details: I'm using PostgreSQL 9.1, Java 6, and the JDBC driver jar is postgresql-9.1-901.jdbc4.jar
Thank you for your help!
In Java you should be able to get back the row count for each statement, per the JDBC API documentation:
http://docs.oracle.com/javase/6/docs/api/java/sql/Statement.html#execute%28java.lang.String%29
The PostgreSQL docs provide an example where this is done in a plpgsql function:
http://www.postgresql.org/docs/9.1/interactive/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING
Look for the merge_db() code sample.
I have been asked in an interview to write a SQL query which fetches the first three records with highest value on some column from a table. I had written a query which fetched all the records with highest value, but didn't get how exactly i can get only first three records of those.
Could you help me in this.
Thanks.
SELECT TOP 3 * FROM Table ORDER BY FieldName DESC
From here, but might be a little out of date:
Postgresql:
SELECT * FROM Table ORDER BY FieldName DESC LIMIT 3
MS SQL Server:
SELECT TOP 3 * FROM Table ORDER BY FieldName DESC
mySQL:
SELECT * FROM Table ORDER BY FieldName DESC LIMIT 3
Select Top 3....
Depending on the database engine, either
select top 3 * from table order by column desc
or
select * from table order by column desc limit 3
The syntax for TOP 3 varies widely from database to database.
Unfortunately, you need to use those constructs for the best performance.
Libraries like Hibernate help here, because they can translate a common API into the various SQL dialects.
Since you are asking about Java, it is possible to just SELECT everything from the database (with an ORDER BY), but just fetch only the first three rows. Depending on how the query needs to be executed this might be good enough (especially if no sorting on the database has to happen thanks to appropriate indexes, for example when you sort by primary key fields).
But in general, you want to go with an SQL solution.
In oracle you can also use where rownum < 4...
Also on mysql there is a Limit keyword (i think)