I have basically the same problem outlined in this question, however I am using Microsoft Access as a database instead of MySQL. The result of which is that SQL_CALC_FOUND_ROWS doesn't seem to be available to me. Believe me, I want to switch, but for the moment it is out of the question.
I have a query that aggregates a number of rows, essentially looking for repeat rows based on certain keys, using a group by. It looks something like this:
Select key1, key2, key3, Count(id)
from table
group by key1, key2, key3
having Count(id) > 1
I need to determine the number of rows (or groupings) that query will return.
The database is being accessed through Java, so in theory I could simply run the query, and cycle through it twice, but I was hoping for something faster and preferably SQL based. Any ideas?
MS Access's record count should give you what you need, or am I missing something?
If you need distinct values from keys, try this
SELECT COUNT(*) AS Expr2
FROM (
SELECT DISTINCT [key1] & "-" & [key2] & "-" & [key3] AS Expr1
FROM Table1
) AS SUB;
When you create the Statement object, you can declare it to be scrollable. Then the first thing you do is scroll to the end and get the record number. As you're looking at the last record, this will be the number of records in the result set. Then scroll back to the beginning and process normally. Something like:
Statement st=connection.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
ResultSet rs=st.executeQuery(myQueryString);
boolean any=rs.last();
int count = any ? count=getRow() : 0;
... do whatever with the record count ...
rs.first();
while (rs.next())
{
... whatever processing you want to do ...
}
rs.close();
... etc ...
I have no idea what the performance implications of doing this with MS Access will be, whether it can jump directly to the end of the result set or if it will have to sequentially read all the records. Still, it should be faster than executing the query twice.
Related
In mysql I want to execute a query like this
SELECT MIN(id) FROM table;
The more I read about JOOQ syntax and the aggregate functions, the confused I get.
I thought something like this would work
select( EVENT.EVENTID , min() ).from( EVENT ).fetch();
or
Result<Integer> er = context.select( EVENT.EVENTID.min()).fetch();
I tried a work around by selecting the whole first record
Result<EventRecord> er2 = context.selectFrom(EVENT).orderBy(EVENT.EVENTID.asc()).limit(1).fetch();
If the result has size 0, a record does not exist, but when it is not 0 I get the right record. I would like to use the min() function but can't get the syntax right.
The query you want to write in SQL is this one:
SELECT MIN(event.eventid) FROM event
This is why your two attempts didn't work
// 1. You cannot combine single columns with aggregate functions in SQL,
// unless you're grouping by those columns
// 2. You didn't pass any cargument column to the MIN() function
context.select( EVENT.EVENTID , min() ).from( EVENT ).fetch();
// 3. This doesn't specify any FROM clause, so your database won't know what
// table you want to select the MIN(eventid) from
context.select( EVENT.EVENTID.min()).fetch();
Note that these thoughts are not specific to jOOQ, they are related to SQL in general. When using jOOQ, always think of the SQL statement you want to express first (the one at the top of my answer). So your jOOQ statement would look like any of these:
// "Postfix notation" for MIN()
context.select(EVENT.EVENTID.min()).from(EVENT).fetch();
// "Prefix notation" for MIN(), where min() is static-imported from
// org.jooq.impl.DSL
context.select(min(EVENT.EVENTID)).from(EVENT).fetch();
It looks like the fetchAny() method will return the record with the first/lowest record id.
EventRecord record = context.selectFrom(EVENT).fetchAny();
As #LukasEder mentioned there are many alternative methods, and he may be generous and follow up on some of those. Thanks Lucas
I have the following prepared statement in java:
with main_select as
(select request_id,rownum iden
from
(select request_id
from queue_requests
where request_status = 0 and
date_requested <= sysdate and
mod(request_id,?) = ?
order by request_priority desc, oper_id, date_requested)
where rownum < ?)
select *
from queue_requests qr, main_select ms
where qr.request_id in ms.request_id
order by ms.iden for update skip locked;
It doesn't execute:
ORA-02014: cannot select FOR UPDATE from view with DISTINCT, GROUP BY, etc.
I'll try to explain why i need all the select statements:
the first (inner) select obtains the data i need
the second one limits the number of lines to a number (i can't put it in the first select, because oracle firstly limits the results and only after orders them, which is not what i want)
the third (outside with) select preserves the order (i tried using 3 nested selects - so, no with clause - but i can't find a way to preserve the order in this case). Also, it should lock the lines in the queue_requests table, but because i selected data from the with clause, it gives the above error.
So, i want to select data from queue_requests, keep the first x lines, preserve the order of the select and lock the lines.
Is there a way to do it?
The problem seems to be, that you want to set a lock on the result of main_select. I would just guess, that you can do select for update in the select in the with clause like:
with main_select as
(select request_id,rownum iden
from (subselect)
where rownum < ?
for update skip locked)
But as I said lucky guess.
How can I retrieve the last entered value of the column in the database (MS ACCESS 2007)
I used the following code
String sql = "SELECT Last(RegNumber) FROM Death ";
but it does not work in MS ACCESS and when I run the program Error generates as
java.sql.SQLException: Column not found
but I have created a column in database as RegNumber
I am using Java for programming in which I used this query
EDIT:
RegNumber is in String form not in integer form so I cant use DESC or ASC
Please help me
Sort your table by whatever criteria you'd like and use SELECT TOP 1 * FROM myTable ORDER BY RegNumber ASC.
Or ORDER BY incrementingId DESC
Basically there must be some logical order to the sorting for what you refer to as the "last entered column" (which I assume means row, not column)
EDIT: Your function is correct in Access, and should return the correct value. However Java may not interpret it correctly. Try your query in an Access native query, then try debugging your Java. If it's simply that Java does not support this function, consider using the built in ResultSet() functions in Java.sql
ResultSet rs = ....;
rs.last();
int RegNumber = rs.getRow();
I do not know about the last() function in MS ACCESS, but I have another idea:
Usually there is an automatically generated id for each table, so you can sort on it and get the first record from the result set like this:
SELECT RegNumber
FROM Death
ORDER BY id DESC
That depend of your database structure.
Typically with table come some unique identifier, if you are sure that it comes always in order to database you could use function MAX to retrieve the identifier and then whole row.
Another scenario is just to a timestamp columns that describe the time when column was created , this approach satisfying if the sequence is really crucial if not the id should be enough.
Following will return the last and lastest RegNumber :
SELECT RegNumber FROM Death ORDER BY RegNumber DESC
I am generating reports in my system but some reports are returning a huge number of results. To remedy this I hit the database with a count first, then in my code I check if this count is above a certain threshold (e.g. 2000), then don't generate the report.
This is fine in most cases but some reports have over a million results, this means it takes the count a good few seconds to return a result.
Ideally, what I would like to do is put my threshold (2000) into my sql statement, stop the count if it reaches this value, and return some value (e.g. true or false, 0 or 1, anything) so that I know it has exceeded its limit. Is this possible in sql, so far I cannot find a solution.
Pseudocode: select count(1) from table while count <= threshold
I am working with java, hibernate, sql server 2005.
Any help will be much appreciated.
Regards,
Eamon
I think something like this should work:
select count(1) from (
select top 2000 *
from table
where ...
)
I don't like the idea of running the query twice, once to check the row count and then again to actually get the data. As a result, if you want to limit the report to 2000 rows, just do something like this:
SELECT TOP 2001
...
from...
in the application check if the rowcount is greater than your limit >2000 return an error, or skip displaying it.
try something like limit clause? this page suggests how that could be done in sql server: http://blogs.msdn.com/b/sqlserver/archive/2006/10/25/limit-in-sql-server.aspx
I'm not really aware of equivalent in Hibernate for TOP instruction, but even if there exists first you will have to retrieve the 2000 rows and then count it what probably will be slow as is now. So you can featch allredy 2001 row and then validate it in application.
But I think that what you should optimize is the query by itself.
If in the query does not exists some outer join just specify index column in statement
SELECT count(*) from test1 inner join test2 on test1.id = test2.id;
888 ms
SELECT count(test.id) from test1 inner join test2 on test1.id = test2.id;
88ms
relation 35 rows in test1 match to 1 in test2;
rows 1252080;
How do I implement paging in Hibernate? The Query objects has methods called setMaxResults and setFirstResult which are certainly helpful. But where can I get the total number of results, so that I can show link to last page of results, and print things such as results 200 to 250 of xxx?
You can use Query.setMaxResults(int results) and Query.setFirstResult(int offset).
Editing too: There's no way to know how many results you'll get. So, first you must query with "select count(*)...". A little ugly, IMHO.
You must do a separate query to get the max results...and in the case where between time A of the first time the client issues a paging request to time B when another request is issued, if new records are added or some records now fit the criteria then you have to query the max again to reflect such. I usually do this in HQL like this
Integer count = (Integer) session.createQuery("select count(*) from ....").uniqueResult();
for Criteria queries I usually push my data into a DTO like this
ScrollableResults scrollable = criteria.scroll(ScrollMode.SCROLL_INSENSITIVE);
if(scrollable.last()){//returns true if there is a resultset
genericDTO.setTotalCount(scrollable.getRowNumber() + 1);
criteria.setFirstResult(command.getStart())
.setMaxResults(command.getLimit());
genericDTO.setLineItems(Collections.unmodifiableList(criteria.list()));
}
scrollable.close();
return genericDTO;
you could perform two queries - a count(*) type query, which should be cheap if you are not joining too many tables together, and a second query that has the limits set. Then you know how many items exists but only grab the ones being viewed.
You can do one thing. just prepare Criteria query as per your busness requirement with all Predicates , sorting , searching etc.
and then do as below :-
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<Feedback> criteriaQuery = criteriaBuilder.createQuery(Feedback.class);
//Just Prepare your all Predicates as per your business need.
//eg :-
yourPredicateAsPerYourBusnessNeed = criteriaBuilder.equal(Root.get("applicationName"), applicationName);
criteriaQuery.where(yourPredicateAsPerYourBusnessNeed).distinct(true);
TypedQuery<Feedback> criteriaQueryWithPredicate = em.createQuery(criteriaQuery);
//Getting total Count Here
Long totalCount = criteriaQueryWithPredicate.getResultStream().distinct().count();
Now we have our actual data with us as above with total count , right.
So now we can apply pagination on the data we have in our hand above , as below :-
List<Feedback> feedbackList = criteriaQueryWithPredicate.setFirstResult(offset).setMaxResults(pageSize).getResultList();
Now You can prepare a wrapper with your List return by DB along with the totalCount , startingPageNo that is offset here in this case, page Size etc and can return to your service / controller class.
I am 101 % sure , this will solve your problem, Because I was facing same problem and sorted it out same way.
Thanks- Sunil Kumar Mali
You can just setMaxResults to the maximum number of rows you want returned. There is no harm in setting this value greater than the number of actual rows available. The problem the other solutions is they assume the ordering of records remains the same each repeat of the query, and there are no changes going on between commands.
To avoid that if you really want to scroll through results, it is best to use the ScrollableResults. Don't throw this object away between paging, but use it to keep the records in the same order. To find out the number of records from the ScrollableResults, you can simply move to the last() position, and then get the row number. Remember to add 1 to this value, since row numbers start counting at 0.
I personally think you should handle the paging in the front-end. I know this isn't that efficiënt but at least it would be less error prone.
If you would use the count(*) thing what would happen if records get deleted from the table in between requests for a certain page? Lots of things could go wrong this way.