How would I randomly select rows from a database table? I'm using JPA and would like to use the Criteria API if possible. I'm aware there is an SQL equivalent, something like:
SELECT TOP 5 Id, Name FROM mNames
ORDER BY NEWID()
But how would I do this with JPQL and the Criteria API?
Possibly, with a NativeQuery? Is there a better way?
If all you need is a single random row, then you can do something like this:
//random is instance of java's Random class, and numberOfRows is total number of rows in the table
long rowIndex = random.nextLong()%numberOfRows;
TypedQuery typedQuery = ...;
typedQuery.setFirstResult(rowIndex);
typedQuery.setMaxResults(1);
I didn't test the code but you should get the idea.
If the primary key of your entity is not an arbitrary number, but something functional, you might be better off, creating a valid key randomly and then querying the specific row, which is fast operation, instead of executing a possibly large number of rows just to filter out everything but one.
Related
as a newbie I want to sum the values of a column pv from a database table evm in my model and store it in a variable. I have tried the SQL code SELECT SUM(pv) FROM evm; but that doesn't seem to work.I would be grateful if you lend me an aid regarding how to pull this one.
You can always write a native query and get the response in the resultset to populate the field of your pojo. Once you have the POJO/DTO created as the list of result set perform your sum on the field by Iterating the list.
You do just use the SQL you have suggested. (The database in an AnyLogic model is a standard HSQLDB database which supports this SQL syntax.)
The simplest way to execute it is to use AnyLogic's in-built functions for such queries (as would be produced by the Insert Database Query wizard), so
mySumVariable = selectFirstValue("SELECT SUM(pv) FROM evm;");
You didn't say what errors you had; obviously the table and column has to exist (and the column you're summing needs to be numeric, though NULLs are OK), as does the variable you're assigning the sum to.
If you wanted to do this in a way which more easily fits one of the standard query 'forms' suggested by the wizard (i.e., not having to know particular SQL syntax) you could just adapt the "Iterate over returned rows and do something" code to 'explicitly' sum the columns; e.g., (using the Query DSL format this time):
List<Tuple> rows = selectFrom(evm).list();
for (Tuple row : rows) {
mySumVariable += row.get(evm.pv);
}
I have an entity that has a CLOB attribute:
public class EntityS {
...
#Lob
private String description;
}
To retrieve certain EntityS from the DB we use a CriteriaQuery where we need the results to be unique, so we do:
query.where(builder.and(predicates.toArray(new Predicate[predicates.size()]))).distinct(true).orderBy(builder.asc(root.<Long> get(EntityS_.id)));
If we do that we get the following error:
ORA-00932: inconsistent datatypes: expected - got CLOB
I know that's because you cannot use distinct when selecting a CLOB. But we need the CLOB. Is there a workaround for this using CriteriaQuery with Predicates and so on?
We are using an ugly workaround getting rid of the .unique(true) and then filtering the results, but that's crap. We are using it only to be able to keep on developing the app, but we need a better solution and I don't seem to find one...
In case you are using Hibernate as persistence provider, you can specify the following query hint:
query.setHint(QueryHints.HINT_PASS_DISTINCT_THROUGH, false);
This way, "distinct" is not passed through to the SQL command, but Hibernate will take care of returning only distinct values.
See here for more information: https://thoughts-on-java.org/hibernate-tips-apply-distinct-to-jpql-but-not-sql-query/
Thinking outside the box - I have no idea if this will work, but perhaps it is worth a shot. (I tested it and it seems to work, but I created a table with just one column, CLOB data type, and two rows, both with the value to_clob('abcd') - of course it should work on that setup.)
To de-duplicate, compute a hash of each clob, and instruct Oracle to compute a row number partitioned by the hash value and ordered by nothing (null). Then select just the rows where the row number is 1. Something like below (t is the table I created, with one CLOB column called c).
I expect that execution time should be reasonably good. The biggest concern, of course, is collisions. How important is it that you not miss ANY of the CLOBs, and how many rows do you have in the base table in the first place? Is something like "one chance in a billion" of having a collision acceptable?
select c
from (
select c, row_number() over (partition by dbms_crypto.hash(c, 3) order by null) as rn
from t
)
where rn = 1;
Note - the user (your application, in your case) must have EXECUTE privilege on SYS.DBMS_CRYPTO. A DBA can grant it if needed.
we know hibernate has this in clause:
Criteria criteria = session.createCriteria(User.class);
criteria.add(Restrictions.in(userIds));
Is there any limit on the size of userIds (which is an ArrayList, say)?
Thanks
It actually depends on the particular database you use. For example in Oracle this limit is 1000.
If you need to pass more values you need to use another approach. For example put the values into a temporary table and then do a select where id in (select id from temptable) query.
I can't figure out how to fetch n random rows from a criteria instance:
Criteria criteria = session.createCriteria(Table.class);
criteria.add(Restrictions.eq('fieldVariable', anyValue));
...
Then what? I can't find any doc with Criteria API
Does it mean I should use HQL instead?
Thanx!
EDIT: I get the number of rows by:
int max = criteria.setProjecxtion(Projections.rowCount()).uniqueResult();
How do I fetch n random rows with indexes between 0 and max?
Thx again!
Actually it is possible with Criteria and a little bit of tweaking. Here is how:
Criteria criteria = session.createCriteria(Table.class);
criteria.add(Restrictions.eq("fieldVariable", anyValue));
criteria.add(Restrictions.sqlRestriction("1=1 order by rand()"));
criteria.setMaxResults(5);
return criteria.list();
any Restrictions.sqlRestriction will add keyword 'and'; so to nullify its effect,
we shall add a dummy condition and inject our rand() function.
First of all, be aware that there is no standard way to do this in SQL, each database engine uses its own proprietary syntax1. With MySQL, the SQL statement to get 5 random rows would be:
SELECT column FROM table
ORDER BY RAND()
LIMIT 5
And you could write this query in HQL because the order by clause in HQL is passed through to the database so you can use any function.
String query = "SELECT e.attribute FROM MyEntity e ORDER BY RAND()";
Query q = em.createQuery(query);
q.setMaxResults(5);
However, unlike HQL, the Criteria API currently doesn't support ORDER BY Native SQL (see HHH-2381) and in the current state, you would have to subclass the Order class to implement this feature. This is doable, refer to the Jira issue, but not available out of the box.
So, if really you need this query, my recommendation would be to use HQL. Just keep in mind it won't be portable.
1 Other readers might want to check the post SQL to Select a random row from a database table to see how to implement this with MySQL, PostgreSQL, Microsoft SQL Server, IBM DB2 and Oracle.
The Criteria API doesn't offer facilities for this. In MySQL however, you can use ORDER BY RAND() LIMIT n for this where n represents the number of random rows you'd like to fetch.
SELECT col1, col2, col3 FROM tbl ORDER BY RAND() LIMIT :n
You indeed need to execute it as HQL.
You can not fetch random rows efficiently, sorry. Hibernate can only do what SQL does, and random row fetch simply is not part of any standard SQL implementation I know - actually it is to my knowledge not part of ANY SQL that I am aware of (anyone please enlight me).
And as Hibernate is an O/R mapper, and not a wonder machine, it can only do what the underlying database supports.
If you have a known filed with ascending numbers and know start and end, you can generate a random number on the computer and ask for that row.
The answer by #PSV Bhat is difficult if you are dynamically generating your Criteria. Here is a solution that extends hibernate Order class:
import org.hibernate.Criteria;
import org.hibernate.criterion.Order;
private void addOrderByToCriteria(Criteria criteria) {
criteria.addOrder(Order.asc("foobar"));
criteria.addOrder(ORDER_RANDOM);
}
private static final OrderRandom ORDER_RANDOM = new OrderRandom();
private static class OrderRandom extends Order {
public OrderRandom() {
super("", false);
}
#Override
public String toSqlString(Criteria criteria, CriteriaQuery criteriaQuery) {
return "RANDOM()"; // or RAND() or whatever this is in your dialect
}
}
How do I implement paging in Hibernate? The Query objects has methods called setMaxResults and setFirstResult which are certainly helpful. But where can I get the total number of results, so that I can show link to last page of results, and print things such as results 200 to 250 of xxx?
You can use Query.setMaxResults(int results) and Query.setFirstResult(int offset).
Editing too: There's no way to know how many results you'll get. So, first you must query with "select count(*)...". A little ugly, IMHO.
You must do a separate query to get the max results...and in the case where between time A of the first time the client issues a paging request to time B when another request is issued, if new records are added or some records now fit the criteria then you have to query the max again to reflect such. I usually do this in HQL like this
Integer count = (Integer) session.createQuery("select count(*) from ....").uniqueResult();
for Criteria queries I usually push my data into a DTO like this
ScrollableResults scrollable = criteria.scroll(ScrollMode.SCROLL_INSENSITIVE);
if(scrollable.last()){//returns true if there is a resultset
genericDTO.setTotalCount(scrollable.getRowNumber() + 1);
criteria.setFirstResult(command.getStart())
.setMaxResults(command.getLimit());
genericDTO.setLineItems(Collections.unmodifiableList(criteria.list()));
}
scrollable.close();
return genericDTO;
you could perform two queries - a count(*) type query, which should be cheap if you are not joining too many tables together, and a second query that has the limits set. Then you know how many items exists but only grab the ones being viewed.
You can do one thing. just prepare Criteria query as per your busness requirement with all Predicates , sorting , searching etc.
and then do as below :-
CriteriaBuilder criteriaBuilder = em.getCriteriaBuilder();
CriteriaQuery<Feedback> criteriaQuery = criteriaBuilder.createQuery(Feedback.class);
//Just Prepare your all Predicates as per your business need.
//eg :-
yourPredicateAsPerYourBusnessNeed = criteriaBuilder.equal(Root.get("applicationName"), applicationName);
criteriaQuery.where(yourPredicateAsPerYourBusnessNeed).distinct(true);
TypedQuery<Feedback> criteriaQueryWithPredicate = em.createQuery(criteriaQuery);
//Getting total Count Here
Long totalCount = criteriaQueryWithPredicate.getResultStream().distinct().count();
Now we have our actual data with us as above with total count , right.
So now we can apply pagination on the data we have in our hand above , as below :-
List<Feedback> feedbackList = criteriaQueryWithPredicate.setFirstResult(offset).setMaxResults(pageSize).getResultList();
Now You can prepare a wrapper with your List return by DB along with the totalCount , startingPageNo that is offset here in this case, page Size etc and can return to your service / controller class.
I am 101 % sure , this will solve your problem, Because I was facing same problem and sorted it out same way.
Thanks- Sunil Kumar Mali
You can just setMaxResults to the maximum number of rows you want returned. There is no harm in setting this value greater than the number of actual rows available. The problem the other solutions is they assume the ordering of records remains the same each repeat of the query, and there are no changes going on between commands.
To avoid that if you really want to scroll through results, it is best to use the ScrollableResults. Don't throw this object away between paging, but use it to keep the records in the same order. To find out the number of records from the ScrollableResults, you can simply move to the last() position, and then get the row number. Remember to add 1 to this value, since row numbers start counting at 0.
I personally think you should handle the paging in the front-end. I know this isn't that efficiënt but at least it would be less error prone.
If you would use the count(*) thing what would happen if records get deleted from the table in between requests for a certain page? Lots of things could go wrong this way.