Again, I'm creating this thread to see if someone has the same issues or if someone knows how to improve performance.
Let me explain first the scenario:
I have a JDO entity named for example Product. In that product I have a field of another entity, a simple one (Example Type: long id, String name, String value) and 1 date.
I created 10000 objects of the product but when i try to query the Product with the dates the performance is good but if I add the type.id == 12 then the performance drops from 100ms to 30 secs... if I instead of having a Type object put a long typeId on the Product, then the same query is also fast... What I'm worried is with scalability, should I flatten the structures and work with ids that are not really connected but needs additional application retrieval or is there a way to improve performance on the query on a Product.Type?
Thank you very much in advance.
I've already tried to define fetchGroups, but they don't really work...
What I wanted to do actually is doing a sort of mapping via an ID on the Product, but I couldn't do it with the embedded annotation...
so, let me explain a little better: I have a class named Reading with (one Date timeStamp and a Product p). this is doing a query. If the filter is something like this:
String filter = "timeStamp > fromDate && timeStamp < toDate";
the query executes in 100ms if I do this
String filter = "timeStamp > fromDate && timeStamp < toDate && prod.id == '941'"
for example, it takes 30secs... I've seen the logs, and without the query for the product he only reads from cache, if I add the clause of the Product he seems to fetch all the objects and starts comparing values I imagine...I don't really know what to do, maybe I should disconnect all these classes and start using like a String productID on the Reading class and then everything starts to be fast again... but in reality, the connection is not there, it would be implemented in an application layer... Thanks again... Any advise?
The query generated is something like this:
SELECT FROM core.jdo.model.Readings WHERE timeStamp > fromDate && timeStamp < toDate && EAN.EAN == '002' PARAMETERS java.util.Date fromDate, java.util.Date toDate import java.util.Date
Related
I would like to create such tables in MySQL with Hibernate/JPA.
SENSOR: PK_SENSOR_ID, SENSOR_NAME, SENSOR_URL
SENSOR_DATA: FK_SENSOR_ID, DATE, TIME, VALUE
SENSOR_DATA will contain readings from multiple sensors. Then I would like to have a method which use this query:
"SELECT AVG(VALUE), DATE, SENSOR_NAME, FROM SENSOR, SENSOR_DATA WHERE DATE BETWEEN '2019-03-01' AND '2019-03-02' AND SENSOR_ID = FK_SENSOR_ID GROUP BY DATE"
It would return an array of an average values of 'value' per each day (with date and sensor_name).
At this moment I know that I have to create a model for SENSOR with #Entity but what about SENSOR_DATA?
So far, I tried to create a simple model, repository and service with date, time and value and then query it but the problem was that I was able to only get single row for a single day. In this way I would have to create lots of objects for each day separately which doesn't make sense. I assume that it was because of that I used #Entity and added id, but I couldn't figure out a different solution. What I want is to create a single Sensor object and then query through SENSOR_DATA table.
Stack: Spring Boot, Spring Boot Data JPA Starter, Hibernate, MySQL, Lombok,
I really don't think the sql you have will work because of the group by. Correct me if I am wrong.
If you using Hibernate and Spring data jpa. You can do new Object in hql.
Say if the entity that mapped to SENSOR_DATA is sensorDate.
And your sensor object has constructor (double ave, Date date, String sensorName)
Example:
#Query("select new path/to/Sensor(avg, date, sensorName)
from sensorDate sd
where sd.date <:fromDate and sd.date > :toDate and sd=:fksd)")
public list<Sensor> findSensor(#Param("fromDate")Date fromDate, #Param("toDate")Date toDate, #Param("fksd")SensorDate fksd);
Let me explain:
when new keyword, hibernate will convert the output of the sql and mapped to sensor object and into list. Also since you are using hibernate, it is better to use ORM, when compare id, you can just compare the object, internally hibernate will convert hql to sql and compare their pk.
I am making a lot of assumption, but you get the idea. As long as you can get the value you want, you can do a new and pass them into the pojos you have.
As I couldn't figure out how to do it with JPA, I just used Spring JDBC and it works great. I based on a example implementation from this link:
Link
I have a database which stores (among others) Presentations, which is mapped using JPA. These presentations have a starting time, and an endtime.
I'd like know whether there's an overlap before adding a new presentation. I am currently doing this with a query that selects all presentations with the same room and same date.
I'd like to add the time to this. My times are made using a custom java.sql.Time subclass (I need it for a custom constructor). I obviously don't want to map those in the database. Now I'd like to check whether a certain time is before or after another.
I'm currently using the query SELECT p FROM Presentatie p WHERE p.lokaal.code = :lokaal AND p.dag = :dag AND ( p.begin <= :einde OR p.einde >= :begin) but that is flawed because it cannot compare Time classes (my guess that it evaluates to NULL, which would explain the zero-length resultset)
I don't know how to do this. I tried adding an int time to the Time class, but I can't reach it as it's not mapped using JPA.
I'd also like to know how it checks p.dag = :dag, is it equivalent to dag == dag2or dag.equals(dag2)? And how does it compare the time? time > time2 (but this doesn't compile in Java, so I'm guessing it goes to time.compareTo(time2)>0?
I am using the CriteriaBuilder and CriteriaQuery to build my query to the database, but I have encountered an issue that I do not know how to solve, since I am very new to this whole ordeal called JPA.
In Java, I have a property called timestamp for a class called Report, and it is set to the same corresponding #TemporalType.
I also have a class called Affiliate which has a list of Report objects.
In my query, I want to fetch all the Affiliate objects that do not have a Report in the last Affiliate.maxSilenceMinutes.
My questions:
Are there any ways in standardized JPA to modify dates? Like a CriteriaBuilder.subtractMilliseconds(Expression<Timestamp>, Long) of sorts?
If not, is there a way to cast Expression<Timestamp> to Expression<Long> so that I can subtract on a currentTimestamp literal to get the minimum value for a CriteriaBuilder.lessThanOrEqualTo(greatestReportTimestampMs, minimumAllowedMs)?
I know this might feel like a confusing question, but the main part is simply: Is it possible to go Expression<Timestamp> to Expression<Long>? It throws an exception for me if I try to use the .as(Long.class) method, but which should be the default underlying data type in most DBs anyway?
Hope you guys can help, since I feel kind of stuck :)
If you know the value you want to subtract at the time of querying,
you can subtract beforehand:
Calendar c = new Calendar();
c.setTime(timestamp.getTimestamp());
c.add(DAY, - someNumberOfDays); //or whatever unit you want
Date d = c.getTime();
If not, you probably need to call a database function to do the subtraction, via
CriteriaBuilder.function()
CriteriaBuilder.lessThanOrEqual() works on Comparables. Timestamps are comparable. So you could construct a Timestamp via new Timestamp(long ms)
and compare it with the other expression.
I hope this helps.
This is not built into Hibernate, so you will need a custom function of some kind.
The JDBC standard includes a function escape {fn TIMESTAMPADD( SQL_TSI_SECOND, secs, timestamp)} which should be translated into the correct SQL for the target database, but not all JDBC implementations provide it. There is therefore a chance you can add a custom StandardJDBCEscapeFunction to Hibernate's Dialect to get the result you need.
If you don't have that available, you'll have to find out what the correct database specific implementation is and there is a lot of variability here. For example:
Oracle: (timestamp + secs/86400)
SQLServer: DATEADD(ss,secs,timestamp)
DB2: (timestamp + secs SECONDS)
MySQL: DATE_ADD(timestamp, INTERVAL secs SECONDS)
Once you know it, you can use the correct expression as an SQL criteria.
The fact that date-time manipulation is not standardised in the Dialect and not fully implemented in many JDBCs means that what you are trying to do will be very difficult to write in a database neutral way.
I need to limit multiple service usages for multiple customers. For example, customer customer1 can send max 1000 SMS per month. My implementation is based on one MySQL table with 3 columns:
date TIMESTAMP
name VARCHAR(128)
value INTEGER
For every service usage (sending SMS) one row is inserted to the table. value holds usage count (eg. if SMS was split to 2 parts then value = 2). name holds limiter name (eg. customer1-sms).
To find out how many times the service was used this month (March 2011), a simple query is executed:
SELECT SUM(value) FROM service_usage WHERE name = 'customer1-sms' AND date > '2011-03-01';
The problem is that this query is slow (0.3 sec). We are using indexes on columns date and name.
Is there some better way how to implement service usage limitation? My requirement is that it must be flexibile (eg. if I need to know usage within last 10 minutes or another within current month). I am using Java.
Thanks in advance
You should have one index on both columns, not two indexes on each of the columns. This should make the query very fast.
If it still doesn't, then you could use a table with a month, a name and a value, and increment the value for the current month each time an SMS is sent. This would remove the sum from your query. It would still need an index on (month, name) to be as fast as possible, though.
I found one solution to my problem. Instead of inserting service usage increment, I will insert the last one incremented:
BEGIN;
-- select last the value
SELECT value FROM service_usage WHERE name = %name ORDER BY date ASC LIMIT 1;
-- insert it to the database
INSERT INTO service_usage (CURRENT_TIMESTAMP, %name, %value + %increment);
COMMIT;
To find out service usage since %date:
SELECT value AS value1 FROM test WHERE name = %name ORDER BY date DESC LIMIT 1;
SELECT value AS value2 FROM test WHERE name = %name AND date <= %date ORDER BY date DESC LIMIT 1;
The result will be value1 - value2
This way I'll need transactions. I'll probably implement it as stored procedure.
Any additional hints are still appreciated :-)
It's worth trying to replace your "=" with "like". Not sure why, but in the past I've seen this perform far more quickly than the "=" operator on varchar columns.
SELECT SUM(value) FROM service_usage WHERE name like 'customer1-sms' AND date > '2011-03-01';
Edited after comments:
Okay, now I can sorta re-create your issue - the first time I run the query, it takes around 0.03 seconds, subsequent runs of the query take 0.001 second. Inserting new records causes the query to revert to 0.03 seconds.
Suggested solution:
COUNT does not show the same slow-down. I would change the business logic so every time the user sends and SMS you insert the a record with value "1"; if the message is a multipart message, simply insert two rows.
Replace the "sum" with a "count".
I've applied this to my test data, and even after inserting a new record, the "count" query returns in 0.001 second.
I have a class User with one field called birthDate which is a java.sql.Date.
How do I do a hql query that will retrieve all Users that are between min and max years old?
(My real scenario is slightly more complex than that but that's where I am stuck right now).
UPDATE
It must be an hql expression so I can put the age expression in a computed property.
Calculate the birth dates corresponding to the min and max ages. Then use the below HQL.
Select u from User u where u.birthDate between :minDate and :maxDate
Setup the the minDate and maxDate to the values you computed before executing the query.
It depends on the database. Most have some way of handling date arithmetic. MySQL has a datediff function that will return a number of days so you could divide that by 365 and get pretty close. The MySQL dialect already has datediff as a registered function. For other databases you may need to use different functions and possibly register them in a custom dialect. But you may be off by a little unless you take leap years into account which is tricky in an HQL expression. Using dates is easier because you can keep the month and day constant, change the year, and then use < or > in HQL.