I'm dabbling in SQL at the moment and would love some help on a problem that I've created.
To practice some programming i'm making an IOU app. Below is the table I store
my IOU records (ignoring some relevant columns). This table allows users to say "Hey, you owe me X amount of money" and "I owe you X amount of money".
| Creator | Type | User_Involved| Amount |
|:-----------|------------:|:------------:|
| 1 | 0 | 2 | 3.0
| 2 | 0 | 1 | 4.0
| 3 | 1 | 1 | 5.0
Note: Type represents if the user has "lent or requested money".
0 = Lent
1 = Requested
Goal: My goal is to get a list of all Users and their total owed amount (negative or positive).
This has become quite difficult as I can't do a simple group by, as there are two columns i need to group on (Creator and User_Involved).
I managed to write an SQL query that gives the correct results, however i'm unable to translate it into a Hibernate version.
The issue mostly comes down to JPA unable to perform unions.
The Query in question:
/** assumption 1 is owed to creator **/
select sum(owed_to_you) as owed_to_you, friend_id
from
(
/** you created **/
select sum(CASE WHEN transaction_type = 0 THEN -amount ELSE amount END) as owed_to_you, friend_involved as friend_id
from iou
where
creator = 3
group by friend_involved
union all
/** someone else created **/
select sum(CASE WHEN transaction_type = 1 THEN -amount ELSE amount END) as owed_to_you, creator as friend_id
from iou
where
friend_involved = 3
group by creator) as theunion
group by friend_id
Besides loading all Ious into memory and sorting it that way, I'm completely stumped. I've done a lot of research today and learned a lot, however, i still have not made progress.
Any help would be much appreciated.
You can execute the two queries separately and combine the results in Java-code.
Related
I have a table with large number of rows. It has column like timestamp(in millis), value, and a siteId(foreign key). I want to fetch data from that of last three months with an interval in timestamp of 8 hours and I want to fetch data of all siteId in the three month timestamp. I have data in there for every 5 minutes of every siteId. If I fetch data of last three months, it is coming in millions. so I want to take data of every 8 hours. Sometimes, there can be a gap too so if a siteId was not there for the 8th hour, it should get its next data which can be 5 minutes past(or 10minutes past...) of that 8th hour.
Its hard to create a query for that and normal fetching and massaging the data in afterwards will take time.
I am using postgres, java and JPA. If I can do it via query or via some JPA utility to ease the CPU? I want to drop the time taken(right now 9 seconds for each query) to the least. Can you guys help me? Thanks in advance
My Table structure:
| timestamp | value | siteId |
|----------------|-------|--------|
| 1610370000000 | 22 | 123 |
| 1610370700000 | 21 | 123 |
| 1610370028000 | 22 | 123 |
| 1610369889000 | 23 | 123 |
| 1610370000000 | 22 | 124 |
| 1613534400000 | 21 | 124 |
| 1610369889000 | 22 | 124 |
| 1610370005000 | 23 | 125 |
So every site is having data for every 5 minutes. I want data of last three months with interval of at least 8 hours of every site. Hope this helps
Assuming you want same data structure like your example in question from last 3 months on 8 hours interval for each siteID.
Try this:
select
distinct on (siteId, "group" ) siteId, value, timestamp_,
ceil((extract(epoch from current_timestamp)*1000-timestamp_)/28800000) "group"
from test
where to_timestamp(timestamp_/1000) between current_timestamp - interval '3 month' and current_timestamp
order by 1,4,3
DEMO
Here I am dividing the difference of current_timestamp and timestamp_ field with 2880000(8*60*60*1000) to get the group and getting the first value of the group by using distinct on.
You can switch order by from order by 1,4,3 (return the value of min timestamp_ of range) to order by 1,4,3 desc (return the value of max timestamp_ of range) to check the correct result.
I am not sure about performance. But is should work better than java fetching.
If you need all data but you face issue with a transfer of huge data amount I would recommend to use pagination approach.
https://www.baeldung.com/jpa-pagination
I know this question has been asked before and most of the answers warn about doing so or suggest a solution for MyISAM but I'm using InnoDB.
I'm trying to generate an Invoice or Bill for the user for him to give out to me (Business plan requirement) !
The thing is that in my country the reference of the ID of the bill for the same person should be ordered or he will be audited for the missing bills. Like for example he gave me one bill with 0001 and the second one is 0005. He will be interrogated for the missing 4.
So I need to have a custom auto-increment per UserID.
User 1 - idUser= 1 ,idBill = 1
User 1 - idUser= 1 ,idBill = 2
User 2 - idUsr = 2 , idBill = 1
Some threads suggested using triggers while others warned about table locks. I personally not familiar with triggers so I steer away from them since they require maintenance.
I am using Java and MySQL.
An example:
CREATE TABLE main (id INT AUTO_INCREMENT PRIMARY KEY,
primary_id CHAR(3),
secondary_id INT) ENGINE = InnoDB;
CREATE TABLE auxiliary (primary_id CHAR(3),
secondary_id INT AUTO_INCREMENT,
PRIMARY KEY (primary_id, secondary_id)) ENGINE = MyISAM;
CREATE TRIGGER generate_secondary_id
BEFORE INSERT
ON main
FOR EACH ROW
BEGIN
INSERT INTO auxiliary (primary_id) VALUES (NEW.primary_id);
SET NEW.secondary_id = LAST_INSERT_ID();
END
INSERT INTO main (primary_id) VALUES
('A01'),
('A01'),
('B01'),
('C01'),
('A01'),
('B01'),
('A01'),
('B01');
SELECT * FROM main;
id | primary_id | secondary_id
-: | :--------- | -----------:
1 | A01 | 1
2 | A01 | 2
3 | B01 | 1
4 | C01 | 1
5 | A01 | 3
6 | B01 | 2
7 | A01 | 4
8 | B01 | 3
db<>fiddle here
Can I run explain analyze on a query in JOOQ? like:
explain analyse select some, columns from some_table
but do it using JOOQ on PostgreSQL database?
I have found an interface org.jooq.Explain, with a method DSLContext.explain​(Query query) - but it seems just to use EXPLAIN on a query:
#Support({AURORA_MYSQL,AURORA_POSTGRES,H2,HSQLDB,MARIADB,MEMSQL,MYSQL,ORACLE,POSTGRES,SQLITE})
Explain explain​(Query query)
Run an EXPLAIN statement in the database to estimate the cardinality of the query.
Is there any sensible way to run an EXPLAIN ANALYZE on the database from the code side?
Yes you can run explain. Example
SelectWhereStep<ModuldefRecord> where = dsl.selectFrom(MODULDEF);
Explain explain = dsl().explain(where);
System.out.println(explain);
The output look like this (for Oracle)
+------------------------------------------------------------------------------+
|PLAN_TABLE_OUTPUT |
+------------------------------------------------------------------------------+
|Plan hash value: 3871168833 |
| |
|------------------------------------------------------------------------------|
|| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time ||
|------------------------------------------------------------------------------|
|| 0 | SELECT STATEMENT | | 61303 | 30M| 1305 (1)| 00:00:01 ||
|| 1 | TABLE ACCESS FULL| MODULDEF | 61303 | 30M| 1305 (1)| 00:00:01 ||
|------------------------------------------------------------------------------|
+------------------------------------------------------------------------------+
Explain also contains rows and cost
/**
* The number of rows (cardinality) that is estimated to be returned by the query.
* <p>
* This returns {#link Double#NaN} if rows could not be estimated.
*/
double rows();
/**
* The cost the database associated with the execution of the query.
* <p>
* This returns {#link Double#NaN} if cost could not be retrieved.
*/
double cost();
It's not supported yet: https://github.com/jOOQ/jOOQ/issues/10424. Use plain SQL templating, instead:
ctx.fetch("explain analyze {0}", select);
For mariadb I needed to do:
SelectConditionStep<TableNameRecord> select =
context.selectFrom(Tables.TABLE_NAME)
.where(filter);
System.out.println(context.fetch("analyze " + select.getSQL(ParamType.INLINED)));
which produced the output:
+----+-----------+----------+-----+-----------------+-----------------+-------+------+----+-------+--------+----------+------------------------+
| id|select_type|table |type |possible_keys |key |key_len|ref |rows|r_rows |filtered|r_filtered|Extra |
+----+-----------+----------+-----+-----------------+-----------------+-------+------+----+-------+--------+----------+------------------------+
| 1|SIMPLE |table_name|range|table_column_name|table_column_name|20 |{null}|1000|1000.00| 100.0| 100.0|Using where; Using index|
+----+-----------+----------+-----+-----------------+-----------------+-------+------+----+-------+--------+----------+------------------------+
If you use context.explain(select) as proposed by another answer you lose a few columns:
+----+-----------+----------+-----+-----------------+-----------------+-------+------+----+------------------------+
| id|select_type|table |type |possible_keys |key |key_len|ref |rows|Extra |
+----+-----------+----------+-----+-----------------+-----------------+-------+------+----+------------------------+
| 1|SIMPLE |table_name|range|table_column_name|table_column_name|20 |{null}|1000|Using where; Using index|
+----+-----------+----------+-----+-----------------+-----------------+-------+------+----+------------------------+
I am making a game and I require a ranking system. I already save all the stats like kills, deaths, wins, innocent shots, etc with MySQL. I am clueless a the moment on how I would be able to rank everyone. I want to have it over MySQL but they will be updated very quickly. I was thinking I could load all ranks in a HashMap when the game starts but that would be very ineffective since there are thousands of players. I want to also use most of the stats to work this out. Could someone explain to me how I would be able to do this? Thanks!
One way would be to use mysql Events to trigger a stored procedure. The stored procedure would execute the ranking and store the rank in the db. You would then just set the event trigger time to whatever you wanted, say 10 minutes.
mysql events: https://dev.mysql.com/doc/refman/5.7/en/events.html
CREATE
[DEFINER = { user | CURRENT_USER }]
EVENT
[IF NOT EXISTS]
event_name
ON SCHEDULE schedule
[ON COMPLETION [NOT] PRESERVE]
[ENABLE | DISABLE | DISABLE ON SLAVE]
[COMMENT 'comment']
DO event_body;
schedule:
AT timestamp [+ INTERVAL interval] ...
| EVERY interval
[STARTS timestamp [+ INTERVAL interval] ...]
[ENDS timestamp [+ INTERVAL interval] ...]
interval:
quantity {YEAR | QUARTER | MONTH | DAY | HOUR | MINUTE |
WEEK | SECOND | YEAR_MONTH | DAY_HOUR | DAY_MINUTE |
DAY_SECOND | HOUR_MINUTE | HOUR_SECOND | MINUTE_SECOND}
Then you would setup a stored procedure to generate the rank and set it for your players: https://dev.mysql.com/doc/refman/5.7/en/stored-programs-views.html
An example procedure would be:
CREATE PROCEDURE simpleproc (OUT param1 INT)
BEGIN
SELECT COUNT(*) INTO param1 FROM t;
END
You can make the procedures as complex as you need to.
This is a simplified task which I have to solve in real project. In this project data is stored in HSQLDB. Data is accessed using JDBC.
I have one table:
name | flag
-----------
aa | 1
bb | 0
cc | 1
dd | 0
ee | 1
ff | 0
I need to compose query to get the following table:
name | flag
-----------
aa | 1
cc | 1
ee | 1
ff | 0
dd | 0
bb | 0
The final table is like rows with flag = 1 were taken and sorted ascending, rows with flag = 0 were taken and sorted descending and results were combined one after another.
Please, pay attention, that rows with flag = 1 and rows with flag = 0 have opposite sort order.
Is it possible to do in SQL? I wouldn`t like to make two queries and merge ResultSets in Java code manually.
In any SQL, only the outermost order by applies so it has to be done there.
I don't know if this works in your SQL dialect (and can't check sorry), but you could do this
SELECT name, flag
FROM 'table'
ORDER BY
flag desc,
CASE WHEN flag = 1 THEN name ELSE '' END,
CASE WHEN flag = 0 THEN name ELSE '' END DESC
Try this:
SELECT name, flag
FROM 'table'
ORDER BY flag desc, name
Let the database do the work whenever you can. Don't do such things in Java. Think "SQL first".
order by can take more than one column:
select *
from table
order by flag desc, name asc