I have an SQLite table like:
+---+-------------+----------------+
|_id| lap_time_ms |formatted_elapse|
+---+-------------+----------------+
| 1 | 5600 | 00:05.6 |
| 2 | 4612 | 00:04.6 |
| 3 | 4123 | 00:04.1 |
| 4 | 15033 | 00:15.0 |
| 5 | 4523 | 00:04.5 |
| 6 | 6246 | 00:06.2 |
Where lap_time_ms is an of the type long and represents the amount of time in milliseconds for a lap while formatter_elapse is a String that represents the formatted (displayable) form of the first column (elapse).
My question is that if I want to add (say) 5 seconds (5000) to each lap_time_ms then I use a statement like:
DB.execSQL("update table_name set KEY_ELAPSE=KEY_ELAPSE+5000);
Which works fine however the problem is that formatted_elapse still retains its outdated value!
So, what is the best way to update the values in the formatted_elapse column if I have a function like:
public static String getFormattedTime(long milliseconds) {
// custom code that processes the long
return processedString;
}
It may be a long shot (metaphorically speaking of course ;) but is it possible to have SQLite link the two columns such that if I update a lap_time_ms row, the formatted_elapse will automatically update appropriately.
Thanks in advance!
In theory, it would be possible to create a trigger to update that column, but only if the formatting can be done with some built-in SQLite function (Android does not allow user-defined functions):
CREATE TRIGGER update_formatted_elapse
AFTER UPDATE OF lap_time_ms ON MyTable
FOR EACH ROW
BEGIN
UPDATE MyTable
SET formatted_elapse = strftime('%M:%f', NEW.lap_time_ms, 'unixepoch')
WHERE _id = NEW._id;
END;
However, it would be bad design to store the formatted string in the database; it would be duplicated information that is in danger of becoming inconsistent.
Drop the formatted_elapse column and just call getFormattedTime in your code whenever you need it.
Related
I have a table with large number of rows. It has column like timestamp(in millis), value, and a siteId(foreign key). I want to fetch data from that of last three months with an interval in timestamp of 8 hours and I want to fetch data of all siteId in the three month timestamp. I have data in there for every 5 minutes of every siteId. If I fetch data of last three months, it is coming in millions. so I want to take data of every 8 hours. Sometimes, there can be a gap too so if a siteId was not there for the 8th hour, it should get its next data which can be 5 minutes past(or 10minutes past...) of that 8th hour.
Its hard to create a query for that and normal fetching and massaging the data in afterwards will take time.
I am using postgres, java and JPA. If I can do it via query or via some JPA utility to ease the CPU? I want to drop the time taken(right now 9 seconds for each query) to the least. Can you guys help me? Thanks in advance
My Table structure:
| timestamp | value | siteId |
|----------------|-------|--------|
| 1610370000000 | 22 | 123 |
| 1610370700000 | 21 | 123 |
| 1610370028000 | 22 | 123 |
| 1610369889000 | 23 | 123 |
| 1610370000000 | 22 | 124 |
| 1613534400000 | 21 | 124 |
| 1610369889000 | 22 | 124 |
| 1610370005000 | 23 | 125 |
So every site is having data for every 5 minutes. I want data of last three months with interval of at least 8 hours of every site. Hope this helps
Assuming you want same data structure like your example in question from last 3 months on 8 hours interval for each siteID.
Try this:
select
distinct on (siteId, "group" ) siteId, value, timestamp_,
ceil((extract(epoch from current_timestamp)*1000-timestamp_)/28800000) "group"
from test
where to_timestamp(timestamp_/1000) between current_timestamp - interval '3 month' and current_timestamp
order by 1,4,3
DEMO
Here I am dividing the difference of current_timestamp and timestamp_ field with 2880000(8*60*60*1000) to get the group and getting the first value of the group by using distinct on.
You can switch order by from order by 1,4,3 (return the value of min timestamp_ of range) to order by 1,4,3 desc (return the value of max timestamp_ of range) to check the correct result.
I am not sure about performance. But is should work better than java fetching.
If you need all data but you face issue with a transfer of huge data amount I would recommend to use pagination approach.
https://www.baeldung.com/jpa-pagination
I know this question has been asked before and most of the answers warn about doing so or suggest a solution for MyISAM but I'm using InnoDB.
I'm trying to generate an Invoice or Bill for the user for him to give out to me (Business plan requirement) !
The thing is that in my country the reference of the ID of the bill for the same person should be ordered or he will be audited for the missing bills. Like for example he gave me one bill with 0001 and the second one is 0005. He will be interrogated for the missing 4.
So I need to have a custom auto-increment per UserID.
User 1 - idUser= 1 ,idBill = 1
User 1 - idUser= 1 ,idBill = 2
User 2 - idUsr = 2 , idBill = 1
Some threads suggested using triggers while others warned about table locks. I personally not familiar with triggers so I steer away from them since they require maintenance.
I am using Java and MySQL.
An example:
CREATE TABLE main (id INT AUTO_INCREMENT PRIMARY KEY,
primary_id CHAR(3),
secondary_id INT) ENGINE = InnoDB;
CREATE TABLE auxiliary (primary_id CHAR(3),
secondary_id INT AUTO_INCREMENT,
PRIMARY KEY (primary_id, secondary_id)) ENGINE = MyISAM;
CREATE TRIGGER generate_secondary_id
BEFORE INSERT
ON main
FOR EACH ROW
BEGIN
INSERT INTO auxiliary (primary_id) VALUES (NEW.primary_id);
SET NEW.secondary_id = LAST_INSERT_ID();
END
INSERT INTO main (primary_id) VALUES
('A01'),
('A01'),
('B01'),
('C01'),
('A01'),
('B01'),
('A01'),
('B01');
SELECT * FROM main;
id | primary_id | secondary_id
-: | :--------- | -----------:
1 | A01 | 1
2 | A01 | 2
3 | B01 | 1
4 | C01 | 1
5 | A01 | 3
6 | B01 | 2
7 | A01 | 4
8 | B01 | 3
db<>fiddle here
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 1 year ago.
Improve this question
I want to insert 5 million data into a 500 million table named t_phone_numbers without duplicated phones, so I select exist phones all and compare them in memory, but I got Out Of Memory ERROR because the MySQL tables have a large data.
How can I solve it?
Restriction: Mysql table t_phone_numbers cannot change, and the column phone_number is not unique. Create a new table is acceptable.
AS an example.
CREATE TABLE existing (phone VARCHAR(255))
SELECT '123456789' phone UNION ALL
SELECT '789456123' UNION ALL
SELECT '456456456' UNION ALL
SELECT '654654645' UNION ALL
SELECT '123321123' ;
SELECT * FROM existing;
| phone |
| :-------- |
| 123456789 |
| 789456123 |
| 456456456 |
| 654654645 |
| 123321123 |
CREATE TABLE new (phone VARCHAR(255) UNIQUE)
SELECT '123456789' phone UNION ALL
SELECT '464646464' UNION ALL
SELECT '123321123' ;
SELECT * FROM new;
| phone |
| :-------- |
| 123321123 |
| 123456789 |
| 464646464 |
INSERT
INTO existing (phone)
SELECT phone
FROM new
WHERE NOT EXISTS ( SELECT NULL
FROM existing
WHERE new.phone = existing.phone );
SELECT * FROM existing;
| phone |
| :-------- |
| 123456789 |
| 789456123 |
| 456456456 |
| 654654645 |
| 123321123 |
| 464646464 |
db<>fiddle here
The most problem is SELECT part in INSERT. If neither existing nor new table's phone column is indexed then the process may be infinite..
So the recommendations - index this column in new table and (if possible) in existing table. Maybe even create a copy of existing table (phone column only), index it and use in SELECT part.
Anycase this SELECT optimization is the most problem (indexing, using LEFT JOIN, etc.), the insertion itself will be relatively fast.
I am using Microsoft SQL Server with already stored data.
In one of my tables I can find data like:
+--------+------------+
| id | value |
+--------+------------+
| 1 | 12-34 |
| 2 | 5678 |
| 3 | 1-23-4 |
+--------+------------+
I realized that the VALUE column was not properly formatted when inserted.
What I am trying to achieve is to get id by given value:
SELECT d.id FROM data d WHERE d.value = '1234';
Is there any way to format data in column just before SELECT clause?
Should I create new view and modify column in that view or maybe use complicated REGEX to get only digits (with LIKE comparator)?
P.S. I manage database in Jakarta EE project using Hibernate.
P.S.2. I am not able to modify stored data.
One method is to use replace() before the comparison:
WHERE REPLACE(d.value, '-', '') = '1234'
Suppose I have the following tables, in an Oracle DB
Foo:
+--------+---------+---------+
| id_foo | string1 | string2 |
+--------+---------+---------+
| 1 | foo | bar |
| 2 | baz | bat |
+--------+---------+---------+
Bar:
+--------+-----------+--------+
| id_bar | id_foo_fk | string |
+--------+-----------+--------+
| 1 | 1 | boo |
| 2 | 1 | bum |
+--------+-----------+--------+
When I insert into Foo, by using a Dataset and JDBC, such as
Dataset<Row> fooDataset = //Dataset is initialized
fooDataset.write().mode(SaveMode.Append).jdbc(url, table, properties)
an ID is auto-generated by the database. Now when I need to save Bar, using the same strategy, I want to be able to link it to Foo, via id_foo_fk.
I looked into some possibilities, such as using monotonically_increasing_id() as suggested in this question, but it won't solve the issue, as I need the ID generated by the database. I tried what was suggested in this question, but it leads to the same issue, of unique non-database IDs
It's also not possible to select from the JDBC again, as string1 and string2 may not be unique. Nor is it possible to change the database. For instance, I can't change it to be UUID, and I can't add a trigger for it. It's a legacy database that we can only use
How can I achieve this? Is this possible with Apache Spark?
I'm not a Java specialist so you will have to look into the database layer on how to proceed exactly but there are 3 ways you can do this:
You can create a store procedure if the database server you are using is capable of (most do) and call it from your code.
Create a trigger that returns the id number on the first insertion and use it in your next DB insertion.
Use UUID and use this as the key instead of the database auto generated key.