I have an sqlite database (on android) where I want to store the latest N entries of some data.
The primary key of the table is a date field. Basically whenever I insert some row after the threshold is reached, I want to delete the oldest entry from the table.
Is there any especially clever/nice way to do this? Note that I always check the invariant (nr rows <= THRESHOLD) after each insert so we don't have to deal with anything but with deleting the oldest entry.
What I'm planning to do is basically:
insert data
if count(*) of table <= THRESHOLD: goto 4
DELETE FROM table WHERE date == (SELECT date from table order by date ASC LIMIT 1);
DONE
Note I'm using ORMlite, but since there's no user data involved I can just use raw SQL, so there shouldn't be a problem.
DELETE FROM table WHERE date = (SELECT MAX(date) from table LIMIT 1);
You can use a trigger to delete the oldest row when a new row is added, and the number of rows is over your threshold.
The count of rows can be kept in a separate accounting table to avoid a COUNT on every insert.
Here's a complete example:
create table bookkeepings (bk_name text primary key, bk_value integer not null);
insert or replace into bookkeepings values ('Max Results', 50);
insert or replace into bookkeepings values ('Qty Results', 0);
create table results
(r_timestamp text primary key default (datetime(current_timestamp)),
result text);
create trigger results_limit_trigger before insert on results"
for each row"
when (select bk_value from bookkeepings where bk_name = 'Qty Results')
>= (select bk_value from bookkeepings where bk_name = 'Max Results')
begin
delete from results
where r_timestamp = (select r_timestamp from results order by r_timestamp limit 1);
end;
create trigger results_count_insert_trigger after insert on results
for each row
begin
update bookkeepings set bk_value = bk_value + 1 where bk_name = 'Qty Results';
end;
create trigger results_count_delete_trigger after delete on results
for each row
begin
update bookkeepings set bk_value = bk_value - 1 where bk_name = 'Qty Results';
end;
How about this?
-- keep the last N records by expiration date
declare #expDate datetime
set #expDate = (select top 100 max(dt) from table order by dt asc);
delete from table where dt > #expDate
Related
I want to reset autoincrement in my table. I tried to do this, but when I created and inserted a new row an id is not 1, but was just autoincremented .
#Query("delete from sqlite_sequence where name='approval';")
void delete();
#Query("DELETE FROM approval")
void nukeTable();
How can I reset autoincrement to start from 1 again?
You can reset by update sequence after deleted rows in your-table
UPDATE SQLITE_SEQUENCE SET SEQ=0 WHERE NAME='table_name';
Reference
I don't know what Room may be bringing to the mix, but at the pure Sqlite3 level, both deleting the record from sqlite_sequence and setting its value to zero does the trick, provided all records are deleted from the table in question.
autoinc.sql3
create table ai ( id integer primary key autoincrement, value ) ;
select 'Normal insertion' ;
insert into ai(value) values ( 1 ), ( 42 ) ;
select * from ai ;
select 'Delete sequence only' ;
delete from sqlite_sequence where name = 'ai' ;
insert into ai(value) values ( 1 ), ( 42 ) ;
select * from ai ;
select 'Delete sequence and data' ;
delete from sqlite_sequence where name = 'ai' ;
delete from ai ;
insert into ai(value) values ( 1 ), ( 42 ) ;
select * from ai ;
Running this with sqlite3 < autoinc.sql3 gives the following output:
Normal insertion
1|1
2|42
Delete sequence only
1|1
2|42
3|1
4|42
Delete sequence and data
1|1
2|42
Exactly the same results are obtained if instead of deleting the record from the sequence table we reset it to zero using update sqlite_sequence set seq=0 where name = 'ai' ;.
However, as the results above show, if there are any records in the table, numbering is persisted (presumably with the highest-used-plus-one that a non autoincrement primary key uses).
Could it be something is inserting records between deleting/resetting the sequence and deleting the records from approval? Assuming Room supports it, try both operations within a single transaction.
Below is a sample of what I did
{
...
Class.forName("com.mysql.jdbc.Driver").newInstance();
con=DriverManager.getConnection(ConnectionStr,"root","root");
String prepareStr="DELETE FROM customer_maintenance where id=?";
PreparedStatement pst= con.prepareStatement(prepareStr);
pst.setInt(1,id);
pst.executeUpdate();
String update_key="SET #count = 0; UPDATE customer_maintenance SET customer_maintenance.id = #count:= #count + 1;";
PreparedStatement pst1=con.prepareStatement(update_key);
pst1.executeUpdate();
System.out.println("UPDATE");
}
It is throwing the following exception
com.mysql.jdbc.exceptions.MySQLSyntaxErrorException:
You have an error in your SQL syntax
and I am not sure if it is possible to update the primary key, every time a row is deleted.
You should provide the exception you get, but the first problem I see in your code is that you may try to update a row setting ID = x, while another row could still have ID = x, thus an error for duplicate key.
That's because that update statement won't guarantee that the rows are updated following the ID order.
With the values in your example:
Start Delete Update1 Update2
ID ID ID ID
1 (del)
2 2 2 2
3 3 1 (upd) 1
4 4 4 2 (upd) <- duplicate key
Edit
Easiest fix is adding an order by clause to your query, so that the rows are update starting from the smaller ID.
You should also get rid of the ; at the end of the query.
String update_key="SET #count = 0;
UPDATE customer_maintenance
SET customer_maintenance.id = #count:= #count + 1
ORDER BY customer_maintenance.id";
If you have set auto increment and primary key for Id column while table creation. Auto increment will handle this automatically while database deletion, no need to worry about it.
I'm trying to implement some code for get friends list.
First:
- I have my String id. E.G: 784717
- I have an string with 100 numbers. E.G: 7781,5913,551949194,4919491,...,444131 (One string separated by ,)
- I have 3000 records in my Database with different numbers. (With numbers I mean some kind of ID)
- Of my 100 numbers only 8 are registered in my database.
Question:
How can I know what numbers are registered in the database and insert in other table the relationship?
My table Relationship have this columns:
*number1 - (Here should be my ID)
*number2 - (1 of the 100 numbers that exists)
So in my table Relationship should be 8 new rows.
I tried with :
EXEC('SELECT * FROM Accounts WHERE ID IN(' +#in_mystring+')')
but i don't know how insert in the other table or if is efficiently
Assuming this is SQL Server, and with the help of a parser function
For example
Select * from [dbo].[udf-Str-Parse]('7781,5913,551949194,4919491,...,444131',',')
Returns
Key_PS Key_Value
1 7781
2 5913
3 551949194
4 4919491
5 ...
6 444131
From this sub-query, you can join the results to your Accounts Table
Perhaps something like this
Select A.*
From Accounts A
Join (Select * from [dbo].[udf-Str-Parse]('7781,5913,551949194,4919491,...,444131',',')) B
on A.Key_Value =A.ID
The UDF -- If 2016, There is a native parser.
CREATE FUNCTION [dbo].[udf-Str-Parse] (#String varchar(max),#delimeter varchar(10))
--Usage: Select * from [dbo].[udf-Str-Parse]('Dog,Cat,House,Car',',')
-- Select * from [dbo].[udf-Str-Parse]('John Cappelletti was here',' ')
-- Select * from [dbo].[udf-Str-Parse]('id26,id46|id658,id967','|')
Returns #ReturnTable Table (Key_PS int IDENTITY(1,1) NOT NULL , Key_Value varchar(max))
As
Begin
Declare #intPos int,#SubStr varchar(max)
Set #IntPos = CharIndex(#delimeter, #String)
Set #String = Replace(#String,#delimeter+#delimeter,#delimeter)
While #IntPos > 0
Begin
Set #SubStr = Substring(#String, 0, #IntPos)
Insert into #ReturnTable (Key_Value) values (#SubStr)
Set #String = Replace(#String, #SubStr + #delimeter, '')
Set #IntPos = CharIndex(#delimeter, #String)
End
Insert into #ReturnTable (Key_Value) values (#String)
Return
End
Im trying to keep track of points / stats by making a column auto increment. However, it's not working as I want it. I want it to auto increment if the row gets updated, not if a new row gets added. For example, if I run the update command it will just add one to the "count" column for the row I updated. If I add a new row it'll start at 0!
Here is my code to create a table:
statement = connection.prepareStatement(
"CREATE TABLE IF NOT EXISTS stats" +
"(" +
"id varchar(100) not null," +
"count int not null auto_increment," +
"PRIMARY KEY (id)," +
"KEY (count)" +
")"
);
statement.execute();
Here is how I update to a specific row:
connection = plugin.getHikari().getConnection();
statement = connection.prepareStatement("INSERT INTO stats (id) VALUES(?) ON DUPLICATE KEY UPDATE id=?");
statement.setString(1, id.toString());
statement.setString(2, id.toString());
statement.execute();
Thanks,
- Nicster
Use Before update trigger in this case. Set default value to 1. and update/increment it by 1 on every update using trigger.
1) Remove the AUTO_INCREMENT attribute from the `count` column.
AUTO_INCREMENT isn't a suitable mechanism for what you are trying to achieve.
2) Add a DEFAULT 1 to the definition of the `count` column
When a new row is inserted into the table, and a value is not supplied for the `count` column, the default value will be assigned to the column.
3) re-write the INSERT statement to increment the `count` column when an attempt is made to add a duplicate `id` value
INSERT INTO atickets_stats (id)
VALUES ( ? )
ON DUPLICATE KEY
UPDATE count = count + 1 ;
From mysql doc (http://dev.mysql.com/doc/refman/5.7/en/example-auto-increment.html):
The AUTO_INCREMENT attribute can be used to generate a unique identity for new rows
So you have to chose a different approach:
Read current value from database
Prepare your statement updating also the count value
Problem: Concurrency
You have to synchronize your method call to make it thread-safe.
I'm not sure why you're having that problem. You're close, IMHO. All you need to do is to make count as INT NOT NULL default 0. Then you should always do INSERT ON DUPLICATE KEY UPDATE like below:
CREATE TABLE IF NOT EXISTS atickets_stats (
`id` VARCHAR(100) NOT NULL,
`count` INT(11) UNSIGNED NOT NULL DEFAULT 0,
PRIMARY KEY (`id`)
);
INSERT INTO atickets_stats (id) VALUES(1) ON DUPLICATE KEY UPDATE count = count + 1;
Note, I changed the count column to be INT(11) UNSIGNED because I assume you won't ever store negative value here. You can take out UNSIGNED if you'll have negative value.
I have a table that consists of a unique id, and a few other attributes. It holds "schedules". Then I have another table that holds a list of all the times each schedule has or will "fire". This isn't the exact schema, but it's close:
create table schedule (
id varchar(40) primary key,
attr1 int,
attr2 varchar(20)
);
create table schedule_times (
id varchar(40) foreign key schedule(id),
fire_date date
);
I want to query the schedule table, getting the attributes and the next and previous fire_dates, in Java, sometimes ordering on one of the attributes, but sometimes ordering on either previous fire_date or the next fire_date. Ordering by the attributes is easy, I just stick an "order by" into the string while I'm building my prepared statement. I'm not even sure how to go about selecting the last fire_date and the next one in a single query - I know that I can find the next fire_date for a given id by doing a
SELECT min(fire_date)
FROM schedule_times
WHERE id = ? AND
fire_date > sysdate;
and the similar thing for previous fire_date using max() and fire_date < sysdate. I'm just drawing a blank on how to incorporate that into a single select from the schedule so I can get both next and previous fire_date in one shot, and also how to order by either of those attributes.
You can do that using two sub-queries in Left Joins.
This has the advantage of returning NULL for your fire_dates if there is no next/previous schedule.
Select id, attr1, attr2, next_fire_date, previous_fire_date
From schedule s
Left Join ( Select id, Min(fire_date) As next_fire_date
From schedule_times st
Where st.fire_date > Sysdate
Group By id ) n
On ( n.id = s.id )
Left Join ( Select id, Max(fire_date) As previous_fire_date
From schedule_times st
Where st.fire_date < Sysdate
Group By id ) p
On ( p.id = s.id )
You can then add your ORDER BY on next_fire_date or previous_fire_date.
If performance matters, create a compound index on schedule_times( id, fire_date ), this will allow the sub-queries to read only from this index.
Try something like this:
select schedule.*,
(
select max(si.fire_date) from schedule_times si where si.id = schedule.id and si.fire_date < sysdate
) as prevfire,
(
select min(si.fire_date) from schedule_times si where si.id = schedule.id and si.fire_date > sysdate
) as nextfire
from schedule
where id = ?
order by attr1
Modify the query to
SELECT Distinct S.ID, ATTR1, ATTR2,
LEAD(FIRE_DATE, 1, SYSDATE) OVER (PARTITION BY S.ID ORDER BY S.ID) NEXT_FIRE_DATE,
LAG(FIRE_DATE, 1, SYADATE) OVER (PARTITION BY S.ID ORDER BY S.ID) PREV_FIRE_DATE
FROM SCHEDULE S,SCHEDULE_TIMES ST WHERE ST.ID=S.ID;
THIS WAS SIMPLE QUERY. YOU CAN TRY THIS.
LEAD has the ability to compute an expression on the next rows (rows which are going to come after the current row) and return the value to the current row. The general syntax of LEAD is shown below:
LEAD (sql_expr, offset, default) OVER (analytic_clause)
sql_expr is the expression to compute from the leading row.
offset is the index of the leading row relative to the current
row. offset is a positive integer
with default 1.
default is the value to return if the offset points to a row
outside the partition range.
The syntax of LAG is similar except that the offset for LAG goes into the previous rows.