This is currently my MySQL UPDATE query, which is called from program written in Java:
String query = "UPDATE maxday SET DatePressureREL = (SELECT " +
"Date FROM ws3600 WHERE PressureREL = (SELECT MAX" +
"(PressureREL) FROM ws3600 WHERE Date >= '" + Date +
"') AND Date >= '" + Date + "' ORDER BY Date DESC LIMIT 1), " +
"PressureREL = (SELECT PressureREL FROM ws3600 WHERE " +
"PressureREL = (SELECT MAX(PressureREL) FROM ws3600 " +
"WHERE Date >= '" + Date + "') AND Date >= '" + Date +
"' ORDER BY Date DESC LIMIT 1), ...";
try {
s.execute(query);
}
catch (SQLException e) {
System.out.println("SQL error");
}
catch(Exception e) {
e.printStackTrace();
}
Let me explain first, what does it do. I have two tables, first is ws3600, which holds columns (Date, PressureREL, TemperatureOUT, Dewpoint, ...). Then I have second table, called maxday, which holds columns like DatePressureREL, PressureREL, DateTemperatureOUT, TemperatureOUT,... Now as you can see from an example, I update each column, the question is, is there a faster way? I am asking this, because I am calling MAX twice, first to find the Date for that value and secondly to find the actual value. Now I know that I could write like that:
SELECT Date, PressureREL FROM ws3600 WHERE PressureREL =
(SELECT MAX(PressureREL) FROM ws3600 WHERE Date >= '" +
Date + "') AND Date >= '" + Date + "'
ORDER BY Date DESC LIMIT 1
That way I get the Date of the max and the max value at the same time and then update with those values the data in maxday table. But the problem of this solution is, that I have to execute many queries, which as I understand takes alot more time compared to executing one long mysql query because of overhead in sending each query to the server.
If there is no better way, which solution beetwen this two should I choose. The first, which only takes one query but is very unoptimized or the second which is beter in terms of optimization, but needs alot more queries which probably means that the preformance gain is lost because of overhead in sending each query to the server?
Doing 2 queries isn't really a problem for me, but they should be in a transaction (the reads and the write), this way you'll be sure that your update values are not wrong. With one query you do not have this problem.
I think the time lost in reading some data is nothing regarding the time lost by performing a write operation. A write operation is not by definition a fast thing, you could have triggers, you're maybe emptying the query cache from all requests impacting this table, the database needs to sync your write on disk, etc.
The more important thing for you is to keep your process simple, readable, and logic.
1) I think the problem goes deeper than just SQL optimization. Do you think this could be modeled differently where you don't have to migrate data like this (this much, and this often too btw) in the first place? Perhaps just using a FK/cross table to link the two together instead of migrating every field?
2) One query is much better than using JDBC to constantly go back and forth over the connection with new statements. That is a very expensive operation (each time). You will always want to stick to condensing queries into one as opposed to using iteration to make execute many statements.
Working from the inside out, it looks like all your subqueries do the same thing.
What's the point of having a where clause that does Date >= '" + Date + "') AND Date >= '" + Date + "' ?
Without going into column names or technical details, what are the purposes of your two tables?
String query = #"UPDATE maxday SET DatePressureREL = (SELECT Date FROM ws3600 WHERE PressureREL = (SELECT MAX(PressureREL) FROM ws3600 WHERE Date >= #Date) AND Date >= #Date ORDER BY Date DESC LIMIT 1), PressureREL = (SELECT PressureREL FROM ws3600 WHERE PressureREL = (SELECT MAX(PressureREL) FROM ws3600 WHERE Date >= #Date) AND Date >= #Date ORDER BY Date DESC LIMIT 1), ...";
After this, ideally if you were using a SelectCommand of some type instead of a string, you would
query.Parameters.Add(new MySqlParameter("#Date", yourdate));
Alternatively, you can just do this, although it opens you up to sql injection
query = query.replace("#Date", "'" + Date "'");
Either way, it makes the query considerably more legible.
If you can get all the values in one select query, this might work. Use a stored procedure accepting one parameter (date) that does:
One select statement, storing the values in a cursor, and
One update statement, using the values in the cursor.
Cursor Example
Related
Statement.excecuteQuery() is taking too much time, no matter what the query is.
athenaQuery = "SELECT DISTINCT date " +
"FROM sid.lvmh_shareofshelf_new_cat_all_dod "+
"where scope='PCD' and date!='' and country='" + countryName+ "' " +
"and rname='" + rname + "' and top_category_lvmh='" + top_category_lvmh +
"' ORDER BY date DESC";
stmt = conn.createStatement();
long startTime = System.nanoTime();
rs = stmt.executeQuery(athenaQuery);
long endTime = System.nanoTime();
System.out.println("Total time taken : "+ (endTime - startTime));
Time taken was 2776930359 nano seconds (2.776930359seconds)
I don't know internal structure of your project. I suggest few things that will help you.
Create prepare statement instead of its compile every time. Prepare query with parameter. This is syntax level optimization you can do.
Where clause parameter every time you want then you can create index of it. This is database level optimization.
Than try and see is performance improve!!
I checked Athena DB driver,
https://s3.amazonaws.com/athena-downloads/drivers/AthenaJDBC41-1.1.0.jar
it has com.amazonaws.athena.jdbc.AthenaConnection class which implements java.sql.Connection interface from JDBC API. it has all overloaded methods of prepareStatement promised by JDBC.
I assume you already know benefits of using Prepared statement.
In you query in Where clause you date != ''
Is it Character data type ?
you may try
date is not null
I'm trying to retrieve a list of different days from table. They are stored in integer.
cal.getTime().getTime()
Returns the number of milliseconds since January 1, 1970, 00:00:00 GMT represented by this Date object.
What I've written
select strftime('%d-%m-%Y'," + date_clmn + ") from "
+ TABLE_NAME +" group by strftime('%d-%m-%Y'," + date_clmn + ")"
Next, I convert these values toString.
What I get:
24-11--471
And after that when I try to make query with
strftime('%d-%m-%Y',date_clmn)=24-11--471
I get nothing.
What's wrong?
Will I run in trouble with timezone using this way?
If there are some other ways make it, I'll be glad to hear you.
Thanks in advance.
UPDATE
Tried "date(" + date_clmn+ ",'unixepoch')"
now retrieved value is "1970-01-01"
Since you are not storing a date in your table, you need to change your field. Try with this code:
"select strftime('%d-%m-%Y'," + date_clmn + "/1000,'unixepoch') from "
+ TABLE_NAME +" group by strftime('%d-%m-%Y'," + date_clmn + "/1000,'unixepoch')"
I'm adding this because I think it could be helpful to someone. As per documentation strftime() returns a string date in a given format. So in this clause:
strftime('%d-%m-%Y',date_clmn) = 24-11--471
^ ^
string this expression returns an integer
There's a type mismatch here that SQLite doesn't detect. Even query will run perfectly (I tested it and runs without errors), but of course it won't retrieve any row.
The right comparisson would be:
strftime('%d-%m-%Y',date_clmn) = '24-11--471'
Testing: Here's a little piece of code to test this behavior:
CREATE TABLE dateTest(myDate VARCHAR);
INSERT INTO dateTest(myDate) VALUES ('2013-08-29 18:57:00');
SELECT strftime('%Y-%m-%d',myDate) AS formatted, myDate FROM dateTest WHERE strftime('%Y-%m-%d') = 2013-08-29; /* this won't retrieve any row */
SELECT strftime('%Y-%m-%d',myDate) AS formatted, myDate FROM dateTest WHERE strftime('%Y-%m-%d') = '2013-08-29'; /* this will retrieve '2013-08-29 | '2013-08-29 18:57:00' */
And here's the output:
The records in the db look like this:
10//12/2013
10/13/2013
10/16/2013
10/20/2013
I want to search 2013 in the above records. Do i have to split first above values?
from CustomerRelations where DATE like'" + input + "'
Can I use like here?
Or how else can I do this?
Query q = session.createQuery("from CustomerPayment where DATE like '" + code + "'");
The Above code does not work.
Since you've saved the dates as VARCHAR do a wildcard String match
Query q = session.createQuery("from CustomerPayment where DATE like '%" + code + "'");
LIKE clause lets you use two wilcards:
_ = matches just one single character
% = matches any number of characters
Ideally, you should have created your table with a TIMESTAMP column. That would allow you to compare dates accurately and build queries like "fetch all customer payments in the last three months" etc.
you can use
datepart(year,datefield)
where datefield is your field from which you want to retrieve the year
Query q = session.createQuery("SELECT date FROM CustomerPayment WHERE datepart(year,"+Convert.ToDateTime(datefield)+")="+ input+";
Try changing your query to
Query q = session.createQuery("SELECT date FROM CustomerPayment WHERE date LIKE '%"+input+"'"
since your date field is a string, you must do the lookup via a string compare
I have a doubt about adding days to Date in MySQL database.
The following is my code:
res=stat.executeQuery("select st_date from tmp1 where st_date = '"+t1.getText()+"'");
while(res.next())
{
System.out.println(res.getDate(1));
int i=0;
while(i<14)
{
statement.executeUpdate("Insert into datetab values(DATE_ADD('"
+res.getDate("st_date")+"',INTERVAL 1 DAY),'"+tempname+"')");
i=i+1;
}
}
All the updates in datetab table occur, but there is a problem. I will explain the problem with an example. If the date from tmp1 table is 28-12-2000, then after executing the insert query with date_add(), what happens is that 13 new inserts are happening, but all those inserts are "29-12-2000".
If the date from tmp1 table is 28-12-2000 Then after executing the insert query with date_add(), what happens is that 13 new inserts are happening, but all those inserts are "29-12-2000".
Because that is exactly what you are asking for. Your insert statement is:
"Insert into datetab values(DATE_ADD('" + res.getDate("st_date") +
"',INTERVAL 1 DAY),'" + tempname + "')"
Since read.getDate is not changing in the loop, the same value is inserted in every interation.
Instead of "Interval 1 DAY", use "Interval " + i + " Day" should insert different days. Is that what you are looking for?
I have a dropdown list that holds all the columns name from the table. When the user selects a certain column name and enters a specific value to search for along with two different dates, I get that information and update the table as shown below.
But before updating I want to prompt a dialog box asking the user "This many rows will be updated, do you want to do this?". Can I modify something in this code to get that count before it's updated or is there a better way to do this?
sql.append(" UPDATE Table_jack ");
sql.append(" set date = to_date('" + this.getNewDate() + "','MM/DD/YYYY')");
if ((this.getSelectedDDL() != null)&& (this.getSelectedDDL().equals("1"))){
sql.append(" where id_nbr =" + this.getValue() + "'");
sql.append(" and date between to_date('" + this.getDateFrom() + "','MM/DD/YYYY') and to_date('" + this.getDateTo() + "','MM/DD/YYYY')");
}
if ((this.getSelectedDDL() != null)&& (this.getSelectedDDL().equals("2"))){
sql.append(" where name =" + this.getValue() + "'");
sql.append(" and date between to_date('" + this.getDateFrom() + "','MM/DD/YYYY') and to_date('" + this.getDateTo() + "','MM/DD/YYYY')");
}
ResultSet rset = db.executeQuery(sql.toString(),true);
Rather than trying to get a count before you issue the update, get a count of the number of rows actually updated and display that to the user before issuing the COMMIT (or ROLLBACK).
The Statement.executeUpdate method returns the number of rows that the UPDATE statement modified. If you used that rather than executeQuery, you could get a row count from the UPDATE statement. You could then present that to the user before ending the transaction.
You would have to build and run a separate select count(*) query to get the number of rows to be updated. There is also the chance that the data could change between the time you do your select count(*) and update the table. If that is an issue, you would need to lock the table for the duration of your queries.