Why SQL select takes more CPU time in java? - java

I have a java web application that selects one column from table (with 6 million rows) and it takes a lot of CPU time. This select (SELECT id FROM mytable WHERE filename = 'unique_filename') takes significantly less time when executed in query browser.
What can cause this?
Where should I start to look for bottlenecks?
Database is MSSQL 2005 Standard
Java container is Tomcat 5.5 (with sqljdbc 1.2)
More details:
1.Java code
ResultSet rs = null;
PreparedStatement stmt = null;
Connection conn = null;
Integer myId=null;
String myVeryUniqueFileName = strFromSomeWhere;
try
{
conn = Database.getConnection();
stmt = conn.prepareStatement("SELECT id FROM mytable WHERE filename = ?");
stmt.setString(1, myVeryUniqueFileName);
rs = stmt.executeQuery();
if (rs.next())
{
myId= new Integer(rs.getInt(1));
} }
if (rs.next())
{
throw new DBException("Duplicate myId: " + myId);
}
return myId;
} catch (Exception e) {
// handle this
}
The Database object uses DriverManager to receive connection object.
2.SQL table has about 30 columns.
CREATE TABLE [dbo].[calls](
[id] [int] NOT NULL,
...
[filename] [varchar](50) NOT NULL,
...
CONSTRAINT [PK_xxxxxxxxxxxx] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY],
CONSTRAINT [UQ_xxxxxxxxxxxx] UNIQUE NONCLUSTERED
(
[filename] ASC
)WITH (IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
filename column is unique so result set from is allways 1 or null.

With help of way smarter developer I was able to solve this problem. Turns out I was misusing PreparedStatement (aricle).
Based on this I changed java code to:
ResultSet rs = null;
Statement stmt = null;
Connection conn = null;
Integer myId=null;
String myVeryUniqueFileName = strFromSomeWhere;
try
{
conn = Database.getConnection();
stmt = conn.createStatement()
//
rs = stmt.executeQuery("SELECT id FROM mytable WHERE filename = '"
+ myVeryUniqueFileName + "'");
if (rs.next())
{
myId= new Integer(rs.getInt(1));
}
if (rs.next())
{
throw new DBException("Duplicate myId: " + myId);
}
return myId;
} catch (Exception e) {
// handle this
}
After this dababase load fell from average 70% to 13%

Can you post your java code where you're executing this query and retrieving the results?
Possible factors causing Java code to appear to take significantly longer are:
Your query returns a large number of records and you're trying to retrieve them all in Java whereas query browser would only show the first 100 (whatever that number may be) and load others on demand.
You're comparing different times, for example "query took X ms" shown by your query browser with time it takes Java from obtaining the connection till closing it.
Your objects (holding the results) may be expensive to create or they may be doing some processing behind the scenes as they're populated.

I can't speak to MSSQL 2005 specifically, but there can be a difference in execution plan between a prepared statement where you're using bind variables and the equivalent statements where values are embedded.
To test this theory, drop the bind parameter, and instead concatenate the SQL query in Java with the actual filename (in quotes). This way you're comparing apples to apples.
Also, it would be useful with an indication of the difference in CPU time you're experiencing. Is it several orders of magnitude or less than 100%.

The symptoms you describe are most often caused by an incorrectly cached query plan.
Rebuild your indexes or update your statistics.

You are using a statement probably and not a prepared statement. A statement does not get precompiled and cached so the query optimizer has to do the work everytime. If you use a prepared statement it will try and find the best way to execute your query and it will store that. The next time you use it it won't bother to try and work out a good way to get your results it will just the execution plan it already has.

Related

H2 Database result set is readonly

I'm getting the SQLNonTransientException error when trying to update one of my rows in a H2 database.
public static void setNewServiceInformationsToShown() {
try (Connection conn = DriverManager.getConnection("jdbc:h2:" + Main.config_db_location,
Main.config_db_username, Main.config_db_password)) {
//read data from database
PreparedStatement stmt = conn.prepareStatement("SELECT * FROM BCSTASKS_SERVICE");
ResultSet rs = stmt.executeQuery();
while (rs.next()) {
if(rs.getString("Status").equals("Neu") && rs.getBoolean("wasShown") == false) {
rs.updateBoolean("WASSHOWN", true);
}
}
} catch (SQLException e) {
e.printStackTrace();
}
}
The error message already suggests that I should use conn.createStatement and set the ResultSet to CONCUR_UPDATABLE. The error occurs at the line with rs.updateBoolean(...);
Error Message:
The result set is readonly. You may need to use conn.createStatement(.., ResultSet.CONCUR_UPDATABLE). [90140-210]
The problem is I don't know where and how I should use this method. In the same function or at the start of the program?
Most DB code I see doesn't attempt to use the fact that resultsets are updatable, and will instead fire off an additional UPDATE query, which works fine.
However, sure, H2 supports updateable resultsets too. However, some of the features that ResultSets have actually have quite a cost; the DB engine needs to do a boatload of additional bookkeeping to enable such features which have a performance cost. Lots of database queries are extremely performance sensitive, so by default you do not get the bookkeeping and therefore these features do not work. You need to enable them explicitly, that's what the error is telling you.
You're currently calling the 'wrong' preparedStatement method. You want the more extended one, where you pick and choose which additional bookkeeping you want H2 to do for you, in order to enable these things. You want this one.
conn.prepareStatement(
"SELECT * FROM BCSTASKS_SERVICE",
ResultSet.TYPE_SCROLL_INSENSITIVE, // [edited]
ResultSet.CONCUR_UPDATABLE);
That CONCUR_UPDATABLE thing is just a flag you pass to say: Please do the bookkeeping so that I can call .update.
[edited] This used to read 0 before, but as #MarkRotteveel pointed out, that's not valid according to the documentation.
You have to put update query for update data in database but you are going with select query that is the problem.
Select query is used if you have to fetch data from database.
Update query is used for update data in database where data already stored in database but you just overwrite data.
Here down is modified code:
public static void setNewServiceInformationsToShown() {
try (Connection conn = DriverManager.getConnection("jdbc:h2:" + Main.config_db_location,
Main.config_db_username, Main.config_db_password)) {
PreparedStatement stmt = conn.prepareStatement("UPDATE BCSTASKS_SERVICE SET wasShown = ? WHERE status = ? AND wasShown = ?");
stmt.setBoolean(1, true);
stmt.setString(2, "Neu");
stmt.setBoolean(3, false);
stmt.executeUpdate();
stmt.close();
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
You need to create a separate query/prepareStatement for an update. In your case as far as I can see you need only one update query:
conn.prepareStatement("UPDATE BCSTASKS_SERVICE SET WASSHOWN=true where
Status = 'Neu' and wasShown = false "

Can't reach temp table with using prepared statement for second time

I am using temporary tables inside my code in order to [some long sequnce of reasons here] in SQL Server, Java. I was executing my sql queries with using Stament object in java. However, recently I decided to use PreparedStatement in order to avoid injection thing.
My problem is when create a temporary table with using PreparedStatement, I can not reach it with the same prepared statement again. Here is a simple illustration:
sql = "select * into #someTable from (select someColumns from someOtherTable where smth = ? and smth2 = ?)"
PreparedStatement preparedStatement = conn.prepareStatement(sql);
for(int i=0; i<parameters.size(); i++){
preparedStatement.setString(i+1, parameters.get(i).toString());
}
this.rs = preparedStatement.executeQuery();
Until here, it is ok. After getting ResultSet and doing something with it, or without getting a resultSet just for preparedStatement.execute() does not makes difference, I can not reach the #someTable object again.
sql = "select count(*) from #someTable"
preparedStatement = conn.prepareStatement(sql);
this.rs = preparedStatement.executeQuery();
Here this.rs = preparedStatement.executeQuery(); part gives 'Invalid object name #someTable'. I am doing all of the things above with using one Connection object only and without closing or reopening it. I need to use that temp table again. Is there any way to create temp table with PreparedStatement object in java and reuse this temp table again and again? Regards,
Rather late to the party, but facing the same problem and finding the above answer wrong:
Read this article about the problem: https://learn.microsoft.com/en-us/sql/connect/jdbc/using-usefmtonly?view=sql-server-2017
I found that using a PreparedStatement to create the temp table wouldn't work, but if I changed to use a Statement to create the temp table it would work (even without the useFmtOnly).
So start with this (from the MS article) and build on it:
final String sql = "INSERT INTO #Bar VALUES (?)";
try (Connection c = DriverManager.getConnection(URL, USERNAME, PASSWORD)) {
try (Statement s = c.createStatement()) {
s.execute("CREATE TABLE #Bar(c1 int)");
}
try (PreparedStatement p1 = c.prepareStatement(sql); PreparedStatement p2 = c.prepareStatement(sql)) {
((SQLServerPreparedStatement) p1).setUseFmtOnly(true);
ParameterMetaData pmd1 = p1.getParameterMetaData();
System.out.println(pmd1.getParameterTypeName(1)); // prints int
ParameterMetaData pmd2 = p2.getParameterMetaData(); // throws exception, Invalid object name '#Bar'
}
}
The temp table you create in the first statement exists for the scope\lifetime of that request. As soon as you call another query, you're in a different scope so it is no longer present as it would have been cleaned up.
Solutions are either make 2 requests in the same call (not great) or create a global temp table that can be accessed by the second query (still not great).
The better solution is to create a stored procedure that does everything you need, with the temp table creation, querying and tidy up encapsulated in the procedure.
PS I can't see any surrounding code, but beware of SQL Injection when building queries in code like this.
Related info:
Scope of temporary tables in SQL Server

How to minimise deadlock on database table when executing operations from JAVA?

I am facing exception as
com.microsoft.sqlserver.jdbc.SQLServerException: Transaction (Process ID 493) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
When high number of users hit my site's particular transaction. This is because there is lock on a table and others requesting to acquire lock on this particular table.
Also 20 tables are used for this particular transaction and for at least five tables first delete query is executing then fresh data is inserted, which might be holding table for long and causing deadlock. Below is sample code.
public void save2(){
con = DBConnFactory.getConnection();
con.setAutoCommit(false);
String deleteQuery1 = "delete from TEST_TABLE1";
String insertQuery1 = "insert into TEST_TABLE1 values ('66','7')";
String deleteQuery2 = "delete from TEST_TABLE2";
String insertQuery2 = "insert into TEST_TABLE2 values ('66','7')";
String deleteQuery3 = "delete from TEST_TABLE3";
String insertQuery3 = "insert into TEST_TABLE3 values ('66','7')";
String deleteQuery4 = "delete from TEST_TABLE4";
String insertQuery4 = "insert into TEST_TABLE4 values ('66','7')";
ps1 = con.prepareStatement(deleteQuery1);
ps2 = con.prepareStatement(insertQuery1);
ps1.executeUpdate();
ps2.executeUpdate();
ps1 = con.prepareStatement(deleteQuery2);
ps2 = con.prepareStatement(insertQuery2);
ps1.executeUpdate();
ps2.executeUpdate();
ps1 = con.prepareStatement(deleteQuery3);
ps2 = con.prepareStatement(insertQuery3);
ps1.executeUpdate();
ps2.executeUpdate();
ps1 = con.prepareStatement(deleteQuery4);
ps2 = con.prepareStatement(insertQuery4);
ps1.executeUpdate();
ps2.executeUpdate();
System.out.println("success2");
con.commit();
}
public class DBConnection {
public static Connection getConnection() {
Connection conn = null;
try {
Context initContext = new InitialContext();
Context envContext = (Context) initContext.lookup("java:/comp/env");
DataSource dataSource = (DataSource) envContext.lookup("jdbc/DBConnection");
if ((conn == null) || conn.isClosed()) {
conn = dataSource.getConnection();
}
} catch (NamingException e) {
LOG.error("DBConnFactory - JNDI namin error in getConnection =>"+ e.getMessage());
} catch (SQLException e) {
LOG.error("DBConnFactory - SQL error in getConnection =>"+ e.getMessage());
}
return conn;
}
}
I was thinking about deleting all the tables data in stored procedure and then inserting through JAVA based on my business logic. Would it help?
Please suggest how to resolve this, do I need to change my approach?
Since you are deleting all of the rows in the table, try using TRUNCATE TABLE TEST_TABLE1 instead. Truncate is much faster than a Delete, though there are additional restrictions on its use since it is a DDL statement.
String deleteQuery1 = "truncate table TEST_TABLE1;";
Another approach you can use is combine your Delete and Insert statement into a single batch:
String query1 = "delete from TEST_TABLE1; insert into TEST_TABLE1 values ('66','7');";
A Delete statement without a where clause takes a full table lock, which will prevent other delete or insert statements from executing. By combining the two statements into a single batch, you'll reduce the number of connections and
latency.
You can combine all of the delete and insert statements into a single batch.
One significant concern is your use of Delete without a where clause. Since you are running into Deadlocks, that means you have multiple connections hitting the same objects at the same time. For example, if connectionA is inserting into Test_Table1 while connectionB is trying to delete from the table, you run tremendous risk of a soft conflict in which the connection deletes the data in connectionA before it can be used for anything else. You really should not use Delete without a Where clause in a transaction system like this.
If you are trying to delete the specific value you're about to insert, you should add a Where clause to limit it. That may also change the locking level from Table to Row level, assuming you have proper indexing, and eliminate the deadlocks.
String query1 = "delete from TEST_TABLE1 where ID = '66'; insert into TEST_TABLE1 values ('66','7');";

SQLite Query With Parameters Not Working in Java

I have a program that selects from a database given a table and column string.
public void selectAllFrom(String table, String column){
String sql = "SELECT ? FROM ?";
try (Connection conn = this.connect();
PreparedStatement pstmt = conn.prepareStatement(sql)){
pstmt.setString(1, column);
pstmt.setString(2, table);
ResultSet rs = pstmt.executeQuery();
while (rs.next()){
System.out.println(rs.getString(column));
}
} catch (SQLException e){
System.out.println(" select didn't work");
System.out.println(e.getMessage());
}
}
For some reason it is not working and it is going right to catch
Here is the connect() function as well:
private Connection connect(){
Connection conn = null;
// SQLite connection string
String url = "jdbc:sqlite:C:/sqlite/db/chinook.db";
try{
// creates connection to the database
conn = DriverManager.getConnection(url);
System.out.println("Connection to SQLite has been established");
} catch (SQLException e){
System.out.println(e.getMessage());
System.out.println("Connection didn't work");
}
return conn;
}
I know the problem is not with the database because I'm able to run other select queries without parameters. It is the parameters that are giving me the problem. Can anyone tell what the problem is?
A table or column name can't be used as a parameter to PreparedStatement. It must be hard coded.
String sql = "SELECT " + column + " FROM " + table;
You should reconsider the design so as to make these two constant and parameterize the column values.
? is a place holder to indicate a bind variable. When a SQL statement is executed, database first checks syntax, and validates the objects being referenced, columns and access permission for specified objects (i.e metadata about objects) and confirms that all are in place and valid. This stage is called parsing.
Post parsing, it substitutes bind variables to query and then proceeds for actual fetch of results.
Bind variables can be substituted in any place in query to replace an actual hard coded data/strings, but not the query constructs them selves. It means
You can not use bind variables for keywords of sql query (ex: SELECT, UPDATE etc.)
You can not use bind variables for objects or their attributes (i.e table names, column names, functions, procedures etc.)
You can use them only in place of a otherwise hard coded data.
ex: SELECT FIRST_NAME, LAST_NAME, 'N' IS_DELETED FROM USER_DATA WHERE COUNTRY ='CANADA' AND VERIFIED_USER='YES'
In above sample query, 'N','CANADA' and 'YES' are the only strings which can be replaced by a bind variable, not any other word.
Using bind variable is best practice of coding. It improves query performance (when used with large no. of queries in tuned database products like Oracle or MSSQL) and also protects your code against sql injection attacks.
Constructing query by concatenating strings (especially data part of query) is never recommended way. You can still construct a query by concatenation for other parts like table name or column name as long as those strings are not directly taken from input.
Below example is acceptable:
query = "Select transaction_id, transaction_date from ";
if (isHistorical(reportType)
{ query = query + "HISTORY_TRANSACTIONS" ;}
else
{query = query + "PRESENT_TRANSACTIONS" ; }
recommended practice is to use
String query_present = "SELECT transaction_id, transaction_date from PRESENT_TRANSACTIONS";
String query_historical = "SELECT transaction_id, transaction_date from HISTORY_TRANSACTIONS";
if (isHisotrical(reportType))
{
ps.executeQuery(query_historical);
}else{
ps.executeQuery(query_present);
}

Does query plan get recomputed if we close and reopen a prepared statement?

Need some insight on how this works
I have the following piece of code
public static void updateOrdersPrepared(int productId , String productName){
Connection con = getConnection();
try {
pstmt = con.prepareStatement
("update Orders set productname = ? where Prod_Id = ?");
pstmt.setInt(2, productId);
pstmt.setString(1, productName);
pstmt.executeUpdate();
pstmt.close();
con.close();
} catch(SQLException ex) {
System.err.println("SQLException: " + ex.getMessage());
}
JOptionPane.showMessageDialog(null,"Data Updated into Orders Table");
}
Now if this method is called 'N' number of times , will the query plan for the prepared statement be computed 'N' times or just once ?
Is there a real advantage of using prepared statements in such scenarios ?
It will depend on the RDBMS you are targeting.
SQL Server is able to cache a parameterised query plan for the statement "update Orders set productname = ? where Prod_Id = ?" so there will be a benefit. However, it is such a simple query that the benefit will likely be quite small.
You should measure in your particular circumstances (is your server CPU bound, or I/O bound? It's usually the latter).
As you are aware. the real advantage to using a PreparedStatement is preparing it once and then calling many times with different parameters set.

Categories