I have a long piece of code in java which uses selenium webdriver and firefox to test my website. Pardon me if I can't reproduce it here. It has an infinite while loop to keep doing its function repeatedly. Thats what its supposed to do. Also, I don't use multi threading.
Sometimes, it gets stuck. I use a windows system and the code runs on command prompt. When it gets stuck, no errors or exceptions are thrown. Its something like "it hangs" (only the window in which the code runs hangs). Then I have to use CTRL + C . Sometimes it resumes working after that, other times it gets terminated and I restart it. It works fine but after some loops it "hangs" again. Also, I've noticed that its usually during the execution of one of the methods querying mysql database.
The code runs an infinite loop. Each time, it queries the mysql database, fetches a value(whose 'status' field is not 'done') from a particular table (one value in each loop) and proceeds with testing with this value.At the end of the loop, the table is updated (the column 'status' is set to 'done' for that value). When there are no new values having 'status' not equal to 'done' in that particular table, it should ideally display "NO NEW VALUE". However, after all the values have been used, it simply takes up the last used value (even though its status is updated to 'done' at the end of previous loop) and goes ahead. I then have to terminate the execution and run the code again. This time when the infinite loop begins, it queries the database and correctly displays "NO NEW VALUE", queries again, displays the message again and so on(which is what it should do)
I close the sql connection using con.close().
It appears that after running the loop for a few times, some resource is getting exhausted somewhere. But this is only a wild guess.
Can anyone suggest what the problem is and how do I fix it ?
Below is a relevant piece of code :
try{
String sql = "select something from somewhere where id = ? and is_deleted = '0';";
System.out.println("\n"+sql + "\n? = " + pID);
PreparedStatement selQuery1 = conn.prepareStatement(sql);
selQuery1.setString(1, pID);
ResultSet rs1 = selQuery1.executeQuery();
//Extract data from result set
while(rs1.next() && i1<6){
//do something
}//end while loop
String sql2 = "select something2 from somewhere2 where id = ? and is_deleted = '0';";
System.out.println("\n"+sql2 + "\n? = " + pjID);
PreparedStatement selQuery2 = conn.prepareStatement(sql2);
selQuery2.setString(1, pjID);
ResultSet rs2 = selQuery2.executeQuery();
//Extract data from result set
while(rs2.next() && i1<6){
//do something
}//end while loop
System.out.println("\nDone.");
conn.close();
}catch (SQLException e) {
flag=false;
}
Please note that no exceptions are thrown anywhere. The window in which the code is running just freezes (that too once in while) after displaying both the query statements.
I forgot to close the query and the resultset. Just closing the connection should implicitly close the query and resultset but it doesn't work always.
I also faced the same problem recently. But in my case the issue was with indexes. I am just pointing out here so that it can be helpful to other folks.
In my case I am fetching the menu items from MenuMaster table from database. So after successfully log in, I am hitting a database to fetch the menu items using MySQL connector driver. Here I need to fetch parent menu with their child menus. In my query, in where clause I have not used any primary key or Unique key. So, it was taking a long time. So just make an index of that key, and it worked as charm...
Related
I am still quite new to the world of java. I am working on my second application which is a program that mass updates a time field in my company's SQL database. I am able to run queries through java, and store each query line in a resultset just fine. The thing is that each line of the result set is an update statement. I want to then run those resultset lines. However over and over I keep getting the "SQL command not properly ended" error message when I know full well these statements are formatted correctly and run just fine in TOAD for oracle. Can anyone help me understand whats going on here? I have also tried batching and continue to get the same error.
This is an example of one of the output lines of my query with table and field names changed.
Update sometable.somefield set COMPLETED_TS ='31-OCT-17 06.00.00.000000000 AM'Where eqact_id ='2559340';
Below you can see the end of my SQL string and my runScript2() method.
"\r\n" +
"\r\n" +
"where \"Center\" = S.CODE and S.TIMEZONE_ID = T.ID"; //This String is named SQL1
public void runScript2(){
try {
PreparedStatement statement0 = Connection1.conn.prepareStatement(SQL1);
ResultSet result0 = statement0.executeQuery();
Connection1.conn.setAutoCommit(false);
while(result0.next()) {
PreparedStatement statementq1=Connection1.conn.prepareStatement(result0.getString(1));
statementq1.executeUpdate();
}
Connection1.conn.commit();
}catch (SQLException e1) {
e1.printStackTrace();
}
}
Well I am angry and happy at the same time as I figured out that the issue was that my result0.getString(1) lines had a semicolon at the end of each and for some reason Java didn't like this. They run just fine without this.
Live and you learn I guess.
I have read the previous questions but none of them seem to match my problem although they might seem similar at first. :/_
So, I am working on a local database on Java(JDBC). When I press a button I should be getting the result of a "SELECT" query. So far so good, but for some reason which my beginner brain does not understand I keep getting only one row from the query. I have even run the same exact query on "DB Browser for SQLite" and it returns the correct result (1+ rows) .
So this is the method I am using to get the result of the query:
public ResultSet returnBill(int no) throws SQLException{
String sql = "SELECT * FROM billList WHERE no = " + no + " ;";
ResultSet thisSet = stmt.executeQuery(sql); // stmt is a 'Statement' type variable
return thisSet;
}
The method does not crash but it only returns the very first row of a query which should return more than 2 ( while (thisSet.next()) RUNS ONCE). I run other "SELECT" queries on the program which are supposed to return more than one rows and they all work fine so it's not a matter of not being able to start/close the connection etc.
Below is the method being used:
int number = table.getModel().getValueAt(rows, 0);
ResultSet thisSet = db.returnBill(number);
while (thisSet.next()){
String name = thisSet.getString("name");
int quantity = thisSet.getInt("quantity");
// do something with the returned data
}
So I get this magical number from a table (of course I made sure it's not 0, -1 etc.) and I run a query using that number. You could think of the structure of the table consisting of columns :
number | name | quantity |
where 'number' is nonzero.
I understand that probably using this method to run a query on a DB might not be safe or might post security threats but it's not the case right now. I have been working on this project for quite a long time already and I have been through many silly mistakes and I think this is yet one of them. Any help is APPRECIATED ! :D
So yes, it was a silly mistake as I expected.
So I had previously initiated a variable
Database db = new Database();
which opened the database for 2 queries (the SELECT query and an UPDATE query on another table as shown) which would then be closed at the end of the following code.
When I removed this UPDATE query however the loop executed the correct amount of times. So it seems like the SQLite JDBC is somehow prone to running a SELECT and UPDATE query on the same Statement (as far as my super mega* brain perceives it.)
So I created 2 connections at the very beginning and closed them at the end using one of them for the SELECT and the other one for the UPDATE query:
Database db = new Database(); // open database
Database db2 = new Database(); // open another creepy one :/
int number = table.getModel().getValueAt(rows, 0);
ResultSet thisSet = db.returnBill(number);
while (thisSet.next()){
String name = thisSet.getString("name");
int quantity = thisSet.getInt("quantity");
// do something with the returned data
// --------> STUPIDO <----------
//** Now executing an UPDATE query on db2 :
// ex.: UPDATE anotherTable SET amount = (current+ "+ quantity+") WHERE name= '" + name+ "' ;";
}
db.closeConn(); // close db ++
db2.closeConn(); // closes db 2
I don't know if this is the best approach but it solved my problem, so I'm leaving it so probably it could help. Any suggestions though would be welcomed :D
I'm starting with SQL and trying to mix it with Java app. I have table ZAMESTNANEC containing 6 rows.
When I issue the command delete from ZAMESTNANEC where ID = 7; in SQL it will delete in no time. A few milliseconds. But when I use this in my Java app, the app will freeze in processing. I waited for 4 minutes and nothing happened (and due to its working state I can't do anything else). Oh and the row wasn't deleted.
I read this topic about deleting but it didn't help me much. In fact it didn't help me at all.
oracle delete query taking too much time
I tried to debug it but it's frozen on this command. I don't understand why in SQL it works fine and in Java app it doesn't. Other commands like SELECT works fine.
JDBC here - http://pastebin.com/BRh06yc8
Code from button here
private void jButtonOdeberZamActionPerformed(java.awt.event.ActionEvent evt) {
try{
OracleConnector.setUpConnection("xxxxxxxx", 1521, "ee11",
"NAME", "PASSWORD");
conn = OracleConnector.getConnection();
stmt = conn.createStatement();
stmt.executeQuery("delete from ZAMESTNANEC where ID = 7");
} catch(SQLException ex){
System.out.println(ex);
}
executeQuery should be used for queries that are expected to return results. Try executeUpdate instead and see if that helps. It could be that your app is waiting to receive results which never come back. By Tom H
Thank you Tom.
I have a wierd behavior in a Java application.
It issues simple queries and modifications to a remote MySQL database. I found that queries, run by executeQuery() work just fine, but inserts or delete to the database run through executeUpdate() will fail.
Ruling out the first thing that comes to mind: the user the app connects with has correct privilledges set up, as the same INSERT run from the same machine, but in DBeaver, will produce the desired modification.
Some code:
Connection creation
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection(url, user, pass);
Problematic part:
Statement parentIdStatement = connection.createStatement();
String parentQuery = String.format(ProcessDAO.GET_PARENT_ID, parentName);
if (DEBUG_SQL) {
plugin.getLogger().log(Level.INFO, parentQuery);
}
ResultSet result = parentIdStatement.executeQuery(parentQuery);
result.first();
parentId = result.getInt(1);
if (DEBUG_SQL) {
plugin.getLogger().log(Level.INFO, parentId.toString()); // works, expected value
}
Statement createContainerStatement = connection.createStatement();
String containerQuery = String.format(ContainerDAO.CREATE_CONTAINER, parentId, myName);
if (DEBUG_SQL) {
plugin.getLogger().log(Level.INFO, containerQuery); // works when issued through DBeaver
}
createContainerStatement.executeUpdate(containerQuery); // does nothing
"DAOs":
ProcessDAO.GET_PARENT_ID = "SELECT id FROM mon_process WHERE proc_name = '%1$s'";
ContainerDAO.CREATE_CONTAINER = "INSERT INTO mon_container (cont_name, proc_id, cont_expiry, cont_size) VALUES ('%2$s', %1$d, CURRENT_TIMESTAMP(), NULL)";
I suspect this might have to do with my usage of Statement and Connection.
This being a lightweight lightly-used app, I went to simplicity, so no framework, and no specific isntructions regarding transactions or commits.
So, in the end, this code was just fine. It worked today.
To answer the question: where to look first in a similar case (SELECT works but UPDATE / INSERT / DELETE does not)
If rights are not the problem, then there is probably a lock on the table you try to modify. In my case, someone left with an uncommited transaction open.
Proper SQL exceptions logging (which was suboptimal in my case) will help you figure it out.
I have a fairly straightforward Java class here which creates 2 thread pools....
Connects to a running URL stream and reads in entries line by line submitting each entry into a back end MySQL DB.
Spawns several threads each of which will carry out the same process (below)
1.Get the oldest DB entry from above
2.Parse and process it accordingly
3.Save several sections to another DB table
4.Delete this DB entry from the running table to signify that analysis is complete for it
5.End Thread
The reason I need 2 pools is because the read process is MUCH faster than the analyse and if I read & analyse each entry as it comes through the entries back up too fast and the incoming stream breaks. By putting in this separation the read can happen as fast as it needs to and the analyse can proceed as fast as it can knowing that the records to catch up with are safe and available to catch up on.
The problem I have is that each concurrent thread is getting the same oldest record. I need to know what the best way would be to ensure the separate threads all run concurrently but each access unique oldest DB entries.
Thanks in advance.
EDIT=================================
Thanks folks for the replies so far...
To further expand on the current setup I was attempting here perhaps this code segment will be helpful...
try
{
String strQuery1 = "SELECT lineID,line FROM lineProcessing ORDER BY lineID ASC LIMIT 1;";
String strQuery2 = "DELETE from lineProcessing WHERE lineID = ?";
DBConnector dbc = new DBConnector(driver,url,userName,passwd);
Connection con = dbc.getConnection();
con.setAutoCommit(false);
PreparedStatement pstmt = con.prepareStatement(strQuery1);
rs = pstmt.executeQuery();
//Now extract the line & Id from the returned result set
while (rs.next()) {
lineID = Integer.parseInt(rs.getString(1));
line = rs.getString(2);
} //end while
//Now delete that entry so that it cannot be analysed again...
pstmt = con.prepareStatement(strQuery2);
pstmt.setString(1, lineID.toString());
int res=pstmt.executeUpdate();
con.commit();
con.setAutoCommit(true);
con.close();
}
catch (SQLException e) {
System.out.println(">>>EXCEPTION FOUND IN QUERY = " + strQuery1 + " __or__ " + strQuery2);
e.printStackTrace();
}
...So as you can see basically opening a DB connection, setting it to "Autocommit = false", execute QUERY1, execute QUERY2, commit both transactions finally closing the connection. This should be all each individual thread will be required to complete. The problem is each of the X threads I have running in the analysis thread pool all get spawned and all execute this batch of code simultaneously (which I would expect) but do not respect the single connection access to the DB I think I have set up above. They all then return with the same line for analysis. When the threads next loop around for iteration #2, they all then return this new last row for analysis following the previous deletion.
Any further suggestions please - including maybe a good example of forced transactional SQL through java?
Thanks again folks.
First, add a nullable datetime column that signifies that the row has been "picked up" at a certain time.
Then in your processing thread:
Start a transaction
Find the oldest row with a "picked up" time of null
Update the picked up time to the current system time
Commit the transaction.
Make sure your isolation level is set to at least READ UNCOMMITTED, and no two threads should get the same row. Also, if a processing thread dies and abandons it's row, you can find that out by periodically querying for rows with a "picked up" time of earlier than some value, and reprocess those by setting the picked up time to null.
Or just switch to a transactional message queue, which does most of this for you automatically.
Another solution is to have the worker threads all wait on a singleton that contains the key to the row. Write the row, place the key in the object, and then notify. The "next" worker thread will pick up the key and operate on it. You will need to make sure that a worker was waiting and what not.