I know this would be a foolish question to ask but still i need to do this.
This is a basic program in java application where I want to use 3 queries simultaneously to print the table.
(I'm not using any Primary key in this case so please help me to resolve this without making my attributes as primary keys - I know this is not a good practice but for now i need to complete it.)
my code:
Connection con = null;
Statement stat1 = null, stat2 = null, stat3 = null;
ResultSet rs1, rs2, rs3;
stat1 = con.createStatement();
stat2 = con.createStatement();
stat3 = con.createStatement();
String str = "\nProduct\tC.P\tS.P.\tStock\tExpenditure\tSales";
info.setText(str);
String s1 = "SELECT type, cp, sp, stock FROM ts_items GROUP BY type ORDER BY type";
String s2 = "SELECT expenditure FROM ts_expenditure GROUP BY type ORDER BY type";
String s3 = "SELECT sales FROM ts_sales GROUP BY type ORDER BY type";
rs1 = stat1.executeQuery(s1);
rs2 = stat2.executeQuery(s2);
rs3 = stat3.executeQuery(s3);
String type;
int cp, sp, stock, expenditure, sales;
while( rs1.next() || rs2.next() || rs3.next() )
{
type = rs1.getString("type");
cp = rs1.getInt("cp");
sp = rs1.getInt("sp");
stock = rs1.getInt("stock");
expenditure = rs2.getInt("expenditure");
sales = rs3.getInt("sales");
info.append("\n" + type + "\t" + cp + "\t" + sp + "\t" + stock + "\t" + expenditure + "\t" + sales);
}
Output:
Runtime Exception: Before start of result set
This is the problem:
while( rs1.next() || rs2.next() || rs3.next() )
If rs1.next() returns true, rs2.next() and rs3.next() won't be called due to short-circuiting. So rs2 and rs3 will both be before the first row. And if rs1.next() returns false, then you couldn't read from that anyway...
I suspect you actually want:
while (rs1.next() && rs2.next() && rs3.next())
After all, you only want to keep going while all three result sets have more information, right?
It's not clear why you're not doing an appropriate join, to be honest. That would make a lot more sense to me... Then you wouldn't be trying to use multiple result sets on a single connection, and you wouldn't be relying on there being the exact same type values in all the different tables.
You do an OR so imagine only one ResultSet has a result.
What you end up with is trying to read from empty result sets.
Suppose rs1 has one result and rs3 has 3 results. Now as per your code it will fail for rs1.getString("type"); during second iteration.
Better to loop over each resultSet separately.
This is going to go badly wrong, in the event that there is a type value that's missing from one of your three tables. Your code just assumes you'll get all of the types from all of the tables. It may be the case for your current data set, but it means that your code is not at all robust.
I would seriously recommend having just one SQL statement, that has each of your three selects as subselects, then joins them all together. Your java can just iterate over the result from this one SQL statement.
Related
I use prepared statements to read/write data in my DB (SQLite). In my table INVENTORY, there are records which have null value in the column paleta (the column is defined as VARCHAR in the table). I want to select these records and I tried:
sq = "SELECT * FROM INVENTORY WHERE paleta = ? AND product = ? AND lot = ?";
//...
stm = c.prepareStatement(sq);
stm.setNull(1, java.sql.Types.VARCHAR);
stm.setString(2, "theIdOftheProduct");
stm.setString(3, "theLotOftheProduct");
ResultSet rs = stm.executeQuery();
The above query doesn't return anything.. I removed the paleta = ? and I get the records I want.. How can I define the query like SELECT * FROM INVENTORY WHERE paleta is null etc.. using the query parameters?
What you are trying to do is equivalent to writing SELECT * FROM INVENTORY WHERE paleta = NULL ..., which doesn't work.
Since you are essentially searching for rows having a constant value in the paleta column (which happens to be NULL), you can eliminate the first query parameter and explicitly check for null:
sq = "SELECT * FROM INVENTORY WHERE paleta IS NULL AND product = ? AND lot = ?";
stm = c.prepareStatement(sq);
stm.setString(1, "theIdOftheProduct");
stm.setString(2, "theLotOftheProduct");
I found my answer in https://stackoverflow.com/a/4215618/1052284
You'll have to decide upon an unused value. I simply kept it at '' since I don't have empty values.
sq = "SELECT * FROM INVENTORY WHERE IFNULL(paleta, '') = ? AND product = ? AND lot = ?";
//...
stm = c.prepareStatement(sq);
stm.setString(1, ""); // '' for NULL, otherwise a specific value
stm.setString(2, "theIdOftheProduct");
stm.setString(3, "theLotOftheProduct");
But beware if you many queries, it's VERY slow. I clock in at about 4000 times slower, on average, than queries without IFNULL. ~50ms instead of microseconds.
I have a table called [Elenco_Aziende] from which I extract all the record in a resultset. [Elenco_Aziende] is in a relation one to many with other two tables called [Elenco_Autisti] and [Elenco_Veicoli] via a field called [Partita_IVA_Azienda] that is also primary key in [Elenco_Aziende] table.
After extracting all records from [Elenco_Aziende] I perform a loop for each value of [Partita_IVA_Azienda] and then open a new result set to try to read values in related fields of [Elenco_Autisti] and [Elenco_Veicoli] tables and do for each of them some operation.
And here comes strange thing: As long as [Partita_IVA_Azienda] (which is defined as a Text field in the Access DB) values are all the same length everything is OK when I try to read in [Elenco_Autisti] and [Elenco_Veicoli] tables, but if some of [Partita_IVA_Azienda] has a different length then I got error:
net.ucanaccess.jdbc.UcanaccessSQLException: UCAExc:::3.0.1 data exception: numeric value out of range
More precisely here is the nested loop scenario:
Connection con = DriverManager.getConnection("jdbc:ucanaccess://"
+ filepath);
String qry = "SELECT * FROM Elenco_Aziende";
ResultSet rs = stmt.executeQuery(qry);
String cognometest = "";
String nometest ="";
while (rs.next()) {
String partitaiva = "Partita IVA: "
+ rs.getString("Partita_IVA_Azienda") + "\n\r";
String partitaivazienda = rs.getString("Partita_IVA_Azienda");
Statement stmtautisti = con.createStatement();
System.out.println("Sto per eseguire la query per partita iva azienda = " + partitaivazienda + "\n\r");
String qryautisti = "SELECT * FROM Elenco_Autisti WHERE Partita_IVA_Azienda="
+ partitaivazienda; /*!!!!! AND HERE WHEN I EXECUTE NEXT QUERY IS WHERE I GET THE EXCEPTION net.ucanaccess.jdbc.UcanaccessSQLException: UCAExc:::3.0.1 data exception: numeric value out of range more!!!!!*/
ResultSet rsautisti = stmtautisti.executeQuery(qryautisti);
while (rsautisti.next()) {
do something here
}
Statement stmtveicoli = con.createStatement();
String qryveicoli = "SELECT * FROM Elenco_Veicoli WHERE Partita_IVA_Azienda="
+ rs.getString("Partita_IVA_Azienda");
ResultSet rsveicoli = stmtveicoli.executeQuery(qryveicoli);
while (rsveicoli.next()) {
do something else here
}
that is as soon as I execute the query
String qryautisti = "SELECT * FROM Elenco_Autisti WHERE Partita_IVA_Azienda="+ partitaivazienda;
for a different length value of [Partita_IVA_Azienda] I get the problem.
I even tried to export the database in a comma separated value and reimporting it in a brand new one but it did not help. Furthermore, the problem seems to happen just for large number of records in tables [Elenco_Autisti] (138 records) and [Elenco_Veicoli] (287 records), while seems not to happen for small number of records. [Elenco_Aziende] is small (no more than 10 records).
According to the little of what I know about SQL, a WHERE with a text field should be written with the value in apostrophes:
String qryautisti = "SELECT * FROM Elenco_Autisti WHERE Partita_IVA_Azienda='"
+ partitaivazienda + "'";
SOLVED (See answer below.)
I did not understand my problem within the proper context. The real issue was that my query was returning multiple ResultSet objects, and I had never come across that before. I have posted code below that solves the problem.
PROBLEM
I have an SQL Server database table with many thousand rows. My goal is to pull the data back from the source database and write it to a second database. Because of application memory constraints, I will not be able to pull the data back all at once. Also, because of this particular table's schema (over which I have no control) there is no good way for me to tick off the rows using some sort of ID column.
A gentleman over at the Database Administrators StackExchange helped me out by putting together something called a database API cursor, and basically wrote this complicated query that I only need to drop my statement into. When I run the query in SQL Management Studio (SSMS) it works great. I get all the data back, a thousand rows at a time.
Unfortunately, when I try to translate this into JDBC code, I get back the first thousand rows only.
QUESTION
Is it possible using JDBC to retrieve a database API cursor, pull the first set of rows from it, allow the cursor to advance, and then pull the subsequent sets one at a time? (In this case, a thousand rows at a time.)
SQL CODE
This gets complicated, so I'm going to break it up.
The actual query can be simple or complicated. It doesn't matter. I've tried several different queries during my experimentation and they all work. You just basically drop it into the the SQL code in the appropriate place. So, let's take this simple statement as our query:
SELECT MyColumn FROM MyTable;
The actual SQL database API cursor is far more complicated. I will print it out below. You can see the above query buried in it:
-- http://dba.stackexchange.com/a/82806
DECLARE #cur INTEGER
,
-- FAST_FORWARD | AUTO_FETCH | AUTO_CLOSE
#scrollopt INTEGER = 16 | 8192 | 16384
,
-- READ_ONLY, CHECK_ACCEPTED_OPTS, READ_ONLY_ACCEPTABLE
#ccopt INTEGER = 1 | 32768 | 65536
,#rowcount INTEGER = 1000
,#rc INTEGER;
-- Open the cursor and return the first 1,000 rows
EXECUTE #rc = sys.sp_cursoropen #cur OUTPUT
,'SELECT MyColumn FROM MyTable'
,#scrollopt OUTPUT
,#ccopt OUTPUT
,#rowcount OUTPUT;
IF #rc <> 16 -- FastForward cursor automatically closed
BEGIN
-- Name the cursor so we can use CURSOR_STATUS
EXECUTE sys.sp_cursoroption #cur
,2
,'MyCursorName';
-- Until the cursor auto-closes
WHILE CURSOR_STATUS('global', 'MyCursorName') = 1
BEGIN
EXECUTE sys.sp_cursorfetch #cur
,2
,0
,1000;
END;
END;
As I've said, the above creates a cursor in the database and asks the database to execute the statement, keep track (internally) of the data it's returning, and return the data a thousand rows at a time. It works great.
JDBC CODE
Here's where I'm having the problem. I have no compilation problems or run-time problems with my Java code. The problem I am having is that it returns only the first thousand rows. I don't understand how to utilize the database cursor properly. I have tried variations on the Java basics:
// Hoping to get all of the data, but I only get the first thousand.
ResultSet rs = stmt.executeQuery(fq.getQuery());
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
I'm not surprised by the results, but all of the variations I've tried produce the same results.
From my research it seems like the JDBC does something with database cursors when the database is Oracle, but you have to set the data type returned in the result set as an Oracle cursor object. I'm guessing there is something similar with SQL Server, but I have been unable to find anything yet.
Does anyone know of a way?
I'm including example Java code in full (as ugly as that gets).
// FancyQuery.java
import java.sql.*;
public class FancyQuery {
// Adapted from http://dba.stackexchange.com/a/82806
String query = "DECLARE #cur INTEGER\n"
+ " ,\n"
+ " -- FAST_FORWARD | AUTO_FETCH | AUTO_CLOSE\n"
+ " #scrollopt INTEGER = 16 | 8192 | 16384\n"
+ " ,\n"
+ " -- READ_ONLY, CHECK_ACCEPTED_OPTS, READ_ONLY_ACCEPTABLE\n"
+ " #ccopt INTEGER = 1 | 32768 | 65536\n"
+ " ,#rowcount INTEGER = 1000\n"
+ " ,#rc INTEGER;\n"
+ "\n"
+ "-- Open the cursor and return the first 1,000 rows\n"
+ "EXECUTE #rc = sys.sp_cursoropen #cur OUTPUT\n"
+ " ,'SELECT MyColumn FROM MyTable;'\n"
+ " ,#scrollopt OUTPUT\n"
+ " ,#ccopt OUTPUT\n"
+ " ,#rowcount OUTPUT;\n"
+ " \n"
+ "IF #rc <> 16 -- FastForward cursor automatically closed\n"
+ "BEGIN\n"
+ " -- Name the cursor so we can use CURSOR_STATUS\n"
+ " EXECUTE sys.sp_cursoroption #cur\n"
+ " ,2\n"
+ " ,'MyCursorName';\n"
+ "\n"
+ " -- Until the cursor auto-closes\n"
+ " WHILE CURSOR_STATUS('global', 'MyCursorName') = 1\n"
+ " BEGIN\n"
+ " EXECUTE sys.sp_cursorfetch #cur\n"
+ " ,2\n"
+ " ,0\n"
+ " ,1000;\n"
+ " END;\n"
+ "END;\n";
public String getQuery() {
return this.query;
}
public static void main(String[ ] args) throws Exception {
String dbUrl = "jdbc:sqlserver://tc-sqlserver:1433;database=MyBigDatabase";
String user = "mario";
String password = "p#ssw0rd";
String driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver";
FancyQuery fq = new FancyQuery();
Class.forName(driver);
Connection conn = DriverManager.getConnection(dbUrl, user, password);
Statement stmt = conn.createStatement();
// We expect to get 1,000 rows at a time.
ResultSet rs = stmt.executeQuery(fq.getQuery());
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
// Alas, we've only gotten 1,000 rows, total.
rs.close();
stmt.close();
conn.close();
}
}
I figured it out.
stmt.execute(fq.getQuery());
ResultSet rs = null;
for (;;) {
rs = stmt.getResultSet();
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
if ((stmt.getMoreResults() == false) && (stmt.getUpdateCount() == -1)) {
break;
}
}
if (rs != null) {
rs.close();
}
After some additional googling, I found a bit of code posted back in 2004:
http://www.coderanch.com/t/300865/JDBC/databases/SQL-Server-JDBC-Registering-cursor
The gentleman who posted the snippet that I found helpful (Julian Kennedy) suggested: "Read the Javadoc for getUpdateCount() and getMoreResults() for a clear understanding." I was able to piece it together from that.
Basically, I don't think I understood my problem well enough at the outset in order to phrase it correctly. What it comes down to is that my query will be returning the data in multiple ResultSet instances. What I needed was a way to not merely iterate through each row in a ResultSet but, rather, iterate through the entire set of ResultSets. That's what the code above does.
If you want all records from the table, just do "Select * from table".
The only reason to retrieve in chunks is if there is some intermediate place for the data: e.g. if you are showing it on the screen, or storing it in memory.
If you are simply reading from one and inserting to another, just read everything from the first.You will not get any better performance by trying to retrieve in batches. If there is a difference, it will be negative. Frame your query in a way that brings back everything. The JDBC software will handle all the other breaking-up and reconstituting that you need.
However, you should batch the update/insert side of things.
The set-up would create two statements on the two connections:
Statement stmt = null;
ResultSet rs = null;
PreparedStatement insStmt = null;
stmt = conDb1.createStatement();
insStmt = conDb2.prepareStament("insert into tgt_db2_table (?,?,?,?,?......etc. ?,?) ");
rs = stmt.executeQuery("select * from src_db1_table");
Then, loop over the select as normal, but use batching on the target.
int batchedRecordCount = 0;
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
//Here you read values from the cursor and set them to the insStmt ...
String field1 = rs.getString(1);
String field2 = rs.getString(2);
int field3 = rs.getInt(3);
//--- etc.
insStmt.setString(1, field1);
insStmt.setString(2, field2);
insStmt.setInt(3, field3);
//----- etc. for all the fields
batchedRecordCount++;
insStmt.addBatch();
if (batchRecordCount > 1000) {
insStmt.executeBatch();
}
}
if (batchRecordCount > 0) {
//Finish of the final (partial) set of records
insStmt.executeBatch();
}
//Close resources...
I'm running the following code:
// Query without parameters
String query = "select data_source from Qc_data_blending_sources where external_object_type_name='well' AND instance_surrogate_key='837410' and attribute_name='preferred_latitude'";
PreparedStatement testPstmt = con.prepareStatement(query);
ResultSet testRs = testPstmt.executeQuery();
if(testRs.next()){
System.out.println("Result: " + testRs.getString(1));
}
// Query with parameters
sources_query = new StringBuilder("select data_source from Qc_data_blending_sources where external_object_type_name=? AND instance_surrogate_key=? AND attribute_name=?");
sourcesPstmt = con.prepareStatement(sources_query.toString());
sourcesPstmt.setString(1, vo.getWellSurrogateKey()); //set to 837410
sourcesPstmt.setString(2, "well");
sourcesPstmt.setString(3, "preferred_latitude");
dataBlendingSources.append("Preferred latitude: ");
sourcesRs = sourcesPstmt.executeQuery();
if(sourcesRs.next()){
dataBlendingSources.append(sourcesRs.getString(1) + " \n");
}
else{
dataBlendingSources.append(" not found, \n");
System.out.println("Preferred latitude not found. Query: " +
sources_query.toString() +
" \ninstance_surrogate_key: " + vo.getWellSurrogateKey() +
" attribute_name: preferred_latitude");
}
When I run it, it gives a valid result for the first query and says "preferred latitude not found" for the second one. What gives? What could be going on here?
Looks like you are using setString() entirely, but you may need to use setInt() if the column type is not a string for the surrogate key, for example.
Per prepared statement documentation:
Note: The setter methods (setShort, setString, and so on) for setting
IN parameter values must specify types that are compatible with the
defined SQL type of the input parameter. For instance, if the IN
parameter has SQL type INTEGER, then the method setInt should be
used.
Totally off the wall guess (I could be sure if I wrote code to test it, but I'm too lazy):
You call con.prepareStatement() twice on the same connection. I suggest closing the first prepareStatement before getting another prepareStatement from the database. they may be interfering with each other. I would also consider closing the resultSet too when I'm done with it.
coding suggestion: If you aren't already doing so, I suggest closing a connection, preparedStatement, and resultSet in the finally block of a try/catch/finally, in the reverse order you obtain them (checking for null before closing each).
int selectie = toernooienUitvoer.getSelectedRow();
int selectiec = toernooienUitvoer.getSelectedColumn();
String primarykey = (String) toernooienUitvoer.getValueAt(selectie, 0).toString();
String waarde = toernooienUitvoer.getValueAt(selectie, selectiec).toString();
String columnaam = toernooienUitvoer.getModel().getColumnName(selectiec).toString();
String input = JOptionPane.showInputDialog("wijzig geselecteerde data", waarde);
toernooienUitvoer.setValueAt(input, selectie, selectiec);
PreparedStatement stat = con.prepareStatement("UPDATE fullhouse.toernooi SET ? = ? WHERE toernooi.T_code = ?");
stat.setString(1, columnaam);
stat.setString(2, input);
stat.setString(3, primarykey);
Guys, i know the query is correct, if i input the values. my guess my mistake is somewhere in the preparedstatement
i am getting a MySQLSyntaxErrorException:
As mentioned in other answer, the placeholder ? can only be used for values, not for table and column names. Since you are not reusing the PreparedStatement this is quite simple.
Change from
PreparedStatement stat = con.prepareStatement("UPDATE fullhouse.toernooi SET ? = ? WHERE toernooi.T_code = ?")
to
PreparedStatement stat = con.prepareStatement("UPDATE fullhouse.toernooi SET " + columnName + " = ? WHERE toernooi.T_code = ?")
And adjust the index parameter in the setString calls.
I don't think you can use place holder for dynamically passing the column name,your query should be:
"UPDATE fullhouse.toernooi SET colname = ? WHERE toernooi.T_code = ?"
When you use bind variables, it means the statement is precompiled and on the next executions, it will be faster. You are trying to make the name of the column to be a bind variable, which is not possible.
because you obviously need to update several different columns, in order to achieve some speed, you should declare several prepared statements, one for each column. Keep them in a HashMap<String, PreparedStatement>
The column name of a prepared statement cannot be dynamic because, depending on the column name, the query plan will be wildly different (e.g. sometimes table scan will be the fastest, sometimes using an index, sometimes something even more esoteric).
If SQL can't rely on a certain plan being the fastest, it needs to come up with a new one every time - which means there's no point in making a prepared statement which is why you can't do it.