String sqlInsertBeacon = "INSERT INTO `beacon` (zone_id, location) VALUE ('(SELECT id FROM zone WHERE GeographicalID = '" + geometry3 + "')', Point(" + x_coordinate + "," + y_coordinate + "))";
System.out.println("The SQL query is: " + sqlInsertBeacon); // Echo for debugging
int countInserted3 = stmt.executeUpdate(sqlInsertBeacon);
System.out.println(countInserted3 + " records inserted.\n");
When I run the above code, the build is successful but the program stops when it reaches the execute line. I am entering using this sql query to insert data into a mysql database. I am not sure where the error is in my query? Can anyone suggest an alternative way or find the mistake?
The output of the program is this, as you can see the program, stops running after the second line:
The SQL query is: INSERT INTO table
(zone_id, location)
VALUES
((SELECT id FROM zone WHERE GeographicalID = '6311599'), Point(-121.9453802,37.3256131) )
;
BUILD SUCCESSFUL (total time: 6 seconds)
For additional information incase it helps:
The stmt, is created like this:
try (
// Step 1: Allocate a database 'Connection' object
Connection conn = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/adhwere2?useSSL=false", "root", "your_new_password"); // MySQL
// Step 2: Allocate a 'Statement' object in the Connection
Statement stmt = conn.createStatement();) {
and the catch exception is :
} catch (SQLException ex) {
}
Try something like this:
String sqlInsertBeacon = "INSERT INTO `beacon` (zone_id, location)" +
" VALUES ( (SELECT id FROM zone WHERE GeographicalID = '" + geometry3 + "'), Point(" +
x_coordinate + "," + y_coordinate + "))";
Just removed the apostrophes aroung the inner SELECT and replaced VALUE with VALUES...
The problem was because the sub-query was returning more than one result, and printing out a stack trace helped debug this error. Using Limit 1 in the sub query also solved this issue.
please use query according to this syntax:
INSERT INTO table
(column1, column2, ... )
VALUES
(expression1, expression2, ... ),
(expression1, expression2, ... ),
...;
your table name is in single quotes and its VALUES not value mind these small things
Correct INSERT INTO SELECT statement looks like this:
INSERT INTO table2
SELECT * FROM table1
WHERE condition;
And you can use PreparedStatement to set parameters in your query.
Related
I am new to using SQL2O with MySQL, but I am having a weird problem, where different queries return same results. Is SQL2O returning me cached results?
My code looks like this:
String sql = "SELECT * " +
"FROM report_A" +
"ORDER BY :order :sequence "+
"LIMIT :from, :limit";
int limit = 5;
int startIndex = (page-1)*limit;
String sequence = "DESC";
try(Connection con = sql2o.open()) {
if(order.contains("-")){
order = order.replace("-", "");
sequence= " ASC";
}
Query query= con.createQuery(sql)
.addParameter("from", startIndex)
.addParameter("limit", limit)
.addParameter("order", order)
.addParameter("sequence", sequence);
List<ReportA> result = query.executeAndFetch(ReportA.class);
con.close();
The 4 parameters always change, but the output remains the same. I have verified the queries in mysql workbench, the data is different, but SQL2O returns me the same set of data. Am I missing something?
Your query is invalid. It wont compile and throw an Sql2oException on execution.
The problem is, basically, that you can use parameters only for values, not for table names, column names or other keywords like "ASC". Changing those would change the structure of the query.
It's possible to construct queries with variable structure by good old string concatenation, i.e.
String sql = "SELECT * " +
"FROM report_A" +
"ORDER BY " + order " " + SEQUENCE +
"LIMIT :from, :limit";
and then
query(sql)
.addParameter("from", from)
.addParameter("limit", limit)
.executeAndFetch(...)
SOLVED (See answer below.)
I did not understand my problem within the proper context. The real issue was that my query was returning multiple ResultSet objects, and I had never come across that before. I have posted code below that solves the problem.
PROBLEM
I have an SQL Server database table with many thousand rows. My goal is to pull the data back from the source database and write it to a second database. Because of application memory constraints, I will not be able to pull the data back all at once. Also, because of this particular table's schema (over which I have no control) there is no good way for me to tick off the rows using some sort of ID column.
A gentleman over at the Database Administrators StackExchange helped me out by putting together something called a database API cursor, and basically wrote this complicated query that I only need to drop my statement into. When I run the query in SQL Management Studio (SSMS) it works great. I get all the data back, a thousand rows at a time.
Unfortunately, when I try to translate this into JDBC code, I get back the first thousand rows only.
QUESTION
Is it possible using JDBC to retrieve a database API cursor, pull the first set of rows from it, allow the cursor to advance, and then pull the subsequent sets one at a time? (In this case, a thousand rows at a time.)
SQL CODE
This gets complicated, so I'm going to break it up.
The actual query can be simple or complicated. It doesn't matter. I've tried several different queries during my experimentation and they all work. You just basically drop it into the the SQL code in the appropriate place. So, let's take this simple statement as our query:
SELECT MyColumn FROM MyTable;
The actual SQL database API cursor is far more complicated. I will print it out below. You can see the above query buried in it:
-- http://dba.stackexchange.com/a/82806
DECLARE #cur INTEGER
,
-- FAST_FORWARD | AUTO_FETCH | AUTO_CLOSE
#scrollopt INTEGER = 16 | 8192 | 16384
,
-- READ_ONLY, CHECK_ACCEPTED_OPTS, READ_ONLY_ACCEPTABLE
#ccopt INTEGER = 1 | 32768 | 65536
,#rowcount INTEGER = 1000
,#rc INTEGER;
-- Open the cursor and return the first 1,000 rows
EXECUTE #rc = sys.sp_cursoropen #cur OUTPUT
,'SELECT MyColumn FROM MyTable'
,#scrollopt OUTPUT
,#ccopt OUTPUT
,#rowcount OUTPUT;
IF #rc <> 16 -- FastForward cursor automatically closed
BEGIN
-- Name the cursor so we can use CURSOR_STATUS
EXECUTE sys.sp_cursoroption #cur
,2
,'MyCursorName';
-- Until the cursor auto-closes
WHILE CURSOR_STATUS('global', 'MyCursorName') = 1
BEGIN
EXECUTE sys.sp_cursorfetch #cur
,2
,0
,1000;
END;
END;
As I've said, the above creates a cursor in the database and asks the database to execute the statement, keep track (internally) of the data it's returning, and return the data a thousand rows at a time. It works great.
JDBC CODE
Here's where I'm having the problem. I have no compilation problems or run-time problems with my Java code. The problem I am having is that it returns only the first thousand rows. I don't understand how to utilize the database cursor properly. I have tried variations on the Java basics:
// Hoping to get all of the data, but I only get the first thousand.
ResultSet rs = stmt.executeQuery(fq.getQuery());
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
I'm not surprised by the results, but all of the variations I've tried produce the same results.
From my research it seems like the JDBC does something with database cursors when the database is Oracle, but you have to set the data type returned in the result set as an Oracle cursor object. I'm guessing there is something similar with SQL Server, but I have been unable to find anything yet.
Does anyone know of a way?
I'm including example Java code in full (as ugly as that gets).
// FancyQuery.java
import java.sql.*;
public class FancyQuery {
// Adapted from http://dba.stackexchange.com/a/82806
String query = "DECLARE #cur INTEGER\n"
+ " ,\n"
+ " -- FAST_FORWARD | AUTO_FETCH | AUTO_CLOSE\n"
+ " #scrollopt INTEGER = 16 | 8192 | 16384\n"
+ " ,\n"
+ " -- READ_ONLY, CHECK_ACCEPTED_OPTS, READ_ONLY_ACCEPTABLE\n"
+ " #ccopt INTEGER = 1 | 32768 | 65536\n"
+ " ,#rowcount INTEGER = 1000\n"
+ " ,#rc INTEGER;\n"
+ "\n"
+ "-- Open the cursor and return the first 1,000 rows\n"
+ "EXECUTE #rc = sys.sp_cursoropen #cur OUTPUT\n"
+ " ,'SELECT MyColumn FROM MyTable;'\n"
+ " ,#scrollopt OUTPUT\n"
+ " ,#ccopt OUTPUT\n"
+ " ,#rowcount OUTPUT;\n"
+ " \n"
+ "IF #rc <> 16 -- FastForward cursor automatically closed\n"
+ "BEGIN\n"
+ " -- Name the cursor so we can use CURSOR_STATUS\n"
+ " EXECUTE sys.sp_cursoroption #cur\n"
+ " ,2\n"
+ " ,'MyCursorName';\n"
+ "\n"
+ " -- Until the cursor auto-closes\n"
+ " WHILE CURSOR_STATUS('global', 'MyCursorName') = 1\n"
+ " BEGIN\n"
+ " EXECUTE sys.sp_cursorfetch #cur\n"
+ " ,2\n"
+ " ,0\n"
+ " ,1000;\n"
+ " END;\n"
+ "END;\n";
public String getQuery() {
return this.query;
}
public static void main(String[ ] args) throws Exception {
String dbUrl = "jdbc:sqlserver://tc-sqlserver:1433;database=MyBigDatabase";
String user = "mario";
String password = "p#ssw0rd";
String driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver";
FancyQuery fq = new FancyQuery();
Class.forName(driver);
Connection conn = DriverManager.getConnection(dbUrl, user, password);
Statement stmt = conn.createStatement();
// We expect to get 1,000 rows at a time.
ResultSet rs = stmt.executeQuery(fq.getQuery());
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
// Alas, we've only gotten 1,000 rows, total.
rs.close();
stmt.close();
conn.close();
}
}
I figured it out.
stmt.execute(fq.getQuery());
ResultSet rs = null;
for (;;) {
rs = stmt.getResultSet();
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
if ((stmt.getMoreResults() == false) && (stmt.getUpdateCount() == -1)) {
break;
}
}
if (rs != null) {
rs.close();
}
After some additional googling, I found a bit of code posted back in 2004:
http://www.coderanch.com/t/300865/JDBC/databases/SQL-Server-JDBC-Registering-cursor
The gentleman who posted the snippet that I found helpful (Julian Kennedy) suggested: "Read the Javadoc for getUpdateCount() and getMoreResults() for a clear understanding." I was able to piece it together from that.
Basically, I don't think I understood my problem well enough at the outset in order to phrase it correctly. What it comes down to is that my query will be returning the data in multiple ResultSet instances. What I needed was a way to not merely iterate through each row in a ResultSet but, rather, iterate through the entire set of ResultSets. That's what the code above does.
If you want all records from the table, just do "Select * from table".
The only reason to retrieve in chunks is if there is some intermediate place for the data: e.g. if you are showing it on the screen, or storing it in memory.
If you are simply reading from one and inserting to another, just read everything from the first.You will not get any better performance by trying to retrieve in batches. If there is a difference, it will be negative. Frame your query in a way that brings back everything. The JDBC software will handle all the other breaking-up and reconstituting that you need.
However, you should batch the update/insert side of things.
The set-up would create two statements on the two connections:
Statement stmt = null;
ResultSet rs = null;
PreparedStatement insStmt = null;
stmt = conDb1.createStatement();
insStmt = conDb2.prepareStament("insert into tgt_db2_table (?,?,?,?,?......etc. ?,?) ");
rs = stmt.executeQuery("select * from src_db1_table");
Then, loop over the select as normal, but use batching on the target.
int batchedRecordCount = 0;
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
//Here you read values from the cursor and set them to the insStmt ...
String field1 = rs.getString(1);
String field2 = rs.getString(2);
int field3 = rs.getInt(3);
//--- etc.
insStmt.setString(1, field1);
insStmt.setString(2, field2);
insStmt.setInt(3, field3);
//----- etc. for all the fields
batchedRecordCount++;
insStmt.addBatch();
if (batchRecordCount > 1000) {
insStmt.executeBatch();
}
}
if (batchRecordCount > 0) {
//Finish of the final (partial) set of records
insStmt.executeBatch();
}
//Close resources...
I try to create a PreparedStatement:
stmt = conn.prepareStatement("SELECT POLBRP, POLTYP, POLNOP, INCPTP, TRMTHP, " +
"CLTKYP , CANDTP, POLSTP, EXPRYP, OINCPP, CANRNP, PAYMDP,
KCNFLP, KCRTSP, KACADP, KSCHMP, EXPRYP FROM "
+ POLHDR + " WHERE POLNOP = " + idNumber +
" AND POLBRP = " + branch + " AND POLTYP = " + product +
" AND OINCPP <= "+date );
And this throws an SQLException: [SQL0206] Column AD not in specified tables.
I have no idea where it's getting column AD from as I never specified it in the select clause (unless I'm being completely blind and stupid)
Can anyone help?
If your variables are strings, e.g. branch
" AND POLBRP = " + branch + " ...
then you forgot to quote the values
" AND POLBRP = '" + branch + "' ...
but the real solution is using placeholders
... AND POLBRP = ? ...
which would prevent such problems once and for all, this is what PreparedStatement is designed for
Try to change your query into this:
SELECT
POLBRP,
POLTYP,
POLNOP,
INCPTP,
TRMTHP,
CLTKYP,
CANDTP,
POLSTP,
EXPRYP,
OINCPP,
CANRNP,
PAYMDP,
KCNFLP,
KCRTSP,
KACADP,
KSCHMP,
EXPRYP
FROM TableName WHERE POLNOP = ? AND POLBRP = ? AND POLTYP = ? AND OINCPP <= ?";
Then use:
stmt.setString(1, "ValueOfPOLNOP");
...
When your query is being executed ? will be replaced with the value you passed into PreparedStatement#setString(int, String) method
Preventing SQL Injection in Java shows the proper use of PreparedStatement:
Prepared Statements Variables passed as arguments to prepared
statements will automatically be escaped by the JDBC driver.
Example: ps.1
String selectStatement = "SELECT * FROM User WHERE userId = ? ";
PreparedStatement prepStmt = con.prepareStatement(selectStatement);
prepStmt.setString(1, userId);
ResultSet rs = prepStmt.executeQuery();
From the same source, following in the same section:
Although Prepared Statements helps in defending against SQL Injection,
there are possibilities of SQL Injection attacks through inappropriate
usage of Prepared Statements. The example below explains such a
scenario where the input variables are passed directly into the
Prepared Statement and thereby paving way for SQL Injection attacks.
Example: ps.2
String strUserName = request.getParameter("Txt_UserName");
PreparedStatement prepStmt = con.prepareStatement("SELECT * FROM user WHERE userId = '+strUserName+'");
I have created table with 3 fields language,country,install type. When I write a query to print the maximum occuring value in each of the field, I am getting a weird problem.Can anyone say the reason.Here is my code.
PreparedStatement ps1= null;
ps1 = conn.prepareStatement("desc Configuration");
ResultSet rs1=ps1.executeQuery();
while(rs1.next()) {
System.out.print(rs1.getString(1)+":");
PreparedStatement ps2= null;
ps2 = conn.prepareStatement("select ? from Configuration c1 "+
" group by language "+
" having count(*) >= all " +
" ( select count(*) from Configuration c2 "+
" group by language )");
ps2.setString(1,rs1.getString(1));
ResultSet rs2=ps2.executeQuery();
while(rs2.next())
System.out.print(rs2.getString(1));
System.out.println();
}
The output I am getting here is language:language But the output what I am expecting is
language:english like that. I am getting later output if i replace '?' with language in the prepare statement.But if i give the same with ? I am getting what ever I have given for ps2.setString.
Why is this happening. Any solutions?
? in prepared statements is not a placeholder for textual substitution, it's a parameter, therefore its value is always interpreted as data, not as an arbitrary part of query syntax.
So, in this case the actual query being executed is an equivalent of select 'language' from ....
If you need to substitute parts of the query other than data, you have to use concatenation (beware of SQL injections!):
ps2 = conn.prepareStatement("select "
+ rs1.getString(1)
+ " from Configuration c1 group by language having count(*) >= all( select count(*)from Configuration c2 group by language )");
You can't set column names using a PreparedStatement. You can only set column values.
Instead of using this approach, you will have to build the sql yourself using concatenation, for example:
String sql = "select "+ rs1.getString(1) + " from Configuration c1 group by language having count(*) >= all( select count(*)from Configuration c2 group by language)";
The '?' mark in ps2 is recognized as literal-string. Not as a column name.
I use Java to do some SQL queries.
in general the queries that i want to perform are:
set #uid=?; set #friendsList=?; IF EXISTS(select 1 from fb_user_friends join fb_user on " +
" fb_user.id = fb_user_friends.fb_user_id where uid=#uid) then " +
"update fb_user_friends set friends = #friendsList; ELSE insert " +
"into fb_user_friends(fb_user_id,friends) values(#uid,#friendsList); END IF;
I get the MySQL Connection using:
Class.forName("com.mysql.jdbc.Driver").newInstance();
this._sqlconn = DriverManager.getConnection(this._url,this._userName,this._password);
and I try to execute the following code:
String sql="set #uid=?; set #friendsList=?; IF EXISTS(select 1 from fb_user_friends join fb_user on " +
" fb_user.id = fb_user_friends.fb_user_id where uid=#uid) then " +
"update fb_user_friends set friends = #friendsList; ELSE insert " +
"into fb_user_friends(fb_user_id,friends) values(#uid,#friendsList); END IF;";
try {
PreparedStatement stmt = this._sqlconn.prepareStatement(sql);
stmt.setLong(1,uid);
stmt.setString(2,StringUtils.join(friendsList.toArray(), ','));
stmt.executeUpdate();
}catch (SQLException e) ....
I get the exception:
class com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'set #friendsList='110633,2018837,6813007,8803501,10711399,500061635,500214841,50'
can't I run several commands with prepareStatement ?
Do i need to find a different approach to set the MySQL variables uid and friendsList?
Thanks!
That looks a bit like a mysql stored procedure to me? If I'm not mistaken, you should register that in mysql directly. That way, you're able to call it using your prepared statement.