We Keep Coding

Java dynamically generate SQL query - ATHENA

I am trying to generate sql query based on user input. There are 4 search fields on the UI:
FIRST_NAME, LAST_NAME, SUBJECT, MARKS
Based on user input I am planning to generate SQL query. Input can be of any combination.
eg: select * from TABLE where FIRST_NAME="some_value";
This query needs to be generated when FIRST_NAME is given and other fields are null
select * from TABLE where FIRST_NAME="some_value" and LAST_NAME="some_value";
This query needs to be generated when FIRST_NAME and LAST_NAME are given and other fields are null
Since there are 4 input fields, number of possible queries that can be generated are 24 (factorial of 4).
One idea is to write if condition for all 24 cases.
Java pseudo code:
String QUERY = "select * from TABLE where ";
if (FIRST_NAME!=null) {
QUERY = QUERY + "FIRST_NAME='use_input_value';"
}
if (LAST_NAME!=null) {
QUERY = QUERY + "LAST_NAME='use_input_value';"
}
if (SUBJECT!=null) {
QUERY = QUERY + "SUBJECT='use_input_value';"
}
if (MARKS!=null) {
QUERY = QUERY + "MARKS='use_input_value';"
}
I am not able to figure out how to generate SQL queries with AND coditions for multiple Input values.
I have been through concepts on dynamically generate sql query but couldn't process further.
Can someone help me on this.
FYI: I have been through How to dynamically generate SQL query based on user's selections?, still not able to generate query string based on user input.

Let's think about what would happen if you just ran the code you wrote and both FIRST_NAME and LAST_NAME are provided. You'll wind up with this:
select * from TABLE where FIRST_NAME='use_input_value';LAST_NAME='use_input_value';
There are two problems here:
The query is syntactically incorrect.
It contains the literals 'use_input_value' instead of the values you want.
To fix the first problem, let's first add and to the start of each expression, and remove the semicolons, something like this:
String QUERY = "select * from TABLE where";
if (FIRST_NAME!=null) {
QUERY = QUERY + " and FIRST_NAME='use_input_value'";
}
Notice the space before the and. We can also remove the space after where.
Now the query with both FIRST_NAME and LAST_NAME will look like this:
select * from TABLE where and FIRST_NAME='use_input_value' and LAST_NAME='use_input_value'
Better but now there's an extra and. We can fix that by adding a dummy always-true condition at the start of the query:
String QUERY = "select * from TABLE where 1=1";
Then we append a semicolon after all the conditions have been evaluated, and we have a valid query:
select * from TABLE where 1=1 and FIRST_NAME='use_input_value' and LAST_NAME='use_input_value';
(It may not be necessary to append the semicolon. Most databases don't require semicolons at the end of a single query like this.)
On to the string literals. You should add a placeholder instead, and simultaneously add the value you want to use to a List.
String QUERY = "select * from TABLE where";
List<String> args = new ArrayList<>();
if (FIRST_NAME!=null) {
QUERY = QUERY + " and FIRST_NAME=?";
args.add(FIRST_NAME);
}
After you've handled all the conditions you'll have a string with N '?' placeholders and a List with N values. At that point just prepare a query from the SQL string and add the placeholders.
PreparedStatement statement = conn.prepareStatement(QUERY);
for (int i = 0; i < args.size(); i++) {
statement.setString(i + 1, args[i]);
}
For some reason columns and parameters are indexed starting at 1 in the JDBC API, so we have to add 1 to i to produce the parameter index.
Then execute the PreparedStatement.

Java Sql 'select sum' giving incorrect value

I'm not experienced in java and I'm stuck with this problem and couldn't figure out the cause for weeks. I have a sql query that selects, counts and groups by the column "name". Next in the 'while(rs.next())' I am trying to select the sum of two other columns, a column named "kolicina" and a column named "cena" here is the code of that method:
public ArrayList<Artikli> getSum(String date){
ArrayList<Artikli> list = new ArrayList<Artikli>();
String query = "select name, count(name) as count from izvestaj where dateZat = '"+date+"' group by name";
try {
rs = st.executeQuery(query);
while(rs.next()){
Artikli art = new Artikli();
art.name = rs.getString("name");
String q = "select sum(kolicina) as total from izvestaj where name = '"+art.name+"' and dateZat = '"+date+"'";
rs2 = st2.executeQuery(q);
while(rs2.next()){
art.kolicina = rs2.getInt("total");
}
String q1 = "select sum(cena) as total1 from izvestaj where name = '"+art.name+"' and dateZat = '"+date+"'";
rs3 = st3.executeQuery(q1);
while(rs3.next()){
art.cena = rs3.getInt("total1")/rs.getInt("count");
}
list.add(art);
}
} catch (SQLException e) {
e.printStackTrace();
}
return list;
}
The problem is that the second query returns the correct sum of the column kolicina, but the third query returns result set with the sum multiplied by the count number (from the first query... That's why i divide it with the count in the end), but still, it doesn't actually return the correct ammount. Column "cena" is actually price in english, so it seems that it's making some average price and than multiplies it with the number of records... I double-checked the insert queries and they are fine, so I am sure i insert the correct values. Any help appreciated.

Instead of issuing two distinct queries per name, just issue
select name,
sum(kolicina) as total,
sum(cena) as total1
from izvestaj
where name is not null and
dateZat = ?
group by name
(re. the questino mark : read up on PreparedStatements and how to set up parameterized queries in order to protect yourself properly against SQL injections.)
(I think you can even skip the first query as its results will also be contained in this query's results.)

Sql2o keep returning a same set of data although query is different

I am new to using SQL2O with MySQL, but I am having a weird problem, where different queries return same results. Is SQL2O returning me cached results?
My code looks like this:
String sql = "SELECT * " +
"FROM report_A" +
"ORDER BY :order :sequence "+
"LIMIT :from, :limit";
int limit = 5;
int startIndex = (page-1)*limit;
String sequence = "DESC";
try(Connection con = sql2o.open()) {
if(order.contains("-")){
order = order.replace("-", "");
sequence= " ASC";
}
Query query= con.createQuery(sql)
.addParameter("from", startIndex)
.addParameter("limit", limit)
.addParameter("order", order)
.addParameter("sequence", sequence);
List<ReportA> result = query.executeAndFetch(ReportA.class);
con.close();
The 4 parameters always change, but the output remains the same. I have verified the queries in mysql workbench, the data is different, but SQL2O returns me the same set of data. Am I missing something?

Your query is invalid. It wont compile and throw an Sql2oException on execution.
The problem is, basically, that you can use parameters only for values, not for table names, column names or other keywords like "ASC". Changing those would change the structure of the query.
It's possible to construct queries with variable structure by good old string concatenation, i.e.
String sql = "SELECT * " +
"FROM report_A" +
"ORDER BY " + order " " + SEQUENCE +
"LIMIT :from, :limit";
and then
query(sql)
.addParameter("from", from)
.addParameter("limit", limit)
.executeAndFetch(...)

Using a database API cursor with JDBC and SQLServer to select batch results

SOLVED (See answer below.)
I did not understand my problem within the proper context. The real issue was that my query was returning multiple ResultSet objects, and I had never come across that before. I have posted code below that solves the problem.
PROBLEM
I have an SQL Server database table with many thousand rows. My goal is to pull the data back from the source database and write it to a second database. Because of application memory constraints, I will not be able to pull the data back all at once. Also, because of this particular table's schema (over which I have no control) there is no good way for me to tick off the rows using some sort of ID column.
A gentleman over at the Database Administrators StackExchange helped me out by putting together something called a database API cursor, and basically wrote this complicated query that I only need to drop my statement into. When I run the query in SQL Management Studio (SSMS) it works great. I get all the data back, a thousand rows at a time.
Unfortunately, when I try to translate this into JDBC code, I get back the first thousand rows only.
QUESTION
Is it possible using JDBC to retrieve a database API cursor, pull the first set of rows from it, allow the cursor to advance, and then pull the subsequent sets one at a time? (In this case, a thousand rows at a time.)
SQL CODE
This gets complicated, so I'm going to break it up.
The actual query can be simple or complicated. It doesn't matter. I've tried several different queries during my experimentation and they all work. You just basically drop it into the the SQL code in the appropriate place. So, let's take this simple statement as our query:
SELECT MyColumn FROM MyTable;
The actual SQL database API cursor is far more complicated. I will print it out below. You can see the above query buried in it:
-- http://dba.stackexchange.com/a/82806
DECLARE #cur INTEGER
,
-- FAST_FORWARD | AUTO_FETCH | AUTO_CLOSE
#scrollopt INTEGER = 16 | 8192 | 16384
,
-- READ_ONLY, CHECK_ACCEPTED_OPTS, READ_ONLY_ACCEPTABLE
#ccopt INTEGER = 1 | 32768 | 65536
,#rowcount INTEGER = 1000
,#rc INTEGER;
-- Open the cursor and return the first 1,000 rows
EXECUTE #rc = sys.sp_cursoropen #cur OUTPUT
,'SELECT MyColumn FROM MyTable'
,#scrollopt OUTPUT
,#ccopt OUTPUT
,#rowcount OUTPUT;
IF #rc <> 16 -- FastForward cursor automatically closed
BEGIN
-- Name the cursor so we can use CURSOR_STATUS
EXECUTE sys.sp_cursoroption #cur
,2
,'MyCursorName';
-- Until the cursor auto-closes
WHILE CURSOR_STATUS('global', 'MyCursorName') = 1
BEGIN
EXECUTE sys.sp_cursorfetch #cur
,2
,0
,1000;
END;
END;
As I've said, the above creates a cursor in the database and asks the database to execute the statement, keep track (internally) of the data it's returning, and return the data a thousand rows at a time. It works great.
JDBC CODE
Here's where I'm having the problem. I have no compilation problems or run-time problems with my Java code. The problem I am having is that it returns only the first thousand rows. I don't understand how to utilize the database cursor properly. I have tried variations on the Java basics:
// Hoping to get all of the data, but I only get the first thousand.
ResultSet rs = stmt.executeQuery(fq.getQuery());
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
I'm not surprised by the results, but all of the variations I've tried produce the same results.
From my research it seems like the JDBC does something with database cursors when the database is Oracle, but you have to set the data type returned in the result set as an Oracle cursor object. I'm guessing there is something similar with SQL Server, but I have been unable to find anything yet.
Does anyone know of a way?
I'm including example Java code in full (as ugly as that gets).
// FancyQuery.java
import java.sql.*;
public class FancyQuery {
// Adapted from http://dba.stackexchange.com/a/82806
String query = "DECLARE #cur INTEGER\n"
+ " ,\n"
+ " -- FAST_FORWARD | AUTO_FETCH | AUTO_CLOSE\n"
+ " #scrollopt INTEGER = 16 | 8192 | 16384\n"
+ " ,\n"
+ " -- READ_ONLY, CHECK_ACCEPTED_OPTS, READ_ONLY_ACCEPTABLE\n"
+ " #ccopt INTEGER = 1 | 32768 | 65536\n"
+ " ,#rowcount INTEGER = 1000\n"
+ " ,#rc INTEGER;\n"
+ "\n"
+ "-- Open the cursor and return the first 1,000 rows\n"
+ "EXECUTE #rc = sys.sp_cursoropen #cur OUTPUT\n"
+ " ,'SELECT MyColumn FROM MyTable;'\n"
+ " ,#scrollopt OUTPUT\n"
+ " ,#ccopt OUTPUT\n"
+ " ,#rowcount OUTPUT;\n"
+ " \n"
+ "IF #rc <> 16 -- FastForward cursor automatically closed\n"
+ "BEGIN\n"
+ " -- Name the cursor so we can use CURSOR_STATUS\n"
+ " EXECUTE sys.sp_cursoroption #cur\n"
+ " ,2\n"
+ " ,'MyCursorName';\n"
+ "\n"
+ " -- Until the cursor auto-closes\n"
+ " WHILE CURSOR_STATUS('global', 'MyCursorName') = 1\n"
+ " BEGIN\n"
+ " EXECUTE sys.sp_cursorfetch #cur\n"
+ " ,2\n"
+ " ,0\n"
+ " ,1000;\n"
+ " END;\n"
+ "END;\n";
public String getQuery() {
return this.query;
}
public static void main(String[ ] args) throws Exception {
String dbUrl = "jdbc:sqlserver://tc-sqlserver:1433;database=MyBigDatabase";
String user = "mario";
String password = "p#ssw0rd";
String driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver";
FancyQuery fq = new FancyQuery();
Class.forName(driver);
Connection conn = DriverManager.getConnection(dbUrl, user, password);
Statement stmt = conn.createStatement();
// We expect to get 1,000 rows at a time.
ResultSet rs = stmt.executeQuery(fq.getQuery());
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
// Alas, we've only gotten 1,000 rows, total.
rs.close();
stmt.close();
conn.close();
}
}

I figured it out.
stmt.execute(fq.getQuery());
ResultSet rs = null;
for (;;) {
rs = stmt.getResultSet();
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
}
if ((stmt.getMoreResults() == false) && (stmt.getUpdateCount() == -1)) {
break;
}
}
if (rs != null) {
rs.close();
}
After some additional googling, I found a bit of code posted back in 2004:
http://www.coderanch.com/t/300865/JDBC/databases/SQL-Server-JDBC-Registering-cursor
The gentleman who posted the snippet that I found helpful (Julian Kennedy) suggested: "Read the Javadoc for getUpdateCount() and getMoreResults() for a clear understanding." I was able to piece it together from that.
Basically, I don't think I understood my problem well enough at the outset in order to phrase it correctly. What it comes down to is that my query will be returning the data in multiple ResultSet instances. What I needed was a way to not merely iterate through each row in a ResultSet but, rather, iterate through the entire set of ResultSets. That's what the code above does.

If you want all records from the table, just do "Select * from table".
The only reason to retrieve in chunks is if there is some intermediate place for the data: e.g. if you are showing it on the screen, or storing it in memory.
If you are simply reading from one and inserting to another, just read everything from the first.You will not get any better performance by trying to retrieve in batches. If there is a difference, it will be negative. Frame your query in a way that brings back everything. The JDBC software will handle all the other breaking-up and reconstituting that you need.
However, you should batch the update/insert side of things.
The set-up would create two statements on the two connections:
Statement stmt = null;
ResultSet rs = null;
PreparedStatement insStmt = null;
stmt = conDb1.createStatement();
insStmt = conDb2.prepareStament("insert into tgt_db2_table (?,?,?,?,?......etc. ?,?) ");
rs = stmt.executeQuery("select * from src_db1_table");
Then, loop over the select as normal, but use batching on the target.
int batchedRecordCount = 0;
while (rs.next()) {
System.out.println(rs.getString("MyColumn"));
//Here you read values from the cursor and set them to the insStmt ...
String field1 = rs.getString(1);
String field2 = rs.getString(2);
int field3 = rs.getInt(3);
//--- etc.
insStmt.setString(1, field1);
insStmt.setString(2, field2);
insStmt.setInt(3, field3);
//----- etc. for all the fields
batchedRecordCount++;
insStmt.addBatch();
if (batchRecordCount > 1000) {
insStmt.executeBatch();
}
}
if (batchRecordCount > 0) {
//Finish of the final (partial) set of records
insStmt.executeBatch();
}
//Close resources...

Error: Before start of result set in Java

I know this would be a foolish question to ask but still i need to do this.
This is a basic program in java application where I want to use 3 queries simultaneously to print the table.
(I'm not using any Primary key in this case so please help me to resolve this without making my attributes as primary keys - I know this is not a good practice but for now i need to complete it.)
my code:
Connection con = null;
Statement stat1 = null, stat2 = null, stat3 = null;
ResultSet rs1, rs2, rs3;
stat1 = con.createStatement();
stat2 = con.createStatement();
stat3 = con.createStatement();
String str = "\nProduct\tC.P\tS.P.\tStock\tExpenditure\tSales";
info.setText(str);
String s1 = "SELECT type, cp, sp, stock FROM ts_items GROUP BY type ORDER BY type";
String s2 = "SELECT expenditure FROM ts_expenditure GROUP BY type ORDER BY type";
String s3 = "SELECT sales FROM ts_sales GROUP BY type ORDER BY type";
rs1 = stat1.executeQuery(s1);
rs2 = stat2.executeQuery(s2);
rs3 = stat3.executeQuery(s3);
String type;
int cp, sp, stock, expenditure, sales;
while( rs1.next() || rs2.next() || rs3.next() )
{
type = rs1.getString("type");
cp = rs1.getInt("cp");
sp = rs1.getInt("sp");
stock = rs1.getInt("stock");
expenditure = rs2.getInt("expenditure");
sales = rs3.getInt("sales");
info.append("\n" + type + "\t" + cp + "\t" + sp + "\t" + stock + "\t" + expenditure + "\t" + sales);
}
Output:
Runtime Exception: Before start of result set

This is the problem:
while( rs1.next() || rs2.next() || rs3.next() )
If rs1.next() returns true, rs2.next() and rs3.next() won't be called due to short-circuiting. So rs2 and rs3 will both be before the first row. And if rs1.next() returns false, then you couldn't read from that anyway...
I suspect you actually want:
while (rs1.next() && rs2.next() && rs3.next())
After all, you only want to keep going while all three result sets have more information, right?
It's not clear why you're not doing an appropriate join, to be honest. That would make a lot more sense to me... Then you wouldn't be trying to use multiple result sets on a single connection, and you wouldn't be relying on there being the exact same type values in all the different tables.

You do an OR so imagine only one ResultSet has a result.
What you end up with is trying to read from empty result sets.

Suppose rs1 has one result and rs3 has 3 results. Now as per your code it will fail for rs1.getString("type"); during second iteration.
Better to loop over each resultSet separately.

This is going to go badly wrong, in the event that there is a type value that's missing from one of your three tables. Your code just assumes you'll get all of the types from all of the tables. It may be the case for your current data set, but it means that your code is not at all robust.
I would seriously recommend having just one SQL statement, that has each of your three selects as subselects, then joins them all together. Your java can just iterate over the result from this one SQL statement.