I am currently trying to use a DECLARE clause in a preparedStatent with jdbc. The code that I wrote is:
statement.executeUpdate(" declare #variable int set #variable = "+timer+" INSERT INTO table1 values (ip, protocol, counter, timer) SELECT ip,protocol,counter,#variable FROM table2 ORDER BY counter DESC LIMIT 5 OFFSET 0 ;");
What I'm trying to get is to create a new table (that is table1) which includes the top 5 from table2 (every 5 secs e.g), with a predefined interval. The interval is the timer variable. The timer variable is passed through a method.
Note: I don't know if it makes any difference to use preparedStatement. I tried both.
Assuming you need to create a new table from a select, then you should use this query instead:
CREATE TABLE table1 SELECT ip,protocol,counter,#variable FROM table2 ORDER BY counter DESC LIMIT 5 OFFSET 0
But if you do this in Java and using PreparedStatement then you can pass the value of #variable as a parameter, thus getting rid of the previous query. So, your query will look like this in Java code:
String sql =
"CREATE TABLE table1"
+ " SELECT ip,protocol,counter,?"
+ " FROM table2"
+ " ORDER BY counter DESC"
+ " LIMIT 5 OFFSET 0";
Assuming you already have the table table1 created and you're just adding the latest results into it from table2, then the query will look like this:
INSERT INTO table1 values (ip, protocol, counter, timer) SELECT ip,protocol,counter,#variable FROM table2 ORDER BY counter DESC LIMIT 5 OFFSET 0
Again, you can pass the value of #variable as a parameter. The query will look like this in Java code:
String sql =
"INSERT INTO table1 (ip, protocol, counter, timer)"
+ " SELECT ip,protocol,counter,?"
+ " FROM table2"
+ " ORDER BY counter DESC"
+ " LIMIT 5 OFFSET 0";
Then, you will prepare the query like this:
PreparedStatement pstmt = con.prepareStatement(sql);
//setting your variable as the parameter in the query
pstmt.setString(1, timer);
In the end, you will use PreparedStatement#execute or PreparedStatement#executeUpdate:
//the former query is a DDL query
pstmt.execute();
//the latter query is a DML query
pstmt.executeUpdate();
Related
I want to fetch data from exasol but only facing this issue when I use limit clause in query.
If i hardcode the limit values in query and don't use prepared statement for then it works fine. But when I try to set int for limit clause in prepared statement it gives me exception
public static final String FROM_DWB_DATA = "SELECT * FROM DWB_DATA a \n"
+ "INNER JOIN DWB_CONN b \n"
+ "ON a.SOURCE_ID=b.ID\n"
+ "WHERE b.PROJECT_ID=? ORDER BY a.TABLE_NAME LIMIT ? , ?";
//and in Prepared statement i am setting these values
PreparedStatement ps = getSQLConnection(projectId, conid)
.prepareStatement(FROM_DWB_DATA_TABLE);
ps.setString(1, projectId);
ps.setInt(2, 0);
ps.setInt(3, 2);
java.sql.SQLException: non-negative integer value expected in LIMIT clause
I guess, it adds your values in quoted form: LIMIT '0', '2'.
Try to build query string normally and run it as simple non-prepared query.
I am having code something like this.
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
Calculation of fullTableName is something like:
public String getFullTableName(final String table) {
if (this.schemaDB != null) {
return this.schemaDB + "." + table;
}
return table;
}
Here schemaDB is the name of the environment(which can be changed over time) and table is the table name(which will be fixed).
Value for schemaDB is coming from an XML file which makes the query vulnerable to SQL injection.
Query: I am not sure how the table name can be used as a prepared statement(like the name used in this example), which is the 100% security measure against SQL injection.
Could anyone please suggest me, what could be the possible approach to deal with this?
Note: We can be migrated to DB2 in future so the solution should compatible with both Oracle and DB2(and if possible database independent).
JDBC, sort of unfortunately, does not allow you to make the table name a bound variable inside statements. (It has its reasons for this).
So you can not write, or achieve this kind of functionnality :
connection.prepareStatement("SELECT * FROM ? where id=?", "TUSERS", 123);
And have TUSER be bound to the table name of the statement.
Therefore, your only safe way forward is to validate the user input. The safest way, though, is not to validate it and allow user-input go through the DB, because from a security point of view, you can always count on a user being smarter than your validation.
Never trust a dynamic, user generated String, concatenated inside your statement.
So what is a safe validation pattern ?
Pattern 1 : prebuild safe queries
1) Create all your valid statements once and for all, in code.
Map<String, String> statementByTableName = new HashMap<>();
statementByTableName.put("table_1", "DELETE FROM table_1 where name= ?");
statementByTableName.put("table_2", "DELETE FROM table_2 where name= ?");
If need be, this creation itself can be made dynamic, with a select * from ALL_TABLES; statement. ALL_TABLES will return all the tables your SQL user has access to, and you can also get the table name, and schema name from this.
2) Select the statement inside the map
String unsafeUserContent = ...
String safeStatement = statementByTableName.get(usafeUserContent);
conn.prepareStatement(safeStatement, name);
See how the unsafeUserContent variable never reaches the DB.
3) Make some kind of policy, or unit test, that checks that all you statementByTableName are valid against your schemas for future evolutions of it, and that no table is missing.
Pattern 2 : double check
You can 1) validate that the user input is indeed a table name, using an injection free query (I'm typing pseudo sql code here, you'd have to adapt it to make it work cause I have no Oracle instance to actually check it works) :
select * FROM
(select schema_name || '.' || table_name as fullName FROM all_tables)
WHERE fullName = ?
And bind your fullName as a prepared statement variable here. If you have a result, then it is a valid table name. Then you can use this result to build a safe query.
Pattern 3
It's sort of a mix between 1 and 2.
You create a table that is named, e.g., "TABLES_ALLOWED_FOR_DELETION", and you statically populate it with all tables that are fit for deletion.
Then you make your validation step be
conn.prepareStatement(SELECT safe_table_name FROM TABLES_ALLOWED_FOR_DELETION WHERE table_name = ?", unsafeDynamicString);
If this has a result, then you execute the safe_table_name. For extra safety, this table should not be writable by the standard application user.
I somehow feel the first pattern is better.
You can avoid attack by checking your table name using regular expression:
if (fullTableName.matches("[_a-zA-Z0-9\\.]+")) {
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
}
It's impossible to inject SQL using such a restricted set of characters.
Also, we can escape any quotes from table name, and safely add it to our query:
fullTableName = StringEscapeUtils.escapeSql(fullTableName);
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
stmt.setString(1, addressName);
StringEscapeUtils comes with Apache's commons-lang library.
I think that the best approach is to create a set of possible table names and check for existance in this set before creating query.
Set<String> validTables=.... // prepare this set yourself
if(validTables.contains(fullTableName))
{
final PreparedStatement stmt = connection
.prepareStatement("delete from " + fullTableName
+ " where name= ?");
//and so on
}else{
// ooooh you nasty haker!
}
create table MYTAB(n number);
insert into MYTAB values(10);
commit;
select * from mytab;
N
10
create table TABS2DEL(tname varchar2(32));
insert into TABS2DEL values('MYTAB');
commit;
select * from TABS2DEL;
TNAME
MYTAB
create or replace procedure deltab(v in varchar2)
is
LvSQL varchar2(32767);
LvChk number;
begin
LvChk := 0;
begin
select count(1)
into LvChk
from TABS2DEL
where tname = v;
if LvChk = 0 then
raise_application_error(-20001, 'Input table name '||v||' is not a valid table name');
end if;
exception when others
then raise;
end;
LvSQL := 'delete from '||v||' where n = 10';
execute immediate LvSQL;
commit;
end deltab;
begin
deltab('MYTAB');
end;
select * from mytab;
no rows found
begin
deltab('InvalidTableName');
end;
ORA-20001: Input table name InvalidTableName is not a valid table name ORA-06512: at "SQL_PHOYNSAMOMWLFRCCFWUMTBQWC.DELTAB", line 21
ORA-06512: at "SQL_PHOYNSAMOMWLFRCCFWUMTBQWC.DELTAB", line 16
ORA-06512: at line 2
ORA-06512: at "SYS.DBMS_SQL", line 1721
String sqlInsertBeacon = "INSERT INTO `beacon` (zone_id, location) VALUE ('(SELECT id FROM zone WHERE GeographicalID = '" + geometry3 + "')', Point(" + x_coordinate + "," + y_coordinate + "))";
System.out.println("The SQL query is: " + sqlInsertBeacon); // Echo for debugging
int countInserted3 = stmt.executeUpdate(sqlInsertBeacon);
System.out.println(countInserted3 + " records inserted.\n");
When I run the above code, the build is successful but the program stops when it reaches the execute line. I am entering using this sql query to insert data into a mysql database. I am not sure where the error is in my query? Can anyone suggest an alternative way or find the mistake?
The output of the program is this, as you can see the program, stops running after the second line:
The SQL query is: INSERT INTO table
(zone_id, location)
VALUES
((SELECT id FROM zone WHERE GeographicalID = '6311599'), Point(-121.9453802,37.3256131) )
;
BUILD SUCCESSFUL (total time: 6 seconds)
For additional information incase it helps:
The stmt, is created like this:
try (
// Step 1: Allocate a database 'Connection' object
Connection conn = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/adhwere2?useSSL=false", "root", "your_new_password"); // MySQL
// Step 2: Allocate a 'Statement' object in the Connection
Statement stmt = conn.createStatement();) {
and the catch exception is :
} catch (SQLException ex) {
}
Try something like this:
String sqlInsertBeacon = "INSERT INTO `beacon` (zone_id, location)" +
" VALUES ( (SELECT id FROM zone WHERE GeographicalID = '" + geometry3 + "'), Point(" +
x_coordinate + "," + y_coordinate + "))";
Just removed the apostrophes aroung the inner SELECT and replaced VALUE with VALUES...
The problem was because the sub-query was returning more than one result, and printing out a stack trace helped debug this error. Using Limit 1 in the sub query also solved this issue.
please use query according to this syntax:
INSERT INTO table
(column1, column2, ... )
VALUES
(expression1, expression2, ... ),
(expression1, expression2, ... ),
...;
your table name is in single quotes and its VALUES not value mind these small things
Correct INSERT INTO SELECT statement looks like this:
INSERT INTO table2
SELECT * FROM table1
WHERE condition;
And you can use PreparedStatement to set parameters in your query.
I have a table with millions of records in it. In order to make the system faster, I need to implement the pagination concept in my Java code. I need to fetch just 1000 records at a time and process them, then pick another 1000 records and do my processing and so on. I have already tried a few things and none of them is working. Some of the things I tried are listed below -
1) String query = "select * from TABLENAME" + " WHERE ROWNUM BETWEEN %d AND %d";
sql = String.format(query, firstrow, firstrow + rowcount);
In the above example, when the query is SELECT * from TABLENAME Where ROWNUM BETWEEN 0 and 10 it gives me a result but when the query is SELECT * from TABLENAME Where ROWNUM BETWEEN 10 and 20, it returns an empty result set. I even tried to run it in the DB, it return Empty result set (not sure why!!)
2) preparedStatement.setFetchSize(100); I have that in my Java code, but it still fetches all the records from the table. Adding this statement didnt affect my code in anyway.
Please help!
It sounds like you are not actually needed to paginate the results but to just process the results in batches. If this is the case then all you need to do is set the fetch size to 1000 using setFetchSize and iterate over the resultset as usual (using resultset.next()) and process the results as you iterate. There are many resources describing setFetchSize and what it does. Do some research:
What does Statement.setFetchSize(nSize) method really do in SQL Server JDBC driver?
How JDBC Statement.SetFetchsize exaclty works
What and when should I specify setFetchSize()?
For oracle pagination there are a lot of resources describing how to do this. Just do a web search. Here are a couple of resources that describe how to do it:
http://www.databasestar.com/limit-the-number-of-rows-in-oracle/
http://ocptechnology.com/how-to-use-row-limiting-clause/
Pagination is not very useful if you do not define a consistent ordering (ORDER BY clause) since you cannot rely on the order they are returned.
This answer explains why your BETWEEN statement is not working: https://stackoverflow.com/a/10318244/908961
From the answer if using oracle older than 12c you need to do a sub select to get your results. Something like:
SELECT c.*
FROM (SELECT c.*, ROWNUM as rnum
FROM (SELECT * FROM TABLENAME ORDER BY id) c) c
WHERE c.rnum BETWEEN %d AND %d
If you are using Oracle 12c or greater I would recommend using the newer OFFSET FETCH syntax instead of fiddling with rownum. See the first link above or
http://www.toadworld.com/platforms/oracle/b/weblog/archive/2016/01/23/oracle-12c-enhanced-syntax-for-row-limiting-a-k-a-top-n-queries
So your query would be something like
String query = "select * from TABLENAME OFFSET %d ROWS FETCH NEXT 1000 ONLY";
String.format(query, firstrow);
or using prepared statements
PreparedStatement statement = con.prepareStatement("select * from TABLENAME OFFSET ? ROWS FETCH NEXT 1000 ONLY");
statement.setInt(1, firstrow);
ResultSet rs = statement.executeQuery();
Alternately you can also use the limit keyword as described here http://docs.oracle.com/javadb/10.10.1.2/ref/rrefjdbclimit.html and your query would be something like
String query = "select * from TABLENAME { LIMIT 1000 OFFSET %d }";
String.format(query, firstrow);
The normal way to implement pagination in Oracle is to use an analytic windowing function, e.g. row_number together with an ORDER BY clause that defines the row ordering. The query with the analytic function is then wrapped into an inline view (or a "window"), from which you can query the row numbers you need. Here's an example that queries the first 1000 rows from my_table (ordering by column_to_sort_by):
select rs.* from
(select t.*,
row_number() over (order by column_to_sort_by) as row_num
from my_table t
) rs
where rs.row_num >= 1 and rs.row_num < 1001
order by rs.row_num
A JDBC implementation could then look like the following:
public void queryWithPagination() throws SQLException {
String query = "select rs.* from"
+ " (select t.*,"
+ " row_number() over (order by column_to_sort_by) as row_num"
+ " from my_table t"
+ " ) rs"
+ " where rs.row_num >= ? and rs.row_num < ?"
+ " order by rs.row_num";
final int pageSize = 1000;
int rowIndex = 1;
try (PreparedStatement ps = myConnection.prepareStatement(query)) {
do {
ps.setInt(1, rowIndex);
ps.setInt(2, rowIndex + pageSize);
rowIndex += pageSize;
} while (handleResultSet(ps, pageSize));
}
}
private boolean handleResultSet(PreparedStatement ps, final int pageSize)
throws SQLException {
int rows = 0;
try (ResultSet rs = ps.executeQuery()) {
while (rs.next()) {
/*
* handle rows here
*/
rows++;
}
}
return rows == pageSize;
}
Note that the table should remain unchanged while you're reading it so that the pagination works correctly across different query executions.
If there are so many rows in the table that you're running out of memory, you probably need to purge/serialize your list after some pages have been read.
EDIT:
If the ordering of rows doesn't matter to you at all, then -- as #bdrx mentions in his answer -- you don't probably need pagination, and the fastest solution would be to query the table without a WHERE condition in the SELECT. As suggested, you can adjust the fetch size of the statement to a larger value to improve throughput.
I have created table with 3 fields language,country,install type. When I write a query to print the maximum occuring value in each of the field, I am getting a weird problem.Can anyone say the reason.Here is my code.
PreparedStatement ps1= null;
ps1 = conn.prepareStatement("desc Configuration");
ResultSet rs1=ps1.executeQuery();
while(rs1.next()) {
System.out.print(rs1.getString(1)+":");
PreparedStatement ps2= null;
ps2 = conn.prepareStatement("select ? from Configuration c1 "+
" group by language "+
" having count(*) >= all " +
" ( select count(*) from Configuration c2 "+
" group by language )");
ps2.setString(1,rs1.getString(1));
ResultSet rs2=ps2.executeQuery();
while(rs2.next())
System.out.print(rs2.getString(1));
System.out.println();
}
The output I am getting here is language:language But the output what I am expecting is
language:english like that. I am getting later output if i replace '?' with language in the prepare statement.But if i give the same with ? I am getting what ever I have given for ps2.setString.
Why is this happening. Any solutions?
? in prepared statements is not a placeholder for textual substitution, it's a parameter, therefore its value is always interpreted as data, not as an arbitrary part of query syntax.
So, in this case the actual query being executed is an equivalent of select 'language' from ....
If you need to substitute parts of the query other than data, you have to use concatenation (beware of SQL injections!):
ps2 = conn.prepareStatement("select "
+ rs1.getString(1)
+ " from Configuration c1 group by language having count(*) >= all( select count(*)from Configuration c2 group by language )");
You can't set column names using a PreparedStatement. You can only set column values.
Instead of using this approach, you will have to build the sql yourself using concatenation, for example:
String sql = "select "+ rs1.getString(1) + " from Configuration c1 group by language having count(*) >= all( select count(*)from Configuration c2 group by language)";
The '?' mark in ps2 is recognized as literal-string. Not as a column name.