I need to initialize a database from my Java application. For reasons of code maintainability, I would like to maintain the SQL code separately from the Java code (it is currently in a separate source file).
The first few lines of the file are as follows:
-- 1 - Countries - COUNTRIES.DAT;
drop table Countries if exists;
create table Countries(
CID integer,
ECC varchar(2),
CCD varchar(1),
NAME varchar(50));
I read the SQL code from the file and store it in a string. Then I do:
PreparedStatement stmt = dbConnection.prepareStatement(sqlString);
This fails with the following exception:
java.sql.SQLSyntaxErrorException: unexpected token: CREATE : line: 2
This looks as if JDBC doesn't like multiple SQL statements in a single PreparedStatement. I have also tried CallableStatement and prepareCall(), with the same result.
Does JDBC provide a way to pass the entire SQL script in one go?
The JDBC standard (and the SQL standard for that matter) assumes a single statement per execute. Some drivers have an option to allow execution of multiple statements in one execute, but technically that option violates the JDBC standard. There is nothing in JDBC itself that supports multi-statement script execution.
You need to separate the statements yourself (on the ;), and execute them individually, or find a third-party tool that does this for you (eg MyBatis ScriptRunner).
You might also want to look at something like flyway or liquibase.
To run a hardcoded / loaded from file queries you can use execute like:
Statement stmt = null;
String query = "drop table Countries if exists;
create table Countries(
CID integer,
ECC varchar(2),
CCD varchar(1),
NAME varchar(50));";
try {
stmt = con.createStatement();
stmt.execute(query);
} catch (SQLException e ) {
JDBCTutorialUtilities.printSQLException(e);
} finally {
if (stmt != null) { stmt.close(); }
}
If you want to run dynamic queries for example to append values you have to use PreparedStatement. For running a query from a file it's not recommended to put dynamic queries in it.
Related
For example, I have the code below:
import java.sql.*; ...
public void main (string[] args){
try {
Class.forName („COM.ibm.db2.jdbc.net.DB2Driver“);} catch (ClassNotFoundException e) {//error handling}
try {
String url = "jdbc:db2://host:6789/myDB2"
Connection con = DriverManager.getConnection(url, "login", "password");
PreparedStatement pStmt = con.prepareStatement("UPDATE PERS SET Salary=Salary*2.0 WHERE PNR=?"
pStmt.setInt (1, 35);
pStmt.executeUpdate();
pStmt.setString (1, args[0]);
pStmt.executeUpdate();
con.close();
} catch (SQLException e) { //error handling}
}
Presumably we have the table like:
+--------+----------+-----------+
|PNR |Name |Salary |
+--------+----------+-----------+
|34 |Tim |20000 |
+--------+----------+-----------+
|35 |John |45000 |
+--------+----------+-----------+
I have a difficult time predicting what will happen if:
args[0]="35 OR Salary<100000"
Doesn't the setString command replace args[0] with 35 OR Salary < 100000 and then all the salary record gets doubled?
Server-Side Prepared Statements
SQL parameters help to avoid SQL injection because the value of the parameter is not combined with the SQL query at all. The SQL query with parameter placeholders is sent to the MySQL server, where it is parsed and analyzed. It does things like check that you wrote valid SQL syntax, that the tables and columns you reference exist, and you have the right privileges to access those tables and columns.
This is why parameters can't be used for table names or column names or other syntax. Because the validation occurs when the parameters are still left as placeholders. The value of the parameters is sent later, so the validation must assume a parameter must replace only a single scalar value in your SQL query.
After this point, the query is stored internally in the MySQL server as non-textual data structures. It is no longer in SQL, it's just a number of internal objects in the MySQL code. The places where you had used ? become query elements that MySQL knows need to be supplied with values before it can execute the query.
When you run pStmt.executeUpdate() the values of the variables you bound to the parameters are sent to the MySQL server. They are filled into the placeholders in the non-textual representation of the query.
This way, the parameter values are not combined until after the parsing is done, therefore there's no way for the content of the parameter to change the SQL syntax. It affects the SQL query only like a single string would, as if there were a type of quote delimiter that could not be broken by unmatched quote characters in the parameter content.
Query parameters are a reliable way to protect against SQL injection.
Emulated Prepared Statements
Some drivers implement an "emulated" prepared statement. This means it does nothing with the SQL query you pass to prepareStatement(), except save the SQL string in the JDBC driver (on the client-side). It does not send the SQL query to the server at this time.
Then when you run executeUpdate() your variables are interpolated into the parameter placeholders in the SQL string, and the full string is sent to the server. Then the MySQL server parses the combined SQL query, with parameter values and all. MySQL Server can't even tell which values were literal values in the original SQL query versus which were combined as parameters. They all appear as literal values to the parser.
In this case, you have to trust that the JDBC driver does correct escaping, so quotes and other characters inside your parameter content can't mix up the SQL parser. The driver should be well-tested to handle all cases, like special character sets, and hex-encoded quote characters and other ways to trick it.
That won't cause SQL injection issues. It will translate to:
UPDATE PERS SET Salary=Salary*2.0 WHERE PNR='35 OR Salary<100000'
The inserted quotes will save you from SQL injection. I am simplifying a bit. JDBC implementation determines how exactly PreparedStatement translates to a real SQL query. It doesn't necessarily actually translate it to the above SQL. But this is one way it can prevent attacks.
Careful, though. If you use user input to create your SQL, you sill still be susceptible to SQL injection. As long as you use user inputs only to call .setXYZ() params, you'll be safe from it.
I want to run a native SQL from a file using Hibernate. The SQL can contain several statements creating the database structure (i.e. tables, constraints but no insert/update/delete statements).
Example, very simple query is below (which contains the following two SQL statements)
CREATE DATABASE test;
CREATE TABLE test.testtbl( id int(5));
I am using MySQL db, and when I run the above query I am gettng syntax error returned. When I run them one by one, its ok.
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:
You have an error in your SQL syntax; check the manual that corresponds to your
MySQL server version for the right syntax to use near
'CREATE TABLE test.testtbl( id int(5))' at line 1
The code to run the query is below (above statement is assigned to 'sql' variable):
session = sf.openSession();
session.beginTransaction();
Query qry = session.createSQLQuery(sql);
qry.executeUpdate();
session.getTransaction().commit();
Any help would be appreciated.
As others have explained
You must run these queries one by one.
The hibernate code gets translated into running one update statement on JDBC.
But you provided two update statements.
In addition,
I personally prefer to have the code that creates tables outside of the Java application, in some DB scripts.
The parameters of the method createSQLQuery is t-sql code;
t-sql code to ensure that in the mysql interface analyzer correctly.
You can try changed the sql :'CREATE TABLE testtbl(id int(5));'
by the way you can use JDBC Connection api (Don't recommend to do so)
Such as:
java.sql.Connection conn=session.connection();
I need to define some variables in Oracle to be used further down our application's database installation scripts. Basically, the way our installer works now is it reads in the script files and calls each one via JDBC in Java. I need to have Oracle do the variable substitution on the database side, as there are procedures, triggers, create statements, etc. that will need to refer to them (So like "CREATE TABLE &&MYSCHEMA.TBL_NAME ...").
The problem I am having is that the DEFINE statement is throwing an error when calling it from Java (example):
private static void testDefineVariables() {
String url = "jdbc:oracle:thin:#localhost:1521:LOCALDEV";
String username = "SYSTEM";
String password = "manager42";
Connection conn = null;
Statement stmt = null;
try {
conn = DriverManager.getConnection(url, username, password);
stmt = conn.createStatement();
//Execute the sql
stmt.execute("DEFINE MYSCHEMA='TESTSCHEMA'");
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally {
try {
if(stmt != null)
stmt.close();
if(conn != null)
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
The error is:
java.sql.SQLSyntaxErrorException: ORA-00900: invalid SQL statement
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:194)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1000)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1882)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1847)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:301)
I am using an Oracle 11g R2 database with the ojdbc6-11.2.0.1.0.jar JDBC driver. I am able to execute the statement successfully from the sqldeveloper tool as well as the SQLPlus console, but not from the Java application. Do I need to configure additional properties on the database driver? Can I make this type of call from this driver?
I know some people right off the bat may suggest using variable bindings on the Java side, but this is not an option. The scripts need to be executable from both the SQL interface and the installer. There are other reasons apart from this one, which I won't go into.
I am also hoping to get this to work with the sql-maven-plugin, but that may not be possible based on this JIRA:
Add Oracle SQLPlus syntax for substitution variables
If anyone has any suggestions or know how to get this to work, I would greatly appreciate it.
I don't think DEFINE will work outside of SQLPLUS - JAVA uses JDBC
and assumes the argument to execute() is valid SQL. If you're able to
use DEFINE outside SQLPLUS it means the utility you're using is
intended to be compatible with or a partial replacement of SQLPLUS.
DEFINE is an SQLPLUS command - SQLPLUS is an ORACLE utility.
According to this URL Define is not an SQL statement
http://www.adp-gmbh.ch/ora/sqlplus/define.html
Both DEFINE and the substitution variable syntax &&MYSCHEMA.TBL_NAME are SQL*Plus commands. They are not valid SQL or PL/SQL constructs. You won't be able to use them as-is via JDBC.
Depending on the reasons that you say you don't want to go into
Your Java application can call out to the operating system to invoke SQL*Plus and pass in the script
Your Java application can implement whatever SQL*Plus features your scripts rely on. Rather than executing DEFINE MYSCHEMA='TESTSCHEMA', your application would need to maintain, say, a HashMap that maps a variable like myschema to a value like TESTSCHEMA. Your application would then have to parse the individual SQL statements looking for text like &&MYSCHEMA, replace that with the value from its local HashMap, and then send the resulting SQL string to the database server. Depending on how much of the SQL*Plus functionality you need to replicate, that could be a reasonably significant undertaking. Many PL/SQL IDEs (such as Toad or SQL Developer) implement a subset of SQL*Plus's functionality-- I don't know of any that have tried to implement every last SQL*Plus feature.
String link = "http://hosted.ap.org";
I want to find whether the given url is already existing in the SQL DB under the table name "urls". If the given url is not found in that table i need to insert it in to that table.
As I am a beginner in Java, I cannot really reach the exact code.
Please advise on this regard on how to search the url in the table.
I am done with the SQL Connection using the java code. Please advise me on the searching and inserting part alone as explained above.
PreparedStatement insert = connectin.preparedStateme("insert into urls(url) vlaues(?)");
PreparedStatement search = connectin.preparedStateme("select * from urls where url = ?");
search.setString(1, <your url value to search>);
ResultSet rs = search.executeQuery();
if (!rs.hasNext()) {
insert.setString(1, <your url value to insert>);
insert.executeUpdate();
}
//finally close your statements and connection
...
i assumed that you only have one field your table and field name is url. if you have more fields you need to add them in insert query.
You need to distinguish between two completely separate things: SQL (Structured Query Language) is the language which you use to communicate with the DB. JDBC (Java DataBase Connectivity) is a Java API which enables you to execute SQL language using Java code.
To get data from DB, you usually use the SQL SELECT statement. To insert data in a DB, you usually use the SQL INSERT INTO statement
To prepare a SQL statement in Java, you usually use Connection#prepareStatement(). To execute a SQL SELECT statement in Java, you should use PreparedStatement#executeQuery(). It returns a ResultSet with the query results. To execute a SQL INSERT statement in Java, you should use PreparedStatement#executeUpdate().
See also:
SQL tutorial
JDBC tutorial
Can anyone out there provide an example of bulk inserts via JConnect (with ENABLE_BULK_LOAD) to Sybase ASE?
I've scoured the internet and found nothing.
I got in touch with one of the engineers at Sybase and they provided me a code sample. So, I get to answer my own question.
Basically here is a rundown, as the code sample is pretty large... This assumes a lot of pre initialized variables, but otherwise it would be a few hundred lines. Anyone interested should get the idea. This can yield up to 22K insertions a second in a perfect world (as per Sybase anyway).
SybDriver sybDriver = (SybDriver) Class.forName("com.sybase.jdbc3.jdbc.SybDriver").newInstance();
sybDriver.setVersion(com.sybase.jdbcx.SybDriver.VERSION_6);
DriverManager.registerDriver(sybDriver);
//DBProps (after including normal login/password etc.
props.put("ENABLE_BULK_LOAD","true");
//open connection here for sybDriver
dbConn.setAutoCommit(false);
String SQLString = "insert into batch_inserts (row_id, colname1, colname2)\n values (?,?,?) \n";
PreparedStatement pstmt;
try
{
pstmt = dbConn.prepareStatement(SQLString);
}
catch (SQLException sqle)
{
displaySQLEx("Couldn't prepare statement",sqle);
return;
}
for (String[] val : valuesToInsert)
{
pstmt.setString(1, val[0]); //row_id varchar(30)
pstmt.setString(2, val[1]);//logical_server varchar(30)
pstmt.setString(3, val[2]); //client_host varchar(30)
try
{
pstmt.addBatch();
}
catch (SQLException sqle)
{
displaySQLEx("Failed to build batch",sqle);
break;
}
}
try {
pstmt.executeBatch();
dbConn.commit();
pstmt.close();
} catch (SQLException sqle) {
//handle
}
try {
if (dbConn != null)
dbConn.close();
} catch (Exception e) {
//handle
}
After following most of your advice we didn't see any improvement over simply creating a massive string and sending that across in batches of ~100-1000rows with a surrounding transaction. we got around:
*Big String Method [5000rows in 500batches]: 1716ms = ~2914rows per second.
(this is shit!).
Our db is sitting on a virtual host with one CPU (i7 underneath) and the table schema is:
CREATE TABLE
archive_account_transactions
(
account_transaction_id INT,
entered_by INT,
account_id INT,
transaction_type_id INT,
DATE DATETIME,
product_id INT,
amount float,
contract_id INT NULL,
note CHAR(255) NULL
)
with four indexes on account_transaction_id (pk), account_id, DATE, contract_id.
Just thought I would post a few comments first we're connecting using:
jdbc:sybase:Tds:40.1.1.2:5000/ikp?EnableBatchWorkaround=true;ENABLE_BULK_LOAD=true
we did also try the .addBatch syntax described above but it was marginally slower than just using java StringBuilder to build the batch in sql manually and then just push it across in one execute statement. Removing the column names in the insert statement gave us a surprisingly large performance boost it seemed to be the only thing that actually effected the performance. As the Enable_bulk_load param didn't seem to effect it at all nor did the EnableBatchWorkaround we also tried DYNAMIC_PREPARE=false which sounded promising but also didn't seem to do anything.
Any help getting these parameters actually functioning would be great! In other words are there any tests we could run to verify that they are in effect? I'm still convinced that this performance isn't close to pushing the boundaries of sybase as mysql out of the box does more like 16,000rows per second using the same "big string method" with the same schema.
Cheers
Rod
In order to get the sample provided by Chris Kannon working, do not forget to disable auto commit mode first:
dbConn.setAutoCommit(false);
And place the following line before dbConn.commit():
pstmt.executeBatch();
Otherwise this technique will only slowdown the insertion.
Don't know how to do this in Java, but you can bulk-load text files with LOAD TABLE SQL statement. We did it with Sybase ASA over JConnect.
Support for Batch Updates
Batch updates allow a Statement object to submit multiple update commands
as one unit (batch) to an underlying database for processing together.
Note: To use batch updates, you must refresh the SQL scripts in the sp directory
under your jConnect installation directory.
CHAPTER
See BatchUpdates.java in the sample (jConnect 4.x) and sample2 (jConnect
5.x) subdirectories for an example of using batch updates with Statement,
PreparedStatement, and CallableStatement.
jConnect also supports dynamic PreparedStatements in batch.
Reference:
http://download.sybase.com/pdfdocs/jcg0420e/prjdbc.pdf
http://manuals.sybase.com/onlinebooks/group-jcarc/jcg0520e/prjdbc/#ebt-link;hf=0;pt=7694?target=%25N%14_4440_START_RESTART_N%25#X
.
Other Batch Update Resources
http://java.sun.com/j2se/1.3/docs/guide/jdbc/spec2/jdbc2.1.frame6.html
http://www.jguru.com/faq/view.jsp?EID=5079