I have a funny problem in creating SP for MariaDB.
When I execute below DDL straight away from HeidiSQL (db client app for MySql), I can create the SP easily.
DROP PROCEDURE IF EXISTS `sp_delete_merchant`;
CREATE PROCEDURE `sp_delete_merchant`(IN `mer_id` BIGINT)
BEGIN
DECLARE stop BIT DEFAULT FALSE;
DECLARE comm_id BIGINT;
DECLARE commodities CURSOR FOR SELECT id FROM tbl_commodity WHERE merchant_id = mer_id;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET stop = TRUE;
# DELETE RECORDS FROM tbl_package2commodity BASED ON COMMODITY ID
OPEN commodities;
read_loop: LOOP
FETCH commodities INTO comm_id;
DELETE FROM tbl_package2commodity WHERE commodity_id = comm_id;
IF stop THEN
LEAVE read_loop;
END IF;
END LOOP;
CLOSE commodities;
# DELETE RECORDS FROM OTHER TABLES BASED ON MERCHANT ID
DELETE FROM tbl_commodity WHERE merchant_id = mer_id;
DELETE FROM tbl_package WHERE merchant_id = mer_id;
DELETE FROM tbl_contact_person WHERE merchant_id = mer_id;
DELETE FROM tbl_sales_force WHERE merchant_id = mer_id;
DELETE FROM tbl_merchant WHERE id = mer_id;
END;
But when I try to execute the above DDL from Spring JdbcTemplate as below code:
public class DbStoreProceduresCreator implements InitializingBean {
private static Logger log = LoggerFactory.getLogger(DbStoreProceduresCreator.class);
#Value("classpath:com/litaal/commerce/config/setup_store_procedures.sql")
private Resource createStoreProcedures;
#Autowired
private DataSource coreDS;
#Override
public void afterPropertiesSet() throws Exception {
BufferedReader br = null;
StringBuffer sql = new StringBuffer();
try {
InputStream is = createStoreProcedures.getInputStream();
br = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = br.readLine()) != null) {
sql.append(line);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (br != null) {
br.close();
}
}
if (sql.length() > 0) {
try {
JdbcTemplate tmp = new JdbcTemplate(coreDS);
tmp.execute(sql.toString());
} catch (Exception e) {
log.error(e.getMessage());
}
}
}
}
It throws error like below:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1
And I already try to googling it and change my script like ten times or more, and still couldn't find what is wrong with my DDL.
The reason why I want to create the SP from my java app is to make my app as portable as possible without any need to do anything to DB.
It turns out the JdbcTemplate cannot execute SQL procedures with comments (#) (--) in them that occur during select/updates, etc. The JdbcTemplate does support the (--, #) comments at the beginning of a statement such that:
-- --------------------------------------------
-- Summary Comments!
-- Note: This comes before a delimited statement (;).
# These are okay too.
-- --------------------------------------------
SELECT *
FROM User
-- This comment fails.
WHERE id > 0;
Here's how I'm breaking down procedures (from MySQL).
Procedure
USE `DbName`;
DROP procedure IF EXISTS `ProcName`;
DELIMITER $$
USE `DbName`$$
CREATE PROCEDURE `ProcName` ()
BEGIN
# Here's a comment that would normally fail.
SELECT
u.id
FROM
User as u
END$$
DELIMITER ;
Parser
StringBuilder builder = new StringBuilder();
String[] lines = script.split("\n");
String delimiter = ";";
for (int i = 0; i < lines.length; i++) {
String line = lines[i];
if (line.contains("DELIMITER")) {
String[] tokens = line.split(" ");
delimiter = tokens[1];
builder = new StringBuilder();
continue;
}
//Comments cannot be executed properly by the JdbcTemplate, strip them out.
if (line.contains("#")) {
line = line.substring(0, line.indexOf('#'));
}
if (line.contains(delimiter)) {
builder.append(line.replace(delimiter, ""));
template.execute(builder.toString());
builder = new StringBuilder();
} else {
builder.append(line);
}
}
Maybe there are several problems with your code, but one is, that JdbcTemplate.execute(String sql) execute only a single statement
/**
* Issue a single SQL execute, typically a DDL statement.
* #param sql static SQL to execute
* #throws DataAccessException if there is any problem
*/
void execute(String sql) throws DataAccessException;
So you must split your execution in two executions, one for DROP PROCEDURE IF EXISTS "sp_delete_merchant"; and one for the rest.
BTW: you should have a look at org.springframework.jdbc.datasource.init.ScriptUtils
An other thing that concerns me a bit, are your string quotes.
At last ... I found the answer.
The problem is lies on how JdbcTemplate executes SQL script. It executes line by line (separated by new line). So I changed part of my code to remove all new lines, like below
....
if (sql.length() > 0) {
String query = sql.toString().replaceAll("\\n", " ");
query = query.replaceAll("\\t", " ");
log.info(query);
try {
JdbcTemplate tmp = new JdbcTemplate(dataSource);
tmp.execute(query);
} catch (Exception e) {
log.error(e.getMessage());
}
}
...
Now everything works like charm.
Related
This is the code where I'm trying to execute a second query on the resultSet of my first lengthy query. I need to upload this
data somewhere.
Is it the right thing to do?
Or is there a better approach apart from querying the database again?
public String createQuery() throws SQLException {
StringBuilder Query = new StringBuilder();
try {
Query.append(" SELECT ...... ")
} catch (Exception e) {
e.printStackTrace();
}
return Query.toString();
}
private void openPreparedStatements() throws SQLException {
myQuery = createQuery();
try {
QueryStatement = dbConnection.prepareStatement(myQuery);
} catch (SQLException e) {
e.printStackTrace();
return;
}
}
public ResultSet selectData(String timestamp) throws SQLException {
openConnection();
ResultSet result = null;
ResultSet rs_new=null;
try {
result = QueryStatement.executeQuery();
while (result.next()) {
String query = "SELECT * FROM " + result + " WHERE " + "ID" + " =" + "ABC";
rs_new =QueryStatementNew.executeQuery(query);
System.out.print(rs_new);
}
} catch (SQLException e) {
LOGGER.info("Exception", e);
}
return result;
}
Instead of running two separate queries (when you don't need the intermediate one) you can combine them.
For example you can do:
SELECT *
FROM (
-- first query here
) x
WHERE ID = 'ABC'
You cannot use two statement objects within one database connection. So you can either open another database connection and execute the second statement in the 2nd connection, or iterate through the resultset from first statement and store the value you need (e.g. in an array/collection) then close that statement and run the second one, this time retrieving the value from the array/collection you saved them in. Refer to Java generating query from resultSet and executing the new query
Make Db2 keep your intermediate result set in a Global Temporary Table, if you have an ability to use it, and you application uses the same database connection session.
DECLARE GLOBAL TEMPORARY TABLE SESSION.TMP_RES AS
(
SELECT ID, ... -- Your first lengthy query text goes here
) WITH DATA WITH REPLACE ON COMMIT PRESERVE ROWS NOT LOGGED;
You may send the result of subsequent SELECT ... FROM SESSION.TMP_RES to FTP, and the result of SELECT * FROM SESSION.TMP_RES WHERE ID = 'ABC' to elastic.
public ResultSet runQuery(String cmd, String table, String[] keys, String[] values, String[] whereKeys, String[] whereValues, int limitStart, int limitCount) throws Exception
{
PreparedStatement stmt;
if(!(cmd.equals("INSERT") || cmd.equals("SELECT") || cmd.equals("UPDATE") || cmd.equals("DELETE")))
{
throw new Exception("CMD UNSUPPORTED! CMD must be INSERT, SELECT, UPDATE, or DELETE.");
}
String sql = "";
if(cmd.equals("INSERT"))
{
//Build Insert statement
sql += "INSERT INTO "+table+" (";
for(int i = 0; i < keys.length; i++)
{
sql += ""+keys[i]+"";
if(i!=keys.length-1)
sql+=",";
}
sql += ") VALUES(";
for(int i = 0; i < values.length; i++)
{
sql += "'?'"; //This line seems to be causing trouble for me...
if(i!=values.length-1)
sql+=",";
}
sql += ")";
}
Context: I am working on a class project, and was assigned the database team. I have very little experience with MySQL and was partnered up with someone that claims to have a ton of experience with MySQL and Java. This is the function he created for Insert, Select, Update and Delete. Every function I have created that uses one of these functions has had errors, and requires modification of this original runQuery function. The main thing I have been trying to work with is the "insert" command. I have to be able to create a User, Admin, or a Class (like a college class), and insert the appropriate information(names, userNames, passwords, tableID). I can create a user or admin without generating an error, but when I create a class, I get an error: http://imgur.com/g99Ru6h
public boolean createUser(String first, String last, String user, String pass)
{
String[] keys = {"FName", "LName", "UName", "Password"};
String[] vals = {first, last, user, pass};
try
{
runQuery("INSERT", "Users", keys, vals, null, null, 0, 30);
}catch(Exception e)
{
e.printStackTrace();
System.out.println(e);
return false;
}
return true;
}
public boolean createClass(int classID, String name, int adminID, String institution, String meetTimes)
{
String[] keys = {"ClassID", "ClassName", "AdminID", "Institution", "MeetTimes"};
String[] vals = {Integer.toString(classID), name, Integer.toString(adminID), institution, meetTimes};
try
{
runQuery("INSERT", "Classes", keys, vals, null, null, 0, 30);
}catch(Exception e)
{
e.printStackTrace();
System.out.println(e);
return false;
}
return true;
}
TL;DR: I am having trouble inserting new 'classes' into a table, and think the error relates back to the runQuery function. I have omitted the rest of the runQuery function for now, as I don't think its necessary for this problem, but I can add it in its entirety of 241 lines. Based on what I am seeing online in regards to this situation, preparedStatements seem like a much easier way to input info, no?
Thanks for reading through this and your possible help!
From the linked image to the stack trace it seems someone tries to execute a statement with arguments with ? place holders. The SQL engine complains that the argument for ClassID is '?'. It should be ? not '?' since numerical values do not need ''. Strange however is the fact that I can't spot any ClassID argument in the query.
I am using an Oracle database and after sometime I receive the following Exception:
java.sql.SQLException: ORA-01000: maximum open cursors exceeded
I analysed my code and I seem to be closing all the ResultSet. This Exception only happens sometimes. Due to this error, I decided to change my code a little bit. Below is the code:
public class Audit {
private Connection connection;
private PreparedStatement insertAuditPreparedStatementSent;
private static int counter;
private static int JDBC_COUNTER;
public Audit() throws Exception {
connection = DriverManager.getConnection("url", "username", "password");
}
public int insertAudit(String message, java.util.Date sent) throws Exception {
PreparedStatment preparedStatement = prepareStatement(new String("INSERT INTO Audit (message, sent) VALUES (?, ?)");
if(JDBC_COUNTER == 0) {
// this is required to be executed so that ORA-08002 SQLException is not thrown
connection.createStatement().executeQuery(new String("SELECT AUDIT_SEQUENCE.NEXTVAL FROM DUAL"));
}
ResultSet resultSet = connection.createStatement().executeQuery(new String("SELECT AUDIT_SEQUENCE.CURRVAL FROM DUAL"));
resultSet.next();
primaryKey = resultSet.getInt(new String("CURRVAL"));
resultSet.close();
return primaryKey;
}
public void executeUpdateAudit(int id, java.util.Date sent) throws Exception {
if(updateAuditPreparedStatement == null) {
updateAuditPreparedStatement = connection.prepareStatement(new String("UPDATE AUDIT SET SENT = ? WHERE AUDIT_ID = " + id));
}
updateAuditPreparedStatement.setTimestamp(1, new java.sql.Timestamp(sent.getDate());
int i = updateAuditPreparedStatement.executeUpdate();
connection.commit();
}
public static void main(String[] args) throws Exception {
Audit audit = new Audit();
int primaryKey = audit.insertAudit("message", new java.util.Date());
audit.executeUpdateAudit(primaryKey, new java.util.Date());
int primaryKey2 = audit.insertAudit("message2", new java.util.Date());
audit.executeUpdateAudit(primaryKey2, new java.util.Date());
}
}
When inserting record 2 and updating record 2 only the updateAuditPreparedStatement.executeUpdate() returns 1, but the database updates the first record rather than the second record.
The reason to change the code is because I believe the PreparedStatement creates a new cursor each time. So, I want the insertAuditPreparedStatementSent to be there on many inserts without closing. I have tried insertAuditPreparedStatementSent.clearBatch() and insertAuditPreparedStatementSent.clearParameters().
I am not sure why it is updating record 1 on the primaryKey of record 2. The SQL is fine.
Any ideas?
You are not closing this:
if(JDBC_COUNTER == 0) {
// this is required to be executed so that ORA-08002 SQLException is not thrown
connection.createStatement().executeQuery(new String("SELECT AUDIT_SEQUENCE.NEXTVAL FROM DUAL"));
}
Also prepared statements needs to be closed as well and you are missing try/catch/finally in all these methods to prevent resource leakage. I strongly discourage using JDBC directly as it is very hard to work with.
In executeUpdateAudit you prepare the statement once using the first ID seen:
if(updateAuditPreparedStatement == null) {
updateAuditPreparedStatement = connection.prepareStatement(
new String("UPDATE AUDIT SET SENT = ? WHERE AUDIT_ID = " + id));
}
updateAuditPreparedStatement.setTimestamp(1,
new java.sql.Timestamp(sent.getDate());
In the second call the ID from the first call is still being used, since it's effectively hard-coded in the SQL.
You should be using a parameter for the ID as well:
if(updateAuditPreparedStatement == null) {
updateAuditPreparedStatement = connection.prepareStatement(
new String("UPDATE AUDIT SET SENT = ? WHERE AUDIT_ID = ?"));
}
updateAuditPreparedStatement.setTimestamp(1,
new java.sql.Timestamp(sent.getDate());
updateAuditPreparedStatement.setInt(2, id);
Not sure why you're using new String explicitly everywhere; it would be a little simpler as:
if(updateAuditPreparedStatement == null) {
updateAuditPreparedStatement = connection.prepareStatement(
"UPDATE AUDIT SET SENT = ? WHERE AUDIT_ID = ?");
}
updateAuditPreparedStatement.setTimestamp(1,
new java.sql.Timestamp(sent.getDate());
updateAuditPreparedStatement.setInt(2, id);
This is nothing to do with your ORA-01000, but this seems to be the main thrust of this question - you shouldn't really ask about two things at once, particularly if they aren't directly related...
Please check the oracle parameter
open_cursors
You will find this in the enterprise manager or by executing the following SQL:
select * from v$parameter a
where a.NAME = 'open_cursors';
If this parameter is very low (e.g. < 300) and you have lots of processes/users working at the same time, this error can happen.
In the below code, values are inserted manually: 13 and Aman. But what I am doing is reading a file and then till the completion of the file, I am inserting values from it into mysql database tables.
public class Main {
public static void main(String[] argv) throws Exception {
String driver = "com.mysql.jdbc.Driver";
Class.forName(driver);
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/jdbctutorial", "root", "root");
Statement st = con.createStatement();
int val = st.executeUpdate("INSERT employee VALUES(" + 13 + "," + "'Aman'" + ")");
System.out.println("1 row affected");
}
}
I was trying to use each line like this:
String query = "INSERT INTO home (home_no, arrival_time, duration, persons, price, origin_city) VALUES("+line+");";
How do I do it?
Depending on how large the contents of the file that you are reading, it may be worth while to check LOAD DATA INFILE syntax, rather than executing queries in a for or while loop.
Edit:
Without seeing your code and line is the current line of the file you are reading and that you are using the syntax above to store the query, I would break down your problems,
Check the line variable prior to executing the query
Check how to insert the values manually opposed to reading the contents of the file, as you had shown above with 13 and Aman.
Figure out how to piece those two together, may need string manipulation.
This should be all you need.
BufferedReader rd = new BufferedReader(new FileReader("yourFile.txt"));
String line;
while ((line = rd.readLine()) != null)
{
// This time this loop runs, you will have the next time of your file.
insertARecord(line.split(" "));
}
rd.close();
// A bit further
public void insertARecord(String[] data)
{
// insert your record.
}
I am starting to use MySQL with JDBC.
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection("jdbc:mysql:///x", "x", "x");
stmt = conn.createStatement();
stmt.execute( "CREATE TABLE amigos" +
"("+
"id int AUTO_INCREMENT not null,"+
"nombre char(20) not null,"+
"primary key(id)" +
")");
I have 3-4 tables to create and this doesn't look good.
Is there a way to run a .sql script from MySQL JDBC?
Ok. You can use this class here (posted on pastebin because of file length) in your project. But remember to keep the apache license info.
JDBC ScriptRunner
It's ripoff of the iBatis ScriptRunner with dependencies removed.
You can use it like this
Connection con = ....
ScriptRunner runner = new ScriptRunner(con, [booleanAutoCommit], [booleanStopOnerror]);
runner.runScript(new BufferedReader(new FileReader("test.sql")));
That's it!
I did a lot of research on this and found a good util from spring. I think using SimpleJdbcTestUtils.executeSqlScript(...) is actually the best solution, as it is more maintained and tested.
Edit: SimpleJdbcTestUtils is deprecated. You should use JdbcTestUtils. Updated the link.
Spring Framework's ResourceDatabasePopulator may help. As you said you're using MySQL and JDBC, let's assume you have a MySQL-backed DataSource instance ready. Further, let's assume your MySQL script files are classpath-locatable. Let's assume you are using WAR layout and the script files are located in a directory src/main/webapp/resources/mysql-scripts/... or src/test/resources/mysql-scripts/.... Then you can use ResourceDatabasePopulator to execute SQL scripts like this:
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
import javax.sql.DataSource;
DataSource dataSource = getYourMySQLDriverBackedDataSource();
ResourceDatabasePopulator rdp = new ResourceDatabasePopulator();
rdp.addScript(new ClassPathResource(
"mysql-scripts/firstScript.sql"));
rdp.addScript(new ClassPathResource(
"mysql-scripts/secondScript.sql"));
try {
Connection connection = dataSource.getConnection();
rdp.populate(connection); // this starts the script execution, in the order as added
} catch (SQLException e) {
e.printStackTrace();
}
For simple sql script splitted by ';' you can use this simple function.
It remove comments and run statements one by one
static void executeScript(Connection conn, InputStream in)
throws SQLException
{
Scanner s = new Scanner(in);
s.useDelimiter("/\\*[\\s\\S]*?\\*/|--[^\\r\\n]*|;");
Statement st = null;
try
{
st = conn.createStatement();
while (s.hasNext())
{
String line = s.next().trim();
if (!line.isEmpty())
st.execute(line);
}
}
finally
{
if (st != null)
st.close();
}
}
#Pantelis Sopasakis
Slightly modified version on GitHub: https://gist.github.com/831762/
Its easier to track modifications there.
Regarding SQL script runner (which I'm also using), I noticed the following piece of code:
for (int i = 0; i < cols; i++) {
String value = rs.getString(i);
print(value + "\t");
}
However, in the API documentation for the method getString(int) it's mentioned that indexes start with 1, so this should become:
for (int i = 1; i <= cols; i++) {
String value = rs.getString(i);
print(value + "\t");
}
Second, this implementation of ScriptRunner does not provide support for DELIMITER statements in the SQL script which are important if you need to compile TRIGGERS or PROCEDURES. So I have created this modified version of ScriptRunner: http://pastebin.com/ZrUcDjSx which I hope you'll find useful.
Another interesting option would be to use Jisql to run the scripts. Since the source code is available, it should be possible to embed it into an application.
Edit: took a careful look at it; embedding it inside something else would require some modification to its source code.
Can you use this:
public static void executeSQL(File f, Connection c) throws Exception {
BufferedReader br = new BufferedReader(new FileReader(f));
String sql = "", line;
while ((line = br.readLine()) != null) sql += (line+"\n");
c.prepareCall(sql).execute(sql);
}
This function gets SQL file and DB connection.
Then it reads the file line-by-line using BufferedReader from java.io.
And, finally, executes the read statements.
Java 8+ version:
public static void executeSQL(Path p, Connection c) throws Exception {
List<String> lines = Files.readAllLines(p);
String s = String.join("\n", lines.toArray(new String[0]));
c.prepareCall(s).execute(s);
}
Write code to:
Read in a file containing a number of SQL statements.
Run each SQL statement.
For Oracle PL/SQL, the Oracle JDBC-driver indeed supports executing entire SQL-scripts including stored procedures and anonymous blocks (PL/SQL specific notation), see
Can the JDBC Drivers access PL/SQL Stored Procedures?
The Oracle JDBC driver FAQ has more info:
Oracle JDBC drivers support execution
of PL/SQL stored procedures and
anonymous blocks. They support both
SQL92 escape syntax and Oracle PL/SQL
block syntax. The following PL/SQL
calls would work with any Oracle JDBC
driver:
// SQL92 syntax
CallableStatement cs1 = conn.prepareCall
( "{call proc (?,?)}" ) ; // stored proc
CallableStatement cs2 = conn.prepareCall
( "{? = call func (?,?)}" ) ; // stored func
// Oracle PL/SQL block syntax
CallableStatement cs3 = conn.prepareCall
( "begin proc (?,?); end;" ) ; // stored proc
CallableStatement cs4 = conn.prepareCall
( "begin ? := func(?,?); end;" ) ; // stored func
It should be possible to read in a file and feed the content to the prepareCall()-method.
Maven SQL Plugin Use this plugin to execute SQL statements a file or list of files through
sqlCommand
srcFiles
3.fileset configurations
There isn't really a way to do this.
You could either run the mysql command line client via Runtime.exec(String[]) and read this article when you decide for this option
Or try using the ScriptRunner (com.ibatis.common.jdbc.ScriptRunner) from ibatis. But it's a bit stupid to include a whole library just to run a script.
Here's a quick and dirty solution that worked for me.
public void executeScript(File scriptFile) {
Connection connection = null;
try {
connection = DriverManager.getConnection(url, user, password);
if(scriptFile.exists()) {
var buffer = new StringBuilder();
var scanner = new Scanner(scriptFile);
while(scanner.hasNextLine()) {
var line = scanner.nextLine();
buffer.append(line);
// If we encounter a semicolon, then that's a complete statement, so run it.
if(line.endsWith(";")) {
String command = buffer.toString();
connection.createStatement().execute(command);
buffer = new StringBuilder();
} else { // Otherwise, just append a newline and keep scanning the file.
buffer.append("\n");
}
}
}
else System.err.println("File not found.");
} catch (SQLException e) {
e.printStackTrace();
} finally {
if(connection != null) connection.close();
}