i have the following snippet:
String insertSql = "LOAD DATA INFILE 'C:/thomsonReuters/compressed/premium-world-check-day.csv'"
+ " INTO TABLE thomsenreuters.worldcheck "
+ " FIELDS TERMINATED BY '\t' "
+ " ENCLOSED BY '\"' "
+ " LINES TERMINATED BY '\r\n' "
+ " IGNORE 1 ROWS;";
This file has 4567 total rows. The fields are terminated by a tab and enclosed by double quotes. I try to insert the csv file by a Prepared-Update stmt. But Java only insert the first 4 rows into MySQL. What is the misstake i've done?
Thanks for help
EDIT
Full Class Code:
Connection con = null;
PreparedStatement stmt = null;
String truncateTable = "Truncate table thomsenreuters.worldcheck;";
String insertSql = "LOAD DATA INFILE 'C:/thomsonReuters/compressed/premium-world-check-day.csv'"
+ " INTO TABLE thomsenreuters.worldcheck "
+ " FIELDS TERMINATED BY '\t' "
+ " ENCLOSED BY '\"' "
+ " LINES TERMINATED BY '\r\n' "
+ " IGNORE 1 ROWS;";
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
System.out.println("Driver was found");
try {
con = DriverManager.getConnection("jdbc:mysql://localhost:3306/thomsenreuters","XXXXXXX","XXXXXXX");
} catch (SQLException e) {
e.printStackTrace();
}
System.out.println("Connection to Database");
System.out.println("Start insert...");
try {
con.setAutoCommit(false);
stmt = con.prepareStatement(truncateTable);
stmt = con.prepareStatement(insertSql);
int total = stmt.executeUpdate();
con.commit();
con.setAutoCommit(true);
System.out.println("Total Rows insert " + total);
} catch (SQLException e) {
e.printStackTrace();
}
finally{
if(stmt != null)
{
try {
stmt.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if(con != null)
{
try {
con.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
Related
I have a sqlite db file created by other program, and checked everything is fine.
Then after doing select query to get some data, some of the row disappear after this process. I tried to use prepareStatement and though it worked but this remained.
my code
private ForecastTableItem selectItemPrepareStatement(String tableName, String columnName, String name) {
ForecastTableItem item = null;
Connection conn = null;
PreparedStatement pstmt = null;
ResultSet rs = null;
try {
Class.forName("org.sqlite.JDBC");
conn = DriverManager.getConnection("jdbc:sqlite:" + dbLocation);
System.out.println("Selecting item from tableName: "+tableName + " of col: "+columnName + " : "+name);
String query = "SELECT * FROM " + tableName + " WHERE " + columnName + "=? COLLATE NOCASE";
pstmt = conn.prepareStatement(query);
pstmt.setString(1, name);
rs = pstmt.executeQuery();
if (rs.next()) {
if (tableName.equalsIgnoreCase("mainTable")) {
item = new ForecastTableItem();
item.setId(rs.getInt("Id"));
item.setTitle(rs.getString("title"));
item.setLink(rs.getString("link").toLowerCase());
item.setPositionType(rs.getString("positionType"));
item.setPackageName(rs.getString("packageName"));
item.setCsvFilePath(rs.getString("csvFilePath"));
item.setSubpackageName(rs.getString("subpackageName"));
item.setTimeFrame(rs.getString("timeFrame"));
item.setForecastDate(rs.getString("forecastDate"));
item.setTargetDate(rs.getString("targetDate"));
}
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
} finally{
if(pstmt != null) try{ pstmt.close();} catch(SQLException e){};
if(rs != null) try{ rs.close();} catch(SQLException e){};
if(conn != null) try{ conn.close();} catch(SQLException e){};
}
return item;
}
after COLLATE NOCASE use semi-column
"SELECT * FROM " + tableName + " WHERE " + columnName + "= ? COLLATE NOCASE;";
My problem is that when I fire a query directly into Oracle, it returns me the rows with proper data. But when I try to fire the same query via Statement, its fetching me a resultset with no rows.
My code is as follows :
public void sendSMStoMobile() {
if (smsGatewayStatus.equals("STOPPED")) {
stringBuilder.append("The control is returning" + "\n");
jTextArea1.setText(stringBuilder.toString());
return;
}
DBCP dbcp = DBCP.getInstance(database_url, username, password);
Connection connection = null;
Statement statement = null;
try {
connection = dbcp.getConnection();
statement = connection.createStatement();
} catch (SQLException ex) {
Logger.getLogger(Send_SMS_Form.class.getName()).log(Level.SEVERE, null, ex);
System.out.println("Message : " + ex.getMessage());
} catch (InterruptedException ex) {
Logger.getLogger(Send_SMS_Form.class.getName()).log(Level.SEVERE, null, ex);
System.out.println("Message : " + ex.getMessage());
}
ResultSet resultSet = null;
try {
sql = new StringBuffer();
sql.append("SELECT SMS_ID, MOBILE_NO, SMS_CONTENT ");
sql.append("FROM SMSDATA.SMS_DATA ");
//sql.append("WHERE SENT_STATUS = 'N' AND Message_Code IN('LOANREPAY') " );
//sql.append("WHERE SENT_STATUS = 'N' AND Message_Code IN('SPECIAL') " );
sql.append("WHERE SENT_STATUS = 'N' ");
//sql.append("WHERE tranum <= (select max(tranum) -25 from smsgtway.sms_all_tran where SENT_STATUS = 'N' AND Message_Code IN('MEMDEP','MEMGRANT','SAMGRANT')) and SENT_STATUS = 'N' AND Message_Code IN('MEMDEP','MEMGRANT','SAMGRANT') " );
sql.append("ORDER BY SMS_ID desc ");
// sql.append("ORDER BY SENT_TIMSTAMP " );
// For Test
//sql.append("SELECT customer_Code,MOBILE_NO, SMS_CONTENT,SMS_ID " );
//sql.append("FROM MB_SMS_DTL ");
//sql.append("WHERE SENT_STATUS = 'N' ");
//sql.append("ORDER BY SMS_ID " );
// End Test
resultSet = statement.executeQuery(sql.toString());
System.out.println("Code comes here");
/* if(!resultSet.next()){
stringBuilder.append("No Data Found" +"\n");
jTextArea1.setText(stringBuilder.toString());
} */
if (!resultSet.next()) {
System.out.println("no data");
} else {
System.out.println("Data Found...");
do {
//statement(s)
} while (resultSet.next());
}
while (resultSet.next()) {
System.out.println("Data Found...");
String smsId = resultSet.getString(1);
String destMobileNo = resultSet.getString(2);
String message = resultSet.getString(3);
OutboundMessage sms = new OutboundMessage(destMobileNo, message);
System.out.println("Code comes here" + smsId);
//System.out.println("sms = " + sms);
try {
Service.getInstance().sendMessage(sms);
} catch (TimeoutException timeoutException) {
System.out.println("1");
stringBuilder.append("Exception : " + timeoutException.getMessage() + "\n");
jTextArea1.setText(stringBuilder.toString());
} catch (GatewayException gatewayException) {
System.out.println("2");
stringBuilder.append("Exception : " + gatewayException.getMessage() + "\n");
jTextArea1.setText(stringBuilder.toString());
} catch (IOException iOException) {
System.out.println("3");
stringBuilder.append("Exception : " + iOException.getMessage() + "\n");
jTextArea1.setText(stringBuilder.toString());
} catch (InterruptedException interruptedException) {
System.out.println("4");
stringBuilder.append("Exception : " + interruptedException.getMessage() + "\n");
jTextArea1.setText(stringBuilder.toString());
} catch (java.lang.StringIndexOutOfBoundsException siobe) {
System.out.println("5");
stringBuilder.append("Exception : " + siobe.getMessage() + "\n");
jTextArea1.setText(stringBuilder.toString());
}
//Update Table After Send Message
String updateQuery = "UPDATE SMSDATA.SMS_DATA SET SENT_STATUS = 'Y', SENT_DATE = sysdate WHERE SMS_ID = " + smsId;
//String updateQuery = "UPDATE MB_SMS_DTL SET SENT_STATUS = 'Y' WHERE SMS_ID = " + tranNumber ;
//System.out.println("Update Query: " + updateQuery);
if (i < 10) {
System.out.print(smsId + ": Y, ");
}
if (i == 10) {
System.out.print(smsId + ": Y, " + "\n");
i = 0;
}
statement.executeUpdate(updateQuery);
connection.commit();
i = i + 1;
}
} catch (Exception e) {
System.out.println("Message: " + e.getMessage());
System.out.println("Cause: " + e.getCause());
stringBuilder.append("Message: " + e.getMessage() + "\n");
stringBuilder.append("Cause: " + e.getCause() + "\n");
jTextArea1.setText(stringBuilder.toString());
e.printStackTrace();
} finally {
if (resultSet != null) {
try {
resultSet.close();
} catch (SQLException ex) {
Logger.getLogger(Send_SMS_Form.class.getName()).log(Level.SEVERE, null, ex);
}
}
if (statement != null) {
try {
statement.close();
} catch (SQLException ex) {
Logger.getLogger(Send_SMS_Form.class.getName()).log(Level.SEVERE, null, ex);
}
}
dbcp.releaseConnection(connection);
//Service.getInstance().stopService();
}
}
When I call this function , "no data" is printed in Console . The sql is correct . When I run the sql in Toad , I have found 2 rows .
Please help me .
The below java method sets the ResultSet data to a bean class and I am fetching the data. But, the method runHiveQuery() returns only one row that is the last record in the table. While debugging the code i found that the resultset is being looped twice as we have two records. But, while returning the bean class object there is some issue as it retrieves only one record.
Unable to find what is going wrong.
public CSPData getCSPData() throws SQLException {
try {
String drivername = "org.apache.hive.jdbc.HiveDriver";
Class.forName(drivername);
connection = DriverManager.getConnection("jdbc:hive2://hddev-c01-edge-01:20000/");
statement = connection.createStatement();
resultset = statement.executeQuery(
"select distinct db_name as db_name,db_server_name as db_server_name,lower(db_name) as l_db_name,lower(db_server_name) as l_server_name,regexp_replace(lower(db_server_name), '-', '_') as server_name,db_server_name_secondary as db_server_name_secondary from csp.curated_input");
} catch (ClassNotFoundException e) {
e.printStackTrace();
System.exit(1);
} catch (SQLException e) {
e.printStackTrace();
System.out.println(e.getMessage());
}
while (resultset.next()) {
cspdata.setDbName(resultset.getString("db_name"));
cspdata.setDbServerName(resultset.getString("db_server_name"));
cspdata.setDbServerNameSecondary(resultset.getString("db_server_name_secondary"));
cspdata.setlDbName(resultset.getString("l_db_name"));
cspdata.setlServerName(resultset.getString("l_server_name"));
cspdata.setServerName(resultset.getString("server_name"));
}
return cspdata;
}
public void runHiveQuery() throws SQLException {
CSPData cspdata = hivedao.getCSPData();
String hive_db = "csp";
String dbname = cspdata.getDbName();
String dbservername = cspdata.getDbServerName();
String servername = cspdata.getlServerName();
String drop = "Drop table if exists " + hive_db + "." + "IB_C3_" + dbname + "_" + dbservername;
String insert = "insert into table " + hive_db + "." + "IB_export_log select " + "\'ib_c3_" + dbname + "_"
+ servername + "\' from " + hive_db + "." + "dual limit 1";
System.out.println(drop);
System.out.println(insert);
}
Your code returns the last record since it only returns a single record. You should return a List :
public List<CSPData> getCSPData() throws SQLException {
List<CSPData> result = new ArrayList<>();
try {
String drivername = "org.apache.hive.jdbc.HiveDriver";
Class.forName(drivername);
connection = DriverManager.getConnection("jdbc:hive2://hddev-c01-edge-01:20000/");
statement = connection.createStatement();
resultset = statement.executeQuery(
"select distinct db_name as db_name,db_server_name as db_server_name,lower(db_name) as l_db_name,lower(db_server_name) as l_server_name,regexp_replace(lower(db_server_name), '-', '_') as server_name,db_server_name_secondary as db_server_name_secondary from csp.curated_input");
} catch (ClassNotFoundException e) {
e.printStackTrace();
System.exit(1);
} catch (SQLException e) {
e.printStackTrace();
System.out.println(e.getMessage());
}
while (resultset.next()) {
CSPData cspdata = new CSPData ();
cspdata.setDbName(resultset.getString("db_name"));
cspdata.setDbServerName(resultset.getString("db_server_name"));
cspdata.setDbServerNameSecondary(resultset.getString("db_server_name_secondary"));
cspdata.setlDbName(resultset.getString("l_db_name"));
cspdata.setlServerName(resultset.getString("l_server_name"));
cspdata.setServerName(resultset.getString("server_name"));
result.add(cspdata);
}
return result;
}
Why i take an empty result set row ?
ResultSet row = dbmd.getTables("%", "dbuser1", "%", types);
while (row.next()) {
System.out.println(result.getString(1));
}
You can use this code :
try {
//must be upper case
String username="dbuser1".toUpperCase();
ResultSet row = dbmd.getTables("%", username , "%", types);
while (row.next()) {
//you need table name for trigger name
result.getString("TABLE_NAME");
}
//catch some errors
} catch (SQLException e) {
while (e != null) {
System.out.println("\n Message: " + e.getMessage());
System.out.println("\n SQLState: " + e.getSQLState());
System.out.println("\n ErrorCode: " + e.getErrorCode());
e = e.getNextException();
}
}
I'm using for the first time the connection pool in java. The application that I'm writing is a web application deployed on oracle glassfish 3.1 and the resource connection pool is handled by it. I set the autocommit to false on my connection object, returned by the javax.sql.DataSource, but in the code the rollback command has no effect at all. All the previous operation of write are committed automatically. Is there any other settings in java code that I have to do to disable autocommit? Or there is some configuration settings that I have to do on glassfish?
The database that I'm using is Oracle 10g.
I post here a part of the code that is giving me problem:
private int importFile(String id, String fileName, String label,
String prosumerId, String districtCodes)
throws IOException,
SQLException {
logger.debug("Importing file [" + fileName + "]");
File f = new File(fileName);
BufferedReader br = null;
Calendar cal = new GregorianCalendar();
SimpleDateFormat sdf = new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
String query = "insert into TB_SDK_USER_FILES values ("
+ id + ", '" + label + "', '"
+ fileName.substring(fileName.lastIndexOf("/") + 1) + "', 'active', "
+ "to_date('" + sdf.format(new Date(cal.getTimeInMillis())) + "', 'dd/mm/yyyy hh24:mi:ss'), "
+ "to_date('" + sdf.format(new Date(cal.getTimeInMillis())) + "', 'dd/mm/yyyy hh24:mi:ss'), "
+ "'" + prosumerId + "'"
+ ") ";
logger.debug("Executing query [" + query + "]");
dbl.openDBConnection();
int x = dbl.set(query);
if (x <= 0) {
dbl.rollback();
dbl.closeDBConnection();
logger.error("Insert in TB_SDK_USER_FILES failed, stopping execution ");
return -3;
}
PreparedStatement ps = null;
String line = new String();
try {
br = new BufferedReader(new FileReader(f));
query = "insert into TB_SDK_CONTENTS "
+ "(id, contentCode, cli, content, contentType, PROSUMERID, cld, email) "
+ "values (?,?,?,?,?,?,?,?)";
logger.debug("Executing query [" + query + "]");
ps = dbl.getPreparedStatement(query);
while ((line = br.readLine()) != null) {
String[] tmp = line.split("\t");
if (tmp.length < 4) {
logger.debug("Sono presenti erroneamente meno di 4 valori!!!!");
br.close();
f.delete();
dbl.rollback();
ps.close();
dbl.closeDBConnection();
return -2;
}
ps.setInt(1, new Integer(id).intValue());
String test = tmp[0].replace("*", "");
logger.debug("first column: test [" + test + "]");
try {
if (test.length() > 0) {
double testDouble = Double.parseDouble(test);
//int testInt = Integer.parseInt(test);
}
} catch (NumberFormatException nfe) {
logger.error("The first column does not satisfy the format requested. Found [" + tmp[0] + "] -- NumberFormatException");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -6;
}
if (tmp[0].length() > 0) {
ps.setString(2, tmp[0].trim());
} else {
logger.error("The first column does not satisfy the format requested. Found [" + tmp[0] + "] -- length() <= 0");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -6;
}
test = tmp[1].replace("*", "").trim();
logger.debug("second column: test [" + test + "]");
try {
if (test.length() > 0) {
double testDouble = Double.parseDouble(test);
//int testInt = Integer.parseInt(test);
}
} catch (NumberFormatException nfe) {
logger.error("The second column does not satisfy the format requested. Found [" + tmp[1] + "]");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -7;
}
if (tmp[1].length() > 0) {
ps.setString(3, tmp[1].trim());
} else {
logger.error("The second column does not satisfy the format requested. Found [" + tmp[1] + "] -- length() <= 0");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -7;
}
logger.debug("third column: tmp[2] [" + tmp[2] + "]");
ps.setString(4, tmp[2]);
if ((tmp[3].length() > 0) && ((tmp[3].compareToIgnoreCase("TTS") == 0)
|| (tmp[3].compareToIgnoreCase("Audio") == 0))) {
ps.setString(5, tmp[3].trim());
logger.debug("fourth column: tmp[3] [" + tmp[3] + "]");
} else {
logger.error("The fourth column does not satisfy the format requested. Found [" + tmp[3] + "]");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -10;
}
ps.setString(6, prosumerId);
if (tmp.length > 4) {
tmp[4] = tmp[4].trim();
if ((tmp[4].length() > 0) && (tmp[4].length() <= 12) && (tmp[4].matches("^\\d*$"))) {
boolean ok = false;
logger.debug("Checking district codes...");
String[] codes = districtCodes.split(",");
for (int i = 0; i < codes.length; i++) {
if (tmp[4].startsWith(codes[i].trim())) {
ok = true;
logger.debug("District code found.");
break;
}
}
if (ok) {
ps.setString(7, tmp[4]);
logger.debug("fifth column: tmp[4] [" + tmp[4] + "]");
} else {
logger.error("The fifth column does not satisfy the format requested. Found [" + tmp[4] + "]");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -8;
}
} else if (tmp[4].length() == 0) {
ps.setString(7, "-");
logger.debug("fifth column: tmp[4] [-]");
} else {
logger.error("The fifth column does not satisfy the format requested. Found [" + tmp[4] + "]");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -8;
}
} else {
ps.setString(7, "-");
}
if (tmp.length == 6) {
tmp[5] = tmp[5].trim();
if ((tmp[5].length() > 0) && (isValidEmailAddress(tmp[5]))) {
ps.setString(8, tmp[5]);
} else if (tmp[5].length() == 0) {
ps.setString(8, "-");
} else {
logger.error("The sixth column does not satisfy the format requested. Found [" + tmp[5] + "]");
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
f.delete();
return -9;
}
} else {
ps.setString(8, "-");
}
x = ps.executeUpdate();
if (x <= 0) {
dbl.rollback();
ps.close();
dbl.closeDBConnection();
br.close();
//f.delete();
logger.error("Insert in TB_SDK_CONTENTS failed, stopping execution ");
return -3;
}
}
br.close();
//f.delete();
ps.close();
dbl.commit();
dbl.closeDBConnection();
return 0;
} catch (Exception ex) {
logger.error("An exception occured during the import of file [" + fileName + "] for line [" + line + "]", ex);
dbl.rollback();
if (br != null) {
br.close();
}
if (ps != null) {
ps.close();
}
dbl.closeDBConnection();
//f.delete();
return -1;
}
}
On the other class that handle the DB operation:
public int set(String query) {
Statement statement = null;
int ris = -1;
try {
if (connection == null || !connection.isValid(1)) connect();
logger.debug("Opening statement");
statement = connection.createStatement();
ris = statement.executeUpdate(query);
} catch (SQLException ex) {
logger.error("ERROR in execution of [" + query + "] due to: ", ex);
ris = -1;
} finally {
try {
logger.debug("Closing statement");
if (statement != null) {
statement.close();
}
} catch (SQLException ex1) {
logger.error("ERROR cannot close statement.", ex1);
}
}
return ris;
}
The connection open:
public boolean connect() {
if (dataSource != null) {
try {
connection = dataSource.getConnection();
logger.info("Connection open to DB");
return true;
} catch (SQLException ex) {
logger.error("An error occured during request of connection to datasource, due to...", ex);
return false;
}
}
String url;
if (connectionString.length() > 0) {
url = connectionString;
} else {
url = "jdbc:oracle:thin:#" + host + ":" + port + ":" + sid;
}
try {
// create an OracleDataSource
OracleDataSource ods = new OracleDataSource();
// set connection properties
Properties prop = new Properties();
prop.put("user", user);
prop.put("password", pwd);
ods.setConnectionProperties(prop);
ods.setURL(url);
// get the connection
connection = ods.getConnection();
connection.setAutoCommit(false);
((OracleConnection) connection).setDefaultRowPrefetch(10);
logger.info("Connection open to DB");
return true;
} catch (Exception ex) {
logger.info("An exception occured in execution of [connect]. ");
if (dataSource == null) {
logger.error("[" + url + ", " + user + ", " + pwd + "] ERROR in execution of [connect] due to: ", ex);
}
return false;
}
}
dataSource getConnection():
public static Connection getConnection() throws SQLException {
Connection con = dataSource.getConnection();
con.setAutoCommit(false);
return con;
}
The lookup for the data source is done in the Context Listener at context creation.
Edit:
I'm reading again the oracle tutorials about the usage of pooling connection and other links about it: every example the open and close of the connection to the pool is done in the same method. I thought that was just a need to be more simplest to explain all the steps, but I'm wondering if this is not also a need to satisfy the "transaction" at connection pool level. Can anyone confirm me this? Or give me a documentation link where this is well explained? Thank you very much!