We Keep Coding

How to a csv file in oracle using sql loader in java

I want to load data from a csv file to oracle database. Here is my code-
void importData(Connection conn) {
Statement stmt;
String query;
String filename = "C:/CSVData/Student.csv";
try {
stmt = conn.createStatement(
ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_UPDATABLE);
query = "LOAD DATA INFILE '" + filename + "' INTO TABLE Student FIELDS terminated by ',' ;";
System.out.println(query);
stmt.executeQuery(query);
} catch (Exception e) {
e.printStackTrace();
stmt = null;
}
}
This code runs perfectly and load data in mysql. But now I want to load data in oracle. what change do i have to make in query. Please help me. Thank you in advance...

First, you need to write a control file.
Control file example FYI:
Load data
infile "D:/Viki/test.CSV" --the input file(s) you need to import
truncate --the option you need do. (truncate, append, insert, replace. insert by default)
into table vk_recon_China_201409_i --table need insert to
fields terminated by "," --
trailing nullcols
(
col_a filler
, col_b "Trim(:col_b)"
, col_c "To_Date(:col_c,'yyyy/mm/dd hh24:mi:ss')"
, seqno sequence(Max,1)
)
Then, call sqlldr command by Runtime.exec or ProcessImpl.start,
public void startUp() {
StringBuffer sb = new StringBuffer();
String path = "sqlldr user/password#sid readsize=10485760 bindsize=10485760 rows=1000 control=controlFileName.ctl log=controlFileName.log direct=true \n pause";
try {
Process pro = Runtime.getRuntime().exec(path);
BufferedReader br = new BufferedReader(new InputStreamReader(pro.getInputStream()), 4096);
String line = null;
int i = 0;
while ((line = br.readLine()) != null) {
if (0 != i)
sb.append("\r\n");
i++;
sb.append(line);
}
} catch (Exception e) {
sb.append(e.getMessage());
}
}

Try making the external table.You can create an external table on your CSV file using ORACLE_LOADER driver and then update your existing table with data in your external table using DML (MERGE for example).

I think below query should work.
query = "LOAD DATA INFILE '" + filename + "' APPEND INTO TABLE Student FIELDS terminated by ',' ;";
For more info:-
http://docs.oracle.com/cd/E11882_01/server.112/e16536/ldr_control_file.htm#SUTIL005

How to use setClob() in PreparedStatement inJDBC

Here are the info:
I have a String
I want to insert a record in a table with the String in a column whose
datatype is CLOB.
I would like to use setClob() method of the preparedstatement.
So my question is how to create a Clob object from this String so that I
can use setClob() method.
Thanks in advance,
Naveen

If you want to write a String to CLOB column just use PreparedStatement.setString.
If you want to know how to create a CLOB from String this is it
Clob clob = connection.createClob();
clob.setString(1, str);

You may create the clob from a connection object as follows
Connection con = null;// write code to make a connection object
Clob clob = con.createClob();
String str = "this is a stirng";
clob.setString(1, str );
PreparedStatement ps = null;// write code to create a prepared statement
ps.setClob(4, clob);
Or you may try the alternative code as follows :
//alternative way
String str = "this is a stirng";
ByteArrayInputStream inputStream = new ByteArrayInputStream(str.getBytes());
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
int parameterIndex = 1;
PreparedStatement ps = null;// write code to create a prepared statement
ps.setClob(parameterIndex, inputStreamReader);

For CLOB it is of String already. So, just use .setString() and that should work. One thing about ORACLE jdbc if you are using it, it like the CLOB INPUT parameter to be the last one in your statement especially with a large data.
Example:
INSERT INTO MY_TABL (NUM_COL, VARC_COL, VARC_COL, TS_COL, CLOB_COL)
VALUES(?,?,?,?,?);
As you can see, the CLOB_COL is of type CLOB and should be last so that when
you do .setString(5) and 5 is the last index.

I had a specific variation of this issue which required to insert a clob into an Oracle database from java code running on that db. None of the answers here quite worked for me.
I eventually found solution, the trick being to use oracle.sql.CLOB
This the approach I discovered:
create table test_clob (
c clob
);
create or replace and compile java source named java_clob_insert as
import java.sql.Connection;
import java.sql.PreparedStatement;
import oracle.sql.CLOB;
import java.io.Writer;
public class JavaClobInsert {
public static void doInsert () {
try {
//create the connection and statement
Connection oracleConn =
(new oracle.jdbc.OracleDriver()).defaultConnection();
String stmt = "INSERT INTO test_clob values (?)";
PreparedStatement oraclePstmt = oracleConn.prepareStatement(stmt);
//Imagine we have a mysql longtext or some very long string
String s = "";
for (int i = 0; i < 32768; i++) {
s += i % 10;
}
//Initialise the Oracle CLOB
CLOB clob;
clob = CLOB.createTemporary(oracleConn, true, CLOB.DURATION_CALL);
//Good idea to check the string is not null before writing to clob
if (s != null) {
Writer w = clob.setCharacterStream( 1L );
w.write(s);
w.close();
oraclePstmt.setClob(1, clob);
} else {
oraclePstmt.setString(1, "");
}
//clean up
oraclePstmt.executeUpdate();
oracleConn.commit();
oraclePstmt.close();
oracleConn.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
/
create or replace procedure clob_insert as language java name
'JavaClobInsert.doInsert()';
/
begin
clob_insert;
end;
/
select *
from test_clob;

Today i had an issue with a Clob field because i was using "setString" to set the parameter, but then i had this error while testing with a very long string: "setString can handle only Strings with less than 32766 characters"
I used connection.createClob but it gave me this exception:
java.lang.AbstractMethodError: org.apache.tomcat.dbcp.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.createClob()Ljava/sql/Clob;
So looking for this exception i found this
using CLOB in java throwing exception and the accepted answer (using setCharacterStream instead of setClob) worked for me
Copy/Pasted from the accepted answer (so all credits are for a_horse_with_no_name )
StringReader reader = new StringReader(userAbout);
PreparedStatement insertClob = dbCon.prepareStatement("UPDATE user_data SET user_about=? WHERE user_id=?");
insertClob.setCharacterStream(1, reader, userAbout.length());
insertClob.setInt(2,userId);

My answer is slightly different than others...
I had a PreparedStatement, stmt, and was using stmt.setString(colIndex, value) for updates to my database that had a CLOB column.
This worked without fail for me when inserting and updating rows in the database table.
When others tested this code though they would occasionally see an exception occur:
ORA-22275: invalid LOB locator
It only seemed to happen on updates, not inserts - not sure why on that, when value was null. And I only ever had this occur with Oracle databases, not MSSQL or DB2.
Anyway to fix it I changed the logic to test for a null value
if (value == null) {
stmt.setNull(colIndex, java.sql.Types.CLOB);
}
else {
stmt.setString(colIndex, value);
}
This worked without fail for me and others!

insert cmr values into database

I have a CMR file with different values, but I don't know how to use the separator.
I want to store "numberPacketsLost", "jitter", and "latency" into a mySQL database using java netbeans with button.
Thanks! :)
Code from cmr file :
"cdrRecordType","globalCallID_callManagerId","globalCallID_callId","nodeId","directoryNum","callIdentifier","dateTimeStamp","numberPacketsSent","numberOctetsSent","numberPacketsReceived","numberOctetsReceived","numberPacketsLost","jitter","latency","pkid","directoryNumPartition","globalCallId_ClusterID","deviceName","varVQMetrics"
INTEGER,INTEGER,INTEGER,INTEGER,VARCHAR(50),INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,INTEGER,UNIQUEIDENTIFIER,VARCHAR(50),VARCHAR(50),VARCHAR(129),VARCHAR(600)
2,2,1732470,2,"4241",47660016,1319556369,192,33024,191,32852,0,0,0,"8ea4f719-c49c-4456-a2a8-972ebcfb57a9","2b494acb-9359-7f52-b0ef-7b66bb672b73","StandAloneCluster","SEP0026CB3C2A16","MLQK=0.0000;MLQKav=0.0000;MLQKmn=0.0000;MLQKmx=0.0000;ICR=0.0000;CCR=0.0000;ICRmx=0.0000;CS=0;SCS=0;MLQKvr=0.95"
2,2,1732447,2,"5352",47659963,1319556371,1409,242348,1408,242176,0,0,0,"61ca6d9f-8e75-4282-b303-3fea2fa75df7","2b494acb-9359-7f52-b0ef-7b66bb672b73","StandAloneCluster","SEP64168D506D26","MLQK=4.5000;MLQKav=4.3554;MLQKmn=4.1440;MLQKmx=4.5000;ICR=0.0000;CCR=0.0029;ICRmx=0.0263;CS=1;SCS=1;MLQKvr=0.95"
2,2,1732134,2,"5502",47658367,1319556373,28529,4906988,28537,4908364,0,0,0,"d1717925-89bf-41b4-b122-6162db89128f","2b494acb-9359-7f52-b0ef-7b66bb672b73","StandAloneCluster","SEP64168D50A4DB","MLQK=4.5000;MLQKav=4.4570;MLQKmn=4.1440;MLQKmx=4.5000;MLQKvr=0.95;CCR=0.0011;ICR=0.0000;ICRmx=0.0267;CS=9;SCS=9"

You need to open up the cmr file, read through it line by line skipping the headers and extract the data. Once you get the needed data, just write a query to insert into the database.
BufferedReader br = new BufferedReader(new FileReader(new File("myCmrFile")));
String line = null;
int linecount = 0;
while ((line = br.readLine()) != null){
if (linecount++ < 2) // skip the headers
continue;
// split the data and convert to integers
String[] data = line.split(",");
Integer packetsLost = Integer.valueOf(data[10]);
Integer jitter = Integer.valueOf(data[11]);
Integer latency = Integer.valueOf(data[12]);
// now insert into the db, query will look something like this
String query = "INSERT INTO myTable (numberPacketsLost, jitter, latency) VALUES(?,?,?)";
PreparedStatement ps = connection.prepareStatment(query);
ps.setInt(1, packetsLost);
ps.setInt(2, jitter);
ps.setInt(3, latency);
ps.executeUpdate();
}
This code won't work exactly, you will need to change it around based on the real values of your database.

Populating database via jdbc

In the below code, values are inserted manually: 13 and Aman. But what I am doing is reading a file and then till the completion of the file, I am inserting values from it into mysql database tables.
public class Main {
public static void main(String[] argv) throws Exception {
String driver = "com.mysql.jdbc.Driver";
Class.forName(driver);
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/jdbctutorial", "root", "root");
Statement st = con.createStatement();
int val = st.executeUpdate("INSERT employee VALUES(" + 13 + "," + "'Aman'" + ")");
System.out.println("1 row affected");
}
}
I was trying to use each line like this:
String query = "INSERT INTO home (home_no, arrival_time, duration, persons, price, origin_city) VALUES("+line+");";
How do I do it?

Depending on how large the contents of the file that you are reading, it may be worth while to check LOAD DATA INFILE syntax, rather than executing queries in a for or while loop.
Edit:
Without seeing your code and line is the current line of the file you are reading and that you are using the syntax above to store the query, I would break down your problems,
Check the line variable prior to executing the query
Check how to insert the values manually opposed to reading the contents of the file, as you had shown above with 13 and Aman.
Figure out how to piece those two together, may need string manipulation.
This should be all you need.

BufferedReader rd = new BufferedReader(new FileReader("yourFile.txt"));
String line;
while ((line = rd.readLine()) != null)
{
// This time this loop runs, you will have the next time of your file.
insertARecord(line.split(" "));
}
rd.close();
// A bit further
public void insertARecord(String[] data)
{
// insert your record.
}

Running a .sql script using MySQL with JDBC

I am starting to use MySQL with JDBC.
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection("jdbc:mysql:///x", "x", "x");
stmt = conn.createStatement();
stmt.execute( "CREATE TABLE amigos" +
"("+
"id int AUTO_INCREMENT not null,"+
"nombre char(20) not null,"+
"primary key(id)" +
")");
I have 3-4 tables to create and this doesn't look good.
Is there a way to run a .sql script from MySQL JDBC?

Ok. You can use this class here (posted on pastebin because of file length) in your project. But remember to keep the apache license info.
JDBC ScriptRunner
It's ripoff of the iBatis ScriptRunner with dependencies removed.
You can use it like this
Connection con = ....
ScriptRunner runner = new ScriptRunner(con, [booleanAutoCommit], [booleanStopOnerror]);
runner.runScript(new BufferedReader(new FileReader("test.sql")));
That's it!

I did a lot of research on this and found a good util from spring. I think using SimpleJdbcTestUtils.executeSqlScript(...) is actually the best solution, as it is more maintained and tested.
Edit: SimpleJdbcTestUtils is deprecated. You should use JdbcTestUtils. Updated the link.

Spring Framework's ResourceDatabasePopulator may help. As you said you're using MySQL and JDBC, let's assume you have a MySQL-backed DataSource instance ready. Further, let's assume your MySQL script files are classpath-locatable. Let's assume you are using WAR layout and the script files are located in a directory src/main/webapp/resources/mysql-scripts/... or src/test/resources/mysql-scripts/.... Then you can use ResourceDatabasePopulator to execute SQL scripts like this:
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
import javax.sql.DataSource;
DataSource dataSource = getYourMySQLDriverBackedDataSource();
ResourceDatabasePopulator rdp = new ResourceDatabasePopulator();
rdp.addScript(new ClassPathResource(
"mysql-scripts/firstScript.sql"));
rdp.addScript(new ClassPathResource(
"mysql-scripts/secondScript.sql"));
try {
Connection connection = dataSource.getConnection();
rdp.populate(connection); // this starts the script execution, in the order as added
} catch (SQLException e) {
e.printStackTrace();
}

For simple sql script splitted by ';' you can use this simple function.
It remove comments and run statements one by one
static void executeScript(Connection conn, InputStream in)
throws SQLException
{
Scanner s = new Scanner(in);
s.useDelimiter("/\\*[\\s\\S]*?\\*/|--[^\\r\\n]*|;");
Statement st = null;
try
{
st = conn.createStatement();
while (s.hasNext())
{
String line = s.next().trim();
if (!line.isEmpty())
st.execute(line);
}
}
finally
{
if (st != null)
st.close();
}
}

#Pantelis Sopasakis
Slightly modified version on GitHub: https://gist.github.com/831762/
Its easier to track modifications there.

Regarding SQL script runner (which I'm also using), I noticed the following piece of code:
for (int i = 0; i < cols; i++) {
String value = rs.getString(i);
print(value + "\t");
}
However, in the API documentation for the method getString(int) it's mentioned that indexes start with 1, so this should become:
for (int i = 1; i <= cols; i++) {
String value = rs.getString(i);
print(value + "\t");
}
Second, this implementation of ScriptRunner does not provide support for DELIMITER statements in the SQL script which are important if you need to compile TRIGGERS or PROCEDURES. So I have created this modified version of ScriptRunner: http://pastebin.com/ZrUcDjSx which I hope you'll find useful.

Another interesting option would be to use Jisql to run the scripts. Since the source code is available, it should be possible to embed it into an application.
Edit: took a careful look at it; embedding it inside something else would require some modification to its source code.

Can you use this:
public static void executeSQL(File f, Connection c) throws Exception {
BufferedReader br = new BufferedReader(new FileReader(f));
String sql = "", line;
while ((line = br.readLine()) != null) sql += (line+"\n");
c.prepareCall(sql).execute(sql);
}
This function gets SQL file and DB connection.
Then it reads the file line-by-line using BufferedReader from java.io.
And, finally, executes the read statements.
Java 8+ version:
public static void executeSQL(Path p, Connection c) throws Exception {
List<String> lines = Files.readAllLines(p);
String s = String.join("\n", lines.toArray(new String[0]));
c.prepareCall(s).execute(s);
}

Write code to:
Read in a file containing a number of SQL statements.
Run each SQL statement.

For Oracle PL/SQL, the Oracle JDBC-driver indeed supports executing entire SQL-scripts including stored procedures and anonymous blocks (PL/SQL specific notation), see
Can the JDBC Drivers access PL/SQL Stored Procedures?
The Oracle JDBC driver FAQ has more info:
Oracle JDBC drivers support execution
of PL/SQL stored procedures and
anonymous blocks. They support both
SQL92 escape syntax and Oracle PL/SQL
block syntax. The following PL/SQL
calls would work with any Oracle JDBC
driver:
// SQL92 syntax
CallableStatement cs1 = conn.prepareCall
( "{call proc (?,?)}" ) ; // stored proc
CallableStatement cs2 = conn.prepareCall
( "{? = call func (?,?)}" ) ; // stored func
// Oracle PL/SQL block syntax
CallableStatement cs3 = conn.prepareCall
( "begin proc (?,?); end;" ) ; // stored proc
CallableStatement cs4 = conn.prepareCall
( "begin ? := func(?,?); end;" ) ; // stored func
It should be possible to read in a file and feed the content to the prepareCall()-method.

Maven SQL Plugin Use this plugin to execute SQL statements a file or list of files through
sqlCommand
srcFiles
3.fileset configurations

There isn't really a way to do this.
You could either run the mysql command line client via Runtime.exec(String[]) and read this article when you decide for this option
Or try using the ScriptRunner (com.ibatis.common.jdbc.ScriptRunner) from ibatis. But it's a bit stupid to include a whole library just to run a script.

Here's a quick and dirty solution that worked for me.
public void executeScript(File scriptFile) {
Connection connection = null;
try {
connection = DriverManager.getConnection(url, user, password);
if(scriptFile.exists()) {
var buffer = new StringBuilder();
var scanner = new Scanner(scriptFile);
while(scanner.hasNextLine()) {
var line = scanner.nextLine();
buffer.append(line);
// If we encounter a semicolon, then that's a complete statement, so run it.
if(line.endsWith(";")) {
String command = buffer.toString();
connection.createStatement().execute(command);
buffer = new StringBuilder();
} else { // Otherwise, just append a newline and keep scanning the file.
buffer.append("\n");
}
}
}
else System.err.println("File not found.");
} catch (SQLException e) {
e.printStackTrace();
} finally {
if(connection != null) connection.close();
}