We Keep Coding

format for Using Sqlite Database (from Java Package) to the Java Code inside `driverManager` to make connection

I have inserted my SQLite.db file inside of package named newpackage
Location of database file I inserted here isCollegeProject\src\operalogsapp\newpackage\SQLite.db
Now I want to get this database, so I want the correct format for inserting it to driver manager.
public class databaseConnection {
public static Connection con;
public static Connection getDBConnection(String username, String password, Integer portNumber, String serviceName){
try {
//Register the JDBC driver
System.out.println("before className");
Class.forName("org.sqlite.JDBC");
System.out.println(con==null);
System.out.println("Registered to the JDBC driver");
//Open the connection
if("V50700_HOTEL".equals(username) && "V50700_HOTEL".equals(password) && portNumber==1501 && "operal".equals(serviceName)){
con=DriverManager.getConnection("jdbc:sqlite:C:\\Users\\absasahu\\Documents\\db\\SQLite.db");
//inside the getConnection (above line) I want to get correct location of database to enter.
System.out.println("DriverManager connected to db");
}
} catch (ClassNotFoundException | SQLException ex) {
System.out.println("Exception : "+ex);
//Logger.getLogger(dbCoonectionCode.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("DFHUDSBFSDF");
return con;
}
Any help will be appreciated.

"jdbc:sqlite:sqlite_database_file_path", in your case from your code:
"jdbc:sqlite:C:/Users/absasahu/Documents/db/SQLite.db"
Regarding your question using a relative path:
"jdbc:sqlite:SQLite.db"
See also the answers at Set path to a jdbc SQLite database, where it will be located
A complete example with class file and driver in the same directory (unfortunately on a unix machine):
// mkdir collegeproject
// cd collegeproject
// curl -LJO https://github.com/xerial/sqlite-jdbc/releases/download/3.36.0.3/sqlite-jdbc-3.36.0.3.jar
// javac SqliteTest.java
// java -classpath ".;sqlite-jdbc-3.36.0.3.jar" SqliteTest # in Windows
// java -classpath ".:sqlite-jdbc-3.36.0.3.jar" SqliteTest # in Unix
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class SqliteTest {
public static void main(String[] args)
{
try (Connection connection = DriverManager.getConnection("jdbc:sqlite:sqltest.db");) {
try (Statement statement = connection.createStatement()) {
statement.executeUpdate("drop table if exists book");
statement.executeUpdate("create table book (id integer, title string)");
statement.executeUpdate("insert into book values(1, 'SQLite with JDBC for Beginners')");
try (ResultSet rs = statement.executeQuery("select * from book")) {
while (rs.next()) {
System.out.println("id = " + rs.getInt("id"));
System.out.println("title = " + rs.getString("title"));
}
} catch (SQLException e) {
System.err.println(e.getMessage());
}
} catch (SQLException e) {
System.err.println(e.getMessage());
}
} catch (SQLException e) {
System.err.println(e.getMessage());
}
}
}
$ javac SqliteTest.java
$ java -classpath ".:sqlite-jdbc-3.36.0.3.jar" SqliteTest
id = 1
title = SQLite with JDBC for Beginners
$

Open CSV Performance to write data

I came through a link: https://github.com/hyee/OpenCSV which drastically improves the writing time of the JDBC ResultSet to CSV due to setAsyncMode, RESULT_FETCH_SIZE
//Extract ResultSet to CSV file, auto-compress if the fileName extension is ".zip" or ".gz"
//Returns number of records extracted
public int ResultSet2CSV(final ResultSet rs, final String fileName, final String header, final boolean aync) throws Exception {
try (CSVWriter writer = new CSVWriter(fileName)) {
//Define fetch size(default as 30000 rows), higher to be faster performance but takes more memory
ResultSetHelperService.RESULT_FETCH_SIZE=10000;
//Define MAX extract rows, -1 means unlimited.
ResultSetHelperService.MAX_FETCH_ROWS=20000;
writer.setAsyncMode(aync);
int result = writer.writeAll(rs, true);
return result - 1;
}
}
But the problem is I don't know how I can merge above into my requirement. As the link has many other classes involved which I am not sure what they do and if I even need it for my requirement. Still, I tried but it fails to compile whenever I enable 2 commented line code. Below is my code.
Any help on how I can achieve this will be greatly appreciated.
package test;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.Date;
import com.opencsv.CSVWriter;
import com.opencsv.ResultSetHelperService;
public class OpenCSVTest1
{
static Connection con =null;
static Statement stmt = null;
static ResultSet rs = null;
public static void main(String args[]) throws Exception
{
connection ();
retrieveData(con);
}
private static void connection() throws Exception
{
try
{
Class.forName("<jdbcdriver>");
con = DriverManager.getConnection("jdbc:","<username>","<pass>");
System.out.println("Connection successful");
}
catch (Exception e)
{
System.out.println("Exception while establishing sql connection");
throw e;
}
}
private static void retrieveData(Connection con) throws Exception
{
try
{
stmt=con.createStatement();
stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
String query = "SELECT * FROM dbo.tablename";
rs=stmt.executeQuery(query);
CSVWriter writer = new CSVWriter(new BufferedWriter(new FileWriter("C:\\Data\\File1.csv")));
ResultSetHelperService service = new ResultSetHelperService();
/*** ResultSetHelperService.RESULT_FETCH_SIZE=10000; ***/ // to add
service.setDateTimeFormat("yyyy-MM-dd HH:mm:ss.SSS");
System.out.println("**** Started writing Data to CSV **** " + new Date());
writer.setResultService(service);
/*** writer.setAsyncMode(aync); ***/ // to add
int lines = writer.writeAll(rs, true, true, false);
writer.flush();
writer.close();
System.out.println("** OpenCSV -Completed writing the resultSet at " + new Date() + " Number of lines written to the file " + lines);
}
catch (Exception e)
{
System.out.println("Exception while retrieving data" );
e.printStackTrace();
throw e;
}
finally
{
rs.close();
stmt.close();
con.close();
}
}
}
UPDATE
I have updated my code. Right now code is writing complete resultset in CSV at once using writeAll method which is resulting in time consumption.
Now what I want to do is write resultset to CSV in batches as resultset's first column will always have dynamically generated via SELECT query Auto Increment column (Sqno) with values as (1,2,3..) So not sure how I can read result sets first column and split it accoridngly to write in CSV. may be HashMap might help, so I have also added resultset-tohashmap conversion code if required.
import com.opencsv.CSVWriter;
import com.opencsv.ResultSetHelperService;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class OpenCSVTest1
{
static int fetchlimit_src = 100;
static Connection con =null;
static Statement stmt = null;
static ResultSet rs = null;
static String filename = "C:\\Data\\filename.csv";
static CSVWriter writer;
public static void main(String args[])
{
try
{
connection();
retrieveData(con);
}
catch(Exception e)
{
System.out.println(e);
}
}
private static void connection() throws Exception
{
try
{
Class.forName("<jdbcdriver>");
con = DriverManager.getConnection("jdbc:","<username>","<pass>");
System.out.println("Connection successful");
}
catch (Exception e)
{
System.out.println("Exception while establishing sql connection");
throw e;
}
}
private static void retrieveData(Connection con) throws Exception
{
try
{
stmt=con.createStatement();
String query = "SELECT ROWNUM AS Sqno, * FROM dbo.tablename "; // Oracle
// String query = "SELECT ROW_NUMBER() OVER(ORDER BY Id ASC) AS Sqno, * FROM dbo.tablename "; // SQLServer
System.out.println(query);
stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(fetchlimit_src);
System.out.println("**** Started querying src **** " + new Date());
rs=stmt.executeQuery(query);
System.out.println("**** Completing querying src **** " + new Date());
// resultset_List(rs); // If required store resultset(rs) to HashMap
writetoCSV(rs,filename);
/** How to write resultset to CSV in batches instead of writing all at once to speed up write performance ?
* Hint: resultset first column is Autoincrement [Sqno] (1,2,3...) which might help to split result in batches.
*
**/
}
catch (Exception e)
{
System.out.println("Exception while retrieving data" );
e.printStackTrace();
throw e;
}
finally
{
rs.close();
stmt.close();
con.close();
}
}
private static List<Map<String, Object>> resultset_List(ResultSet rs) throws SQLException
{
ResultSetMetaData md = rs.getMetaData();
int columns = md.getColumnCount();
List<Map<String, Object>> rows = new ArrayList<Map<String, Object>>();
while (rs.next())
{
Map<String, Object> row = new HashMap<String, Object>(columns);
for(int i = 1; i <= columns; ++i)
{
row.put(md.getColumnName(i), rs.getObject(i));
}
rows.add(row);
}
// System.out.println(rows.toString());
return rows;
}
private static void writetoCSV(ResultSet rs, String filename) throws Exception
{
try
{
writer = new CSVWriter(new BufferedWriter(new FileWriter(filename)));
ResultSetHelperService service = new ResultSetHelperService();
service.setDateTimeFormat("yyyy-MM-dd HH:mm:ss.SSS");
long batchlimit = 1000;
long Sqno = 1;
ResultSetMetaData rsmd = rs.getMetaData();
String columnname = rsmd.getColumnLabel(1); // To retrieve columns with labels (for example SELECT ROWNUM AS Sqno)
System.out.println("**** Started writing Data to CSV **** " + new Date());
writer.setResultService(service);
int lines = writer.writeAll(rs, true, true, false);
System.out.println("** OpenCSV -Completed writing the resultSet at " + new Date() + " Number of lines written to the file " + lines);
}
catch (Exception e)
{
System.out.println("Exception while writing data" );
e.printStackTrace();
throw e;
}
finally
{
writer.flush();
writer.close();
}
}
}

You should be able to use the OpenCSV sample, pretty much exactly as it is provided in the documentation. So, there should be no need for you to write any of your own batching logic.
I was able to write a 6 million record result set to a CSV file in about 10 seconds. To be clear -that was just the file-write time, not the DB data-fetch time - but I think that should be fast enough for your needs.
Here is your code, with adaptations for using OpenCSV based on its documented approach... But please see the warning at the end of my notes!
import com.opencsv.CSVWriter;
import com.opencsv.ResultSetHelperService;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.Date;
import java.text.SimpleDateFormat;
public class OpenCSVDemo {
static int fetchlimit_src = 100;
static Connection con = null;
static Statement stmt = null;
static ResultSet rs = null;
static String filename = "C:\\Data\\filename.csv";
public static void main(String args[]) {
try {
connection();
retrieveData(con);
} catch (Exception e) {
System.out.println(e);
}
}
private static void connection() throws Exception {
try {
final String jdbcDriver = "YOURS GOES HERE";
final String dbUrl = "YOURS GOES HERE";
final String user = "YOURS GOES HERE";
final String pass = "YOURS GOES HERE";
Class.forName(jdbcDriver);
con = DriverManager.getConnection(dbUrl, user, pass);
System.out.println("Connection successful");
} catch (Exception e) {
System.out.println("Exception while establishing sql connection");
throw e;
}
}
private static void retrieveData(Connection con) throws Exception {
try {
stmt = con.createStatement();
String query = "select title_id, primary_title from imdb.title";
System.out.println(query);
stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(fetchlimit_src);
System.out.println("**** Started querying src **** " + new Date());
rs = stmt.executeQuery(query);
System.out.println("**** Completing querying src **** " + new Date());
// resultset_List(rs); // If required store resultset(rs) to HashMap
System.out.println();
String timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());
System.out.println("Started writing CSV: " + timeStamp);
writeToCsv(rs, filename, null, Boolean.FALSE);
timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());
System.out.println("Finished writing CSV: " + timeStamp);
System.out.println();
} catch (Exception e) {
System.out.println("Exception while retrieving data");
e.printStackTrace();
throw e;
} finally {
rs.close();
stmt.close();
con.close();
}
}
public static int writeToCsv(final ResultSet rs, final String fileName,
final String header, final boolean aync) throws Exception {
try (CSVWriter writer = new CSVWriter(fileName)) {
//Define fetch size(default as 30000 rows), higher to be faster performance but takes more memory
ResultSetHelperService.RESULT_FETCH_SIZE = 1000;
//Define MAX extract rows, -1 means unlimited.
ResultSetHelperService.MAX_FETCH_ROWS = 2000;
writer.setAsyncMode(aync);
int result = writer.writeAll(rs, true);
return result - 1;
}
}
}
Points to note:
1) I used "async" set to false:
writeToCsv(rs, filename, null, Boolean.FALSE);
You may want to experiment with this and the other settings to see if they make any significant difference for you.
2) Regarding your comment "the link has many other classes involved": The OpenCSV library's entire JAR file needs to be included in your project, as does the related disruptor JAR:
opencsv.jar
disruptor-3.3.6.jar
To get the JAR files, go to the GitHub page, click on the green button, select the zip download, unzip the zip file, and look in the "OpenCSV-master\release" folder.
Add these two JARs to your project in the usual way (depends on how you build your project).
3) WARNING: This code runs OK when you use Oracle's Java 8 JDK/JRE. If you try to use OpenJDK (e.g. for Java 13 or similar) it will not run. This is because of some changes behind the scenes to hidden classes. If you are interested, there are more details here.
If you need to use an OpenJDK version of Java, you may therefore have better luck with the library on which this CSV library is based: see here.

Java Excel Manipulation

Hello i a am curently trying to read an excel file and manipulate it as a database. I wrote the following code whitch i supposed to fetch the first row of the datasheet and print it
import java.sql.Connection;
import java.sql.Statement;
import java.sql.ResultSet;
import java.sql.DriverManager;
public class ExcelReader {
public static void main( String [] args ) {
Connection c = null; Statement stmnt = null;
try
{
Class.forName( "sun.jdbc.odbc.JdbcOdbcDriver" );
//using DSN connection. Here qa is the name of DSN
//c = DriverManager.getConnection( "jdbc:odbc:qa", "", "" );
//using DSN-less connection
c = DriverManager.getConnection( "jdbc:odbc:Driver={Microsoft Excel Driver (*.xls)};DBQ=C:/Users/Μιχάλης/Documents/mike1.xls");
stmnt = c.createStatement();
String query = "select * from [Ledis$] ;";
ResultSet rs = stmnt.executeQuery( query );
while( rs.next() )
{
System.out.println(rs.getString(1));
}
}
catch( Exception e )
{
//System.err.println( e );
}
finally
{
try
{
stmnt.close();
c.close();
}
catch( Exception e )
{
System.err.println( e );
}
}
}
}
When i execute the upper code i get the following print out
90-02-00-0000-2000
90-02-00-0000-2400
90-02-00-0000-2500
90-02-00-0000
90-02-00
90-02
90-06-00-0000-6000
90-06-00-0000-6100
90-06-00-0000-6200
90-06-00-0000-6300
90-06-00-0000-6400
90-06-00-0000-6500
90-06-00-0000-6600
90-06-00-0000
90-06-00
90-06
null
The problem is that in the last column the price that was supposed to be printed is 90 instead of null.
Does anybody have any idea of what is happening;

MySQL Netbeans Java ResultSet Failure

Hi I a have MySql installed with Netbeans and have been trying to use Java with MySQL, however I am running into an issue when I run it. My database is called "test" and my table is "task". The two columns I have are: "id", and "task" (and I realized that naming a variable the same as the table probably is not a good idea). I also have a side question in the code area asking what it does. This is the error:
run:
May 22, 2015 11:52:25 PM databasetest.DatabaseTest main
SEVERE: Operation not allowed after ResultSet closed
java.sql.SQLException: Operation not allowed after ResultSet closed
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:988)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:974)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919)
at com.mysql.jdbc.ResultSetImpl.checkClosed(ResultSetImpl.java:804)
at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:6986)
at databasetest.DatabaseTest.main(DatabaseTest.java:43)
BUILD SUCCESSFUL (total time: 41 seconds)
This is my code:
package databasetest;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.logging.Level;
import java.util.logging.Logger;
public class DatabaseTest {
public static void main(String[] args) {
Connection con = null;
Statement st = null;
ResultSet rs = null;
PreparedStatement pst = null;
String url = "jdbc:mysql://localhost:3306/test";
String user = "root";
String password = "cinder";
try {
String author = "Trygve Gulbranssen";
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery("SELECT VERSION()");
//^^ what is VERSION? What is this supposed to be doing?
for (int i=1; i<=1000; i++) {
String query;
query = "INSERT INTO task(task) VALUES(" + 2*i + ")";
st.executeUpdate(query);
}
if (rs.next()) {
System.out.println(rs.getString(1));
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(DatabaseTest.class.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(DatabaseTest.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
}
}

SELECT VERSION() is meant to tell you your MySQL version. First, print the result of the SELECT then run your other queries. Running intermediate insert queries with the Statement implicitly closes the ResultSet, hence your error. Move
if (rs.next()) {
System.out.println(rs.getString(1));
}
before you run
for (int i=1; i<=1000; i++) {
// String query;
String query = "INSERT INTO task(task) VALUES(" + 2*i + ")";
st.executeUpdate(query);
}

Java Oracle database connection error

I am new to Java and Oracle. I am trying to make an application that lists serial numbers of a product and when you click on a product's serial number from the list, it shows the other column informations from the database in a textbox. I have a form named CRUD.I am using Oracle 10g. Code is here:
package project1;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.event.ListSelectionEvent;
import javax.swing.event.ListSelectionListener;
public class CRUD extends javax.swing.JFrame {
Connection connection = null;
public CRUD() {
try {
initComponents();
String driverName = "oracle.jdbc.driver.OracleDriver";
Class.forName(driverName);
String serverName = "192.168.0.36";
String portNumber = "1521";
String sid = "XE";
String url = "jdbc:oracle:thin:#"+serverName+":"+portNumber+":"+sid;
String userName = "HR";
String password = "hr";
try {
connection = DriverManager.getConnection(url,userName,password);
} catch (SQLException ex) {
Logger.getLogger(CRUD.class.getName()).log(Level.SEVERE, null, ex);
}
try {
String temp="";
Statement stmt = connection.createStatement();
ResultSet rs = stmt.executeQuery("SELECT SERI_NO FROM KART");
while(rs.next()) // dönebildiği süre boyunca
{
String s = rs.getString("SERI_NO") ; //kolon isimleri oluşturuldu
temp+=s+"_";
}
Object [] tem_obj;
tem_obj=temp.split("_");
listOgrenciler.setListData(tem_obj);
} catch (SQLException ex) {
Logger.getLogger(edit.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (ClassNotFoundException ex) {
Logger.getLogger(CRUD.class.getName()).log(Level.SEVERE, null, ex);
}
listOgrenciler.addListSelectionListener(new ListSelectionListener() {
#Override
public void valueChanged(ListSelectionEvent arg0) {
if (!arg0.getValueIsAdjusting()) {
try {
Statement stmtx = connection.createStatement();
Object[] sss=listOgrenciler.getSelectedValues();
String swhere="" ;
for (int i = 0; i < sss.length; i++) {
swhere+=sss[i].toString()+",";
}
swhere=swhere.substring(0,swhere.length()-1);
ResultSet rsx = stmtx.executeQuery("SELECT * FROM KART where SERI_NO in ("+swhere+")") ;
String temp="";
Object [] tem_obj;
tem_obj=temp.split("_");
String ara="";
for (int i = 0; i < tem_obj.length; i++) {
ara+=tem_obj[i].toString()+"\n";
}
texttoarea.setText(ara);
} catch (SQLException ex)
{
Logger.getLogger(edit.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
});
}
Errors i get are here:
20.Şub.2014 11:22:11 project1.CRUD$1 valueChanged
SEVERE: null
java.sql.SQLSyntaxErrorException: ORA-00904: "SNS080961097": invalid identifier
.....
at project1.CRUD$1.valueChanged(CRUD.java:78)
......
As I said before, I am new to both Java and Oracle. If the errors are so obvious don't laugh:)

Your this query
ResultSet rsx = stmtx.executeQuery("SELECT * FROM KART where SERI_NO in ("+swhere+")") ;
should be like this:
ResultSet rsx = stmtx.executeQuery("SELECT * FROM KART where SERI_NO in ('"+swhere+"')") ;

Actually there is no problems with you connection step to Oracle DB, its connected successfully, Your problem is within the query. make sure that you have a SERI_NO column in your KART table.
i suggest you to RUN the both same queries in you code from any SQL client such SQLDeveloper and see what these queries retrieve.

Observe this statement once,
swhere=swhere.substring(0,swhere.length()-1);
replace the above statement with the following
shere=swhere.substring(0,swhere.length()-2);
Because an extra comma(,) is included in your sql statement.

There is no issue with your connection.
Please add some logging to your code and you will know exactly where the error is throwing.
I guess the error is throwing in this line..
SELECT * FROM KART where SERI_NO in ("+swhere+")
You have to specify this as a string with '',where you actual select should look like below.
SELECT * FROM KART where SERI_NO in ('ABC','XCV');
so please check with this one check the value of the "swhere"