I have written this code to read from a table and write onto a file, but I am unable to write to a file. The file gets created, but its empty. I don't have a mysql problem tried to insert comments and debug. Code compiles fine, but has a problem in creating a file:
import java.io.File;
import java.io.FileNotFoundException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.Formatter;
import java.util.Scanner;
import java.util.logging.Level;
import java.util.logging.Logger;
public class tabletocsv {
public static void main(String[] args) {
System.out.println("enter the table name");
Scanner input = new Scanner(System.in);
String table = input.next();
System.out.println("enter number of columns");
int columns = input.nextInt();
createcsv(table, columns);
System.out.print(" csv created");
displaycsv(table, columns);
}
static void displaycsv(String table, int count) {
Scanner file123;
try {
file123 = new Scanner(new File("/Users/Tannishk/Documents/csv/" + table + ".csv"));
//System.out.printf("reading");
while (file123.hasNext()) {
System.out.println("going inside");
for (int i = 1; i <= count; i++) {
String a = file123.next();
System.out.print(a + " ");
}
System.out.println();
}
} catch (FileNotFoundException ex) {
Logger.getLogger(tabletocsv.class.getName()).log(Level.SEVERE, null, ex);
}
}
static void createcsv(String table, int count) {
Formatter x;
Connection connection;
Statement st;
try {
x = new Formatter("/Users/Tannishk/Documents/csv/" + table + ".csv");
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/db", "root", "password");
st = connection.createStatement();
ResultSet rs = st.executeQuery("select * from " + table + ";");
while (rs.next()) {
for (int i = 1; i <= count; i++) {
String a = rs.getString(i);
x.format("%s ", a);
// System.out.print(a);
}
x.format("\n");
}
} catch (Exception e) {
} finally {
}
}
}
Try flushing and closing the formatter
I think you have missed to flush the file.
static void createcsv(String table,int count)
{
Formatter x = null;
Connection connection;
Statement st;
try{
x = new Formatter("/Users/Tannishk/Documents/csv/"+table+".csv");
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/db","root","password");
st = connection.createStatement();
ResultSet rs= st.executeQuery("select * from "+table+";");
while(rs.next())
{
for(int i=1;i<=count;i++)
{
String a = rs.getString(i);
x.format("%s ",a);
// System.out.print(a);
}
x.format("\n");
}
}
catch(Exception e)
{
}
finally
{
try {
x.flush();
x.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
Try this ...
You need to close the output stream that you're using to write to the file. For example if you're using a FileWriter object named fout to write to the file, you'll have to do fout.close() to actually have anything written to the file as otherwise it would just be in memory.
Related
I came through a link: https://github.com/hyee/OpenCSV which drastically improves the writing time of the JDBC ResultSet to CSV due to setAsyncMode, RESULT_FETCH_SIZE
//Extract ResultSet to CSV file, auto-compress if the fileName extension is ".zip" or ".gz"
//Returns number of records extracted
public int ResultSet2CSV(final ResultSet rs, final String fileName, final String header, final boolean aync) throws Exception {
try (CSVWriter writer = new CSVWriter(fileName)) {
//Define fetch size(default as 30000 rows), higher to be faster performance but takes more memory
ResultSetHelperService.RESULT_FETCH_SIZE=10000;
//Define MAX extract rows, -1 means unlimited.
ResultSetHelperService.MAX_FETCH_ROWS=20000;
writer.setAsyncMode(aync);
int result = writer.writeAll(rs, true);
return result - 1;
}
}
But the problem is I don't know how I can merge above into my requirement. As the link has many other classes involved which I am not sure what they do and if I even need it for my requirement. Still, I tried but it fails to compile whenever I enable 2 commented line code. Below is my code.
Any help on how I can achieve this will be greatly appreciated.
package test;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.Date;
import com.opencsv.CSVWriter;
import com.opencsv.ResultSetHelperService;
public class OpenCSVTest1
{
static Connection con =null;
static Statement stmt = null;
static ResultSet rs = null;
public static void main(String args[]) throws Exception
{
connection ();
retrieveData(con);
}
private static void connection() throws Exception
{
try
{
Class.forName("<jdbcdriver>");
con = DriverManager.getConnection("jdbc:","<username>","<pass>");
System.out.println("Connection successful");
}
catch (Exception e)
{
System.out.println("Exception while establishing sql connection");
throw e;
}
}
private static void retrieveData(Connection con) throws Exception
{
try
{
stmt=con.createStatement();
stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
String query = "SELECT * FROM dbo.tablename";
rs=stmt.executeQuery(query);
CSVWriter writer = new CSVWriter(new BufferedWriter(new FileWriter("C:\\Data\\File1.csv")));
ResultSetHelperService service = new ResultSetHelperService();
/*** ResultSetHelperService.RESULT_FETCH_SIZE=10000; ***/ // to add
service.setDateTimeFormat("yyyy-MM-dd HH:mm:ss.SSS");
System.out.println("**** Started writing Data to CSV **** " + new Date());
writer.setResultService(service);
/*** writer.setAsyncMode(aync); ***/ // to add
int lines = writer.writeAll(rs, true, true, false);
writer.flush();
writer.close();
System.out.println("** OpenCSV -Completed writing the resultSet at " + new Date() + " Number of lines written to the file " + lines);
}
catch (Exception e)
{
System.out.println("Exception while retrieving data" );
e.printStackTrace();
throw e;
}
finally
{
rs.close();
stmt.close();
con.close();
}
}
}
UPDATE
I have updated my code. Right now code is writing complete resultset in CSV at once using writeAll method which is resulting in time consumption.
Now what I want to do is write resultset to CSV in batches as resultset's first column will always have dynamically generated via SELECT query Auto Increment column (Sqno) with values as (1,2,3..) So not sure how I can read result sets first column and split it accoridngly to write in CSV. may be HashMap might help, so I have also added resultset-tohashmap conversion code if required.
import com.opencsv.CSVWriter;
import com.opencsv.ResultSetHelperService;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class OpenCSVTest1
{
static int fetchlimit_src = 100;
static Connection con =null;
static Statement stmt = null;
static ResultSet rs = null;
static String filename = "C:\\Data\\filename.csv";
static CSVWriter writer;
public static void main(String args[])
{
try
{
connection();
retrieveData(con);
}
catch(Exception e)
{
System.out.println(e);
}
}
private static void connection() throws Exception
{
try
{
Class.forName("<jdbcdriver>");
con = DriverManager.getConnection("jdbc:","<username>","<pass>");
System.out.println("Connection successful");
}
catch (Exception e)
{
System.out.println("Exception while establishing sql connection");
throw e;
}
}
private static void retrieveData(Connection con) throws Exception
{
try
{
stmt=con.createStatement();
String query = "SELECT ROWNUM AS Sqno, * FROM dbo.tablename "; // Oracle
// String query = "SELECT ROW_NUMBER() OVER(ORDER BY Id ASC) AS Sqno, * FROM dbo.tablename "; // SQLServer
System.out.println(query);
stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(fetchlimit_src);
System.out.println("**** Started querying src **** " + new Date());
rs=stmt.executeQuery(query);
System.out.println("**** Completing querying src **** " + new Date());
// resultset_List(rs); // If required store resultset(rs) to HashMap
writetoCSV(rs,filename);
/** How to write resultset to CSV in batches instead of writing all at once to speed up write performance ?
* Hint: resultset first column is Autoincrement [Sqno] (1,2,3...) which might help to split result in batches.
*
**/
}
catch (Exception e)
{
System.out.println("Exception while retrieving data" );
e.printStackTrace();
throw e;
}
finally
{
rs.close();
stmt.close();
con.close();
}
}
private static List<Map<String, Object>> resultset_List(ResultSet rs) throws SQLException
{
ResultSetMetaData md = rs.getMetaData();
int columns = md.getColumnCount();
List<Map<String, Object>> rows = new ArrayList<Map<String, Object>>();
while (rs.next())
{
Map<String, Object> row = new HashMap<String, Object>(columns);
for(int i = 1; i <= columns; ++i)
{
row.put(md.getColumnName(i), rs.getObject(i));
}
rows.add(row);
}
// System.out.println(rows.toString());
return rows;
}
private static void writetoCSV(ResultSet rs, String filename) throws Exception
{
try
{
writer = new CSVWriter(new BufferedWriter(new FileWriter(filename)));
ResultSetHelperService service = new ResultSetHelperService();
service.setDateTimeFormat("yyyy-MM-dd HH:mm:ss.SSS");
long batchlimit = 1000;
long Sqno = 1;
ResultSetMetaData rsmd = rs.getMetaData();
String columnname = rsmd.getColumnLabel(1); // To retrieve columns with labels (for example SELECT ROWNUM AS Sqno)
System.out.println("**** Started writing Data to CSV **** " + new Date());
writer.setResultService(service);
int lines = writer.writeAll(rs, true, true, false);
System.out.println("** OpenCSV -Completed writing the resultSet at " + new Date() + " Number of lines written to the file " + lines);
}
catch (Exception e)
{
System.out.println("Exception while writing data" );
e.printStackTrace();
throw e;
}
finally
{
writer.flush();
writer.close();
}
}
}
You should be able to use the OpenCSV sample, pretty much exactly as it is provided in the documentation. So, there should be no need for you to write any of your own batching logic.
I was able to write a 6 million record result set to a CSV file in about 10 seconds. To be clear -that was just the file-write time, not the DB data-fetch time - but I think that should be fast enough for your needs.
Here is your code, with adaptations for using OpenCSV based on its documented approach... But please see the warning at the end of my notes!
import com.opencsv.CSVWriter;
import com.opencsv.ResultSetHelperService;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.Date;
import java.text.SimpleDateFormat;
public class OpenCSVDemo {
static int fetchlimit_src = 100;
static Connection con = null;
static Statement stmt = null;
static ResultSet rs = null;
static String filename = "C:\\Data\\filename.csv";
public static void main(String args[]) {
try {
connection();
retrieveData(con);
} catch (Exception e) {
System.out.println(e);
}
}
private static void connection() throws Exception {
try {
final String jdbcDriver = "YOURS GOES HERE";
final String dbUrl = "YOURS GOES HERE";
final String user = "YOURS GOES HERE";
final String pass = "YOURS GOES HERE";
Class.forName(jdbcDriver);
con = DriverManager.getConnection(dbUrl, user, pass);
System.out.println("Connection successful");
} catch (Exception e) {
System.out.println("Exception while establishing sql connection");
throw e;
}
}
private static void retrieveData(Connection con) throws Exception {
try {
stmt = con.createStatement();
String query = "select title_id, primary_title from imdb.title";
System.out.println(query);
stmt = con.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(fetchlimit_src);
System.out.println("**** Started querying src **** " + new Date());
rs = stmt.executeQuery(query);
System.out.println("**** Completing querying src **** " + new Date());
// resultset_List(rs); // If required store resultset(rs) to HashMap
System.out.println();
String timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());
System.out.println("Started writing CSV: " + timeStamp);
writeToCsv(rs, filename, null, Boolean.FALSE);
timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());
System.out.println("Finished writing CSV: " + timeStamp);
System.out.println();
} catch (Exception e) {
System.out.println("Exception while retrieving data");
e.printStackTrace();
throw e;
} finally {
rs.close();
stmt.close();
con.close();
}
}
public static int writeToCsv(final ResultSet rs, final String fileName,
final String header, final boolean aync) throws Exception {
try (CSVWriter writer = new CSVWriter(fileName)) {
//Define fetch size(default as 30000 rows), higher to be faster performance but takes more memory
ResultSetHelperService.RESULT_FETCH_SIZE = 1000;
//Define MAX extract rows, -1 means unlimited.
ResultSetHelperService.MAX_FETCH_ROWS = 2000;
writer.setAsyncMode(aync);
int result = writer.writeAll(rs, true);
return result - 1;
}
}
}
Points to note:
1) I used "async" set to false:
writeToCsv(rs, filename, null, Boolean.FALSE);
You may want to experiment with this and the other settings to see if they make any significant difference for you.
2) Regarding your comment "the link has many other classes involved": The OpenCSV library's entire JAR file needs to be included in your project, as does the related disruptor JAR:
opencsv.jar
disruptor-3.3.6.jar
To get the JAR files, go to the GitHub page, click on the green button, select the zip download, unzip the zip file, and look in the "OpenCSV-master\release" folder.
Add these two JARs to your project in the usual way (depends on how you build your project).
3) WARNING: This code runs OK when you use Oracle's Java 8 JDK/JRE. If you try to use OpenJDK (e.g. for Java 13 or similar) it will not run. This is because of some changes behind the scenes to hidden classes. If you are interested, there are more details here.
If you need to use an OpenJDK version of Java, you may therefore have better luck with the library on which this CSV library is based: see here.
i am trying to connect to teradata and execute a statement and reading the data using stringbuilder. I want the get this as output when i define the outparameter as string the output is getting truncated as it is greater than 50K characters, i want this to be coverted as CLOB and give me the output as CLOB, could some one please help me how i can achieve this.
here is the code i am using ..
import java.sql.*;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.PreparedStatement;
import java.util.*;
import java.sql.*;
public class getDDLinfo {
public static void getDDL (int [] status, String objectType, String objName, String [] ddl)
throws SQLException {
ddl[0] = "Unknown failure in XSP.";
status[0] = 1;
Connection conn = DriverManager.getConnection("jdbc:default:connection");
try {
String sql = "show " + objectType + " " + objName + ";";
PreparedStatement stmt = conn.prepareStatement(sql);
StringBuilder result = new StringBuilder();
ResultSet rs = stmt.executeQuery();
while (rs.next()) {
result.append(rs.getString(1));
result.append("\n");
}
if (result.length() > 0) {
ddl[0] = result.toString();
}
rs.close();
stmt.close();
conn.close();
status[0] = 0;
} catch (SQLException e) {
status[0] = e.getErrorCode();
ddl[0] = e.getMessage();
}
}
public static String toHexString(byte[] ba)
{
StringBuilder str = new StringBuilder();
for(int i = 0; i < ba.length; i++)
str.append(String.format("%x ", ba[i]));
return str.toString();
}
}
I made a program that can parse .csv file to database,
I want to make semicolon as delimiter but I got some trouble here ~
This is my code CSVLoader.java
package id.co.lolo.coreservice;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.util.Date;
import org.apache.commons.lang.StringUtils;
import au.com.bytecode.opencsv.CSVReader;
public class CSVLoader {
private static final String SQL_INSERT = "INSERT INTO ${table}(${keys}) VALUES(${values})";
private static final String TABLE_REGEX = "\\$\\{table\\}";
private static final String KEYS_REGEX = "\\$\\{keys\\}";
private static final String VALUES_REGEX = "\\$\\{values\\}";
private Connection connection;
private char seprator;
/**
* Public constructor to build CSVLoader object with Connection details. The
* connection is closed on success or failure.
*
* #param connection
*/
public CSVLoader(Connection connection) {
this.connection = connection;
// Set default separator
this.seprator = ',';
}
/**
* Parse CSV file using OpenCSV library and load in given database table.
*
* #param csvFile
* Input CSV file
* #param tableName
* Database table name to import data
* #param truncateBeforeLoad
* Truncate the table before inserting new records.
* #throws Exception
*/
#SuppressWarnings("resource")
public void loadCSV(String csvFile, String tableName,
boolean truncateBeforeLoad) throws Exception {
CSVReader csvReader = null;
if (null == this.connection) {
throw new Exception("Not a valid connection.");
}
try {
csvReader = new CSVReader(new FileReader(csvFile), this.seprator);
} catch (Exception e) {
e.printStackTrace();
throw new Exception("Error occured while executing file. "
+ e.getMessage());
}
String[] headerRow = csvReader.readNext();
if (null == headerRow) {
throw new FileNotFoundException(
"No columns defined in given CSV file."
+ "Please check the CSV file format.");
}
String questionmarks = StringUtils.repeat("?,", headerRow.length);
questionmarks = (String) questionmarks.subSequence(0,
questionmarks.length() - 1);
String query = SQL_INSERT.replaceFirst(TABLE_REGEX, tableName);
query = query
.replaceFirst(KEYS_REGEX, StringUtils.join(headerRow, ","));
query = query.replaceFirst(VALUES_REGEX, questionmarks);
System.out.println("Query: " + query);
String[] nextLine;
Connection con = null;
PreparedStatement ps = null;
try {
con = this.connection;
con.setAutoCommit(false);
ps = con.prepareStatement(query);
if (truncateBeforeLoad) {
// delete data from table before loading csv
con.createStatement().execute("DELETE FROM " + tableName);
}
final int batchSize = 1000;
int count = 0;
Date date = null;
while ((nextLine = csvReader.readNext()) != null) {
if (null != nextLine) {
int index = 1;
for (String string : nextLine) {
date = DateUtil.convertToDate(string);
if (null != date) {
ps.setDate(index++,
new java.sql.Date(date.getTime()));
} else {
ps.setString(index++, string);
}
}
ps.addBatch();
}
if (++count % batchSize == 0) {
ps.executeBatch();
}
}
ps.executeBatch(); // insert remaining records
con.commit();
} catch (Exception e) {
con.rollback();
e.printStackTrace();
throw new Exception(
"Error occured while loading data from file to database."
+ e.getMessage());
} finally {
if (null != ps)
ps.close();
if (null != con)
con.close();
csvReader.close();
}
}
public char getSeprator() {
return seprator;
}
public void setSeprator(char seprator) {
this.seprator = seprator;
}
}
this is Main.java
package id.co.lolo.coreservice;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
public class Main {
#SuppressWarnings("unused")
private static String JDBC_CONNECTION_URL =
"jdbc:mysql://localhost:3306/bni";
public static void main(String[] args) {
try {
CSVLoader loader = new CSVLoader(getCon());
loader.setSeprator(';');
loader.loadCSV("C:\\Log\\Logtima.csv", "coreservice", true);
} catch (Exception e) {
e.printStackTrace();
}
}
private static Connection getCon() {
Connection connection = null;
try {
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/bni","root","shikamaru");
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
return connection;
}
}
I got this Error,
Query: INSERT INTO coreservice(MODULE_NAME,SERVICE_NAME,COUNTER,LOG_DATE,UPDATE_DATE) VALUES(?)
java.sql.BatchUpdateException: Column count doesn't match value count at row 1
at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1693)
at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1108)
at id.co.bni.coreservice.CSVLoader.loadCSV(CSVLoader.java:118)
at id.co.bni.coreservice.Main.main(Main.java:20)
java.lang.Exception: Error occured while loading data from file to database.Column count doesn't match value count at row 1
at id.co.bni.coreservice.CSVLoader.loadCSV(CSVLoader.java:123)
at id.co.bni.coreservice.Main.main(Main.java:20)
I'm trying to build a class that retrieves each row from a MySQL table and store it to a List but my code doesn't compile and throws me an error
Error: Main method not found in class
MySQLAccountsDatabankReader,
please define the main method as: public static void main(String[]
args)
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
public class MySQLAccountsDatabankReader {
private static List<String[]> accounts;
public MySQLAccountsDatabankReader(String[] args) {
Connection con = null;
Statement st = null;
ResultSet rs = null;
String url = "jdbc:mysql://localhost:3306/test";
String user = "root";
String password = "mysql";
try {
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery("SELECT * FROM test.accounts;");
if (rs.next()) {
ArrayList<String[]> accounts = new ArrayList<String[]>();
rs = st.executeQuery("SELECT * FROM test.accounts;");
int columnCount = rs.getMetaData().getColumnCount();
while (rs.next()) {
String[] row = new String[columnCount];
for (int i = 0; i < columnCount; i++) {
row[i] = rs.getString(i + 1);
}
accounts.add(row);
}
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQLAccountsDatabankReader.class
.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQLAccountsDatabankReader.class
.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
return accounts;
}
}
Revision 0.2 - I've got the following compilation error - Exception in thread "main" java.lang.Error: Unresolved compilation problems:
accounts cannot be resolved
accounts cannot be resolved to a variable
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
public class MySQLAccountsDatabankReader {
public static void main(String[] args) {
MySQLAccountsDatabankReader reader = new MySQLAccountsDatabankReader();
List<String[]> accounts = reader.getAccounts();
}
public List<String[]> getAccounts() {
Connection con = null;
Statement st = null;
ResultSet rs = null;
String url = "jdbc:mysql://localhost:3306/test";
String user = "root";
String password = "mysql";
try {
con = DriverManager.getConnection(url, user, password);
st = con.createStatement();
rs = st.executeQuery("SELECT * FROM test.accounts;");
if (rs.next()) {
rs = st.executeQuery("SELECT * FROM test.accounts;");
int columnCount = rs.getMetaData().getColumnCount();
while (rs.next()) {
String[] row = new String[columnCount];
for (int i = 0; i < columnCount; i++) {
row[i] = rs.getString(i + 1);
}
accounts.add(row);
}
return accounts;
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQLAccountsDatabankReader.class
.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(MySQLAccountsDatabankReader.class
.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
}
}
Compiler errors are usually a good indicator of what is wrong - you need a main method.
Also, non-declarative statements need to be in a method, constructor or static initializer rather than in the class code block. Place your JDBC code within a method. Here you need a method that will return accounts to match your return statement. Also you need to create an instance of List, e.g. ArrayList to return.
private List<String[]> getAccounts() {
List<String[]> accounts = new ArrayList<String[]>();
...
return accounts;
}
and add a main method entry point:
public static void main(String[] args) {
MySQLAccountsDatabankReader reader = new MySQLAccountsDatabankReader();
List<String[]> accounts = reader.getAccounts();
...
}
Add main method:
public static void main(String [] args) {
new MySQLAccountsDatabankReader(args);
}
I have some problems in managing the storage of data via query (NetBeans-> Java-> Sqlite):
1) I have a folder with some txt file, containing several lines of text (the files do not exceed 2 Kb)
2) The program opens the files in sequence, and stores each word in a table
3)When the program comes to analyze too much data (some time more than 40 files ore more then 82) returns the following error
Exception in thread "main" java.sql.SQLException: unable to open database file
at org.sqlite.DB.throwex (DB.java: 288)
at org.sqlite.DB.executeBatch (DB.java: 236)
at org.sqlite.PrepStmt.executeBatch (PrepStmt.java: 83)
The error is in int [] upCountsb = prepb.executeBatch();
Here the code:
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
public static void main(String[] args) throws ClassNotFoundException, SQLException, IOException, InterruptedException {
Class.forName("org.sqlite.JDBC");
Connection conn = DriverManager.getConnection("jdbc:sqlite:C:/Users/.../test.db");
Statement stmt;
stmt = conn.createStatement();
stmt.executeUpdate("DROP TABLE IF EXISTS words");
stmt.executeUpdate("CREATE TABLE words (words)");
String path_dir ="C:/Users/.../good";
File currentDIR = new File("C:/Users/.../good");
File files[]=currentDIR.listFiles();
String tmp="";
ArrayList app = new ArrayList();
//Search in DIR for Files
for( File f1 : files ){
String nameFile = f1.getName();
FileReader f = null;
BufferedReader fIN = null;
String s;
//Open the file xxx.txt
try{
f = new FileReader(path_dir+"/"+nameFile);
fIN = new BufferedReader(f);
s = fIN.readLine();
while(s != null) {
StringTokenizer st = new StringTokenizer(s);
while(st.hasMoreTokens()) {
String str = st.nextToken().toString().toLowerCase();
Pattern pattern =Pattern.compile("\\W", Pattern.MULTILINE);
String newAll = pattern.matcher(str).replaceAll("").trim();
tmp=newAll;
app.add(tmp); //Add all data in the ArrayList app
} // Close While 'hasMoreTokens'
s = fIN.readLine();
} //Close While on File
} //Close TRAY
catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
f.close(); //Close FileReader
} //Close Scan DIR for FILE
//Add all data in the Tbl od Database
PreparedStatement prep = conn.prepareStatement("insert into words values (?);");
for (int z=0; z<app.size();z++){
prep.setString(1,app.get(z).toString().toLowerCase());
prep.addBatch();
conn.setAutoCommit(false);
prep.executeBatch(); ***//Here I get the error although i use int [] Count =prep.executeBatch();***
conn.setAutoCommit(true);
}
prep.close();
} //Close MAIN
You need to release the resources you are using once you are done with them. E.g. prepb.close() after executing the statement.
The same thing goes for your file handles.
Also, the point of batching is lost if you execute the statement for every insert.
Since your files are very small you might as well prepare all the data in memory before you persist it to a database.
package stackoverflow.wordanalysis;
import java.io.*;
import java.sql.*;
import java.util.*;
import java.util.regex.*;
public class WordFrequencyImporter {
public static void main(String[] args) throws Exception {
List<String> words = readWords("the-directory-from-which-to-read-the-files");
Class.forName("org.sqlite.JDBC");
Connection conn = DriverManager.getConnection("jdbc:sqlite:test.db");
try {
createWordsTable(conn);
persistWords(words, conn);
} finally {
conn.close();
}
}
private static void persistWords(List<String> words, Connection conn)
throws SQLException {
System.out.println("Persisting " + words.size() + " words");
PreparedStatement prep = conn
.prepareStatement("insert into words values (?);");
try {
for (String word : words) {
prep.setString(1, word);
prep.addBatch();
}
conn.setAutoCommit(false);
prep.executeBatch();
conn.setAutoCommit(true);
} finally {
prep.close();
}
}
private static void createWordsTable(Connection conn) throws SQLException {
Statement stmt = conn.createStatement();
try {
stmt.executeUpdate("DROP TABLE IF EXISTS words");
stmt.executeUpdate("CREATE TABLE words (words)");
} finally {
stmt.close();
}
}
private static List<String> readWords(String path_dir) throws IOException {
Pattern pattern = Pattern.compile("\\W", Pattern.MULTILINE);
List<String> words = new ArrayList<String>();
for (File file : new File(path_dir).listFiles()) {
BufferedReader reader = new BufferedReader(new FileReader(file));
System.out.println("Reading " + file);
try {
String s;
while ((s = reader.readLine()) != null) {
StringTokenizer st = new StringTokenizer(s);
while (st.hasMoreTokens()) {
String token = st.nextToken().toString().toLowerCase();
String word = pattern.matcher(token).replaceAll("")
.trim();
words.add(word);
}
}
} finally {
reader.close();
}
}
return words;
}
}