Connection conn = SqlConnection.getConnection("jdbc:mysql://localhost:3306/stocks");
Statement statement = conn.createStatement();
File path = new File("/Users/Zack/Desktop/JavaDB/BALANCESHEETS");
for(File file: path.listFiles()) {
if (file.isFile()) {
String fileName = file.getName();
String ticker = fileName.split("\\_")[0];
if (ticker.equals("ASB") || ticker.equals("FRC")) {
if (ticker.equals("ASB")) {
ticker = ticker + "PRD";
}
if (ticker.equals("FRC")) {
ticker = ticker + "PRD";
}
}
//CSVReader reader = new CSVReader(new FileReader(file));
//List entries = reader.readAll();
//ArrayList<String> entry = new ArrayList<String>();
Reader reader = new BufferedReader(new FileReader(file));
StringBuilder builder = new StringBuilder();
int c;
while ((c = reader.read()) != -1) {
builder.append((char) c);
}
String string = builder.toString();
ArrayList<String> stringResult = new ArrayList<String>();
if (string != null) {
String[] splitData = string.split("\\s*,\\s*|\\n");
for (int i = 0; i <splitData.length; i++) {
if (!(splitData[i] == null) || !(splitData[i].length() ==0)) {
stringResult.add(splitData[i].trim());
}
}
}
String columnName = null;
int yearCount = 0;
for (int i = 0; i < stringResult.size(); i++) {
int sL = stringResult.get(i).length();
for (int x = 0; x < sL; x++) {
if (Character.isLetter(stringResult.get(i).charAt(x))) {
yearCount = 0;
System.out.println("index: " + i);
columnName = stringResult.get(i);
columnName = columnName.replace(" ", "_");
System.out.println(columnName);
i++;
break;
}
}
yearCount++;
String value = stringResult.get(i);
System.out.println("Year: " + stringResult.get(yearCount) + " Value: " + value + " Stock: " + ticker + " Column: " + columnName );
if (!(columnName == null)) {
String writeValues = "INSERT INTO BalanceSheet (ticker, Year, " + columnName + ") "
+ "VALUE ('" + ticker + "','" + stringResult.get(yearCount) + "','" + value + "')";
String writeValues2 = "UPDATE BalanceSheet "
+ "SET ticker = '" + ticker + "', "
+ "Year = '" + stringResult.get(yearCount) + "', "
+ columnName + " = '" + value + "' "
+ "WHERE ticker = '" + ticker + "'";
statement.executeUpdate(writeValues2);
}
}
Towards the bottom of the code are two queries I tried, I'm trying to get all data organized by ticker and year into a table, "writeColumns" works but it's making a new row for every new "value" put into "columnName". My second attempt "writeColumns2" doesn't do anything.
I want to update the same row with a certain year for all values and then move onto the next year, then next ticker.
If I have understood your question correctly, you want to insert a row if it doesn't exists but update the values if it already does. You need to use ON DUPLICATE KEY UPDATE
String writeValues = "INSERT INTO BalanceSheet (ticker, Year, " + columnName + ") "
+ "VALUES (?,?,?) "
+"ON DUPLICATE KEY UPDATE " + columnName +"=?";
Statement statement = conn.prepareStatement(writeValues);
statement.setString(1,ticker);
statement.setString(2,stringResult.get(yearCount));
statement.setString(3, value);
This will solve your immidiate problem provided you create a UNIQUE index on ticker,year
However there are lot's of other issues here.
An update for each column - Currently you are doing an insert/update for each column on the table. What you are supposed to do is to insert update all the columns at one.
You are not using prepared statements addressed in my code sample
You shouldn't be doing this at all the best way to batch process data is to use MYSQL's built in LOAD DATA INFILE command. If your data is not in a format that can be easily imported into mysql, what your Java code can do is to transform it into a format that can be. Such a code will be a lot simpler and neater than what you have now
Related
I am trying to run the below query and insert the output of it into another table (postgres db):
String ssnQuery = "SELECT period_year, period_name, period_num, NULL as count_of_issues,
ledger_id,
balancing_segment,
Count(*) AS count_of_account_segments,
Sum(accounted_period_net_dr) AS balance_accounted_period_net_dr,
Sum(accounted_period_net_cr) AS balance_accounted_period_net_cr,
Round(Sum(accounted_period_net_dr_cr)) AS balance_accounted_period_net_dr_cr_diff,
Sum(entered_period_net_dr) AS balance_entered_period_net_dr,
Sum(entered_period_net_cr) AS balance_entered_period_net_cr,
Round(Sum(entered_period_net_dr_cr)) AS balance_entered_period_net_dr_cr_diff,
Sum(begin_balance_dr) AS begin_balance_dr,
Sum(begin_balance_cr) AS begin_balance_cr,
Round(Sum(net_beginning_balance)) AS net_beginning_balance,
Round(Sum(net_closing_balance)) AS net_closing_balance
FROM schema.tablename";
try {
pstmnt = financeConnection.prepareStatement(ssnQuery);
rs = pstmnt.executeQuery();
rsmd = rs.getMetaData();
for(int i=1; i<=rsmd.getColumnCount(); i++) {
columnNames.add(rsmd.getColumnName(i));
if(i == 1) {
queryColumns = rsmd.getColumnName(i);
} else if(i<7) {
queryColumns += ", " + rsmd.getColumnName(i);
} else {
queryColumns += ", value_" + (i-7);
}
}
while (rs.next()) {
queryValues = " ";
for(String colname: columnNames) {
if(queryValues.isEmpty()) {
queryValues = rs.getString(colname);
} else {
queryValues += rs.getString(colname) + ", ";
}
}
remCommas = queryValues.replaceAll(", $", "");
insertQuery = "INSERT INTO bdmerge.gen_audit_func_hive_results (run_id, run_date, run_date_ist" + queryColumns + ") VALUES (" + runid +"," + utcTimeStamp + "," + istTimeStamp + "," + remCommas + ")";
System.out.println("Final Insert query: " + insertQuery);
}
} catch (SQLException e) {
e.printStackTrace();
} catch(Exception e) {
e.printStackTrace();
}
To insert the output of above query, I formed the insert query according the column names in the destination table as below:
INSERT INTO schema.destinationTable (run_id, run_date, run_date_ist, source_system_type, source_system, module, source_table_name, period_year, period_name, period_num, count_of_issues, count_of_accounted_issues, count_of_entered_issues, value_0, value_1, value_2, value_3, value_4, value_5, value_6, value_7, value_8, value_9, value_10, value_11, value_12) VALUES (781,2018-11-12 08:15:32.0,2018-11-12 13:45:32.0, 2018, OCT-18, 10, null, 1, 1, 2073, ATRS, 6135, 6.2778220466107E11, 6.277946274560101E11, -1.2422795E7, 5.929031383587201E11, 5.9291556115366E11, -1.2422795E7, 3.931397937759116E13, 3.9313979377591164E13, 0.0)
But the destination table's columns:
run_id, count_of_issues, count_of_accounted_issues, count_of_entered_issues
are in numeric format (working on postgres db) and all others are varchar(1000).
In order to insert the varchar data, I need to enclose the column values from value_0 till value_12 in double quotes.
Without properly modifying the data, I am getting SQLException while inserting which is expected.
Is there any way I can just enclose those varchar column values from the resultSet in double quotes and insert them into destination table?
You need to add double quotes to the string values; Use Escape characters, like this:
String quotedStr = "Non Quoted," + " \"Quoted\".";
Modifying your code to address the problem:
while (rs.next()) {
queryValues = " ";
int i = 0;
for(String colname: columnNames) {
if(queryValues.isEmpty()) {
queryValues = rs.getString(colname);
} else {
if (i >= 7)
queryValues += "\"" + rs.getString(colname) +"\"" + ", ";
}
else
queryValues += rs.getString(colname) + ", ";
i++;
}
This method is working perfectly as my requirement.But I want to query with DB only once.
I have 2 conditions to check with DB for querying and displaying List data in UI.Each time I am accessing DB and checking list size and performing action.
For one data list with same table I am querying 3 times.Which is not efficient way I think. And wrote sub queries for it.But could not succeed with it.
So any efficient way for writing this and optimizing my method..?
query1: "FROM Produce where produceId not in (:produceIdList) and itemName in (:itemNameList)" + " and farmerUuid not in(" + merchantUuid + ") and lastDateForBid>=CURDATE() order by lastDateForBid Asc";
Condition: If(result>=10) return data
else check for query2
query2: "FROM Produce where produceId not in (:produceIdList) and farmerUuid not in(" + merchantUuid + ")" + "and itemName in (:itemNameList) and categoryId in (:categoryList) and lastDateForBid>=CURDATE() order by lastDateForBid Asc";
Condition: If(result>=10) return data
else check for query3
query3:
"FROM Produce where produceId not in (:produceIdList) and farmerUuid not in(" + merchantUuid + ") and lastDateForBid>=CURDATE() order by lastDateForBid Asc ";
This is my method
mId = merchantUuid;
Long produceId = null;
String itemName = null;
Long categoryId = null;
List<Long> categoryList = new ArrayList<Long>();
List<Long> produceIdList = new ArrayList<Long>();
List<Produce> produceList = new ArrayList<Produce>();
List<String> itemNameList = new ArrayList<String>();
List<Bidding> biddingList = getBiddingForMerchant();
int count = biddingList.size();
if (count > 0) {
for (int i = 0; i < count; i++) {
produceId = biddingList.get(i).getProduce().getProduceId();
if(!produceIdList.contains(produceId)){
produceIdList.add(produceId);
}
itemName = biddingList.get(i).getProduce().getItemName();
if (!itemNameList.contains(itemName)) {
itemNameList.add(itemName);
}
categoryId = biddingList.get(i).getProduce().getCategory().getCategoryId();
if (!categoryList.contains(categoryId)) {
categoryList.add(categoryId);
}
}
String ProduceQuery1 = "FROM Produce where produceId not in (:produceIdList)"
+ " and itemName in (:itemNameList)" + " and farmerUuid not in(" + merchantUuid + ")"
+ " and lastDateForBid>=CURDATE() order by lastDateForBid Asc";
Query q1 = sessionFactory.getCurrentSession().createQuery(ProduceQuery1);
q1.setParameterList("itemNameList", itemNameList);
q1.setParameterList("produceIdList", produceIdList);
//q1.setMaxResults(10);
#SuppressWarnings("unchecked")
List<Produce> produceList1 = q1.list();
produceList.addAll(produceList1);
if (produceList.size() ==10) {
return produceList;
}
else if (produceList.size() < 10) {
String produceQuery2 = "FROM Produce where produceId not in (:produceIdList)"
+ " and farmerUuid not in(" + merchantUuid + ")" + "and itemName in (:itemNameList)"
+ " and categoryId in (:categoryList) "
+ " and lastDateForBid>=CURDATE() order by lastDateForBid Asc";
Query q2 = sessionFactory.getCurrentSession().createQuery(produceQuery2);
System.out.println("produceQuery::" + produceQuery2);
q2.setParameterList("produceIdList", produceIdList);
q2.setParameterList("itemNameList", itemNameList);
q2.setParameterList("categoryList", categoryList);
//q2.setMaxResults(10);
#SuppressWarnings("unchecked")
List<Produce> produceList2 = q2.list();
produceList.clear();
produceList.addAll(produceList2);
if (produceList.size() > 9) {
List<Produce> produceListNew = produceList.subList(0,10);
return produceListNew;
}
else {
String produceQuery3 = "FROM Produce where produceId not in (:produceIdList)"
+ " and farmerUuid not in(" + merchantUuid + ")"
+ " and lastDateForBid>=CURDATE() order by lastDateForBid Asc ";
Query q3 = sessionFactory.getCurrentSession().createQuery(produceQuery3);
q3.setParameterList("produceIdList", produceIdList);
//q3.setMaxResults(10);
#SuppressWarnings("unchecked")
List<Produce> produceList3 = q3.list();
produceList.clear();
produceList.addAll(produceList3);
if(produceList.size()>10){
produceList = produceList.subList(0,10);
}
return produceList;
}
}
}
Thank you..!!
am trying to edit and update jtable cells as in my code below. my problem is that a single row when updated all the other rows get the same values. i mean only one row is updated and all other a duplicated. can any one help with a good approach. thanks
int count = Table_purchase.getRowCount();
int col = Table_purchase.getColumnCount();
String pod_id[] = new String[count];
String po_id[] = new String[count];
String order_qty[] = new String[count];
String item_id[] = new String[count];
String unit_price[] = new String[count];
String recived_qty[] = new String[count];
String rejected_qty[] = new String[count];
for (int i = 0; i < count; i++) {
po_id[i] = Table_purchase.getValueAt(i,0).toString();
pod_id[i] = Table_purchase.getValueAt(i,1).toString();
order_qty[i] = Table_purchase.getValueAt(i,2).toString();
item_id[i] = Table_purchase.getValueAt(i,3).toString();
unit_price[i] = Table_purchase.getValueAt(i,4).toString();
recived_qty[i] = Table_purchase.getValueAt(i, 5).toString();
rejected_qty[i] = Table_purchase.getValueAt(i,6).toString();
try {
String sql = "update purchase.purchase_detail set pod_id='" + pod_id[i] + "',order_qty='" + order_qty[i] + "',item_id='" + item_id[i] + "', unit_price='" + unit_price[i] + "', recived_qty='" + recived_qty[i] + "',rejected_qty='" + rejected_qty[i] + "'where po_id= '" + po_id[i] + "'";
pst = conn.prepareStatement(sql);
pst.execute();
JOptionPane.showMessageDialog(null, "updated");
} catch (Exception e) {
JOptionPane.showMessageDialog(null, e);
}
}
Your sql statement does not contain a where-clause, hence all rows in the database table are updated for each iteration of the swing-table-rows, and in the end, all the database-rows will have the values from the last swing-table-row.
(And, use pst.setParameter (http://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html) and do not mix sql into gui-code.)
I'm trying to make my own ORM framework. I get the values from object into arrayList and I'm trying to save them in one time. I have to make for loop to save them all, but I'm confused how to make it?
prepareteState = connect.prepareStatement(Query);
for (int y = 1; y <= obs.size() ; y++) {
for(Object obj : obs){
prepareteState.setObject(y, obj);
System.out.println(Query);
System.out.println(prepareteState.toString());
}
}
prepareteState.execute();
thanks for good advices but, i found the solution :) it is a little bit different than the first idea but, works fine for me:) instead of using prepareState and setObject one by one i`m using StringBuilder to make String and execute query
private String makeInsertQuery(List<String> listOfColumnsNames, List<Object> listOfParameters, String tableName){
StringBuilder columns = new StringBuilder();
StringBuilder parameters = new StringBuilder();
String query = null;
if(listOfColumnsNames != null && listOfColumnsNames.size() != 0 && listOfParameters != null && listOfParameters.size() != 0 && tableName != null){
for(String string : listOfColumnsNames){
columns.append(string + ",");
}
columns.deleteCharAt(columns.length() - 1);
for(Object object : listOfParameters){
parameters.append ("'" + object + "'" + ",");
}
parameters.deleteCharAt(parameters.length() - 1);
query = "INSERT " + "INTO " + tableName + " ( " + columns.toString() + " ) " + " VALUES " + "( " + parameters.toString() + " )" ;
}
//TODO if you need check for null
return query;
}
I've come across a weird situation. The code is as below:
public static int add(String trcd, String tlcd, String dept, String doDate,
String doTime, String andConfirm, Teller admin) throws
Exception {
try {
String table1 = "table1";
String table2 = "table2";
String trap = null;
String trtype = null;
String sql = "select * from " + table2;
DataSet dataset = DBOper.DBQuery("taUtil", sql);
if (dataset.isEmpty()) {
return -1;
}
else {
HashMap map = dataset.getRow(0);
trap = (String) map.get("aut_ap_code");
trtype = (String) map.get("aut_type_code");
//point 1
sql = "insert into " + table1 + " values("+trtype + "','" + doDate + "','" + doTime + "','N','Y')";
DBOper.DBUpdate("taUtil", sql);
if (andConfirm.equals("Y")) {
//point 2
sql = "select * " + table1 +" where tr_create_date='" + doDate + "' and tr_create_time='" + doTime + "' and tr_stcd='Y'";
//point 3
DataSet dataset2 = DBOper.DBQuery("taUtil", sql);
if (dataset2.isEmpty()) {
return -2;
}
else {
String trNo = null;
HashMap map2 = dataset2.getRow(0);
trNo = (String) map2.get("tr_no");
confirm(admin, trNo, "N");
}
}
return 0;
}
}
catch (Exception e) {
throw e;
}
}
The problem is:
at point 3, it
always prints "insert" ie the previous sql value instead of the latest assignment of "select".
Does anybody knows why is it so ?
Thanks
You have a syntax error in your assignment statement:
sql = "insert into " + table1 + " values(trtype + "','" + doDate + "','" + doTime + "','N','Y')";
Try to replace it with:
sql = "insert into " + table1 + " values(" +trtype + "',' " + doDate + "','" + doTime + "','N','Y')";
I'm not sure how you even managed to compile this...
EDIT: If this syntax error does stop the code from compiling and your IDE (assuming you are using one) executes older version of the class that could not be compiled (has happened to me using Eclipse on occasions), I think it could end up doing something completely unpredictable and possibly explain this odd behavior.