I have different tables in a database(say mysql). i need to extract some columns from different table and write it in a csv file. consider table 1
A 1
B 2
C 3
table 2 table 3
X 7 AB A1
Y 8 BC B2
Z 10 CD C3
U 11 DE D4
V 12
W 13
i want to write 1st column from table 1,2nd col from table 2, and 1st col from table 3 in a csv file such that empty rows are made null.
output:
A,7,AB
B,8.BC
C,10,CD
null,11,DE
null,12,null
null,13,null
i can do the basic reading and writing from mysql to csv, need help in the logic or code to get the above output. "Looking for a generic solution for say 'n' number of columns from 'n' number of tables". above is jus a example.
I do not know how you read your database but assume you do it with JDBC:
ResultSet tableAReusltSet= null;
ResultSet tableBReusltSet= null;
ResultSet tableCReusltSet= null;
List<PseudeContainer> container = new ArrayList<>();
while (tableAReusltSet.next()) {
PseudeConteiner ps = new PseudoContainer();
ps.col1 = tableAReusltSet.getString("ColumnName");
container.add(ps);
}
int i = 0;
while (tableBReusltSet.next()) {
if(container.size() <= i){
container.add(new PseudeContainer());
}
container.get(i).col2 = tableBReusltSet.getString("ColumnName");
}
i = 0;
while (tableBReusltSet.next()) {
if(container.size() <= i){
container.add(new PseudeContainer());
}
container.get(i).col2 = tableBReusltSet.getString("ColumnName");
}
//.. now you have a collection to work with which you can write
public PseudeContainer {
String col1 = null;
String col2 = null;
String col3 = null;
}
Above should work.. still pseudo code...
Since you just want the logic here is some pseudo code that may help you in whatever language you are using. Since I don't know how you are exporting to csv I made it pretty generic.
arr1 = select column1 from table1;
arr2 = select column2 from table2;
arr3 = select column1 from table3;
max = isBigger(arr1.length, arr2.length);
max = isBigger(max, arr3.length);
for(i=0; i<max; i++)
{
if(arr1[i]=="") arr1[i]=null;
if(arr2[i]=="") arr2[i]=null;
if(arr3[i]=="") arr3[i]=null;
print arr1[i] + "," + arr2[i] + "," + arr3[i];
}
Recommend 3rd party libraries to handle CSV files:
Apache Commons CSV
Open CSV
Related
Check my Excel Tables SNAP above pasted
By below CODE of java I have read the data and fetch in a arraylist;
workbook = WorkbookFactory.create(new File(SAMPLE_XLSX_FILE_PATH));
ArrayList<String> comps = new ArrayList<String>();
for (Sheet sheet : workbook) {
System.out.println("=> " + sheet.getSheetName());
}
Sheet sheet = workbook.getSheetAt(0);
DataFormatter dataFormatter = new DataFormatter();
for (Row row : sheet) {
for (Cell cell : row) {
String tempValue = dataFormatter.formatCellValue(cell);
// System.out.println(tempValue);
comps.add(tempValue);
}
}
for (int i = 2; i < comps.size(); i++) {
System.out.println(comps.get(i));
}
Output is like below after reading and storing into arraylist
A
B
1
AA
2
BB
3
CC
Now, In first table in my snap, I want to compare Column A to Second Table Column A and when there is a match then in D column of first table I need to put the data of Column B from Second Table. Simply this vlookup between 2 excels. I want to do it by java.
As per my understanding I need to read the first table as well like I did for second table and store it in a Arraylist then start comparing and write it in Column D of first table and save it.
Anyone can help me with next step? Appreciate the code on this context.
i want to write my data in excel.but it didnot write in formatted way
driver.get("https://careernavigator.naukri.com/sales-executive-retail-careers-in-mahindra-and-mahindra-financial-services-15731");
List<WebElement> row = driver.findElements(By.xpath("(//*[name()='svg'])[2]//*[name()='rect' and #height='40']"));
List<WebElement> column = driver.findElements(By.xpath("(//*[name()='svg'])[2]//*[name()='text']//*[name()='tspan' and (#dy=4 or #dy='3.5')]"));
for (int i=0;i<column.size();i++) {
System.out.println(column.get(i).getText());
XSSFRow row1 = sheet.createRow(i);
for(int j=0;j<4;j++)
{
Cell cell1 = row1.createCell(j);
cell1.setCellValue(column.get(j).getText());
The issue appears to be with your for loop structure. A couple of notes, first, on some observations.
You need a high implicit wait here because the site is slow to load the findings.
Your rows xpath finds nothing, but your column xpath works, even though it did not when I ported to Protactor.
I am sketching an alternative using locator by id "f1" and splitting the resulting text, but it turns out that is equivalent to what your xpath "column" finds.
The key here is to know when a new row begins and when the items are no longer really data you want. Rows start when the index is a multiple of 3, since each row has 3 fields (a name and 2 numbers). Stuff about which we do not care begins with the number 1. I did not write code to put the data in Excel, but I indicated where you would.
Here is my code (yours will vary slightly because of where your chrome driver or whichever is located, as well as because you have the Excel sheet pieces):
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
public class ShonaDriver {
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver", "bin/chromedriver");
WebDriver driver = new ChromeDriver();
driver.manage().timeouts().implicitlyWait(90, TimeUnit.SECONDS);
driver.manage().window().maximize();
driver.get(
"https://careernavigator.naukri.com/sales-executive-retail-careers-in-mahindra-and-mahindra-financial-services-15731");
/// the row xpath that ws once here found nothing
List<WebElement> column = driver.findElements(
By.xpath("(//*[name()='svg'])[2]//*[name()='text']//*[name()='tspan' and (#dy=4 or #dy='3.5')]"));
int spreadSheetRowNum = 0;
int spreadSheetColumnNum = 0;
WebElement f1 = driver.findElement(By.id("f1"));
for (int i = 0; i < column.size(); i++) {
if (column.get(i).getText().equalsIgnoreCase("1")) {
// reached end of meaningful fields
break;
}
if (i % 3 == 0) {// start of new row
spreadSheetRowNum++;
System.out.println("here create row: " + spreadSheetRowNum);
spreadSheetColumnNum = 1;// assuming excel column A is column 1
} else {
spreadSheetColumnNum++;
}
System.out.println("for column list " + i + " text is:");
System.out.println(column.get(i).getText());
System.out.println("write it to row " + spreadSheetRowNum + " column " + spreadSheetColumnNum);
}
String[] f1Arr = f1.getText().split("\n");
System.out.println("if you prefer to use the f1 array, its contents are:");
for (int i = 0; i < f1Arr.length; i++) {
System.out.println("f1[" + i + "] = " + f1Arr[i]);
}
driver.close();
}
}
and here is the output I get indicating what is there and what you need to do at the various steps:
here create row: 1
for column list 0 text is:
Mahindra and Mahindra Financia..
write it to row 1 column 1
for column list 1 text is:
4.8
write it to row 1 column 2
for column list 2 text is:
2.4
write it to row 1 column 3
here create row: 2
for column list 3 text is:
Tata Motors
write it to row 2 column 1
for column list 4 text is:
5.0
write it to row 2 column 2
for column list 5 text is:
2.6
write it to row 2 column 3
here create row: 3
for column list 6 text is:
Mahindra and Mahindra
write it to row 3 column 1
for column list 7 text is:
4.6
write it to row 3 column 2
for column list 8 text is:
2.9
write it to row 3 column 3
if you prefer to use the f1 array, its contents are:
f1[0] = Mahindra and Mahindra Financia..
f1[1] = 4.8
f1[2] = 2.4
f1[3] = Tata Motors
f1[4] = 5.0
f1[5] = 2.6
f1[6] = Mahindra and Mahindra
f1[7] = 4.6
f1[8] = 2.9
f1[9] = 1
f1[10] = 1
f1[11] = 2
f1[12] = 2
f1[13] = 3
f1[14] = 3
f1[15] = 4
f1[16] = 4
f1[17] = 5
f1[18] = 5
f1[19] = 6
f1[20] = 6
f1[21] = 7
f1[22] = 7
f1[23] = 8
f1[24] = 8
f1[25] = Avg.Exp
f1[26] = Avg.Sal
f1[27] = In lacs
f1[28] = Avg.Exp
f1[29] = Avg.Sal
f1[30] = In lacs
f1[31] = Top Companies
f1[32] = Top Companies
f1[33] = Top Companies
f1[34] = Top Companies
f1[35] = 1.6
f1[36] = 4.9
f1[37] = 2.4
f1[38] = 1.6
f1[39] = 7.0
f1[40] = 2.6
f1[41] = 1.3
f1[42] = 7.2
f1[43] = 2.9
f1[44] = View 22 more Companies.
I want to get table label of column .like string display in mysql. Like this
However, when I use getColumnName, It turn out there is some difference between returned string and below string. Like this:
But It is right in Variables explorer in eclipse when I debug, Like this:
I can't find other way to get column. It seem returned string is originalColumnName, but how to get ColumnName? Anyone know how to fix it?
There is my code, I know there is other problem in code. Please just assume the type of all column is String.
public ResultSet DisplayShowTables() throws SQLException
{
ResultSet Res = Sta.executeQuery("DESC Code2Name");
ResultSetMetaData ResMeta = Res.getMetaData();
String [] ColumnName = new String [ResMeta.getColumnCount()];
int MetaCount = ResMeta.getColumnCount();
for (int i = 0; i < MetaCount; i++) {
ColumnName [i] = ResMeta.getColumnName(i+1);
}
String LeftAlignFormat = "|";
String Separator = "+";
for (int i = 0; i < MetaCount; i++) {
LeftAlignFormat = LeftAlignFormat.concat(" %-20s |");
Separator =Separator.concat("----------------------+");
}
LeftAlignFormat = LeftAlignFormat.concat("%n");
Separator = Separator.concat("%n");
if(Res.isBeforeFirst()){
System.out.format(Separator);
System.out.format(LeftAlignFormat, ColumnName);
System.out.format(Separator);
}
while (Res.next()) {
Vector<String> RowData = new Vector<String>();
for (int i = 0; i < MetaCount; i++) {
RowData.add(Res.getString(i+1).toString());
}
System.out.format(LeftAlignFormat, RowData);
}
if(Res.isAfterLast())
System.out.format(Separator);
return Res;
}
It looks like DESC is just a shortcut for a query of the information schema with aliases for the columns.
Column aliases can be retrieved with ResultSetMetadata.getColumnLabel(int).
JDBC defines column label as:
Gets the designated column's suggested title for use in printouts and displays. The suggested title is usually specified by the SQL AS clause. If a SQL AS is not specified, the value returned from getColumnLabel will be the same as the value returned by the getColumnName method.
This also means that in almost all situation you should be using getColumnLabel instead of getColumnName.
I have a column defined in a table in postgres DB as:
counterparty_company_number character varying(255)
I have written a JAVA code to populate this table with values from corresponding columns of an excel file. In the sample excel file, this column has 8 rows,out of which some are blank,like:
counterparty_company_number
blank
234567
345678
456789
567890
1.03E+09
blank
blank
In my model class,I have defined the attribute for this column as:
private String counterparty_company_number;
In my code, I replaced blank values with empty String while saving.After reading the data from excel and saving in DB,the values in corresponding column of DB table appear as:
""
"234567.0"
"345678.0"
"456789.0"
"567890.0"
"1.0304388E9"
""
""
This is the snippet from the code that reads the excel file and populates the table in DB:
..
//reads excel file and stores values in a list
FileInputStream file = new FileInputStream(new File(filepath));
XSSFWorkbook workbook = new XSSFWorkbook(file);
XSSFSheet xssfSheet = workbook.getSheetAt(0);
if (xssfSheet.getLastRowNum() != 0) {
for (int i = 1; i <= xssfSheet.getLastRowNum(); i++) {
Row row = xssfSheet.getRow(i);
for (int j = 0; j < row.getLastCellNum(); j++) {
Cell cell = row.getCell(j);
if (cell != null) {
valueHolder.add(cell.toString());
} else {valueHolder.add("");
}
}
}
}
..
//sets attribute values in model class
if (!valueHolder.isEmpty() && valueHolder != null) {
for (int i = 0; i < valueHolder.size(); i += 10) {
CardsCounterparty ccp = new CardsCounterparty();
..
ccp.setCounterparty_company_number(valueHolder.get(i+3).toString());
..
counterpartyList.add(ccp);//counterpartyList is a list of counterparty objects
}
}
return counterpartyList;//this list is passed to the code that will save the objects in DB table
}
..
//code to persist in DB
Configuration c = new Configuration();
c.configure("/hibernate.cfg.xml");
SessionFactory sf = c.buildSessionFactory();
Session s = sf.openSession();
Transaction tx = s.beginTransaction();
try {
for (int i = 0; i < counterparties.size(); i++) {//counterparties is the list of all counterparty objects to be saved
CardsCounterparty ccp = counterparties.get(i);
s.save(ccp);
}
tx.commit();
s.close();
} catch (Exception e) {
tx.rollback();
}
}
I want to know how can I get the values in proper numeric format like 234567 instead of 234567.0 and 1030438800 instead of 1.0304388E9. I cannot change the datatype to int from String as the values may contain characters also and there may be blank values also.
Examine your SQL to ensure that the value that is getting inserted is surrounded by quotes to prevent any conversion from happening.
"update tablex set counterparty_company_number = '" + value + "'"
I have a sqlite databse, with alot of tables. Each table has many rows. In one column I have something like this:
[58458, 65856, 75658, 98456, 98578, ... N]
I made a mistake when created the databse (I don't have acces to the initial data anymore ), what I need is to make these numbers with a punct after the second digit and have something like this:
[58.458, 65.856, 75.658, 98.456, 98.578, ... N]
Is there any way I can do this? I prefer Java. Or is it already any tools that can do this?
Use this function to parse the information from each column.
public static String convertColumn(String textF)
{
String textAux = "";
String newText = "[";
int i = 0;
textF = textF.substring(1, textF.length() - 1);
while(i < textF.length())
{
textAux = textF.substring(i, i + 5);
int nrAux = Integer.parseInt(textAux);
i+=7;
int a;
int b;
a = nrAux / 1000;
b = nrAux - a * 1000;
double newNr;
newNr = a + b * 0.001;
newText = newText + newNr + ", ";
}
newText = newText.substring(0, newText.length() - 2);
newText += "]";
return newText;
}
The function will have as parameter a string like [58458, 65856, 75658, 98456, 98578], which you will get from
the SQL table, and the return value will be [58.458, 65.856, 75.658, 98.456, 98.578] which is the value that you need to update the column with.
For SQL the base idea is this:
UPDATE table
SET column = convertColumn(column);
You can use CAST as REAL on the column, and then update as advised in the other answer.
select CAST(YOUR_COL AS REAL) from YOUR_TABLE
Search for CAST in this doc for more info on it: SQLite language guide
This should work if it's a NUMERIC column:
UPDATE <TABLE NAME> SET <COLUMN> = <COLUMN>/1000;
If it is NOT a NUMERIC or REAL column then this should work:
UPDATE <TABLE NAME> SET <COLUMN> = CAST(<COLUMN> AS REAL)/1000;
(Thanks to Goibniu for the pointer)