We Keep Coding

Create CSV file with columns and values from HashMap

Be gentle,
This is my first time using Apache Commons CSV 1.7.
I am creating a service to process some CSV inputs,
add some additional information from exterior sources,
then write out this CSV for ingestion into another system.
I store the information that I have gathered into a list of
HashMap<String, String> for each row of the final output csv.
The Hashmap contains the <ColumnName, Value for column>.
I have issues using the CSVPrinter to correctly assign the values of the HashMaps into the rows.
I can concatenate the values into a string with commas between the variables;
however,
this just inserts the whole string into the first column.
I cannot define or hardcode the headers since they are obtained from a config file and may change depending on which project uses the service.
Here is some of my code:
try (BufferedWriter writer = Files.newBufferedWriter(
Paths.get(OUTPUT + "/" + project + "/" + project + ".csv"));)
{
CSVPrinter csvPrinter = new CSVPrinter(writer,
CSVFormat.RFC4180.withFirstRecordAsHeader());
csvPrinter.printRecord(columnList);
for (HashMap<String, String> row : rowCollection)
{
//Need to map __record__ to column -> row.key, value -> row.value for whole map.
csvPrinter.printrecord(__record__);
}
csvPrinter.flush();
}
Thanks for your assistance.

You actually have multiple concerns with your technique;
How do you maintain column order?
How do you print the column names?
How do you print the column values?
Here are my suggestions.
Maintain column order.
Do not use HashMap,
because it is unordered.
Instead,
use LinkedHashMap which has a "predictable iteration order"
(i.e. maintains order).
Print column names.
Every row in your list contains the column names in the form of key values,
but you only print the column names as the first row of output.
The solution is to print the column names before you loop through the rows.
Get them from the first element of the list.
Print column values.
The "billal GHILAS" answer demonstrates a way to print the values of each row.
Here is some code:
try (BufferedWriter writer = Files.newBufferedWriter(
Paths.get(OUTPUT + "/" + project + "/" + project + ".csv"));)
{
CSVPrinter csvPrinter = new CSVPrinter(writer,
CSVFormat.RFC4180.withFirstRecordAsHeader());
// This assumes that the rowCollection will never be empty.
// An anonymous scope block just to limit the scope of the variable names.
{
HashMap<String, String> firstRow = rowCollection.get(0);
int valueIndex = 0;
String[] valueArray = new String[firstRow.size()];
for (String currentValue : firstRow.keySet())
{
valueArray[valueIndex++] = currentValue;
}
csvPrinter.printrecord(valueArray);
}
for (HashMap<String, String> row : rowCollection)
{
int valueIndex = 0;
String[] valueArray = new String[row.size()];
for (String currentValue : row.values())
{
valueArray[valueIndex++] = currentValue;
}
csvPrinter.printrecord(valueArray);
}
csvPrinter.flush();
}

for (HashMap<String,String> row : rowCollection) {
Object[] record = new Object[row.size()];
for (int i = 0; i < columnList.size(); i++) {
record[i] = row.get(columnList.get(i));
}
csvPrinter.printRecord(record);
}

Appending to text file [duplicate]

This question already has answers here:
How to append text to an existing file in Java?
(31 answers)
Closed 6 years ago.
This code will iterate through a number of pages to find and extract elements on the page. Once the loop has completed it will generate a log file with these elements from a HashMap, but the results aren't being appended and are rather being overwritten.
int d = new Integer(0);
for (int i = 0; i <= 100; d += 10) {
String url = Constants.FilterUrl + "&startIndex=" + d;
this.getAuthors();
driver.get(url);
if (!driver.getPageSource().contains("h3")) break;
}
/* Send HashMap values to text file */
File file = new File(Constants.FILEPATH + Constants.dateFormat.format(new Date()) + ".txt");
try{
if(!file.exists()){
System.out.println("We had to make a new file.");
file.createNewFile();
}
PrintWriter out = new PrintWriter(new FileWriter(file), true);
map.forEach((k, v) -> out.println(k + ", " + v));
out.append("************** " + "\n");
out.close();
} catch(IOException e) {
System.out.println("COULD NOT LOG!!");
}
}
public void getAuthors(){
List<WebElement> allElements = driver.findElements(By.tagName("h3"));
/* Create HashMap and store H3 elements in the key set */
this.map = new HashMap<String, String>();
for (WebElement element1 : allElements) {
map.put(element1.getText(), element1.findElement(By.tagName("a")).getAttribute("href"));
}
/* Visit pages for H3 elements and retrieve names of the authors */
for (Map.Entry<String, String> entry : map.entrySet()) {
driver.get(entry.getValue());
entry.setValue(driver.findElement(By.className("userlink-0")).getText());
}
}
Any ideas?

map.put(element1.getText(),
element1.findElement(By.tagName("a")).getAttribute("href"));
If there is any entry in HashMap with the same text as element1.getText(), it will override it.
Also you are creating map for each call, it will create a new Map each time and lead to data loss for earlier content.
/* Create HashMap and store H3 elements in the key set */
this.map = new HashMap<String, String>();
You should create this at instance level.
For generating unique key, define a number variable at instance level and increment that for each put.
long counter = 0;
map.put(counter++, element1.findElement(By.tagName("a")).getAttribute("href"));
May be change the HashMap to take long as Key instead of String.

for (WebElement element1 : allElements) {
i++
map.put(element1.getText()+i, element1.findElement(By.tagName("a")).getAttribute("href"));
}
add i++ so it doesnt override

Import Excel Data to Mongodb using Java

Tried Importing Excel Data to Mongo db in the Following Document Format
[
{"productId":"",
"programeName":"",
"programeThumbImageURL":"",
"programeURL":"",
"programEditors":["editor1","editor2"],
"programChapters":[
{
"chapterName":"chapter1",
"authorNames":["authorName1","authorname2"]
},
{"chapterName":"chapter2"},
"authorNames":["authorName1","authorName2"]
}
,...
]},...]
There are many products in the Excel with with chapterNames has multiple authors. following is the code which tried executing and i could do inserting data. But the i couldn't merge the authorNames corresponding to a particular chapterName as above. So currently there are programChapters array contains objects as duplicated chapterNames. Following code shows my experiment towards this.
private static XSSFWorkbook myWorkBook;
public static void main(String[] args) throws IOException {
String[] programs = {"programName1","programName2","programName3","programName4",...};
#SuppressWarnings("deprecation")
Mongo mongo = new Mongo("localhost", 27017);
#SuppressWarnings("deprecation")
DB db = mongo.getDB("dbName");
DBCollection collection = db.getCollection("programsCollection");
File myFile =
new File("dsm_article_author_details.xlsx");
FileInputStream fis = new FileInputStream(myFile); // Finds the workbook instance for XLSX file
myWorkBook = new XSSFWorkbook(fis);
XSSFSheet mySheet = myWorkBook.getSheetAt(0); // Get iterator to all the rows in current sheet
#SuppressWarnings("unused")
Iterator<Row> rowIterator = mySheet.iterator(); // Traversing over each row of XLSX file
for (String program : programs) {
String programName = "";
String chapterName = "";
String authorName = "";
BasicDBObject product = new BasicDBObject();
BasicDBList programChaptersList = new BasicDBList();
// For Each Row , Create Chapters Object here
for (int i = 0; i <= mySheet.getLastRowNum(); i++) { // points to the starting of excel i.e
// excel first row
Row row = (Row) mySheet.getRow(i); // sheet number
System.out.println("Row is :" + row.getRowNum());
BasicDBObject programChapters = new BasicDBObject();
if (row.getCell(0).getCellType() == Cell.CELL_TYPE_STRING) {
programName = row.getCell(0).getStringCellValue();
System.out.println("programName : " + programName);
}
if (row.getCell(1).getCellType() == Cell.CELL_TYPE_STRING) {
chapterName = row.getCell(1).getStringCellValue().replaceAll("\n", "");
System.out.println("chapterName : " + chapterName);
}
if (row.getCell(2).getCellType() == Cell.CELL_TYPE_STRING) {
authorName = row.getCell(2).getStringCellValue();
System.out.println("authorName : " + authorName);
}
List<String> authors = new ArrayList<String>();
programChapters.put("chapterName", chapterName);
authors.add(authorName);
programChapters.put("authorName", authors);
if (programName.trim().equals(program.trim())) {
programChaptersList.add(programChapters);
}
}
product.put("programName", program);
product.put("programThumbImageURL", "");
product.put("programeURL", "");
product.put("programChapters", programChaptersList);
collection.insert(product);
System.out.println("*#*#*#*#*#");
}
}
I hope this is the part went wrong. Need to store all chapterNames in an array and compare with each upcoming value and according to that create new objects and store it in a list
List<String> authors = new ArrayList<String>();
programChapters.put("chapterName", chapterName);
authors.add(authorName);
programChapters.put("authorName", authors);
Can someone suggest me, available solutions :-)

I hope this is the part went wrong. Need to store all chapterNames in an array and compare with each upcoming value and according to that create new objects and store it in a list
List<String> authors = new ArrayList<String>();
programChapters.put("chapterName", chapterName);
authors.add(authorName);
programChapters.put("authorName", authors);

writing json file using arrays with large data using java

I am trying to form a json file to source an autocomplete controlled textbox.
The file will have millions of elements so I am trying to eliminate duplicates while saving on memory and time. For small amount the following code works yet since I am using an array, the execution gets really slow as the array gets larger.
int i = 0;
JSONObject obj = new JSONObject();
JSONArray array = new JSONArray();
while (iter.hasNext()) {
Map<String,String>forJson = new HashMap<String, String>();
Statement stmt = iter.nextStatement();
object = stmt.getObject();
forJson.put("key", object.asResource().getLocalName());
forJson.put("value", object.asResource().getURI());
i++;
System.out.println(i);
if(!array.contains(forJson))
{
array.add(forJson);
}
}
obj.put("objects", array);
FileWriter file = new FileWriter("/homeDir/data.json");
file.write(obj.toJSONString());
file.flush();
file.close();
The array.contains control eliminates duplicates but it has a considerable negative effect on execution time.
The json file should have tokens like
[{"key": "exampleText1", "value": "exampleValue1"},
{"key": "exampleText2", "value": "exampleValue2"}]

Use a HashSet to contain the keys you have already added:
...
Set<String> usedKeys = new HashSet<String>();
while (iter.hasNext()) {
Map<String,String>forJson = new HashMap<String, String>();
Statement stmt = iter.nextStatement();
object = stmt.getObject();
String key = object.asResource().getLocalName();
if(!usedKeys.contains(key)) {
usedKeys.add(key);
forJson.put("key", key);
forJson.put("value", object.asResource().getURI());
array.add(forJson);
}
i++;
System.out.println(i);
}
If you need to uniqueness check to include the value, you could append the two using a character separator that you know cannot exist in the keys. For example:
String key = object.asResource().getLocalName();
String value = object.asResource().getURI();
String unique = key + "|#|#|" + value;
if(!usedKeys.contains(unique)) {
usedKeys.add(unique);
forJson.put("key", key);
forJson.put("value", value);
array.add(forJson);
}

how to get highest version of record from a list

I have nearly 500 file. each file has nearly 5 version or 10 version with the same name. each file distinguish start with file name followed by semicolon then version number.
example:
Test.pdf;1
Test.pdf;2
Test.pdf;3
so i am getting list of file by doing this:
File file = new File(Path + "\\");
File[] fileList = file.listFiles();
List<String> namesList = new ArrayList<String>();
for (int i = 0; i < fileList.length; i++) {
String name=fileList[i].getName();
if(name.contains(".pdf")||name.contains(".excel")){
namesList.add(name);
}
for (int i = 0; i < namesList.size(); i++) {
String name = namesList.get(i);
}
How to get only highest version file i.e test.pdf;3

Use a Map to map each file name to a version. Iterate over your file names extract the name part and the version part and then compare the version with a previously saved one in the map. If there is no version for a name in the map then put the actual version in the Map as the highest.
You could try something like this:
#Test
public void listHighestVersion() {
String[] fileNames = {
"Test.pdf;1", "Test.pdf;2", "Test.pdf;3",
"Test.excel;1", "Test.excel;4", "Test.excel;3",
};
Map<String, Integer> fileVersionMap = new HashMap<>();
for(String fileNameWithVersion : fileNames) {
String[] split = fileNameWithVersion.split(";", 2);
if(split.length > 1) {
String fileName = split[0];
Integer fileVersion;
try {
fileVersion = Integer.parseInt(split[1]);
} catch(NumberFormatException e) {
fileVersion = 0;
}
Integer version = fileVersionMap.get(fileName);
if(version == null || fileVersion > version) {
fileVersionMap.put(fileName, fileVersion);
}
}
}
System.out.println(fileVersionMap);
}
output is:
{Test.excel=4, Test.pdf=3}

You can use the code shown below:
String[] files = {"test.pdf;3","test.pdf;1","differentTest.pdf;2","test.pdf;5","test.pdf;4", "test.excel;3","test.excel;1","test.excel;6","test.excel;5","test.excel;4"};
HashMap <String, Integer> newestFiles = new HashMap<String, Integer>();
for (String currentString : files) {
String currentFileName = currentString.split(";")[0];
Integer currentFileVersion = Integer.parseInt(currentString.split(";")[1]);
if (newestFiles.containsKey(currentFileName)) {
if (newestFiles.get(currentFileName) < currentFileVersion) {
newestFiles.put(currentFileName, currentFileVersion);
}
}
else {
newestFiles.put(currentFileName, currentFileVersion);
}
}
for (Entry<String, Integer> maxValue : newestFiles.entrySet()) {
String result = new StringBuilder("Max version of file: ").append(maxValue.getKey()).append(" is: ").append( maxValue.getValue()).toString();
System.out.println(result);
}
Result:
Max version of file: test.excel is: 6
Max version of file: test.pdf is: 5
Max version of file: differentTest.pdf is: 2
What is done here:
For every file get its name and version - using splitting.
Then check if you already processed files with that name (all processed are stored in newestFiles Map).
If it was not processed add it to the Map with current file
version (in else statement).
If it was processed check if
current version is bigger. If it is, substitute the file in the map.

Create a class say FileDetail with 2 attributs i.e. String fileName and integer version as attributes.
create List<FileDetail> as you are creating namesList
Implement comparable or comparator for class FileDetail and implement your sorting logic.