We Keep Coding

Use Scanner on csv, iterate every time method is called

I am trying to parse a text document that has release notes and grab specific ones. To do that I have a csv with the desired release note keys. I want to scan the csv and use each key to find the matching section of the release note, and print the description that follows.
I would like to use the Scanner class for this to practice with it.
The csv looks like:
foobar-123,foobar-127,foobar-129
The release note text doc looks like:
foobar-123: ewkjhlq kghlhrekgh
foobar-124: lkjhfgrelgkj nberg
foobar-127: ljdfgl kjwneglkjn fdg
foobar-129: lguwlrkguj gwrlekgj werlktj
The issue I am running into is iterating through the csv. I seem to keep grabbing the first string in the csv. I am trying to figure out how to save my place in the csv, so every time the method is called, it goes to the next string.
I'm thinking I could create a variable that saves the last found string, and then use scanner to find that and then grab the next string. But that would require scanning through the csv each time I want to progress and which does not seem efficient. What would be the best way to iterate through the csv using the Scanner class?
Here is the code I have so far:
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.util.Scanner;
public class ReleaseNotesScan {
public static void main(String[] args) {
//Open csv file with issue keys
Scanner getIssueKeys = null;
try {
getIssueKeys = new Scanner(new FileReader("resources/Issues.txt"));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
//Open release notes
Scanner releaseNotes = null;
try {
releaseNotes = new Scanner(new FileReader("resources/Release notes text.txt"));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
//Get issue key from csv
String issueKey = Finders.issueKey(getIssueKeys);
//The below three lines are just for testing if I am iterating through the csv
System.out.println(issueKey);
Finders.issueKey(getIssueKeys);
System.out.println(issueKey);
//Get issue key and description
String description = Finders.sectionContent(releaseNotes, issueKey);
System.out.println(issueKey + ": " + description);
//Close csv
getIssueKeys.close();
//Close release notes
releaseNotes.close();
}
}
My Finders class:
import java.util.Scanner;
public class Finders {
//parse csv
public static String issueKey(Scanner findIssues) {
findIssues.useDelimiter(",");
String issue = findIssues.next();
return issue;
}
public static String sectionContent(Scanner releaseNotes, String heading) {
while (releaseNotes.hasNextLine()){
String found = releaseNotes.findInLine(heading);
if (found != null){
releaseNotes.findInLine(": ");
String grabIt = releaseNotes.nextLine();
return grabIt;
}
releaseNotes.nextLine();
}
releaseNotes.close();
return "Not found";
}
}

Here is some example code to demonstrate how the application can be structured. I made some assumptions that the input file "issues" as a string (instead of a file, for brevity). The issues are stored in an array and release notes in HashMap collection. The release notes are read from the file, tokenized (split with ":" as delimiter) as the issue and its release-note text. The issue is the key and the release-note is the value in the map.
Finally, iterate each issue and get the corresponding release-note from the map.
Example Code:
import java.util.*;
import java.io.*;
public class MatchIssues {
private static String [] issues;
private static Map<String,String> releseNotes = new HashMap<>();
public static void main(String [] args)
throws IOException {
getIssues();
getReleaseNotes();
for(String issue : issues) {
// Match release notes for the issue
System.out.println(releseNotes.get(issue));
}
}
private static void getIssues() {
String s = "foobar-123,foobar-127,foobar-129"; // use string for demo
issues = s.split(",");
System.out.println(Arrays.toString(issues));
}
private static void getReleaseNotes()
throws IOException {
BufferedReader reader =
new BufferedReader(new FileReader("release_notes.txt"));
String line = null;
while ((line = reader.readLine()) != null) {
String [] tokens = line.split(":");
releseNotes.put(tokens[0].trim(), tokens[1].trim());
}
System.out.println(releseNotes);
}
}
release_notes.txt:
foobar-123: aa ewk jhlq kghlhrekgh aa
foobar-124: bb lkjh fgrelgkj nberg bb
foobar-127: yy ljdfgl kjw neglkjn fdg yy
foobar-129: zz lgu wlrkguj gw rlekgj werlktj zz

How to get data from csv file based on first column name using java

In attached csv image I am trying to fetch the data and store it hashmap based on testcase name which is in first column. And in second column I have given rowcount value which says number of test data available for that particular testcase, so that I can get the value of rowcount and loop it for that many times to get all those data available for that particular testcase. I have written the code to get the value of rowcount. But I do no how to fetch the data for that particular test and store it in hashmap.
Click here to view CSV File Image
String path = "./TestData.csv";
String TestCaseName="Login";
String CurrentLine;
try (BufferedReader br = new BufferedReader(new FileReader(path))) {
while ((CurrentLine = br.readLine()) != null) {
String Data[] = CurrentLine.split(",");
if(TestCaseName.equalsIgnoreCase(Data[0])){
System.out.println("Details in this row :"+ " " +CurrentLine);
String rowcount_value = Data[1];
System.out.println(rowcount_value);
}
}
}
catch (IOException e) {
e.printStackTrace();
}
In the attached csv image I wanted to fetch data for Testcase Login, So I gave Testcase name as Login in the above code. And now I want to fetch all the highlighted data in the image which belongs to Login testcase and store it in a hashmap. The above code just gives the rowcount value of the passed testcase name.
Please help me to fetch those highlighted data and store it in hashmap. I am new to this field and Thanks in advance.

As your csv file is having fixed number of data points for different test cases, you may need to update the below code in case of adding more test cases.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public class ReadCsv {
public static void main(String[] args) {
String path = "sample.csv";
// String TestCaseName="Login";
String CurrentLine;
//Map to store TestCase -> Map(Key ->value)
Map<String,Map<String,String>> testcaseMap = new HashMap<String,Map<String,String>>();
try (BufferedReader br = new BufferedReader(new FileReader(path))) {
br.readLine();
while ((CurrentLine = br.readLine()) != null) {
String Data[] = CurrentLine.trim().split(",");
String testCase_DataNames = "";
for(int j=2;j<Data.length;j++){
testCase_DataNames = testCase_DataNames +"_"+Data[j];
}
int dataCount = Integer.parseInt(Data[1]);
Map<String,String> testdata = new HashMap<String,String>();
for(int i=0;i<dataCount;i++){
String nextLine = br.readLine();
String feilds[] = nextLine.trim().split(",");
testdata.put(feilds[2], feilds.length ==4 ? feilds[3]:null);
}
testcaseMap.put(Data[0]+testCase_DataNames, testdata);
/*if(TestCaseName.equalsIgnoreCase(Data[0])){
System.out.println("Details in this row :"+ " " +CurrentLine);
String rowcount_value = Data[1];
System.out.println(rowcount_value);
} */
}
for(Map.Entry<String,Map<String,String>> entry:testcaseMap.entrySet()){
String key[] = entry.getKey().split("_");
System.out.println("Test case for:"+key[0]);
for(Map.Entry<String, String> dataPair: entry.getValue().entrySet()){
System.out.println("Data header:"+key[1]+" with value:"+dataPair.getKey());
if(dataPair.getValue()!= null ) {
System.out.println("Data header:"+key[2]+" with value:"+dataPair.getValue());
}
}
}
}
catch (IOException e) {
e.printStackTrace();
}
}
}
Output:
Test case for:Login
Data header:UserName with value:uname2
Data header:Password with value:pwd2
Data header:UserName with value:uname1
Data header:Password with value:pwd1
Test case for:Hotel
Data header:HotelNames with value:Ambur
Data header:HotelNames with value:BlackPerl
Data header:HotelNames with value:Zingro
Hope this will work for you ! Good luck

Convert CSV values to a HashMap key value pairs in JAVA

HI I have a csv called test.csv . I am trying to read the csv line by line and convert the values into a hash key value pairs .
Here is the code :-
public class Example {
public static void main(String[] args) throws ParseException, IOException {
// TODO Auto-generated method stub
BufferedReader br = new BufferedReader(new FileReader("test.csv"));
String line = null;
HashMap<String,String> map = new HashMap<String, String>();
while((line=br.readLine())!=null){
String str[] = line.split(",");
for(int i=0;i<str.length;i++){
String arr[] = str[i].split(":");
map.put(arr[0], arr[1]);
}
}
System.out.println(map);
}
}
The csv file is as follows :-
1,"testCaseName":"ACLTest","group":"All_Int","projectType":"GEN","vtName":"NEW_VT","status":"ACTIVE","canOrder":"Yes","expectedResult":"duplicateacltrue"
2,"testCaseName":"DCLAddTest","group":"India_Int","projectType":"GEN_NEW","vtName":"OLD_VT","status":"ACTIVE","canOrder":"Yes","expectedResult":"invalidfeaturesacltrue"
When I run this code I get this error :-
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 1
Example.main(Example.java:33)
Can anyone please help me to fix the code and find out the error in my program ?

Using FasterXML's CSV package:
https://github.com/FasterXML/jackson-dataformats-text/tree/master/csv
public static List<Map<String, String>> read(File file) throws JsonProcessingException, IOException {
List<Map<String, String>> response = new LinkedList<Map<String, String>>();
CsvMapper mapper = new CsvMapper();
CsvSchema schema = CsvSchema.emptySchema().withHeader();
MappingIterator<Map<String, String>> iterator = mapper.reader(Map.class)
.with(schema)
.readValues(file);
while (iterator.hasNext()) {
response.add(iterator.next());
}
return response;
}

In your String when you split it on first time only contains arr[0] as 1 nothing in arr[1] so it will cause an Exception
If you does not need the 1,2, etc.. You can look the following code:
String str[] = line.split(",");
for(int i=1;i<str.length;i++){
String arr[] = str[i].split(":");
map.put(arr[0], arr[1]);
}

The problem is that when you split your str, the first element in each line is alone (i.e 1 and 2). So arr only contains ["1"], and hence arr[1] doesn't exists.
I.e for the example input :
1,"testCaseName":"ACLTest"
split by , => str contains {1, testCaseName:ACLTest}
split by : at the first iteration => arr contains {1}
Example :
String s = "1,testCaseName:ACLTest";
String str[] = s.split(",");
System.out.println(Arrays.toString(str));
for(String p : str){
String arr[] = p.split(":");
System.out.println(Arrays.toString(arr));
}
Output :
[1, testCaseName:ACLTest]
[1] //<- here arr[1] doesn't exists, you only have arr[0] and hence the ArrayIndexOutOfBoundsException when trying to access arr[1]
[testCaseName, ACLTest]
To fix your code (if you don't want to use a CSV parser), make your loop starting at 1 :
for(int i=1;i<str.length;i++){
String arr[] = str[i].split(":");
map.put(arr[0], arr[1]);
}
Another problem is that the HashMap use the hashCode of the keys to store the (key, value) pairs.
So when insering "testCaseName":"ACLTest" and "testCaseName":"DCLAddTest", the first value will be erased and replace by the second one :
Map<String, String> map = new HashMap<>();
map.put("testCaseName","ACLTest");
map.put("testCaseName","DCLAddTest");
System.out.println(map);
Output :
{testCaseName=DCLAddTest}
So you have to fix that too.

Look at the output of the call
String arr[] = str[i].split(":");
arr[1] does not exists for the first element in your CSV file which happens to be 1, 2... You can start the loop with int i=0 to fix this issue.

String.split is rubbish for parsing CSV. Either use the Guava Splitter or a proper CSV parser. You can parse CSV into beans using the Jackson CSV mapper like this:
public class CSVPerson{
public String firstname;
public String lastname;
//etc
}
CsvMapper mapper = new CsvMapper();
CsvSchema schema = CsvSchema.emptySchema().withHeader().withColumnSeparator(delimiter);
MappingIterator<CSVPerson> it = = mapper.reader(CSVPerson).with(schema).readValues(input);
while (it.hasNext()){
CSVPerson row = it.next();
}
more info at http://demeranville.com/how-not-to-parse-csv-using-java/

Beside the problem you have with the first number which its not a pair and its causing the Exception, you will not want to use Hashmap, since hashmap use a unique key, so line 2 will replace values from line 1.
You should use a MultiMap, or a List of pairs in this case.

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.*;
public class Example {
public static void main(String[] args) {
String csvFile = "test.csv";
String line = "";
String cvsSplitBy = ",";
HashMap<String, String> list = new HashMap<>();
try (BufferedReader br = new BufferedReader(new FileReader(csvFile))) {
while ((line = br.readLine()) != null) {
// use comma as separator
String[] country = line.split(cvsSplitBy);
//System.out.println(country[0] +" " + country[1]);
list.put(country[0], country[1]);
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(list);
}
// enter code here
}

using openCSV would be one way to do it
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import au.com.bytecode.opencsv.CSVReader;
public class CsvFileReader {
public static void main(String[] args) {
try {
System.out.println("\n**** readLineByLineExample ****");
String csvFilename = "C:/Users/hussain.a/Desktop/sample.csv";
CSVReader csvReader = new CSVReader(new FileReader(csvFilename));
String[] col = null;
while ((col = csvReader.readNext()) != null)
{
System.out.println(col[0] );
//System.out.println(col[0]);
}
csvReader.close();
}
catch(ArrayIndexOutOfBoundsException ae)
{
System.out.println(ae+" : error here");
}catch (FileNotFoundException e)
{
System.out.println("asd");
e.printStackTrace();
} catch (IOException e) {
System.out.println("");
e.printStackTrace();
}
}
}
the jar is available here

How to populate an ArrayList from words in a text file?

I have a text file containing words separated by newline , like the following format:
>hello
>world
>example
How do i create an ArrayList and store each word as an element?

You can use apache commons FileUtils.readLines().
I think the List it returns is already an ArrayList, but you can use the constructor ArrayList(Collection) to make sure you get one.

public static void main(String[] args) throws IOException{
// TODO Auto-generated method stub
File file = new File("names.txt");
ArrayList<String> names = new ArrayList<String>();
Scanner in = new Scanner(file);
while (in.hasNextLine()){
names.add(in.nextLine());
}
Collections.sort(names);
for(int i=0; i<names.size(); ++i){
System.out.println(names.get(i));
}

The simplest way is to use Guava:
File file = new File("foo.txt");
List<String> words = Files.readLines(file, Charsets.UTF_8);
(It's not guaranteed to be an ArrayList, but I'd hope that wouldn't matter.)

You read the file line-by-line, create an ArrayList for Strings, and add line.substring(1) to the defined ArrayList if line.length>0.

I put the file at "C:\file.txt"; if you run the following it fils an ArrayList with the words and prints them.
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.util.ArrayList;
import java.util.List;
public class Main {
public static void main(String[] args) throws Exception {
File file = new File("C:\\file.txt");
BufferedReader br = new BufferedReader(new FileReader(file));
List<String> lines = new ArrayList<String>();
String line = br.readLine();
while(line != null) {
lines.add(line.replace(">", ""));
line = br.readLine();
}
for(String l : lines) {
System.out.println(l);
}
}
}

I'm sure they're lots of libraries that do this with 1 line, but here's a "pure" Java implementation:
Notice that we've "wrapped"/"decorated" etc. a standard FileReader (which only has read one byte at a time) with a BufferedReader which gives us a nicer readLine() method.
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(
new FileInputStream("test.txt"),
Charset.forName("ISO-8859-1")));
List<String> lines = new ArrayList<String>();
String line;
while ((line = reader.readLine()) != null) {
lines.add(line);
}
System.out.println(lines);
} catch (IOException e) {
e.printStackTrace();
} finally {
if (reader != null) {
reader.close();
}
}

Read file and get key=value without using java.util.Properties

I'm building a RMI game and the client would load a file that has some keys and values which are going to be used on several different objects. It is a save game file but I can't use java.util.Properties for this (it is under the specification). I have to read the entire file and ignore commented lines and the keys that are not relevant in some classes. These properties are unique but they may be sorted in any order. My file current file looks like this:
# Bio
playerOrigin=Newlands
playerClass=Warlock
# Armor
playerHelmet=empty
playerUpperArmor=armor900
playerBottomArmor=armor457
playerBoots=boot109
etc
These properties are going to be written and placed according to the player's progress and the filereader would have to reach the end of file and get only the matched keys. I've tried different approaches but so far nothing came close to the results that I would had using java.util.Properties. Any idea?

This will read your "properties" file line by line and parse each input line and place the values in a key/value map. Each key in the map is unique (duplicate keys are not allowed).
package samples;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.util.TreeMap;
public class ReadProperties {
public static void main(String[] args) {
try {
TreeMap<String, String> map = getProperties("./sample.properties");
System.out.println(map);
}
catch (IOException e) {
// error using the file
}
}
public static TreeMap<String, String> getProperties(String infile) throws IOException {
final int lhs = 0;
final int rhs = 1;
TreeMap<String, String> map = new TreeMap<String, String>();
BufferedReader bfr = new BufferedReader(new FileReader(new File(infile)));
String line;
while ((line = bfr.readLine()) != null) {
if (!line.startsWith("#") && !line.isEmpty()) {
String[] pair = line.trim().split("=");
map.put(pair[lhs].trim(), pair[rhs].trim());
}
}
bfr.close();
return(map);
}
}
The output looks like:
{playerBoots=boot109, playerBottomArmor=armor457, playerClass=Warlock, playerHelmet=empty, playerOrigin=Newlands, playerUpperArmor=armor900}
You access each element of the map with map.get("key string");.
EDIT: this code doesn't check for a malformed or missing "=" string. You could add that yourself on the return from split by checking the size of the pair array.

I 'm currently unable to come up with a framework that would just provide that (I'm sure there are plenty though), however, you should be able to do that yourself.
Basically you just read the file line by line and check whether the first non whitespace character is a hash (#) or whether the line is whitespace only. You'd ignore those lines and try to split the others on =. If for such a split you don't get an array of 2 strings you have a malformed entry and handle that accordingly. Otherwise the first array element is your key and the second is your value.

Alternately, you could use a regular expression to get the key/value pairs.
(?m)^[^#]([\w]+)=([\w]+)$
will return capture groups for each key and its value, and will ignore comment lines.
EDIT:
This can be made a bit simpler:
[^#]([\w]+)=([\w]+)

After some study i came up with this solution:
public static String[] getUserIdentification(File file) throws IOException {
String key[] = new String[3];
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
String lines;
try {
while ((lines = br.readLine()) != null) {
String[] value = lines.split("=");
if (lines.startsWith("domain=") && key[0] == null) {
if (value.length <= 1) {
throw new IOException(
"Missing domain information");
} else {
key[0] = value[1];
}
}
if (lines.startsWith("user=") && key[1] == null) {
if (value.length <= 1) {
throw new IOException("Missing user information");
} else {
key[1] = value[1];
}
}
if (lines.startsWith("password=") && key[2] == null) {
if (value.length <= 1) {
throw new IOException("Missing password information");
} else {
key[2] = value[1];
}
} else
continue;
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
return key;
}
I'm using this piece of code to check the properties. Of course it would be wiser to use Properties library but unfortunately I can't.

The shorter way how to do that:
Properties properties = new Properties();
String confPath = "src/main/resources/.env";
try {
properties.load(new FileInputStream(confPath));
} catch (IOException e) {
e.printStackTrace();
}
String specificValueByKey = properties.getProperty("KEY");
Set<Object> allKeys = properties.keySet();
Collection<Object> values = properties.values();