I made a java program that will check contents of directory and generate for each file a md5 checksum. When the program is done it will save it to a CSV file. So far the lookup of files is working perfectly except that when writing to the CSV i want to make to only add new detected files. I think the issue lies with the md5 string used as key is not correctly found.
Here is an excerpt of the CSV file:
4d1954a6d4e99cacc57beef94c80f994,uiautomationcoreapi.h;E:\Tools\Strawberry-perl-5.24.1.1-64\c\x86_64-w64-mingw32\include\uiautomationcoreapi.h;N/A
56ab7135e96627b90afca89199f2c708,winerror.h;E:\Tools\Strawberry-perl-5.24.1.1-64\c\x86_64-w64-mingw32\include\winerror.h;N/A
146e5c5e51cc51ecf8d5cd5a6fbfc0a1,msimcsdk.h;E:\Tools\Strawberry-perl-5.24.1.1-64\c\x86_64-w64-mingw32\include\msimcsdk.h;N/A
e0c43f92a1e89ddfdc2d1493fe179646,X509.pm;E:\Tools\Strawberry-perl-5.24.1.1-64\perl\vendor\lib\Crypt\OpenSSL\X509.pm;N/A
As you can see first is the MD5 as key and afterwards is a long string containing name, location and score that will be split with the ; character.
and here is the code that should make sure only new ones are added:
private static HashMap<String, String> map = new HashMap<String,String>();
public void UpdateCSV(HashMap<String, String> filemap) {
/*Set set = filemap.entrySet();
Iterator iterator = set.iterator();
while(iterator.hasNext()) {
Map.Entry mentry = (Map.Entry) iterator.next();
String md = map.get(mentry.getKey());
System.out.println("checking key:" + md);
if (md == null) {
String[] line = mentry.getValue().toString().split(";");
System.out.println("Adding new File:" + line[0]);
map.put(mentry.getKey().toString(), mentry.getValue().toString());
}
}*/
for (final String key : filemap.keySet()) {
String md = map.get(key.toCharArray());
if (md == null) {
System.out.println("Key was not found:" + key);
String[] line = filemap.get(key).toString().split(";");
System.out.println("Adding new File:" + line[0]);
map.put(key, filemap.get(key));
}
}
}
As you can see from the commented code i tried in different ways already. hashmap filemap is the current status of the folder structure.
To read the already saved CSV file is use the following code:
private void readCSV() {
System.out.println("Reading CSV file");
BufferedReader br = new BufferedReader(filereader);
String line = null;
try {
while ((line = br.readLine()) != null) {
String str[] = line.split(",");
for (int i = 0; i < str.length; i++) {
String arr[] = str[i].split(":");
map.put(arr[0], arr[1]);
System.out.println("just added to map" + arr[0].toString() + " with value "+ arr[0].toString() );
}
}
}
catch(java.io.IOException e) {
System.out.println("Can't read file");
}
}
So when i run the program it will say that all files are new even tough they are already known in the CSV. So can anyone help to get this key string checked correctly?
As #Ben pointed out, your problem is that you use String as key when putting, but char[] when getting.
It should be something along the lines:
for (final String key : filemap.keySet()) {
map.computeIfAbsent(key, k -> {
System.out.println("Key was not found:" + k);
String[] line = filemap.get(k).toString().split(";");
System.out.println("Adding new File:" + line[0]);
return filemap.get(k);
});
}
Since you need both key as well as value from filemap, you actually better iterate over entrySet. This will save you additional filemap.gets:
for (final Map.Entry<String, String> entry : filemap.entrySet()) {
final String key = entry.getKey();
final String value = entry.getValue();
map.computeIfAbsent(key, k -> {
System.out.println("Key was not found:" + k);
String[] line = value.split(";");
System.out.println("Adding new File:" + line[0]);
return value;
});
}
Related
I need to write to csv all the keys from one map in one column, and all the values from a different map in the next column.
I can do either column individually with this code but when I combine, how do I explain this(?), if I have 10 keys and 10 values the keys will repeat 10 of each key.
What do I need to do to my loops?
private static void generateCourseCounts() throws IOException {
ArrayList<StudentCourse> lsc = loadStudentCourses();
Map<Integer, Integer> countStudents = new TreeMap<Integer, Integer>();
for (StudentCourse sc : lsc) {
Integer freq = countStudents.get(sc.getCourseId());
countStudents.put(sc.getCourseId(), (freq == null) ? 1 : freq + 1);
}
ArrayList<Course> lc = loadCourses();
Map<String, String> courses = new LinkedHashMap<String, String>();
for (Course c : lc) {
String freq = courses.get(c.getCourseName());
courses.put(c.getCourseName(), freq);
}
FileWriter writer = new FileWriter("CourseCounts.csv");
PrintWriter printWriter = new PrintWriter(writer);
printWriter.println("Course Name\t# Students");
for (Entry<String, String> courseKey : courses.entrySet())
for (Entry<Integer, Integer> numberKey : countStudents.entrySet()) {
printWriter.println(courseKey.getKey() + "\t" + numberKey.getValue());
}
printWriter.close();
writer.close();
}
So, as per comments below, I edited to this:
for (String courseKey : courses.keySet()) {
Integer count = countStudents.get(courseKey) ;
printWriter.println(courseKey + "\t" + count);
}
However, this writes an empty file.
Try this. It does presume that the number of map entries in each map is the same.
Otherwise, you will either get an index out of bounds exception or you won't print all the values.
int i = 0;
Integer[] counts = countStudents.values().stream().toArray(Integer[]::new);
for (String courseKey : courses.keySet()) {
printWriter.println(courseKey + "\t" + counts[i++]);
}
printWriter.close();
writer.close();
You don't need embedded cycles. You can just iterate by keys from 1st map and get values from 2nd:
for (String courseKey: courses.keySet())
String count = countStudents.get(courseKey);
// ... output courseKey and count to file
}
I am working to read a file of about 2 million entries. Not all are valid lines, but that's the cap. I am using a Map to store key value pairings from the file. I however find that the last term is at line 1932513. I am coding with eclipse, in Java, and went over to bump up the Xmx parameters in the eclipse.ini. This did not solve the problem. Here is the code I am working with:
BufferedReader bufferedRead = new BufferedReader(inputStream);
String data;
try {
while((data = bufferedRead.readLine()) != null) {
String[] meterReadings = data.split(";");
if("Date".equals(meterReadings[0])) continue;
/*
* Creating the key object using concatenation of date-time
*/
if(newApp.isEmpty(meterReadings)) continue;
List<Object> valueList = new ArrayList<Object>();
String[] subDate = meterReadings[0].split("/");
StringBuffer dateTime = new StringBuffer();
for(String s : subDate) {
dateTime.append(s + ":");
}
dateTime.append(meterReadings[1]);
for(int i=2; i < meterReadings.length -1; ++i) {
valueList.add(meterReadings[i]);
}
newApp.textMap.put(dateTime.toString(), valueList);
}
} catch (IOException e) {
log.error("Unable to read from file: ", App.DATA_DIR + App.FILENAME);
e.printStackTrace();
}
/*
* Checking to see if Map has been created
*/
System.out.println("*****************************************");
Set<String> newSet = newApp.textMap.keySet();
Object[] setArray = newSet.toArray();
System.out.println("The last term's key is : " + (String) setArray[setArray.length - 1]);
System.out.println(newApp.textMap.size());
log.info("The size of the reading is : ", newApp.textMap.size());
System.out.println("*****************************************");
}
This is my code. I am not able to write it back to a csv. I have figured out that the problem is with the header matching in the if loop. The values are being read correctly nut the if loop is not getting satisfied when the headers are getting matched. Any help is much appreciated. Thanks.
public class FactsChecker {
public static void main(String[] args) {
run();
}
static void run()
{
Map <String,String> csvMap = new HashMap<String,String>();
String csvPath = "resource/Sample.csv";
String outputPath = "resource/Output.csv";
String[] headers;
try{
CsvReader csvreader = new CsvReader("resource/Sample.csv");
csvreader.readHeaders();
headers = csvreader.getHeaders();
String[] values;
String key = null;
String value = null;
while (csvreader.readRecord()){`
values = csvreader.getValues();
System.out.println(headers[0]);
for (int i = 0; i < headers.length; i++) {
if (headers[i].equals("FCT_NAME")) {
key = values[i];
}
if (headers[i].equals("FCT_TYPE")) {
value = values[i];
}
}
csvMap.put(key,value);
}
for (Map.Entry<String, String> entry : csvMap.entrySet())
{
System.out.println(entry.getKey() + "/" + entry.getValue());
}
Map<String,Entry> entryMap= new HashMap<String, Entry>();
for (String factKey:csvMap.keySet()){
Entry entry = new Entry();
entry.setFactName(factKey);
String factType = csvMap.get(factKey);
System.out.println(factKey);
entry.setFactType(factType);
entryMap.put(factKey, entry);
}
CsvWriter writer = new CsvWriter(outputPath);
for(String key1 : entryMap.keySet()){
Entry entry = entryMap.get(key1);
writer.writeRecord(new String[]{entry.getFactName(),entry.getFactType()});
}
writer.close();
}
catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
try using contains instead of equals in case there is trailing or leading white space
I also faced the same issue .Try the below. Just remove this appended character from your header when you read it from csv and then compare.
if(headers[i].startswith("\uFEFF"){
headers[i].replace("\uFEFF","");
}
I am wondering if I can access the values I get after iterating the hashmap in the following code (I know they are stored in map but want to use them outside loop). The key and corresponding values are iterated inside a loop. Can concurrent hashmap help me to get the values and use them outside the loop.
Thank You.
public static void main(String[] args) {
Map<String, List<String>> maleMap = new LinkedHashMap<String, List<String>>();
Map<String, List<String>> femaleMap = new LinkedHashMap<String, List<String>>();
try {
Scanner scanner = new Scanner(new FileReader(.txt));
while (scanner.hasNextLine()) {
String nextLine = scanner.nextLine();
String[] column = nextLine.split(":");
if (column[0].equals("male") && (column.length == 4)) {
maleMap.put(column[1],
Arrays.asList(column[2], column[3]));
} else if (column[0].equals("female") && (column.length == 4)) {
femaleMap.put(column[1],
Arrays.asList(column[2], column[3]));
}
}
Set<Entry<String, List<String>>> entries = maleMap.entrySet();
Iterator<Entry<String, List<String>>> entryIter = entries
.iterator();
while (entryIter.hasNext()) {
Map.Entry entry = (Map.Entry) entryIter.next();
Object key = entry.getKey(); // Get the key from the entry.
List<String> value = (List<String>) entry.getValue();
Object value1 = " ";
Object value2 = " ";
int counter = 0;
for (Object listItem : (List) value) {
Writer writer = null;
Object Name = key;
Object Age = null;
Object ID = null;
if (counter == 0) {// first pass assign value to value1
value1 = listItem;
counter++;// increment for next pass
} else if (counter == 1) {// second pass assign value to
// value2
value2 = listItem;
counter++;// so we dont keep re-assigning listItem for
// further iterations
}
}
System.out.println(key + ":" + value1 + "," + value2);
scanner.close();
Writer writer = null;
Object Name = key;
Object Age = value1;
Object ID = value2;
try {
String filename = ".txt";
FileWriter fw = new FileWriter(filename, true);
fw.write("# Table" + Name + "\n" + "map:"+ Name + " a d2rq:ClassMap;" + "\n"
+ " dataStorage map:database;" + "\n"+ "Pattern " +"\""+ Name + "/##"+ Age +
"." + ID + "##\";" + "\n"+ " class :" + Name +";"+"\n"+ " ."+"\n");//
fw.close();
} catch (Exception e) {
e.printStackTrace();
}
}
ConcurrentHashMap is designed to be thread-safe.
If you aren't using multiple threads, it's worse than useless.
You should just use a MultiMap.
You can always access values outside the loop.
ConcorrentHashMap is fail-safe. it won't give any concurrent modification exceptions. it works good for multi threaded operations. The whole implementation of ConcurrentHashMap is same as HashMap but the while retrieving the elements , HashMap locks whole map restricting doing further modifications which gives concurrent modification exception.' But in ConcurrentHashMap, the locking happens at bucket level so the chance of giving concurrent modification exception is not present.
I've been coding Perl and Python a lot and this time I got an assignment to code in Java instead. So I'm not too familiar with handling data in Java.
My task involves having a input file where I need to check dependencies and then output those with transitive dependencies. Clearer ideas below:
Input File:
A: B C
B: C E
C: G
D: A
Output File:
A: B C E G
B: C E G
C: G
D: A B C E G
So far this is what I've got (separating the first and second token):
import java.util.StringTokenizer;
import java.io.*;
public class TestDependency {
public static void main(String[] args) {
try{
FileInputStream fstream = new FileInputStream("input-file");
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
while ((strLine = br.readLine()) != null) {
StringTokenizer items = new StringTokenizer(strLine, ":");
System.out.println("I: " + items.nextToken().trim());
StringTokenizer depn = new StringTokenizer(items.nextToken().trim(), " ");
while(depn.hasMoreTokens()) {
System.out.println( "D: " + depn.nextToken().trim() );
}
}
} catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
}
}
Any help appreciated. I can imagine Perl or Python to handle this easily. Just need to implement it in Java.
This is not very efficient memory-wise and requires good input but should run fine.
public class NodeParser {
// Map holding references to nodes
private Map<String, List<String>> nodeReferenceMap;
/**
* Parse file and create key/node array pairs
* #param inputFile
* #return
* #throws IOException
*/
public Map<String, List<String>> parseNodes(String inputFile) throws IOException {
// Reset list if reusing same object
nodeReferenceMap = new HashMap<String, List<String>>();
// Read file
FileInputStream fstream = new FileInputStream(inputFile);
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
// Parse nodes into reference mapping
while((strLine = br.readLine()) != null) {
// Split key from nodes
String[] tokens = strLine.split(":");
String key = tokens[0].trim();
String[] nodes = tokens[1].trim().split(" ");
// Set nodes as an array list for key
nodeReferenceMap.put(key, Arrays.asList(nodes));
}
// Recursively build node mapping
Map<String, Set<String>> parsedNodeMap = new HashMap<String, Set<String>>();
for(Map.Entry<String, List<String>> entry : nodeReferenceMap.entrySet()) {
String key = entry.getKey();
List<String> nodes = entry.getValue();
// Create initial node set
Set<String> outSet = new HashSet<String>();
parsedNodeMap.put(key, outSet);
// Start recursive call
addNode(outSet, nodes);
}
// Sort keys
List<String> sortedKeys = new ArrayList<String>(parsedNodeMap.keySet());
Collections.sort(sortedKeys);
// Sort nodes
Map<String, List<String>> sortedParsedNodeMap = new LinkedHashMap<String, List<String>>();
for(String key : sortedKeys) {
List<String> sortedNodes = new ArrayList<String>(parsedNodeMap.get(key));
Collections.sort(sortedNodes);
sortedParsedNodeMap.put(key, sortedNodes);
}
// Return sorted key/node mapping
return sortedParsedNodeMap;
}
/**
* Recursively add nodes by referencing the previously generated list mapping
* #param outSet
* #param nodes
*/
private void addNode(Set<String> outSet, List<String> nodes) {
// Add each node to the set mapping
for(String node : nodes) {
outSet.add(node);
// Get referenced nodes
List<String> nodeList = nodeReferenceMap.get(node);
if(nodeList != null) {
// Create array list from abstract list for remove support
List<String> referencedNodes = new ArrayList<String>(nodeList);
// Remove already searched nodes to prevent infinite recursion
referencedNodes.removeAll(outSet);
// Recursively search more node paths
if(!referencedNodes.isEmpty()) {
addNode(outSet, referencedNodes);
}
}
}
}
}
Then, you can call this from your program like so:
public static void main(String[] args) {
try {
NodeParser nodeParser = new NodeParser();
Map<String, List<String>> nodeSet = nodeParser.parseNodes("./res/input.txt");
for(Map.Entry<String, List<String>> entry : nodeSet.entrySet()) {
String key = entry.getKey();
List<String> nodes = entry.getValue();
System.out.println(key + ": " + nodes);
}
} catch (IOException e){
System.err.println("Error: " + e.getMessage());
}
}
Also, the output is not sorted but that should be trivial.
String s = "A: B C D";
String i = s.split(":")[0];
String dep[] = s.split(":")[1].trim().split(" ");
System.out.println("i = "+i+", dep = "+Arrays.toString(dep));