We Keep Coding

How can two CSV files with the same data but different column orders be validated in Java?

I am trying to compare two CSV files that have the same data but columns in different orders. When the column orders match, the following code works: How can I tweak my following code to make it work when column orders don't match between the CSV files?
Set<String> source = new HashSet<>(org.apache.commons.io.FileUtils.readLines(new File(sourceFile)));
Set<String> target = new HashSet<>(org.apache.commons.io.FileUtils.readLines(new File(targetFile)));
return source.containsAll(target) && target.containsAll(source)
For example, the above test pass when the source file and target file are in this way:
source file:
a,b,c
1,2,3
4,5,6
target file:
a,b,c
1,2,3
4,5,6
However, the source file is same, but if the target file is in the following way, it doesn't work.
target file:
a,c,b
1,3,2
4,6,5

A Set relies on properly functioning .equalsmethod for comparison, whether detecting duplicates, or comparing it's elements to those in another Collection. When I saw this question, my first thought was to create a new class for Objects to put into your Set Objects, replacing the String Objects. But, at the time, it was easier and faster to produce the code in my previous answer.
Here is another solution, which is closer to my first thought. To start, I created a Pair class, which overrides .hashCode () and .equals (Object other).
package comparecsv1;
import java.util.Objects;
public class Pair <T, U> {
private final T t;
private final U u;
Pair (T aT, U aU) {
this.t = aT;
this.u = aU;
}
#Override
public int hashCode() {
int hash = 3;
hash = 59 * hash + Objects.hashCode(this.t);
hash = 59 * hash + Objects.hashCode(this.u);
return hash;
}
#Override
public boolean equals(Object obj) {
if (this == obj) { return true; }
if (obj == null) { return false; }
if (getClass() != obj.getClass()) { return false; }
final Pair<?, ?> other = (Pair<?, ?>) obj;
if (!Objects.equals(this.t, other.t)) {
return false;
}
return Objects.equals(this.u, other.u);
} // end equals
} // end class pair
The .equals (Object obj) and the .hashCode () methods were auto-generated by the IDE. As you know, .hashCode() should always be overridden when .equals is overridden. Also, some Collection Objects, such as HashMap and HashSet rely on proper .hashCode() methods.
After creating class Pair<T,U>, I created class CompareCSV1. The idea here is to use a Set<Set<Pair<String, String>>> where you have Set<String> in your code.
A Pair<String, String> pairs a value from a column with the header for the column in which it appears.
A Set<Pair<String, String>> represents one row.
A Set<Set<Pair<String, String>>> represents all the rows.
package comparecsv1;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
public final class CompareCSV1 {
private final Set<Set<Pair<String, String>>> theSet;
private final String [] columnHeader;
private CompareCSV1 (String columnHeadings, String headerSplitRegex) {
columnHeader = columnHeadings.split (headerSplitRegex);
theSet = new HashSet<> ();
}
private Set<Pair<String, String>> createLine
(String columnSource, String columnSplitRegex) {
String [] column = columnSource.split (columnSplitRegex);
Set<Pair<String, String>> lineSet = new HashSet<> ();
int i = 0;
for (String columnValue: column) {
lineSet.add (new Pair (columnValue, columnHeader [i++]));
}
return lineSet;
}
public Set<Set<Pair<String, String>>> getSet () { return theSet; }
public String [] getColumnHeaders () {
return Arrays.copyOf (columnHeader, columnHeader.length);
}
public static CompareCSV1 createFromData (List<String> theData
, String headerSplitRegex, String columnSplitRegex) {
CompareCSV1 result =
new CompareCSV1 (theData.get(0), headerSplitRegex);
for (int i = 1; i < theData.size(); ++i) {
result.theSet.add(result.createLine(theData.get(i), columnSplitRegex));
}
return result;
}
public static void main(String[] args) {
String [] sourceData = {"a,b,c,d,e", "6,7,8,9,10", "1,2,3,4,5"
,"11,12,13,14,15", "16,17,18,19,20"};
String [] targetData = {"c,b,e,d,a", "3,2,5,4,1", "8,7,10,9,6"
,"13,12,15,14,11", "18,17,20,19,16"};
List<String> source = Arrays.asList(sourceData);
List<String> target = Arrays.asList (targetData);
CompareCSV1 sourceCSV = createFromData (source, ",", ",");
CompareCSV1 targetCSV = createFromData (target, ",", ",");
System.out.println ("Source contains target? "
+ sourceCSV.getSet().containsAll (targetCSV.getSet())
+ ". Target contains source? "
+ targetCSV.getSet().containsAll (sourceCSV.getSet())
+ ". Are equal? " + targetCSV.getSet().equals (sourceCSV.getSet()));
} // end main
} // end class CompareCSV1
This code has some things in common with the code in my first answer:
Except for the column header lines, which must be first in the "source" and "Target" data, matching lines in one file can be in a different order in the other file.
I used String [] Objects, with calls to Arrays.asList method as substitutes for your data sources.
It does not contain code to guard against errors, such as lines in the file having different numbers of columns from other lines, or no header line.
I hard coded "," as the String split expression in main. But, the new methods allow the String split expression to be passed. It allows a separate String split expressions for the column header line and the data lines.

Here is some code that could work. It relies on the first line of each file containing column headers.
It's a bit more than a tweak, though. It's an "old dog" approach.
The original code in the question has these lines:
Set<String> source = new HashSet<>(org.apache.commons.io.FileUtils.readLines(new File(sourceFile)));
Set<String> target = new HashSet<>(org.apache.commons.io.FileUtils.readLines(new File(targetFile)));
With this solution, the data coming in needs more processing before it will be ready to be put into a Set. Those two lines get changed as follows:
List<String> source = (org.apache.commons.io.FileUtils.readLines(new File(sourceFile)));
List<String> target = (org.apache.commons.io.FileUtils.readLines(new File(targetFile)));
This approach will compare column headers in the target file and the source file. It will use that to build an int [] that indicates the difference in column order.
After the order difference array is filled, the data in the file will be put into a pair of Set<List<String>>. Each List<String> will represent one line from the source and target data files. Each String in the List will be data from one column.
In the following code, main is the test driver. Only for my testing purposes, the data files have been replaced by a pair of String [] and reading the file with org.apache.commons.io.FileUtils.readLines has been replaced with Arrays.asList.
package comparecsv;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
public class CompareCSV {
private static int [] columnReorder;
private static void headersOrder
(String sourceHeader, String targetHeader) {
String [] columnHeader = sourceHeader.split (",");
List<String> sourceColumn = Arrays.asList (columnHeader);
columnReorder = new int [columnHeader.length];
String [] targetColumn = targetHeader.split (",");
for (int i = 0; i < targetColumn.length; ++i) {
int j = sourceColumn.indexOf(targetColumn[i]);
columnReorder [i] = j;
}
}
private static Set<List<String>> toSet
(List<String> data, boolean reorder) {
Set<List<String>> dataSet = new HashSet<> ();
for (String s: data) {
String [] byColumn = s.split (",");
if (reorder) {
String [] reordered = new String [byColumn.length];
for (int i = 0; i < byColumn.length; ++i) {
reordered[columnReorder[i]] = byColumn [i];
}
dataSet.add (Arrays.asList (reordered));
} else {
dataSet.add (Arrays.asList(byColumn));
}
}
return dataSet;
}
public static void main(String[] args) {
String [] sourceData = {"a,b,c,d,e", "1,2,3,4,5", "6,7,8,9,10"
,"11,12,13,14,15", "16,17,18,19,20"};
String [] targetData = {"c,b,e,d,a", "3,2,5,4,1", "8,7,10,9,6"
,"13,12,15,14,11", "18,17,20,19,16"};
List<String> source = Arrays.asList(sourceData);
List<String> target = Arrays.asList (targetData);
headersOrder (source.get(0), target.get(0));
Set<List<String>> sourceSet = toSet (source, false);
Set<List<String>> targetSet = toSet (target, true);
System.out.println ( sourceSet.containsAll (targetSet)
+ " " + targetSet.containsAll (sourceSet) + " " +
( sourceSet.containsAll (targetSet)
&& targetSet.containsAll (sourceSet)));
}
}
MethodheadersOrder compares the headers, column by column, and populates the columnReorder array. Method toSet creates the Set<List<String>>, either reordering the columns or not, according to the value of the boolean argument.
For the sake of simplification, this assumes lines are easily split using comma. Data such as dog, "Reginald, III", 3 will cause failure.
In testing this, I found lines in the file can be matched with their counterpart in the other file, regardless of order of the lines. Here is an example:
Source:
a,b,c
1,2,3
4,5,6
7,8,9
Target:
a,b,c
4,5,6
7,8,9
1,2,3
The result would be the contents match.
I believe this would match a result from the O/P question code. However, for this solution to work, the first line in each file must contain column headers.

separating unique values in an algorithm

I am decomposing a series of 90,000+ strings into a discrete list of the individual, non-duplicated pairs of words that are included in the strings with the rxcui id values associated with each string. I have developed a method which tries to accomplish this, but it is producing a lot of redundancy. Analysis of the data shows there are about 12,000 unique words in the 90,000+ source strings, after I clean and format the contents of the strings.
How can I change the code below so that it avoids creating the redundant rows in the destination 2D ArrayList (shown below the code)?
public static ArrayList<ArrayList<String>> getAllWords(String[] tempsArray){//int count = tempsArray.length;
int fieldslenlessthan2 = 0;//ArrayList<String> outputarr = new ArrayList<String>();
ArrayList<ArrayList<String>> twoDimArrayList= new ArrayList<ArrayList<String>>();
int idx = 0;
for (String s : tempsArray) {
String[] fields = s.split("\t");//System.out.println(" --- fields.length is: "+fields.length);
if(fields.length>1){
ArrayList<String> row = new ArrayList<String>();
System.out.println("fields[0] is: "+fields[0]);
String cleanedTerms = cleanTerms(fields[1]);
String[] words = cleanedTerms.split(" ");
for(int j=0;j<words.length;j++){
String word=words[j].trim();
word = word.toLowerCase();
if(isValidWord(word)){//outputarr.add(word);
System.out.println("words["+j+"] is: "+word);
row.add(word_id);//WORD_ID NEEDS TO BE CREATED BY SOME METHOD.
row.add(fields[0]);
row.add(word);
twoDimArrayList.add(row);
idx += 1;
}
}
}else{fieldslenlessthan2 += 1;}
}
System.out.println("........... fieldslenlessthan2 is: "+fieldslenlessthan2);
return twoDimArrayList;
}
The output of the above method currently looks like the following, with many rxcui values for some name values, and with many name values for some rxcui:
How do I change the code above so that the output is a list of unique pairs of name/rxcui values, summarizing all relevant data from the current output while removing only the redundancies?

If you just need a Collection of all words, use a HashSet Sets are primarily used for contains logic. If you need to associate a value with your string use a HashMap
public HashSet<String> getUniqueWords(String[] stringArray) {
HashSet<String> uniqueWords = new HashSet<String>();
for (String str : stringArray) {
uniqueWords.add(str);
}
return uniqueWords;
}
This will give you a collection of all the unique Strings in your array. If you need an ID use a HashMap
String[] strList; // your String array
int idCounter = 0;
HashMap<String, Integer> stringIDMap = new HashMap<String, Integer>();
for (String str : strList) {
if (!stringIDMap.contains(str)) {
stringIDMap.put(str, new Integer(idCounter));
idCounter++;
}
}
This will provide you a HashMap with unique String keys and unique Integer values. To get an id for a String you do this:
stringIDMap.get("myString"); // returns the Integer ID associated with the String "myString"
UPDATE
Based on the question update from the OP. I recommend creating an object that holds the String value and the rxcui. You can then place these in a Set or HashMap using a similar implementation to the one provided above.
public MyObject(String str, int rxcui); // The constructor for your new object
MyObject mo1 = new MyObject("hello", 5);
Either
mySet.add(myObject);
will work or
myMap.put(mo1.getStr, mo1.getRxcui);

What is the purpose of the unique word ID? Is the word itself not unique enough since you are not keeping duplicates?
A very basic way would be to keep a counter going as you are checking new words. For each word that doesn't already exist you could increase the counter and use the new value as the unique id.
Lastly, might I suggest you use a HashMap instead. It would allow you to both insert and retrieve words in O(1) time. I am not entirely sure what you are going for, but I think the HashMap might give you more range.
Edit2:
It would be something a little more along these lines. This should help you out.
public static Set<DataPair> getAllWords(String[] tempsArray) {
Set<DataPair> set = new HashSet<>();
for (String row : tempsArray) {
// PARSE YOUR STRING DATA
// the way you were doing it seemed fine but something like this
String[] rowArray = row.split(" ");
String word = row[1];
int id = Integer.parseInt(row[0]);
DataPair pair = new DataPair(word, id);
set.add(pair);
}
return set;
}
class DataPair {
private String word;
private int id;
public DataPair(String word, int id) {
this.word = word;
this.id = id;
}
public boolean equals(Object o) {
if (o instanceof DataPair) {
return ((DataPair) o).word.equals(word) && ((DataPair) o).id == id;
}
return false;
}
}

Java Generating Strings Recursively - All 1s then 1 and 0

Hey guys I'm trying to get the concept of recursion down by making a program that generates String of an ArrayList recursively. My basic algorithm is:
public static ArrayList<String> generateListOfAll1sStrings(int maxBits)
terminal condition: if maxBits is 1, return the simplest case: a list containing just "1"
otherwise:
recursively call generateListOfAll1sStrings() for the next-smallest bit-length, saving the list that is returned
find the longest string in that list and create a new string with "1" appended to it (making the next-longest string)
return a new list that contains all the elements of the shorter list along with the new string just added.
The code I have so far is:
package bincomb.model;
import java.util.ArrayList;
public class BinaryCombinationGenerator {
public static ArrayList<String> generateListOfAll1sStrings(int maxBits) {
String string = null;
ArrayList<String> listofJust1 = new ArrayList<String>();
ArrayList<String> otherArray = new ArrayList<String>();
int i = 1;
if (maxBits == 1) {
listofJust1.add("1");
return listofJust1;
}
if (maxBits > 1) {
for (String string2 : listofJust1) {
String comp = "";
if (!(comp.equals(string2))) {
comp = string2;
}
string = comp;
}
listofJust1.add(i, (string + "1"));
i++;
listofJust1 = BinaryCombinationGenerator.generateListOfAll1sStrings((maxBits-1));
System.out.println(listofJust1);
return listofJust1;
}
return listofJust1;
}
public static void main(String[] args) {
generateListOfAll1sStrings(10);
}
}
However, currently, I'm returning an IndexOutOfBoundsException. I think my for loop is causing the problem, but I'm not certain how to go about fixing it.

You're getting an java.lang.IndexOutOfBoundsException at this line listofJust1.add(i, (string + "1"));.
This is because the method list.add(index, objects) tries to add the object at index "1" but your array has 0 elements.
Either change it to listofJust1.add(i-1, (string + "1")); or simply listofJust1.add((string + "1"));
#Edit: here:
listofJust1.add(i, (string + "1"));
You want to add the string for the current (N) level of recursion but below you substitute this array with:
listofJust1 = BinaryCombinationGenerator.generateListOfAll1sStrings((maxBits-1));
Which basically says "get the result for (maxBits-1) and replace with it listofJust1" therefore you are losing what you added before.
Instead you should first get the list for level N-1 and then add the string for the current level:
listofJust1 = BinaryCombinationGenerator.generateListOfAll1sStrings((maxBits-1));
listofJust1.add(stringForThisLevel);
Also you need to rething how you are computing "string" at level N, doesn't seem right.

Android: Find a value inside an Array with a "a like" value

I'm having problem with this part
My Code:
String[] sample = {'name=NAME', 'add=ADD', 'age=AGE', 'gender=GENDER'};
for(int a = 0; a < sample.length; a++) {
if(Arrays.asList(sample).contains("name")) {
Log.d(tag, "successful");
} else {
Log.d(tag, "failed");
}
}
When I'm using this code, it doesn't return true, but when I use .contains("name=NAME")
it returns true.
Is there any possibility to compare a string value using not too specific string?
BTW, those string values came from a file.txt.

If you use Arrays.asList(sample) you will have a list containing the String "name=NAME" hence it doesn't contain the String "name"
You should loop over the array (not needed to create the list)
boolean found = false;
for (String s: sample)
if (s.contains("name"))
found=true;

To get a value based on key:
1) You can use HashMap object
import java.util.*;
import java.lang.*;
import java.io.*;
/* Name of the class has to be "Main" only if the class is public. */
class Ideone
{
public static void main(String args[]) {
// create hash map
HashMap newmap = new HashMap();
// populate hash map
newmap.put("name", "abcde");
newmap.put("age", "XX");
newmap.put("Gender", "Male");
// get value of key Gender
String val=(String)newmap.get("Gender");
// check the value
System.out.println("Value for key 3 is: " + val);
}
}
2) You can also use Map object
Map<String, String> map = new HashMap<String, String>();
map.put("name", "abcde");
map.put("age", "XX");
String val=(String)map.get("name");
3) You can also use two dimentional array of string
String data = "name=Name,age=Age";
String[] rows = data.split(",");
String[][] matrix = new String[rows.length][];
int r = 0;
for (String row : rows) {
matrix[r++] = row.split("\\=");
}
System.out.println(matrix[1][1]);

.contains method will check whether the object is present in the ArrayList or not.
Here in your case objects are : "name=NAME","add=ADD","age=AGE", "gender=GENDER" which are of String type. So it is obvious that it returns false.
For you, better practice is to create a class named Person which has attributes like name,add,age and gender. Then store the object of it in ArrayList and you can check whether object is in ArrayList of not using .contains() function. Like below :
class Person{
String name;
String add;
int age;
String gen;
// All getters and setters methods will be here.
}
public static void main(String[] args){
Person p = new Person();
//Here you can set the properties of person using p.setXXX() methods
//Now suppose you have ArrayList of Person object named "per" then you can check
// whether Person exist in it or not using contains like below
per.contains(p) //returns true if per contains object p
//or you can check name by below code
for(Person temp:per){
if(temp.getName().equals(name)){ //returns true if name matchs with Persons name
//do something
}
}

Complex string split in Java

Consider the following String :
5|12345|value1|value2|value3|value4+5|777|value1|value2|value3|value4?5|777|value1|value2|value3|value4+
Here is how I want to split string, split it with + so I get this result :
myArray[0] = "5|12345|value1|value2|value3|value4";
myArray[1] = "5|777|value1|value2|value3|value4?5|777|value1|value2|value3|value4";
if string has doesn't contain char "?" split it with "|" and continue to part II, if string does contain "?" split it and for each part split it with "|" and continue to part II.
Here is part II :
myObject.setAttribute1(newString[0]);
...
myObject.setAttribute4(newString[3]);
Here what I've got so far :
private static String input = "5|12345|value1|value2|value3|value4+5|777|value1|value2|value3|value4?5|777|value1|value2|value3|value4+";
public void mapObject(String input){
String[] myArray = null;
if (input.contains("+")) {
myArray = input.split("+");
} else {
myArray = new String[1];
myArray[0] = input;
}
for (int i = 0; i < myArray.length; i++) {
String[] secondaryArray = null;
String[] myObjectAttribute = null;
if (myArray[i].contains("?")) {
secondaryArray = temporaryString.myArray[i].split("?");
for (String string : secondaryArray) {
myObjectAttribute = string.split("\\|");
}
} else {
myObjectAttribute = myArray[i].toString().split("\\|");
}
myObject.setAttribute1(myObjectAttribute[0]);
...
myObject.setAttribute4(myObjectAttribute[3]);
System.out.println(myObject.toString());
}
Problem :
When I split myArray, going trough for with myArray[0], everything set up nice as it should.
Then comes the myArray[1], its split into two parts then the second part overrides the value of the first(how do I know that?). I've overridden toString() method of myObject, when I finish I print the set values so I know that it overrides it, does anybody know how can I fix this?

I'm not quite sure what the intention is here, but in this snippet of code
secondaryArray = temporaryString.split("?");
for (String string : secondaryArray) {
myObjectAttribute = string.split("\\|");
}
if secondaryArray has two elements after the split operation, you are iterating over each half and re-assigning myObjectAttribute to the output of string.split("\|") each time. It doesn't matter what is in the first element of secondaryArray, as after this code runs myObjectAttribute is going to contain the result of split("\\|") on the last element in the array.
Also, there is no point in calling .toString() on a String object as you do in temporaryString = myArray[i].toString().

The code doesn't seem to be able to handle the possible expansion of strings in the secondary case. To make the code clearer, I would use a List rather than array.
private static String input = "5|12345|value1|value2|value3|value4+5|777|value1|value2|value3|value4?5|777|value1|value2|value3|value4+";
private void split(List<String> input, List<String> output, String split) {
for (String s: input) {
if (s.contains(split))
{
output.addAll(Arrays.asList(s.split(Pattern.quote(split)));
}
else
output.add(s);
}
}
public void mapObject(String input) {
List<String> inputSrings = new ArrayList<String>();
List<String> splitPlus = new ArrayList<String>();
inputStrings.add(input);
split(inputStrings, splitPlus);
List<String> splitQuest = new ArrayList<String>();
split(splitPlus, splitQuest, "?");
for (String s: splitQuest) {
// you can now set the attributes from the values in the list
// splitPipe
String[] attributes = s.split("\\|");
myObject.setAttribute1(attributes[0]);
....
myObject.setAttribute4(attributes[3]);
System.out.println(myObject);
}
}