Best way to implement a Reverse MultiMap - java

My Java8 program has several stages:
A CSV file is parsed. The CSV file looks like this:
123,[Foo:true; Bar:true; Foobar:false; Barfoo:false]
456,[Foobar:true; Barfoo:false; Foo:false; Bar:false]
789,[Foobar:true; Barfoo:false; Foo:false]
where 123, 546 and 789 are unique identifiers of each datastructure, one datastructure is represented by the column identifiers Foo Bar Foobar and Barfoo, where each boolean indicates, if the column is a key-colum.
While the CSV file is parsed, for each line a datastructure must be created.
Later, in runtime, wich needs to be fast, the following will happen:
An ArrayList<String> containing column data is given. Data needs to be added to a specific datastructure. (I do now the unique identifier 123).
Say: the ArrayList<String> needs to be added to 123: 1-> foo, 2->bar, 3-> foobar, 4 ->barfoo.
Say: another ArrayList<String> needs to be added to 456: 1-> foobar, 2-> barfoo
Say: another ArrayList<String> needs to be added to 789: 1-> foobar, 2-> null, 3-> foo
The tasks that the datastructure needs to provide are the following:
add(ArrayList<String>) : void
remove(ArrayList<String>) : boolean (if successfull)
contains(Arraylist<String>) : boolean
get(ArrayList<String>) : ArrayList<String>
Notes:
The combination of all keys inside of one datastructure 123 are unique. Meaning: If in 123 is one entry with foo,bar,foobar, barfoo, Another enrty with foo,bar,doesnt matter, neither will not be allowed. Another enrty with foo1,bar, foobar,barfoo is allowed, as well as an entry with foo1,bar1,foobar, barfoo is also allowed.
It won't happen, that wile parsing, a column name not beeing a key (true), is in front of a key. This will not happen:
[Foobar:true;Barfoo:false;Foo:true;Bar:false]
It won't happen, at runtime, that a column marked as a key will not get data: This will not happen: an ArrayList<String> added to 123 with data looks like this: 1->foo, 2->null, 3->foobar.
I tried: storing at each datastructure-class two arrays. One with the Column Names, and one with Numbers of the Columns, which are keys. At runtime the key-indicating array will be processed to get all key values (at the first add example above it would be foo and bar) and they will be concatenated. (to a String "foo,bar"). This is a new key for a second HashMap<String,ArrayList<String>> where the value (ArrayList<String>) contains the data of all columns (foo,bar,foobar,barfoo).
I have a getKeyString method:
String getKeyString(ArrayList<String> keys, ArrayList<Integer> keyPos){
if (keyPos.get(keyPos.size()-1) >= keys.size()) //if the last entry from orders arraylist keyPos is greater than size of keys
throw new Exception();
String collect = keyPos.stream().map(i -> keys.get(i))
.map(string ->{
try{
if(string.equals("null")) // happens not very often, ~1time in 1,000
return "";
}
catch(NullPointerException e) { //happens even less 1 in 100,000
return "";
}
return string;
})
.collect(Collectors.joining(","));
if(collect.length()<keyPos.size())
throw new Exception("results in an empty key: ");
return collect;
and the addDataListEnty looks quite similar to this:
HashMap<String, ArrayList<String>> dataLists = new HashMap<>();
ArrayList<Integer> keyPos = new ArrayList<>();
...
public void addDataListEntry(ArrayList<String> values) {
// will overwrite Entry if it already exists
try {
this.dataLists.put(getKeyString(values, keyPos), values);
} catch (Exception e) {
logger.info(e.getMessage());
}
}
This does actually work, but is really slow, since the key (foo,bar) needs to created at every datastructure-access.
Which combination of HashMaps, Lists, Sets, (even Google Guava) is the best to make it as fast as possible?

Related

Retrieve String from values in HashMap at a specific occurrence of special character

So I'm trying retrieve specific substrings in values in a Hashmap constructed like this..
HashMap<ID, "Home > Recipe > Main Dish > Chicken > Chicken Breasts">
Which is passed from a different method that returns a HashMap
In above example, I need to retrieve Chicken.
Thus far, I have..
public static ArrayList<String> generalize() {
HashMap<String, String> items = new HashMap<>();
ArrayList<String> cats = new ArrayList<>();
items = RecSys.readInItemProfile("PATH", 0, 1);
for(String w : items.values()) {
cats.add(w);
}
for(String w : cats) {
int e = w.indexOf('>', 1 + w.indexOf('>', 1 + w.indexOf('>')));
String k = w.substring(e+1);
System.out.print(k);
e = 0;
}
System.out.println("k" + cats);
return cats;
}
Where I try to nullify String e for each iteration (I know it's redundant but it was just to test).
In my dataset, the first k-v pair is
3880=Home  >  Recipes  >  Main Dish  >  Pasta,
My output is
Pasta
Which is ok. If there are more than 3x ">", it'll return all following categories. Optimally it wouldn't do that, but it's ok if it does. However, further down the line, it (seemingly) randomly returns
Home > Recipe
Along with the rest of the data...
This happens at the 6th loop, I believe.
Any help is greatly appreciated..
Edit:
To clarify, I have a .csv file containing 3 columns, whereas 2 are used in this function (ID and Category). These are passed to this function by a read method in another class.
What I need to do is extract a generalized description of each category, which in all cases is the third instance of category specification (that is, always between the third and fourth ">" in every k-v pair).
My idea was to simply put all values in an arraylist, and for every value extract a string from between the third and fourth ">".
I recommend using the following map:
Map<Integer, List> map = new HashMap<>();
String[] vals = new String[] { "HomeRecipe", "Main Dish", "Chicken",
"Chicken Breasts" };
map.put(1, Arrays.asList(vals));
Then, if you need to find a given value in your original string using an ID, you can simply call ArrayList#get() at a certain position. If you don't care at all about order, then a map of integers to sets might make more sense here.
If you can. change your data structure to a HashMap<Integer, List<String>> or HashMap<Integer, String[]>. It's better to store the categories (by cats you mean categories right?) in a collection instead of a string.
Then you can easily get the third item.
If this is not possible. You need to do some debugging. Start by printing every input and output pair and find out which input caused the unexpected output. Your indexOf method seems to work at first glance.
Alternatively, try this regex method:
String k = cats.replaceAll("(?:[^>]+\\s*>\\s*){3}([^>]+).*", "$1");
System.out.println(k);
The regex basically looks for a xxx > yyy > zzz > aaa ... pattern and replaces that pattern with aaa (whatever that is in the original string).

Comparing Keys in a Hashmap

I have a test.csv file that is formatted as:
Home,Owner,Lat,Long
5th Street,John,5.6765,-6.56464564
7th Street,Bob,7.75,-4.4534564
9th Street,Kyle,4.64,-9.566467364
10th Street,Jim,14.234,-2.5667564
I have a hashmap that reads a file that contains the same header contents such as the CSV, just a different format, with no accompanying data.
In example:
Map<Integer, String> container = new HashMap<>();
where,
Key, Value
[0][NULL]
[1][Owner]
[2][Lat]
[3][NULL]
I have also created a second hash map that:
BufferedReader reader = new BufferedReader (new FileReader("test.csv"));
CSVParser parser = new CSVParser(reader, CSVFormat.DEFAULT);
Boolean headerParsed = false;
CSVRecord headerRecord = null;
int i;
Map<String,String> value = new HashMap<>();
for (final CSVRecord record : parser) {
if (!headerParsed = false) {
headerRecord = record;
headerParsed = true;
}
for (i =0; i< record.size(); i++) {
value.put (headerRecord.get(0), record.get(0));
}
}
I want to read and compare the hashmap, if the container map has a value that is in the value map, then I put that value in to a corresponding object.
example object
public DataSet (//args) {
this.home
this.owner
this.lat
this.longitude
}
I want to create a function where the data is set inside the object when the hashmaps are compared and when a value map key is equal to a contain map key, and the value is placed is set into the object. Something really simply that is efficient at handling the setting as well.
Please note: I made the CSV header and the rows finite, in real life, the CSV could have x number of fields(Home,Owner,Lat,Long,houseType,houseColor, ect..), and a n number of values associated to those fields
First off, your approach to this problem is too unnecessarily long. From what I see, all you are trying to do is this:
Select a two columns from a CSV file, and add them to a data structure. I highlighted the word two because in a map, you have a key and a value. One column becomes the key, and the other becomes the value.
What you should do instead:
Import the names of columns you wish to add to the data structure into two strings. (You may read them from a file).
Iterate over the CSV file using the CSVParser class that you did.
Store the value corresponding to the first desired column in a string, repeat with the value corresponding to the second desired column, and push them both into a DataSet object, and push the DataSet object into a List<DataSet>.
If you prefer to stick to your way of solving the problem:
Basically, the empty file is supposed to hold just the headers (column names), and that's why you named the corresponding hash map containers. The second file is supposed to contain the values and hence you named the corresponding hash map values.
First off, where you say
if (!headerParsed = false) {
headerRecord = record;
headerParsed = true;
}
you probably mean to say
if (!headerParsed) {
headerRecord = record;
headerParsed = true;
}
and where you say
for (i =0; i< record.size(); i++) {
value.put(headerRecord.get(0), record.get(0));
}
you probably mean
for (i =0; i< record.size(); i++) {
value.put(headerRecord.get(i), record.get(i));
}
i.e. You iterate over one record and store the value corresponding to each column.
Now I haven't tried this code on my desktop, but since the for loop also iterates over Home and Longitude, I think it should create an error and you should add an extra check before calling value.put (i.e. value.put("Home", "5th Street") should create an error I suppose). Wrap it inside an if conditional and check of the headerRecord(i) even exists in the containers hash map.
for (i =0; i< record.size(); i++) {
if (container[headerRecord.get(i)] != NULL) {
value.put(headerRecord.get(i), record.get(i));
}
}
Now thing is, that the data structure itself depends on which values from the containers hash map you want to store. It could be Home and Lat, or Owner and Long. So we are stuck. How about you create a data structure like below:
struct DataSet {
string val1;
string val2;
}
Also, note that this DataSet is only for storing ONE row. For storing information from multiple rows, you need to create a Linked List of DataSet.
Lastly, the container file contains ALL the column names. Not all these columns will be stored in the Data Set (i.e. You chose to NULL Home and Long. You could have chosen to NULL Owner and Lat), hence the header file is not what you need to make this decision.
If you think about it, just iterate over the values hash map and store the first value in string val1 and the second value in val2.
List<DataSet> myList;
DataSet row;
Iterator it = values.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pair = (Map.Entry)it.next();
row.val1 = pair.getKey();
row.val2 = pair.getValue();
myList.add(row);
it.remove();
}
I hope this helps.

My arraylist is only outputting the last value

I created a HashMap to store a text file with the columns of information. I compared the key to a specific name and stored the values of the HashMap into an ArrayList. When I try to println my ArrayList, it only outputs the last value and leaves out all the other values that match that key.
This isn't my entire code just my two loops that read in the text file, stores into the HashMap and then into the ArrayList. I know it has something to do with my loops.
Did some editing and got it to output, but all my values are displayed multiple times.
My output looks like this.
North America:
[ Anguilla, Anguilla, Antigua and Barbuda, Antigua and Barbuda, Antigua and Barbuda, Aruba, Aruba, Aruba,
HashMap<String, String> both = new HashMap<String, String>();
ArrayList<String> sort = new ArrayList<String>();
//ArrayList<String> sort2 = new ArrayList<String>();
// We need a try catch block so we can handle any potential IO errors
try {
try {
inputStream = new BufferedReader(new FileReader(filePath));
String lineContent = null;
// Loop will iterate over each line within the file.
// It will stop when no new lines are found.
while ((lineContent = inputStream.readLine()) != null) {
String column[]= lineContent.split(",");
both.put(column[0], column[1]);
Set set = both.entrySet();
//Get an iterator
Iterator i = set.iterator();
// Display elements
while(i.hasNext()) {
Map.Entry me = (Map.Entry)i.next();
if(me.getKey().equals("North America"))
{
String value= (String) me.getValue();
sort.add(value);
}
}
}
System.out.println("North America:");
System.out.println(sort);
System.out.println("\n");
}
Map keys need to be unique. Your code is working according to spec.
if you need to have many values for a key, you may use
Map<key,List<T>>
here T is String (not only list you can use any collection)
Some things seems wrong with your code :
you are iterating on the Map EntrySet to get just one value (you could just use the following code :
if (both.containsKey("North America"))
sort.add(both.get("North America"));
it seems that you can have "North America" more than one time in your input file, but you are storing it in a Map, so each time you store a new value for "North America" in your Map, it will overwrite the current value
I don't know what the type of sort is, but what is printed by System.out.print(sort); is dependent of the toString() implementation of this type, and the fact that you use print() instead of println() may also create problems depending on how you run your program (some shells may not print the last value for instance).
If you want more help, you may want to provide us with the following things :
sample of the input file
declaration of sort
sample of output
what you want to obtain.

How to "join" Hashtables in Java?

I have two strings:
A { 1,2,3,4,5,6 }
B { 6,7,8,9,10,11 }
it doesnt really matter what the numbers are in the strings. So then the user is going to pick what to join:
A hashjoin A.a1 = B.b5 B
I think I put the A into a hashtable by the A.a1 as the key and then iterate through B? The keys will be what the user wants then to join on and the data will be whats in the strings.
Are you sure you're trying to join hashtables? Perhaps you have the wrong data structure?
Look into java.util.Set (and java.util.HashSet). If you want the items that are in both tables, then it's a simple Set operation like so:
Collection A = new ...
...fill the A up...
Collection B = new ...
...fill the B up...
Set join = new HashSet();
join.addAll(A);
join.retainAll(B);
If you mean something more like a SQL table join, then the output will depend on what type of join you mean to perform, and what the equals sign means in this case. Note you'll have to write a Pair class (which you should make more descriptive than Pair for your exact case)
For a full join:
ArrayList pairs = new ArrayList();
for (Number numberA : A) {
for (Number numberB : B) {
pairs.add(new Pair(numberA, numberB));
}
}
For a full join with a where clause:
ArrayList pairs = new ArrayList();
for (Number numberA : A) {
for (Number numberB : B) {
if (check the condition of the where clause here) {
pairs.add(new Pair(numberA, numberB));
}
}
}
That's about as good an answer that can be given under the circumstances, as your question isn't very specific. If these general answers don't help you out, then you'll need to explain your question in more detail to get a more detailed answer.
--- First Edit, after some clarification ---
Ok, so it's an SQL-like equi-join.
Hashtables are Maps, which means they have an element in one "domain" which can be used to look up an element in another "domain". In a hash table, the first domain is the set of keys, and the second domain is the set of values. Think of it as a bunch of labels and a bunch of items. If the equi-join is to be performed, it must join like elements. That means it will either join one key to another key, or it will join one value to another value.
For keys:
Hashtable A = ...
Hashtable B = ...
Set keyJoin = new HashSet();
keyJoin.addAll(A.keySet());
keyJoin.retainAll(B.keySet());
For values:
Hashtable A = ...
Hashtable B = ...
Set valueJoin = new HashSet();
valueJoin.addAll(A.values());
valueJoin.retainAll(B.values());
It doesn't make sense to join the hashtables themselves; because, one "matching" value may live in both hashtables but be referenced by two different keys. Likewise, one "matching" key found in two different hashtables might not refer to the same value.
Your question doesn't make much sense. A hashtable (or hashmap), stores data as keys and values. You've said nothing about which of those values should be keys, and which should be values.

Sorting of 2 or more massive resultsets?

I need to be able to sort multiple intermediate result sets and enter them to a file in sorted order. Sort is based on a single column/key value. Each result set record will be list of values (like a record in a table)
The intermediate result sets are got by querying entirely different databases.
The intermediate result sets are already sorted based on some key(or column). They need to be combined and sorted again on the same key(or column) before writing it to a file.
Since these result sets can be massive(order of MBs) this cannot be done in memory.
My Solution broadly :
To use a hash and a random access file . Since the result sets are already sorted, when retrieving the result sets , I will store the sorted column values as keys in a hashmap.The value in the hashmap will be a address in the random access file where every record associated with that column value will be stored.
Any ideas ?
Have a pointer into every set, initially pointing to the first entry
Then choose the next result from the set, that offers the lowest entry
Write this entry to the file and increment the corresponding pointer
This approach has basically no overhead and time is O(n). (it's Merge-Sort, btw)
Edit
To clarify: It's the merge part of merge sort.
If you've got 2 pre-sorted result sets, you should be able to iterate them concurrently while writing the output file. You just need to compare the current row in each set:
Simple example (not ready for copy-and-paste use!):
ResultSet a,b;
//fetch a and b
a.first();
b.first();
while (!a.isAfterLast() || !b.isAfterLast()) {
Integer valueA = null;
Integer valueB = null;
if (a.isAfterLast()) {
writeToFile(b);
b.next();
}
else if (b.isAfterLast()) {
writeToFile(a);
a.next();
} else {
int valueA = a.getInt("SORT_PROPERTY");
int valueB = b.getInt("SORT_PROPERTY");
if (valueA < valueB) {
writeToFile(a);
a.next();
} else {
writeToFile(b);
b.next();
}
}
}
Sounds like you are looking for an implementation of the Balance Line algorithm.

Categories