We Keep Coding

Find duplicates in first column and take average based on third column

My issue here is I need to compute average time for each Id and compute average time of each id.
Sample data
T1,2020-01-16,11:16pm,start
T2,2020-01-16,11:18pm,start
T1,2020-01-16,11:20pm,end
T2,2020-01-16,11:23pm,end
I have written a code in such a way that I kept first column and third column in a map.. something like
T1, 11:16pm
but I could not able to compute values after keeping those values in a map. Also tried to keep them in string array and split into line by line. By same issue facing for that approach also.
**
public class AverageTimeGenerate {
public static void main(String[] args) throws IOException {
File file = new File("/abc.txt");
try (BufferedReader reader = new BufferedReader(new FileReader(file))) {
while (true) {
String line = reader.readLine();
if (line == null) {
break;
}
ArrayList<String> list = new ArrayList<>();
String[] tokens = line.split(",");
for (String s: tokens) {
list.add(s);
}
Map<String, String> map = new HashMap<>();
String[] data = line.split(",");
String ids= data[0];
String dates = data[1];
String transactionTime = data[2];
String transactionStartAndEndTime = data[3];
String[] transactionIds = ids.split("/n");
String[] timeOfEachTransaction = transactionTime.split("/n");
for(String id : transactionIds) {
for(String time : timeOfEachTransaction) {
map.put(id, time);
}
}
}
}
}
}
Can anyone suggest me is it possible to find duplicates in a map and compute values in map, Or is there any other way I can do this so that the output should be like
`T1 2:00
T2 5:00'

I don't know what is your logic to complete the average time but you can save data in map for one particular transaction. The map structure can be like this. Transaction id will be the key and all the time will be in array list.
Map<String,List<String>> map = new HashMap<String,List<String>>();

You can do like this:
Map<String, String> result = Files.lines(Paths.get("abc.txt"))
.map(line -> line.split(","))
.map(arr -> {
try {
return new AbstractMap.SimpleEntry<>(arr[0],
new SimpleDateFormat("HH:mm").parse(arr[2]));
} catch (ParseException e) {
return null;
}
}).collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.collectingAndThen(Collectors
.mapping(Map.Entry::getValue, Collectors.toList()),
list -> toStringTime.apply(convert.apply(list)))));
for simplify I've declared two functions.
Function<List<Date>, Long> convert = list -> (list.get(1).getTime() - list.get(0).getTime()) / 2;
Function<Long, String> toStringTime = l -> l / 60000 + ":" + l % 60000 / 1000;

how to avoid duplicates and count the list once stored in LinkedHashMap

I have a scenario to count number of names which is stored in the LinkedHashMap , and names can be duplicated but i should not count the duplicate name.
Below is Sample code:
LinkedHashMap<Long,MyApplicationDTO> myApps = (LinkedHashMap<Long,MyApplicationDTO>) request.getAttribute("data");
for (Map.Entry app : myApps.entrySet()) {
Long ID = (Long)app.getKey() ;
MyApplicationDTO singleMyApp = (MyApplicationDTO) app.getValue();
LinkedHashMap<Long, MyDTO> myList = singleMyApp.getMyList();
String name = "";
for (Map.Entry details : myList.entrySet()) {
Long id1 = (Long)details.getKey() ;
MyDTO myDetails = (MyDTO) details.getValue();
name = myDetails.getName(); // For first time it stores A
//how to loop so that i can only get the count of names as 3 by avoiding duplicate names from the below shown list.
//A B A B A B C
}
}
On the Screen i have something as below:
Name :
A
B
A
B
A
B
C
I have to print the count of the name as 3(non repeating names)

As you iterate over the entrySet, add all names to a Set<String>. Then output set.size().
The Set will not add duplicates when you add names by set.add(name), so the size of the set will be the count of uniqe names.

LinkedHashMap<Long,MyApplicationDTO> myApps = (LinkedHashMap<Long,MyApplicationDTO>) request.getAttribute("data");
for (Map.Entry app : myApps.entrySet()) {
Long ID = (Long)app.getKey() ;
MyApplicationDTO singleMyApp = (MyApplicationDTO) app.getValue();
LinkedHashMap<Long, MyDTO> myList = singleMyApp.getMyList();
String name = "";
Set<String> uniqueNames = new HashSet<String>();
for (Map.Entry details : myList.entrySet()) {
Long id1 = (Long)details.getKey() ;
MyDTO myDetails = (MyDTO) details.getValue();
name = myDetails.getName(); // For first time it stores A
//how to loop so that i can only get the count of names as 3 by avoiding duplicate names from the below shown list.
//A B A B A B C
uniqueNames.add(name);
}
}
To get size do =
uniqueNames.size();

I'm not sure I'm understanding your question entirely, but if you're just looking to count the occurrences of unique values in the LinkedHashmap you can do something like this `
public static void main(String[] args) {
LinkedHashMap<Long, String> myApps = new LinkedHashMap<Long, String>();
myApps.put(4L, "A");
myApps.put(14L, "B");
myApps.put(44L, "A");
myApps.put(54L, "B");
myApps.put(46L, "A");
myApps.put(543L, "B");
myApps.put(144L, "C");
ArrayList<String> names = new ArrayList<String>();
for (Map.Entry app : myApps.entrySet()) {
if (!(names.contains(app.getValue()))) {
names.add(app.getValue().toString());
}
}
System.out.println(names.size());
for (String s : names ) {
System.out.print(s + " ");
}
}

One liner with JAVA 8 Stream API
LinkedHashMap<Long, MyDTO> myList = singleMyApp.getMyList();
long disctinctNamesCount = myList.values().stream().map(MyDTO::getNAme).distinct().count();

how to avoid ConcurrentHashMap usage

I have written this code inside the run() method of the Reducer class in Hadoop
#Override
public void run(Context context) throws IOException, InterruptedException {
setup(context);
ConcurrentHashMap<String, HashSet<Text>> map = new ConcurrentHashMap<String, HashSet<Text>>();
while (context.nextKey()) {
String line = context.getCurrentKey().toString();
HashSet<Text> values = new HashSet<Text>();
for (Text t : context.getValues()) {
values.add(new Text(t));
}
map.put(line, new HashSet<Text>());
for (Text t : values) {
map.get(line).add(new Text(t));
}
}
ConcurrentHashMap<String, HashSet<Text>> newMap = new ConcurrentHashMap<String, HashSet<Text>>();
for (String keyToMerge : map.keySet()) {
String[] keyToMergeTokens = keyToMerge.split(",");
for (String key : map.keySet()) {
String[] keyTokens = key.split(",");
if (keyToMergeTokens[keyToMergeTokens.length - 1].equals(keyTokens[0])) {
String newKey = keyToMerge;
for (int i = 1; i < keyTokens.length; i++) {
newKey += "," + keyTokens[i];
}
if (!newMap.contains(newKey)) {
newMap.put(newKey, new HashSet<Text>());
for (Text t : map.get(keyToMerge)) {
newMap.get(newKey).add(new Text(t));
}
}
for (Text t : map.get(key)) {
newMap.get(newKey).add(new Text(t));
}
}
}
//call the reducers
for (String key : newMap.keySet()) {
reduce(new Text(key), newMap.get(key), context);
}
cleanup(context);
}
my problem is that even if my input is too small it takes 30 minutes to run epsecially because of the newMap.put() call. If I put this command in comments then it runs quickly without any problems.
As you can see I use ConcurrentHashMap. I didn't want to use it because I think that run() is called only once at each machine (it doesn't run concurrently) so I would not have any problem with a simple HashMap but if I replace the concurrentHashMap with the simple HashMap I am getting an error (concurrentModificationError).
Does anyone have an idea about how to make it work without any delays ?
thanks in advance!
*java6
*hadoop 1.2.1

I don't know if it would solve your performance problems, but I see one inefficient thing you are doing :
newMap.put(newKey, new HashSet<Text>());
for (Text t : map.get(keyToMerge)) {
newMap.get(newKey).add(new Text(t));
}
It would be more efficient to keep the HashSet in a variable instead of searching for it in newMap :
HashSet<Text> newSet = new HashSet<Text>();
newMap.put(newKey, newSet);
for (Text t : map.get(keyToMerge)) {
newSet.add(new Text(t));
}
Another inefficient thing you are doing is create a HashSet of values and then create another identical HashSet to put in the map. Since the original HashSet (values) is never used again, you are constructing all those Text objects for no reason at all.
Instead of:
while (context.nextKey()) {
String line = context.getCurrentKey().toString();
HashSet<Text> values = new HashSet<Text>();
for (Text t : context.getValues()) {
values.add(new Text(t));
}
map.put(line, new HashSet<Text>());
for (Text t : values) {
map.get(line).add(new Text(t));
}
}
You can simply write :
while (context.nextKey()) {
String line = context.getCurrentKey().toString();
HashSet<Text> values = new HashSet<Text>();
for (Text t : context.getValues()) {
values.add(new Text(t));
}
map.put(line, values);
}
EDIT :
I just saw the additional code you posted as an answer (from your cleanup() method) :
//clear map
for (String s : map.keySet()) {
map.remove(s);
}
map = null;
//clear newMap
for (String s : newMap.keySet()) {
newMap.remove(s);
}
newMap = null;
The reason this code gives you ConcurrentModificationError is that foreach loops don't support modification of the collection you are iterating over.
To overcome this, you can use an Iterator :
//clear map
Iterator<Map.Entry<String, HashSet<Text>>> iter1 = map.entrySet ().iterator ();
while (iter1.hasNext()) {
Map.Entry<String, HashSet<Text>> entry = iter1.next();
iter1.remove();
}
map = null;
//clear newMap
Iterator<Map.Entry<String, HashSet<Text>>> iter2 = newMap.entrySet ().iterator ();
while (iter2.hasNext()) {
Map.Entry<String, HashSet<Text>> entry = iter2.next();
iter2.remove();
}
newMap = null;
That said, you don't really have to remove each item separately.
You can simply write
map = null;
newMap = null;
When you remove the reference to the maps, the garbage collector can garbage collect them. Removing the items from the maps makes no difference.

Is there a convenient way to convert comma separated string to hashmap

String format is (not json format)：
a="0PN5J17HBGZHT7JJ3X82", b="frJIUN8DYpKDtOLCwo/yzg="
I want convert this string to a HashMap:
key a with value 0PN5J17HBGZHT7JJ3X82
key b with value frJIUN8DYpKDtOLCwo/yzg=
Is there a convenient way? Thanks
What I've tried:
Map<String, String> map = new HashMap<String, String>();
String s = "a=\"00PN5J17HBGZHT7JJ3X82\",b=\"frJIUN8DYpKDtOLCwo/yzg=\"";
String []tmp = StringUtils.split(s,',');
for (String v : tmp) {
String[] t = StringUtils.split(v,'=');
map.put(t[0], t[1]);
}
I get this result:
key a with value "0PN5J17HBGZHT7JJ3X82"
key b with value "frJIUN8DYpKDtOLCwo/yzg
for key a, the start and end double quotation marks(") is unwanted; for key b, the start double quotation marks(") is unwanted and the last equals sign(=) is missing.
Sorry for my poor english.

Probably you don't care that it's a HashMap, just a Map, so this will do it, since Properties implements Map:
import java.io.StringReader;
import java.util.*;
public class Strings {
public static void main(String[] args) throws Exception {
String input = "a=\"0PN5J17HBGZHT7JJ3X82\", b=\"frJIUN8DYpKDtOLCwo/yzg=\"";
String propertiesFormat = input.replaceAll(",", "\n");
Properties properties = new Properties();
properties.load(new StringReader(propertiesFormat));
System.out.println(properties);
}
}
Output:
{b="frJIUN8DYpKDtOLCwo/yzg=", a="0PN5J17HBGZHT7JJ3X82"}
If you absolutely need a HashMap, you can construct one with the Properties object as input: new HashMap(properties).

Added few changes in Ryan's code
public static void main(String[] args) throws Exception {
String input = "a=\"0PN5J17HBGZHT7JJ3X82\", b=\"frJIUN8DYpKDtOLCwo/yzg=\"";
input=input.replaceAll("\"", "");
String propertiesFormat = input.replaceAll(",", "\n");
Properties properties = new Properties();
properties.load(new StringReader(propertiesFormat));
Set<Entry<Object, Object>> entrySet = properties.entrySet();
HashMap<String,String > map = new HashMap<String, String>();
for (Iterator<Entry<Object, Object>> it = entrySet.iterator(); it.hasNext();) {
Entry<Object,Object> entry = it.next();
map.put((String)entry.getKey(), (String)entry.getValue());
}
System.out.println(map);
}

Split the String on the Basis of commas (",") and then with with ("=")
String s = "Comma Separated String";
HashMap<String, String> map = new HashMap<String, String>();
String[] arr = s.split(",");
String[] arStr = arr.split("=");
map.put(arr[0], arr[1]);

You can also use the regex as below.
Map<String,String> data = new HashMap<String,String>();
Pattern p = Pattern.compile("[\\{\\}\\=\\, ]++");
String[] split = p.split(text);
for ( int i=0; i+2 <= split.length; i+=2 ){
data.put( split[i], split[i+1] );
}
return data;

How to sort a string into a map and print the results

I have a string in the format nm=Alan&hei=72&hair=brown
I would like to split this information up, add a conversion to the first value and print the results in the format
nm Name Alan
hei Height 72
hair Hair Color brown
I've looked at various methods using the split function and hashmaps but have had no luck piecing it all together.
Any advice would be very useful to me.

Map<String, String> aliases = new HashMap<String, String>();
aliases.put("nm", "Name");
aliases.put("hei", "Height");
aliases.put("hair", "Hair Color");
String[] params = str.split("&"); // gives you string array: nm=Alan, hei=72, hair=brown
for (String p : params) {
String[] nv = p.split("=");
String name = nv[0];
String value = nv[1];
System.out.println(nv[0] + " " + aliases.get(nv[0]) + " " + nv[1]);
}
I really do not understand what you problem was...

Try something like this:
static final String DELIMETER = "&"
Map<String,String> map = ...
map.put("nm","Name");
map.put("hei","Height");
map.put("hair","Hair color");
StringBuilder builder = new StringBuilder();
String input = "nm=Alan&hei=72&hair=brown"
String[] splitted = input.split(DELIMETER);
for(Stirng str : splitted){
int index = str.indexOf("=");
String key = str.substring(0,index);
builder.append(key);
builder.append(map.get(key));
builder.append(str.substring(index));
builder.append("\n");
}

A HashMap consists of many key, value pairs. So when you use split, devise an appropriate regex (&). Once you have your string array, you can use one of the elements as the key (think about which element will make the best key). However, you may now be wondering- "how do I place the rest of elements as the values?". Perhaps you can create a new class which stores the rest of the elements and use objects of this class as values for the hashmap.
Then printing becomes easy- merely search for the value of the corresponding key. This value will be an object; use the appropriate method on this object to retrieve the elements and you should be able to print everything.
Also, remember to handle exceptions in your code. e.g. check for nulls, etc.
Another thing: your qn mentions the word "sort". I don't fully get what that means in this context...

Map<String, String> propsMap = new HashMap<String, String>();
Map<String, String> propAlias = new HashMap<String, String>();
propAlias.put("nm", "Name");
propAlias.put("hei", "Height");
propAlias.put("hair", "Hair Color");
String[] props = input.split("&");
if (props != null && props.length > 0) {
for (String prop : props) {
String[] propVal = prop.split("=");
if (propVal != null && propVal.length == 2) {
propsMap.put(propVal[0], propVal[1]);
}
}
}
for (Map.Entry tuple : propsMap.getEntrySet()) {
if (propAlias.containsKey(tuple.getKey())) {
System.out.println(tuple.getKey() + " " + propAlias.get(tuple.getKey()) + " " + tuple.getValue());
}
}