We Keep Coding

Find duplicates in first column and take average based on third column

My issue here is I need to compute average time for each Id and compute average time of each id.
Sample data
T1,2020-01-16,11:16pm,start
T2,2020-01-16,11:18pm,start
T1,2020-01-16,11:20pm,end
T2,2020-01-16,11:23pm,end
I have written a code in such a way that I kept first column and third column in a map.. something like
T1, 11:16pm
but I could not able to compute values after keeping those values in a map. Also tried to keep them in string array and split into line by line. By same issue facing for that approach also.
**
public class AverageTimeGenerate {
public static void main(String[] args) throws IOException {
File file = new File("/abc.txt");
try (BufferedReader reader = new BufferedReader(new FileReader(file))) {
while (true) {
String line = reader.readLine();
if (line == null) {
break;
}
ArrayList<String> list = new ArrayList<>();
String[] tokens = line.split(",");
for (String s: tokens) {
list.add(s);
}
Map<String, String> map = new HashMap<>();
String[] data = line.split(",");
String ids= data[0];
String dates = data[1];
String transactionTime = data[2];
String transactionStartAndEndTime = data[3];
String[] transactionIds = ids.split("/n");
String[] timeOfEachTransaction = transactionTime.split("/n");
for(String id : transactionIds) {
for(String time : timeOfEachTransaction) {
map.put(id, time);
}
}
}
}
}
}
Can anyone suggest me is it possible to find duplicates in a map and compute values in map, Or is there any other way I can do this so that the output should be like
`T1 2:00
T2 5:00'

I don't know what is your logic to complete the average time but you can save data in map for one particular transaction. The map structure can be like this. Transaction id will be the key and all the time will be in array list.
Map<String,List<String>> map = new HashMap<String,List<String>>();

You can do like this:
Map<String, String> result = Files.lines(Paths.get("abc.txt"))
.map(line -> line.split(","))
.map(arr -> {
try {
return new AbstractMap.SimpleEntry<>(arr[0],
new SimpleDateFormat("HH:mm").parse(arr[2]));
} catch (ParseException e) {
return null;
}
}).collect(Collectors.groupingBy(Map.Entry::getKey,
Collectors.collectingAndThen(Collectors
.mapping(Map.Entry::getValue, Collectors.toList()),
list -> toStringTime.apply(convert.apply(list)))));
for simplify I've declared two functions.
Function<List<Date>, Long> convert = list -> (list.get(1).getTime() - list.get(0).getTime()) / 2;
Function<Long, String> toStringTime = l -> l / 60000 + ":" + l % 60000 / 1000;

function needs to be void with if, else if, else? hash map, java

OK, very odd. Not come across this before from what I can remember.
OK, so the compiler is telling me that the following method should be void for some reason:
public static HashMap<String, ArrayList<FlightData>> mapper(ArrayList<String> lineBuffer) {
HashMap<String, ArrayList<FlightData>> mapdata = new HashMap<>(); //array list for Mapdata object
for (String flightData : lineBuffer) {
String[] str = flightData.split(",");
FlightData flight = new FlightData(str[0], str[1], str[2].toCharArray(), str[3].toCharArray(), new Date(Long.valueOf(str[4])), Long.valueOf(str[5]).longValue()); //creating the object
mapdata.get(flight.getFlightID()); //getting the flight data
if (mapdata.containsKey(flight.getFlightID())) { //checking if the data for the oject contains hash key Flightdata
mapdata.get(flight.getFlightID()).add(flight);
}
else if (mapdata.containsKey(flight.getFromID())) {
mapdata.get(flight.getFromID()).add(flight);
ArrayList<FlightData> noID2 = new ArrayList<>(); //creating an array for noID
noID2.add(flight);
mapdata.put(flight.getFlightID(), noID2);
}
else {
ArrayList<FlightData> noID = new ArrayList<>(); //creating an array for noID
noID.add(flight);
mapdata.put(flight.getFlightID(), noID);
// System.out.println(mapdata);
}
return mapdata;
}
Which is odd, because when I remove the additional if (if else to just else) its fine:
public static HashMap<String, ArrayList<FlightData>> mapper(ArrayList<String> lineBuffer) {
HashMap<String, ArrayList<FlightData>> mapdata = new HashMap<>(); //array list for Mapdata object
for (String flightData : lineBuffer) {
String[] str = flightData.split(",");
FlightData flight = new FlightData(str[0], str[1], str[2].toCharArray(), str[3].toCharArray(), new Date(Long.valueOf(str[4])), Long.valueOf(str[5]).longValue()); //creating the object
mapdata.get(flight.getFlightID()); //getting the flight data
if (mapdata.containsKey(flight.getFlightID())) { //checking if the data for the oject contains hash key Flightdata
mapdata.get(flight.getFlightID()).add(flight);
} else {
ArrayList<FlightData> noID = new ArrayList<>(); //creating an array for noID
noID.add(flight);
mapdata.put(flight.getFlightID(), noID);
}
// System.out.println(mapdata);
}
return mapdata;
}
I get the following:
the error is telling my missing return statement/type. Hence void suggestion. any ideas why??
Any help would be great.
Cheers,
Glenn

Yeah, so I'm an idiot and didn't spot that my return statement was inside my for loop... Just wow.
Cheers to all who commented with a response.
markspace
ayush Gupta
ifly6
apparently I'm blind.
Cheers guys.
Glenn

How to Loop next element in hashmap

I have a set of strings like this
A_2007-04, A_2007-09, A_Agent, A_Daily, A_Execute, A_Exec, B_Action, B_HealthCheck
I want output as:
Key = A, Value = [2007-04,2007-09,Agent,Execute,Exec]
Key = B, Value = [Action,HealthCheck]
I'm using HashMap to do this
pckg:{A,B}
count:total no of strings
reports:set of strings
Logic I used is nested loop:
for (String l : reports[i]) {
for (String r : pckg) {
String[] g = l.split("_");
if (g[0].equalsIgnoreCase(r)) {
report.add(g[1]);
dirFiles.put(g[0], report);
} else {
break;
}
}
}
I'm getting output as
Key = A, Value = [2007-04,2007-09,Agent,Execute,Exec]
How to get second key?
Can someone suggest logic for this?

Assuming that you use Java 8, it can be done using computeIfAbsent to initialize the List of values when it is a new key as next:
List<String> tokens = Arrays.asList(
"A_2007-04", "A_2007-09", "A_Agent", "A_Daily", "A_Execute",
"A_Exec", "P_Action", "P_HealthCheck"
);
Map<String, List<String>> map = new HashMap<>();
for (String token : tokens) {
String[] g = token.split("_");
map.computeIfAbsent(g[0], key -> new ArrayList<>()).add(g[1]);
}

In terms of raw code this should do what I think you are trying to achieve:
// Create a collection of String any way you like, but for testing
// I've simply split a flat string into an array.
String flatString = "A_2007-04,A_2007-09,A_Agent,A_Daily,A_Execute,A_Exec,"
+ "P_Action,P_HealthCheck";
String[] reports = flatString.split(",");
Map<String, List<String>> mapFromReportKeyToValues = new HashMap<>();
for (String report : reports) {
int underscoreIndex = report.indexOf("_");
String key = report.substring(0, underscoreIndex);
String newValue = report.substring(underscoreIndex + 1);
List<String> existingValues = mapFromReportKeyToValues.get(key);
if (existingValues == null) {
// This key hasn't been seen before, so create a new list
// to contain values which belong under this key.
existingValues = new ArrayList<>();
mapFromReportKeyToValues.put(key, existingValues);
}
existingValues.add(newValue);
}
System.out.println("Generated map:\n" + mapFromReportKeyToValues);
Though I recommend tidying it up and organising it into a method or methods as fits your project code.

Doing this with Map<String, ArrayList<String>> will be another good approach I think:
String reports[] = {"A_2007-04", "A_2007-09", "A_Agent", "A_Daily",
"A_Execute", "A_Exec", "P_Action", "P_HealthCheck"};
Map<String, ArrayList<String>> map = new HashMap<>();
for (String rep : reports) {
String s[] = rep.split("_");
String prefix = s[0], suffix = s[1];
ArrayList<String> list = new ArrayList<>();
if (map.containsKey(prefix)) {
list = map.get(prefix);
}
list.add(suffix);
map.put(prefix, list);
}
// Print
for (Map.Entry<String, ArrayList<String>> entry : map.entrySet()) {
String key = entry.getKey();
ArrayList<String> valueList = entry.getValue();
System.out.println(key + " " + valueList);
}

for (String l : reports[i]) {
String[] g = l.split("_");
for (String r : pckg) {
if (g[0].equalsIgnoreCase(r)) {
report = dirFiles.get(g[0]);
if(report == null){ report = new ArrayList<String>(); } //create new report
report.add(g[1]);
dirFiles.put(g[0], report);
}
}
}
Removed the else part of the if condition. You are using break there which exits the inner loop and you never get to evaluate the keys beyond first key.
Added checking for existing values. As suggested by Orin2005.
Also I have moved the statement String[] g = l.split("_"); outside inner loop so that it doesn't get executed multiple times.

how to avoid ConcurrentHashMap usage

I have written this code inside the run() method of the Reducer class in Hadoop
#Override
public void run(Context context) throws IOException, InterruptedException {
setup(context);
ConcurrentHashMap<String, HashSet<Text>> map = new ConcurrentHashMap<String, HashSet<Text>>();
while (context.nextKey()) {
String line = context.getCurrentKey().toString();
HashSet<Text> values = new HashSet<Text>();
for (Text t : context.getValues()) {
values.add(new Text(t));
}
map.put(line, new HashSet<Text>());
for (Text t : values) {
map.get(line).add(new Text(t));
}
}
ConcurrentHashMap<String, HashSet<Text>> newMap = new ConcurrentHashMap<String, HashSet<Text>>();
for (String keyToMerge : map.keySet()) {
String[] keyToMergeTokens = keyToMerge.split(",");
for (String key : map.keySet()) {
String[] keyTokens = key.split(",");
if (keyToMergeTokens[keyToMergeTokens.length - 1].equals(keyTokens[0])) {
String newKey = keyToMerge;
for (int i = 1; i < keyTokens.length; i++) {
newKey += "," + keyTokens[i];
}
if (!newMap.contains(newKey)) {
newMap.put(newKey, new HashSet<Text>());
for (Text t : map.get(keyToMerge)) {
newMap.get(newKey).add(new Text(t));
}
}
for (Text t : map.get(key)) {
newMap.get(newKey).add(new Text(t));
}
}
}
//call the reducers
for (String key : newMap.keySet()) {
reduce(new Text(key), newMap.get(key), context);
}
cleanup(context);
}
my problem is that even if my input is too small it takes 30 minutes to run epsecially because of the newMap.put() call. If I put this command in comments then it runs quickly without any problems.
As you can see I use ConcurrentHashMap. I didn't want to use it because I think that run() is called only once at each machine (it doesn't run concurrently) so I would not have any problem with a simple HashMap but if I replace the concurrentHashMap with the simple HashMap I am getting an error (concurrentModificationError).
Does anyone have an idea about how to make it work without any delays ?
thanks in advance!
*java6
*hadoop 1.2.1

I don't know if it would solve your performance problems, but I see one inefficient thing you are doing :
newMap.put(newKey, new HashSet<Text>());
for (Text t : map.get(keyToMerge)) {
newMap.get(newKey).add(new Text(t));
}
It would be more efficient to keep the HashSet in a variable instead of searching for it in newMap :
HashSet<Text> newSet = new HashSet<Text>();
newMap.put(newKey, newSet);
for (Text t : map.get(keyToMerge)) {
newSet.add(new Text(t));
}
Another inefficient thing you are doing is create a HashSet of values and then create another identical HashSet to put in the map. Since the original HashSet (values) is never used again, you are constructing all those Text objects for no reason at all.
Instead of:
while (context.nextKey()) {
String line = context.getCurrentKey().toString();
HashSet<Text> values = new HashSet<Text>();
for (Text t : context.getValues()) {
values.add(new Text(t));
}
map.put(line, new HashSet<Text>());
for (Text t : values) {
map.get(line).add(new Text(t));
}
}
You can simply write :
while (context.nextKey()) {
String line = context.getCurrentKey().toString();
HashSet<Text> values = new HashSet<Text>();
for (Text t : context.getValues()) {
values.add(new Text(t));
}
map.put(line, values);
}
EDIT :
I just saw the additional code you posted as an answer (from your cleanup() method) :
//clear map
for (String s : map.keySet()) {
map.remove(s);
}
map = null;
//clear newMap
for (String s : newMap.keySet()) {
newMap.remove(s);
}
newMap = null;
The reason this code gives you ConcurrentModificationError is that foreach loops don't support modification of the collection you are iterating over.
To overcome this, you can use an Iterator :
//clear map
Iterator<Map.Entry<String, HashSet<Text>>> iter1 = map.entrySet ().iterator ();
while (iter1.hasNext()) {
Map.Entry<String, HashSet<Text>> entry = iter1.next();
iter1.remove();
}
map = null;
//clear newMap
Iterator<Map.Entry<String, HashSet<Text>>> iter2 = newMap.entrySet ().iterator ();
while (iter2.hasNext()) {
Map.Entry<String, HashSet<Text>> entry = iter2.next();
iter2.remove();
}
newMap = null;
That said, you don't really have to remove each item separately.
You can simply write
map = null;
newMap = null;
When you remove the reference to the maps, the garbage collector can garbage collect them. Removing the items from the maps makes no difference.

Searching String Array item in a nested HashMap

I have a nested HashMap (outer_map), which has another HashMap inside of it as a value (inner_map), such implemented as
Map<String, HashMap<String, String>> outer_map = new HashMap<String, HashMap<String, String>>();
Map<String, String> inner_map = new HashMap<String, String>();
The figure below illustrates the whole structures of the maps:
To make the long story short, I need to compare and search values inside the outer_map's vlaue (inner_map) by String Array items, and then produce another String Array to add matched items.
If the String Array has there elements which are the same as one of the inner_map's random (for example; value2, value1, and value7) values, how can I search and compare these items to add to another String Array?
The latest code snippet I tried and I couldn't succeed:
if( !( theStringArray.equals("") ) )
{
while( outer_map.keySet().iterator().hasNext() )
{
for( int i=0; i <= theStringArray.length; i++)
{
// outer_map keys are order as 1,2,3,..,8
theStringArray[i] = outer_map.get(String.valueOf(i+1)).get("key1");
...
}
}
}
EDIT: Map generating function
private void parse(String in) throws IOException
{
reader = new JsonReader(new StringReader(in));
...
int nodeCounter = 1;
while(reader.hasNext())
{
...
String nameAsKey1 = "blabla"; // value1
inner_map.put("name", nameAsKey1);
String surnameAsKey2 = "blabla"; // value2
inner_map.put("surname", surnameAsKey2);
...
outer_map.put(String.valueOf(nodeCounter), (HashMap<String, String>) inner_map);
inner_map = new HashMap<String, String>();
nodeCounter++;
}
}
EDIT: I don't know how I can explain the issue more clearly, but may be this will help to understand about it: Map structure

I assume you have an array of String and a Map of map. Now you want to search the value fields of inner map against the array of String and if make a new string array with matching values.
If that is the case the below program will help you..
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
public class InnerMapSearch {
public static void main(String[] args) {
Map<String, HashMap<String, String>> outer_map = new HashMap<String, HashMap<String, String>>();
Map<String, String> inner_map = new HashMap<String, String>();
String[] searchParams = {"blabla1", "blabla3", "blabla20"};
//Populating the map
int reader = 1;
while (reader < 10) {
String nameAsKey1 = "blabla" + reader; // value1
inner_map.put("name", nameAsKey1);
String surnameAsKey2 = "blabla" + reader; // value2
inner_map.put("surname", surnameAsKey2);
outer_map.put(String.valueOf(reader), (HashMap<String, String>) inner_map);
inner_map = new HashMap<String, String>();
reader++;
}
//Searching
Set<String> searchResults = new HashSet<String>(); // Using set to avoid duplicate
// Iterate over the outer map
for(String key : outer_map.keySet()){
// Iterate through each inner_map value of outer map
for(Entry<String, String> innerEntry : outer_map.get(key).entrySet()){
// Iterate through the list of search params and see if its present in inner_hashmap
for(String searchParam : searchParams){
if(searchParam.equals(innerEntry.getValue())){
// The search parameter is in inner map so adding to result.
searchResults.add(searchParam);
}
}
}
}
// Converting the list to an array.
String[] searchResultsArray = searchResults.toArray(new String[searchResults.size()]);
}
}