We Keep Coding

Most Effective Way to get matched and unmatched objects in 2 ArrayLists

I have a task to read 2 files and match the contents of the files and provide a list of unmatched entries of both files. That means I have to present how many matched entries in the two files and how many unmatched entries in file 1 which is not in file 2 , how many unmatched entries in file 2 which is not in file 1.
My apporach is reading the files , creating java objects out of it, putting the contents of 2 files to 2 separate arraylists and compare them. My current code is listed below. For clarification, I want to check the content of the object ( eg : check EmployeeID and match from both files).
In below code, I have matched file1 content with file2, and removed the matched contents from file2.Works fine to match entries and get the unmatched count of file1 compared to file2.
I am plannning to match the remaining items in file2 and go another round in the same compareByEmpIdandDOBmethod using fileTwoEmpList as first parameter and fileOneEmpList as second parameter get the unmatched count of file2 compared to file1. But I feel this is an overkill and not very efficient. Can someone point out a different approach if any pelase ?
Both of the arraylists are sorted. Thanks in advance !
public class EmpMatching {
public void compareLists(List<EmployeeDetails> fileOneEmpList, List<EmployeeDetails> fileTwoEmpList){
Collections.sort(fileOneEmpList);
Collections.sort(fileTwoEmpList);
List<EmployeeDetails> unmatchedFromListTwo = compareByEmpIdandDOB(fileOneEmpList,fileTwoEmpList);
}
public List<EmployeeDetails> compareByEmpIdandDOB(List<EmployeeDetails> fileOneEmpList,List<EmployeeDetails> fileTwoEmpList){
int matchEmpCountFromTwoFiles = 0;
System.out.println("File One List Size Before Recon " + fileTwoEmpList.size());
for(EmployeeDetails fileOneEmp : fileOneEmpList){
for(int index = 0;index < fileTwoEmpList.size();index++ ){
EmployeeDetails fileTwoEmp= fileTwoEmpList.get(index);
if(fileOneEmp.getEmpID().equals(fileTwoEmp.getEmpID()) && fileOneEmp.getEmpDOB().equals(fileTwoEmp.getEmpDOB())){
matchEmpCountFromTwoFiles++;
fileTwoEmpList.remove(fileTwoEmp);
System.out.println("Match Found " + fileOneEmp.getEmpID());
}
}
System.out.println("File Two List Size " + fileTwoEmpList.size());
}
System.out.println("Match Count >>>>> " + matchEmpCountFromTwoFiles);
System.out.println("File Two List Size >>>>> " + fileTwoEmpList.size());
return fileTwoEmpList;
}
}
//Model class
public class EmployeeDetails implements Comparable<EmployeeDetails>{
private String EmpID;
private String EmpName;
private String EmpDOB;
#Override
public int compareTo(EmployeeDetails o) {
return 0;
}
}

You don't need to sort these lists for this task.
In terms of the Set theory, you need to find the set difference. I.e. to find all unique objects that appear only in the first or in the second list.
This task can be solved in a few lines of code with liner time complexity. But it is important to implement the equals/hashCode contract in the EmployeeDetails.
public List<EmployeeDetails> compareLists(List<EmployeeDetails> fileOneEmpList,
List<EmployeeDetails> fileTwoEmpList) {
Set<EmployeeDetails> emp1 = new HashSet<>(fileOneEmpList);
Set<EmployeeDetails> emp2 = new HashSet<>(fileTwoEmpList);
emp1.removeAll(emp2);
emp2.removeAll(emp1);
emp1.addAll(emp2);
return new ArrayList<>(emp1);
}
The approach above is both the most efficient and the simplest.
If you are comfortable with Streams API, you can try another approach and implement this method in the following way:
public List<EmployeeDetails> compareLists(List<EmployeeDetails> fileOneEmpList,
List<EmployeeDetails> fileTwoEmpList) {
return Stream.of(new HashSet<>(fileOneEmpList), new HashSet<>(fileTwoEmpList)) // wrapping with sets to ensure uniqueness (if objects in the list are guaranteed to be unique - use lists instead)
.flatMap(Collection::stream)
.collect(Collectors.groupingBy(Function.identity(), Collectors.counting()))
.entrySet().stream()
.filter(entry -> entry.getValue() == 1) // i.e. object appear only once either in the first or in the second list
.map(Map.Entry::getKey)
.collect(Collectors.toList()); // .toList(); for Java 16+
}
Time complexity of the stream based solution would be linear as well. But as I've said, the first solution based on the Collections API is simpler and slightly more performant.
If for some reason, there's no proper implementation of equals() and hashCode() in the EmployeeDetails. And you have no control over this class and can't change it. Then you can declare a wrapper class and perform the same actions.
Below is an example of how to create the wrapper using Java 16 records.
Methods equals() and hashCode() will be generated by the compiler based on empId and empDob.
public record EmployeeWrapper(String empId, String empDob) {
public EmployeeWrapper(EmployeeDetails details) {
this(details.getEmpID(), details.empDOB);
}
}
The implementation of the equals/hashCode for the EmployeeDetails class based on the empID and empDOB might look like this (also, you can use the facilities of your IDE to generate these methods):
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
EmployeeDetails that = (EmployeeDetails) o;
return empID.equals(that.empID) && empDOB.equals(that.empDOB);
}
#Override
public int hashCode() {
return Objects.hash(empID, empDOB);
}

How change Iterable to ArrayList with filter

I have code :
#GetMapping("/goal/{id}")
public String goalInfo(#PathVariable(value = "id") long id, Model model) {
if (!goalRepository.existsById(id)) {
return "redirect:/goal";
}
Iterable<SubGoal> subGoal = subGoalRepository.findAll();
ArrayList<SubGoal> subGoals = new ArrayList<>();
//How refactor this?
for(SubGoal sub : subGoal){
if(sub.getParentGoal().getId().equals(id)){
subGoals.add(sub);
}
}
if(subGoals.size() > 0) {
goalPercent(id, subGoal);
}
Optional<Goal> goal = goalRepository.findById(id);
ArrayList<Goal> result = new ArrayList<>();
goal.ifPresent(result::add);
model.addAttribute("goal", result);
model.addAttribute("subGoal",subGoals);
return "goal/goal-info";
}
Here I get sub-goals from the repository and filter these values.
How I can do it without foreach? I want to use Streams or something else.

You don't need to declare an iterable on your code to filter your ArrayList. The filter method already provides one for you. You can use:
subGoals = subGoals.stream().filter(subGoal ->
/*Here goes your filter condition*/ ).collect(Collectors.toList());

To convert Iterable to Stream use StreamSupport.stream(iter.spliterator(), par).
Iterable<SubGoal> subGoal = subGoalRepository.findAll();
List<SubGoal> subGoals = StreamSupport
.stream(subGoal.spliterator(), false)
.filter(sub -> sub.getParentGoal().getId().equals(id))
.collect(toList()) // static import `Collectors.toList()`
...
Additionally, this part can be also single statement.
before (three statement)
Optional<Goal> goal = goalRepository.findById(id);
ArrayList<Goal> result = new ArrayList<>();
goal.ifPresent(result::add);
after (single statement)
List<Goal> result = goalRepository.findById(id)
.map(goal -> singletonList(goal)) // Collections.singletonList()
.orElse(emptyList()); // Collections.emptyList()
Updates
1. singletonList(), emptyList()
These are just factory methods used when creating single entity list and empty list.
you can change this part any kind of function that has Goal as input and List as output and any empty list.
For example,
.map(goal -> Arrays.asList(goal))
.orElse(new ArrayList<>());
or
.map(goal -> {
ArrayList<Goal> l = new ArrayList<>();
l.add(goal);
return l;
})
...
2. I changed the List Type to List<Goal>, not ArrayList<Goal>
Sorry, I missed explanation about that.
In OOP, using Interface will be better practices than using Concrete Class in many situation.
If you have to use ArrayList<> Type explicitly or want to specify actual list instance in some reason, you can also use toCollection() like below.
.collect(toCollection(ArrayList::new)) // you can specify the actual list instance
Thanks to #John Bollinger #hfontanez for pointing this out.

This is client-side filtering and is extremely inefficient. Instead, simply declare this method on your repository interface:
Collection<SubGoal> findByParentId(Long id); // or Stream, Iterable

Create list of an element of an Object from another list of Objects

The model:
public class MyModel{
private int id;
private String name;
....
....
//getters and setters
}
I have a list of MyModel object:
//First Object to be added to list
MyModel myModelObject1 = new MyModel();
myModelObject1.setId(1);
myModelObject1.setName("abc");
//Second Object to be added to list
MyModel myModelObject2 = new MyModel();
myModelObject1.setId(2);
myModelObject1.setName("pqr");
List<MyModel> myModelList = new ArrayList<MyModel>();
myModelList.add(myModelObject1);
myModelList.add(myModelObject2);
I want to get a list of names present in the MyModel List i.e. I want to create a list of names (List<String> in this case) from myModelList. So, I want my list to have:
{"abc", "pqr"}
There is always a way to iterate and create another list but is there any better way to do that? (Not necessarily to be efficient but if it can be done in a line using streams, foreach e.t.c.).
EDIT:
The answers worked for me but I have some modifications in my use case: If I want to add a condition that only name which contains character 'a' should be added to the list and I want to add a logger message to debug for each element then how should I approach this?
I tried doing the following using a method (charAPresent()) which checks that if the passed String contains character 'a' but it didn't work:
List<String> modelNameList = myModelList.stream()
.map(model -> {
if (charAPresent(model.getName)) {
model.getName()
}
})
.collect(Collectors.toList());
Let me know if I am missing something.

Using Java 8 Stream:
List<String> modelNameList = myModelList.stream()
.map(Model::getName)
.collect(Collectors.toList());
Model::getName is called as method reference. It equivalents to model -> model.getName()

You can use streams and map your object to its name and collect to a list as:
List<String> names = myModelList.stream()
.map(MyModel::getName)
.collect(Collectors.toList());
There is always a way to iterate and create another list but is there
any better way to do that
Even with the use of streams, you would have to iterate the complete collection. There is no better way of doing it than iterating the complete collection.

You can use Stream.map() to get only the names of your model. Use Stream.filter() to get only the names matching your charAPresent() method. To log the entries before collecting you can use Stream.peek():
List<String> modelNameList = myModelList.stream()
.map(Model::getName) // map only the name
.filter(YourClass::charAPresent) // filter the items
.peek(System.out::println) // print all items which are in the filter
.collect(Collectors.toList());

You can also use foreach like this:
public static List<String> modelNames(List<MyModel> myModelList)
List<String> list = new ArrayList<String>();
for(MyModel mm : myModelList) {
if(mm.getName().contains("a") {
list.add(mm.getName());
}
}
return list;
}

Remake list with some condition

There are two entities:
class GiftCertificate {
Long id;
List<Tag> tags;
}
class Tag {
Long id;
String name;
}
There is a list
List<GiftCertificate>
which contains, for example, the following data:
<1, [1, "Tag1"]>, <2, null>, <1, [2, "Tag2"]>. (It does not contain a set of tags, but only one tag or does not have it at all).
I need to do so that in the result it was this:
<1, {[1," Tag1 "], [2," Tag2 "]}>, <2, null>. I mean, add to the set of the first object a tag from the third GiftCertificate and at the same time delete the 3rd one. I would like to get at least some ideas on how to do this. it would be nice to use stream.

Probably not the most effective way, but it might help
private List<GiftCertificate> joinCertificates(List<GiftCertificate> giftCertificates) {
return giftCertificates.stream()
.collect(Collectors.groupingBy(GiftCertificate::getId))
.entrySet().stream()
.map(entry -> new GiftCertificate(entry.getKey(), joinTags(entry.getValue()))).collect(Collectors.toList());
}
private List<Tag> joinTags(List<GiftCertificate> giftCertificates) {
return giftCertificates.stream()
.flatMap(giftCertificate -> Optional.ofNullable(giftCertificate.getTags()).stream().flatMap(Collection::stream))
.collect(Collectors.toList());
}

You can do what you want with streams and with the help of a dedicated custom constructor and a couple of helper methods in GiftCertificate. Here's the constructor:
public GiftCertificate(GiftCertificate another) {
this.id = another.id;
this.tags = new ArrayList<>(another.tags);
}
This just works as a copy constructor. We're creating a new list of tags, so that if the list of tags of either one of the GiftCertificate instances is modified, the other one won't. (This is just basic OO concepts: encapsulation).
Then, in order to add another GiftCertificate's tags to this GiftCertificate's list of tags, you could add the following method to GiftCertificate:
public GiftCertificate addTagsFrom(GiftCertificate another) {
tags.addAll(another.tags);
return this;
}
And also, a helper method that returns whether the list of tags is empty or not will come in very handy:
public boolean hasTags() {
return tags != null && !tags.isEmpty();
}
Finally, with these three simple methods in place, we're ready to use all the power of streams to solve the problem in an elegant way:
Collection<GiftCertificate> result = certificates.stream()
.filter(GiftCertificate::hasTags) // keep only gift certificates with tags
.collect(Collectors.toMap(
GiftCertificate::getId, // group by id
GiftCertificate::new, // use our dedicated constructor
GiftCertificate::addTagsFrom)) // merge the tags here
.values();
This uses Collectors.toMap to create a map that groups gift certificates by id, merging the tags. Then, we keep the values of the map.
Here's the equivalent solution, without streams:
Map<Long, GiftCertificate> map = new LinkedHashMap<>(); // preserves insertion order
certificates.forEach(cert -> {
if (cert.hasTags()) {
map.merge(
cert.getId(),
new GiftCertificate(cert),
GiftCertificate::addTagsFrom);
}
});
Collection<GiftCertificate> result = map.values();
And here's a variant with a slight performance improvement:
Map<Long, GiftCertificate> map = new LinkedHashMap<>(); // preserves insertion order
certificates.forEach(cert -> {
if (cert.hasTags()) {
map.computeIfAbsent(
cert.getId(),
k -> new GiftCertificate(k)) // or GitCertificate::new
.addTagsFrom(cert);
}
});
Collection<GiftCertificate> result = map.values();
This solution requires the following constructor:
public GiftCertificate(Long id) {
this.id = id;
this.tags = new ArrayList<>();
}
The advantage of this approach is that new GiftCertificate instances will be created only if there's no other entry in the map with the same id.

Java 9 introduced flatMapping collector that is particularly well-suited for problems like this. Break the task into two steps. First, build a map of gift certificate IDs to list of tags and then assemble a new list of GiftCertificate objects:
import static java.util.stream.Collectors.flatMapping;
import static java.util.stream.Collectors.groupingBy;
import static java.util.stream.Collectors.toList;
......
Map<Long, List<Tag>> gcIdToTags = gcs.stream()
.collect(groupingBy(
GiftCertificate::getId,
flatMapping(
gc -> gc.getTags() == null ? Stream.empty() : gc.getTags().stream(),
toList()
)
));
List<GiftCertificate> r = gcIdToTags.entrySet().stream()
.map(e -> new GiftCertificate(e.getKey(), e.getValue()))
.collect(toList());
This assumes that GiftCertificate has a constructor that accepts Long id and List<Tag> tags
Note that this code deviates from your requirements by creating an empty list instead of null in case there are no tags for a gift certificate id. Using null instead of an empty list is just a very lousy design and forces you to pollute your code with null checks everywhere.
The first argument to flatMapping can also be written as gc -> Stream.ofNullable(gc.getTags()).flatMap(List::stream) if you find that more readable.

Creating a list of booleans on the fly

Imagine we are pulling data about people and their favourite foods.
The data would come to us in the format: "Name, FavFood1, FavFood2..FavFoodn".
e.g. "James, Beans, Chicken".Notice how we do not know how many foods a person will favour.
From this data we create an instance of a Person object which captures the person's name and favourite foods. After we have pulled data on every person, we want to create a spreadsheet whose columns would be: Name|Potato|Chicken|Beans|Curry etc.
All of the values to the right of the person's name will be simple boolean values representing whether or not that food was one of the person's favourites.
The problem is: we do not know in advance; all the foods that someone could possibly favour, and as such cannot just set up boolean instance variables in the Person class.
I've given this some thought, implementing sets,hash-sets and hash-maps, however every solution I think of ends up being horribly inelegant and so I've turned to the genius of stackoverflow for help on this one.
My question is: What design pattern / approach can I use to cleanly achieve the outcome I desire? Whilst this is a language-agnostic question I am programming this in Java, so if there's anything in the Java API or elsewhere built for this, do let me know.
Thanks in advance!

Try this. It generates data in csv form.
class Person {
final String name;
final Set<String> foods;
Person(String name, Set<String> foods) {
this.name = name;
this.foods = foods;
}
Stream<Boolean> getBooleans(List<String> foods) {
return foods.stream().map(food -> this.foods.contains(food));
}
#Override
public String toString() {
return "Person(" + name + ", " + foods +")";
}
}
class Test {
public static void main(String[] args) throws Exception {
List<String> data = Arrays.asList(
"James, Beans, Chicken",
"Emily, Potato, Curry",
"Clara, Beans, Curry"
);
List<String> foodNames = Arrays.asList(
"Potato", "Chicken", "Beans", "Curry"
);
Stream<Person> persons = data.stream().map(d -> {
String[] split = d.split(",");
for(int i = 0; i < split.length; i++) {
split[i] = split[i].trim();
}
String name = split[0];
Set<String> foods = Stream.of(split).skip(1).collect(Collectors.toSet());
return new Person(name, foods);
});
Stream<String> csvData = persons.map(p ->
p.name + ", " + p.getBooleans(foodNames)
.map(b -> b.toString())
.collect(Collectors.joining(", "))
);
csvData.forEach(System.out::println);
}
}

First of all, I highly recommend that whatever you do it in a separate class with methods like addFavoriteFood(String food) and boolean isFavoriteFood(String food) getFavorites(String food).
Personally I think the implementation of this class should contain both an instance HashSet (to hold the foods this person likes) and a SortedSet that is common to all the foods that can contain a list of ALL foods. (See notes at end)
Add would add it to both sets, getFavorites would return those in the first Hash set.
Hmm, it may also need a static getAllFavorites() method to return the SortedSet
Since your FavoiteFoods class knows the master list AND the person's favorites, you could even have it do most of the work by having a getFormattedRow() and static getFormattedHeaderRow() method. then your implementaiton is just:
System.out.println(FavoriteFoods.getFormattedHeaderRow());
for(Person person:people)
System.out.println(person.favoriteFood.getFormattedRow());
Again, the best thing here is that you can just use the Simplest Thing That Could Possibly Work for your implementation and re-do it later if need be since, being isolated in another class, it doesn't infect all your code with nasty implementation-specific sets, classes, booleans, etc.
Notes about the master list: This master list could naively be implemented as a Static but that's a bad idea--optimally the same masterList SortedSet would be passed into each instance on construction. Also since it is shared among all instances and is mutable it brings in issues if your solution is threaded!

What is so inelegant about this pseudocode?
Set<String> allFoods = new TreeSet<String>();
List<Person> allPersons = new ArrayList<Person>();
while (hasMorePersons()) {
Person person = getNextPerson();
allPersons.add(person);
allFoods.addAll(person.getFoods());
}
spreadSheet.writeHeader("Name", allFoods);
for (Person person : allPersons) {
spreadSheet.writeName(person.getName());
for (String food : allFoods) {
// assume getFoods() return a Set<String>,
// not necessarily ordered (could be a HashSet)
boolean yourBooleanHere = person.getFoods().contains(food);
spreadSheet.writeBoolean(yourBooleanHere);
}
spreadSheet.nextLine();
}
If you need a table of booleans or whatever else, you can easily store them anywhere you want during the second loop.
Note: TreeSet orders foods according to the natural order (that is, alphabetically). To output them in the order they are encountered, use a LinkedHashSet instead.