We Keep Coding

TreeSet Comparator failed to remove duplicates in some cases?

I have the following comparator for my TreeSet:
public class Obj {
public int id;
public String value;
public Obj(int id, String value) {
this.id = id;
this.value = value;
}
public String toString() {
return "(" + id + value + ")";
}
}
Obj obja = new Obj(1, "a");
Obj objb = new Obj(1, "b");
Obj objc = new Obj(2, "c");
Obj objd = new Obj(2, "a");
Set<Obj> set = new TreeSet<>((a, b) -> {
System.out.println("Comparing " + a + " and " + b);
int result = a.value.compareTo(b.value);
if (a.id == b.id) {
return 0;
}
return result == 0 ? Integer.compare(a.id, b.id) : result;
});
set.addAll(Arrays.asList(obja, objb, objc, objd));
System.out.println(set);
It prints out [(1a), (2c)], which removed the duplicates.
But when I changed the last Integer.compare to Integer.compare(b.id, a.id) (i.e. switched the positions of a and b), it prints out [(2a), (1a), (2c)]. Clearly the same id 2 appeared twice.
How do you fix the comparator to always remove the duplicates based on ids and sort the ordered set based on value (ascending) then id (descending)?

You're askimg:
How do you fix the comparator to always remove the duplicates based on ids and sort the ordered set based on value (ascending) then id (descending)?
You want the comparator to
remove duplicates based on Obj.id
sort the set by Obj.value and Obj.id
Requirement 1) results in
Function<Obj, Integer> byId = o -> o.id;
Set<Obj> setById = new TreeSet<>(Comparator.comparing(byId));
Requirement 2) results in
Function<Obj, String> byValue = o -> o.value;
Comparator<Obj> sortingComparator = Comparator.comparing(byValue).thenComparing(Comparator.comparing(byId).reversed());
Set<Obj> setByValueAndId = new TreeSet<>(sortingComparator);
Let's have a look on the JavaDoc of TreeSet. It says:
Note that the ordering maintained by a set [...] must be consistent with equals if it is to
correctly implement the Set interface. This is so
because the Set interface is defined in terms of the equals operation,
but a TreeSet instance performs all element comparisons using its
compareTo (or compare) method, so two elements that are deemed equal
by this method are, from the standpoint of the set, equal.
The set will be ordered according to the comparator but its elements are also compared for equality using the comparator.
As far as I can see there is no way to define a Comparator which satisfies both requirements. Since a TreeSet is in the first place a Set requirement 1) has to match. To achieve requirement 2) you can create a second TreeSet:
Set<Obj> setByValueAndId = new TreeSet<>(sortingComparator);
setByValueAndId.addAll(setById);
Or if you don't need the set itself but to process the elements in the desired order you can use a Stream:
Consumer<Obj> consumer = <your consumer>;
setById.stream().sorted(sortingComparator).forEach(consumer);
BTW:
While it's possible to sort the elements of a Stream according to a given Comparator there is no distinct method taking a Comparator to remove duplicates according to it.
EDIT:
You have two different tasks: 1. duplicate removal, 2. sorting. One Comparator cannot solve both tasks. So what alternatives are there?
You can override equals and hashCode on Obj. Then a HashSet or a Stream can be used to remove duplicates.
For the sorting you still need a Comparator (as shown above). Implementing Comparable just for sorting would result in an ordering which is not "consistent with equals" according to Comparable JavaDoc.
Since a Stream can solve both tasks, it would be my choice. First we override hashCode and equals to identify duplicates by id:
public int hashCode() {
return Integer.hashCode(id);
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Obj other = (Obj) obj;
if (id != other.id)
return false;
return true;
}
Now we can use a Stream:
// instantiating one additional Obj and reusing those from the question
Obj obj3a = new Obj(3, "a");
// reusing sortingComparator from the code above
Set<Obj> set = Stream.of(obja, objb, objc, objd, obj3a)
.distinct()
.sorted(sortingComparator)
.collect(Collectors.toCollection(LinkedHashSet::new));
System.out.println(set); // [(3a), (1a), (2c)]
The returned LinkedHashSet has the semantics of a Set but it also preserved the ordering of sortingComparator.
EDIT (answering the questions from comments)
Q: Why it didn't finish the job correctly?
See it for yourself. Change the last line of your Comparator like follows
int r = result == 0 ? Integer.compare(a.id, b.id) : result;
System.out.println(String.format("a: %s / b: %s / result: %s -> %s", a.id, b.id, result, r));
return r;
Run the code once and then switch the operands of Integer.compare. The switch results in a different comparing path. The difference is when (2a) and (1a) are compared.
In the first run (2a) is greater than (1a) so it's compared with the next entry (2c). This results in equality - a duplicate is found.
In the second run (2a) is smaller than (1a). Thus (2a) would be compared as next with a previous entry. But (1a) is already the smallest entry and there is no previous one. Hence no duplicate is found for (2a) and it's added to the set.
Q: You said one comparator can't finish two tasks, my 1st comparators in fact did both tasks correctly.
Yes - but only for the given example. Add Obj obj3a to the set as I did and run your code. The returned sorted set is:
[(1a), (3a), (2c)]
This violates your requirement to sort for equal values descending by id. Now it's ascending by id. Run my code and it returns the right order, as shown above.
Struggling with a Comparator a time ago I got the following comment: "... it’s a great exercise, demonstrating how tricky manual comparator implementations can be ..." (source)

Treemap's headmap method in Java

I was checking headMap method of TreeMap which returns a portion of Map whose keys are strictly less than toKey. So I was expecting output to be B, C but it returns only B. Here is a thing I did weird I changed compareTo method like this return this.priority > o.priority ? 1 : -1; then it started returning C, B which is I was expecting. I am sure this is not correct but how can I get both B, C which has lower priority than A. Where I am getting it wrong. Thanks.
NavigableMap<PolicyTypePriorityWrapper, String> treeMap = new TreeMap();
PolicyTypePriorityWrapper a = new PolicyTypePriorityWrapper("A", 2);
PolicyTypePriorityWrapper b = new PolicyTypePriorityWrapper("B", 1);
PolicyTypePriorityWrapper c = new PolicyTypePriorityWrapper("C", 1);
treeMap.put(a, "A");
treeMap.put(b, "B");
treeMap.put(c, "C");
NavigableMap<PolicyTypePriorityWrapper, String> map = treeMap.headMap(a, false);
Set<PolicyTypePriorityWrapper> policyTypePriorityWrappers = map.keySet();
for (PolicyTypePriorityWrapper pol: policyTypePriorityWrappers) {
System.out.println(pol.getPolicyType());
}
PolicyTypePriorityWrapper.java
class PolicyTypePriorityWrapper implements Comparable<PolicyTypePriorityWrapper> {
private String policyType;
private int priority;
public PolicyTypePriorityWrapper(final String policyType, final int priority) {
this.policyType = policyType;
this.priority = priority;
}
public String getPolicyType() {
return this.policyType;
}
public int getPriority() {
return this.priority;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
PolicyTypePriorityWrapper that = (PolicyTypePriorityWrapper) o;
if (priority != that.priority) return false;
return policyType.equals(that.policyType);
}
#Override
public int hashCode() {
int result = policyType.hashCode();
result = 31 * result + priority;
return result;
}
#Override
public int compareTo(final PolicyTypePriorityWrapper o) {
return Integer.compare(this.priority, o.priority);
}
}

That's because you are not following JDK documentation guidelines, from Comprarable:
It is strongly recommended (though not required) that natural orderings be consistent with equals. This is so because sorted sets (and sorted maps) without explicit comparators behave "strangely" when they are used with elements (or keys) whose natural ordering is inconsistent with equals. In particular, such a sorted set (or sorted map) violates the general contract for set (or map), which is defined in terms of the equals method.
As you can see you have circumstances in which a.compareTo(b) == 0 but !a.equals(b). Both "B", 1 and "C", 1 are considered equal for the TreeMap:
Note that the ordering maintained by a tree map, like any sorted map, and whether or not an explicit comparator is provided, must be consistent with equals if this sorted map is to correctly implement the Map interface. (See Comparable or Comparator for a precise definition of consistent with equals.) This is so because the Map interface is defined in terms of the equals operation, but a sorted map performs all key comparisons using its compareTo (or compare) method, so two keys that are deemed equal by this method are, from the standpoint of the sorted map, equal. The behavior of a sorted map is well-defined even if its ordering is inconsistent with equals; it just fails to obey the general contract of the Map interface.
For example, if one adds two keys a and b such that (!a.equals(b) && a.compareTo(b) == 0) to a sorted set that does not use an explicit comparator, the second add operation returns false (and the size of the sorted set does not increase) because a and b are equivalent from the sorted set's perspective.
So what happens is that you compareTo is not able to distinguish two elements with same priority but different type, but since a TreeMap is using ONLY that method to decide if two elments are equal then you are not adding them both to the map in the first place.
Did you try if treeMap.size() == 3? My guess is that it's 2 in the first place.

The new PolicyTypePriorityWrapper("B", 1) is not qualified because it does it even not make it into the treeMap.
Why? Because the keys are the PolicyTypePriorityWrapper objects which are compared according to their integer priority value. Since b and c have the same priority, only the last one is saved to the treeMap. Compared a, b and c has a lower priority than a and equal to b. The key remains and the value is replaced. So in the map appears an entry PolicyTypePriorityWrapper b with the newly replaced value C.
It's the behavior of Map::put(K key, V value) method.
If the map previously contained a mapping for the key, the old value is replaced by the specified value.
Now the NavigableMap::headMap(K toKey, boolean inclusive) which returns a view of the portion of this map whose keys are less than (or equal to, if inclusive is true) toKey (taken from the documentation). The result is obvious. Only a and b stay in the treeMap, so the a is filtered out since it has smaller priority to b and only b is qualified to be returned.

Sorting a list of objects in java [duplicate]

This question already has answers here:
Sorting an ArrayList of objects using a custom sorting order
(12 answers)
Closed 8 years ago.
I want to sort this list of Emp objects in ascending order based on the marks field.
List<Emp> emp= new ArrayList<Emp>();
public class Emp implements Serializable {
private String empname;
private String section;
private int empId;
private int marks;
...

You need to write a comparator, otherwise the Sort method assumes which fields you want use when sorting.
Collections.sort(emp, new Comparator<Emp>() { public int compare(Emp one, Emp two) {
return one.marks.compareTo(two.marks);
});
In my example i treated the field marks as public, replace one.marks with a getter if you so choose.
Also since you're using ints which do not have a compareTo, do like so:
Collections.sort(list, new Comparator<Emp>() {
public int compare(Emp one, Emp two) {
int cmp = one.getMarks() > two.getMarks() ? +1 : one.getMarks() < two.getMarks() ? -1 : 0;
return cmp;
}
});

You can use a comparator object to sort.
Collections.sort();
does the sorting.
This will work with your List. The method to be used is compareTo.
if (list.size() > 0) {
Collections.sort(list, new Comparator<Emp>() {
#Override
public int compare(final Emp object1, final Emp object2) {
return object1.getMarks().compareTo(object2.getMarks());
}
} );
}

There are two main ways of supporting object comparisons in Java.
You can have your class implement the Comparable interface, which is acceptable when your objects have a natural ordering that you're relying on (for example, alphabetical ordering for strings). This requires classes to implement a compareTo method which defines the comparison rule between instances.
The standard alternative is to instantiate a Comparator for your class, and specify the comparison rule in a compare method.
In your case, the latter option seems more appropriate. The mechanics of the compare method are fairly simple: it takes two instances of your class, returns a negative value if the first is "less than" the second, a positive number if it is "greater", and 0 if they are "equal". For integer-based comparisons, like comparing by marks, the quick trick is to return the difference of the numbers.
Once you have your Comparator defined, sorting is as simple as invoking the Collections.sort method, opting for the method signature which takes a List and a specified Comparator.
List<Emp> emps = new ArrayList<Emp>();
// populate list...
Comparator<Emp> empComparator = new Comparator<Emp>() {
public int compare(Emp e1, Emp e2) {
return e2.getMarks() - e2.getMarks();
}
};
Collections.sort(emps, empComparator);

How to sort bidimensional ArrayList?

I have an ArrayList<ArrayList<String>> that is a list of couple of values:
John, 12.3
Marcus, 35.0
Sue, 11.4
How to sort the list by amount?
If in this case there is a better way than using an ArrayList of an ArrayList, please tell me and tell me then how to sort it. Thank you.

Instead I will recommend you to use a class and use class like comparable or comparator to sort.
Something like this:
class Person implements Comparable<Person> {
String name;
double amount;
Person(String n, double d) {
name = n;
amount = d;
}
public int compareTo(Person other) {
if (amount != other.amount)
return Double.compare(amount, other.amount);
return name.compareTo(other.name);
}
}
and this is easy to implement and understand.

Use a Comparator:
A comparison function, which imposes a total ordering on some
collection of objects. Comparators can be passed to a sort method
(such as Collections.sort or Arrays.sort) to allow precise control
over the sort order. Comparators can also be used to control the order
of certain data structures (such as sorted sets or sorted maps), or to
provide an ordering for collections of objects that don't have a
natural ordering.
It'd be great if you could add more information about why you are using those values then I am sure a better approach can be suggested.

I solved with this:
Collections.sort(data, new Comparator<ArrayList<String>>() {
#Override
public int compare(ArrayList<String> one, ArrayList<String> two) {
// Replacements for using Double.parseDouble(string) later
String value1 = one.get(1).replace(",", ".");
String value2 = two.get(1).replace(",", ".");
if (Double.parseDouble(value1) < Double.parseDouble(value2))
return -1;
else if (Double.parseDouble(value1) > Double.parseDouble(distanza2))
return 1;
else
return 0;
}
});

Remove array elements belonging to the same group leaving only 1 of the group?

I have an array myArray[] of objects MyThing which contains X elements. I need to remove elements belonging to the same group, but leaving one representative of each group.
MyThing class has a field groupId
public class MyThing {
private int groupId;
//...other fields
public int getGroupId(){return groupId;}
//getter and setter
}
So I have to compare groupId integer value of array elements (myArray[x].getGroupId()) and remove all element belonging the same group except the first such element in the array.
This way I will get an array of unique elements with only 1 from the same group. For example, if I have an array with a.getGroupId()=1, b.getGroupId()=2, c.getGroupId()=1 after purification, the array will contain only {a,b}, and c will be removed since it's of the same group as a.
Because this is the custom object, I cannot use Set<T>.
Any ideas?
PS. please let me know if I explained this clearly since it's kind of confusing.

A set by definition doesn't contain any duplicates. A set determines if two items are alike, by using either the objects equals()/compareTo(..) method or by using a Comparator. If you only want unique items in your set, implementing the Comparable interface and overriding equals() is what you want to do. BUT in your case, you're only interested in objects in unique groups, so it's then better to create a custom Comparator for the occasion, which you then supply to the Set, telling it to use it, instead of "natural ordering".
Set<MyThing> myThings = new TreeSet<>(new Comparator<MyThing>() {
#Override
public int compare(MyThing o1, MyThing o2)
{
return o1.getGroupId() - o2.getGroupId();
}
});
myThings.addAll(Arrays.asList(myArray));
After creating the set, you add your entire array into it, using the convinience method addAll(..).
(How the comparator sorts your objects is completely up to you to decide.)

You could loop through your array and use a map to keep track of which IDs have already occurred. Then if one was already added to the set, remove it from the array:
Set<Integer> uniqueIDs = new HashSet<Integer>();
for(MyThing thing : MyThings){
int groupID = thing.getGroupId();
if(!uniqueIDs.add(groupID)){
// DUPLICATE, REMOVE IT
}
}

Use a TreeSet and a custom Comparator class that inspects your objects and treats two with the same group as equal.
http://docs.oracle.com/javase/6/docs/api/java/util/TreeSet.html
Algorithm psuedocode:
Create TreeSet
Add all array elements to TreeSet
Convert TreeSet back to array
For a sample implementation: see Martin's answer

Just rewrote the Martin's solution because the comparator is broken, it might overflow
Set<MyThing> myThings = new TreeSet<>(new Comparator<MyThing>() {
#Override
public int compare(MyThing o1, MyThing o2) {
return Integer.compare(o1.getGroupId(), o2.getGroupId());
}
});
myThings.addAll(Arrays.asList(myArray));

Why don't you try something like (semi-pseudo-code here):
List<Integer> uniqGroups = new ArrayList<Integer>();
for (int i = 0; i < myArray.length; i++) {
int groupId = myArray[i].getGroupId();
if (!uniqGroups.contains(groupId)) {
// Hasn't been seen before, keep around
uniqGroups.add(groupId);
}
else {
// Already seen, remove or otherwise clean up the array
myArray[i] = null;
}
}

As you just need to distinguish your objects by groupId, you might override the hashCode() and equals() methods in your class:
class MyThing {
private int groupId;
public int getGroupId(){return groupId;}
// new code to add...
#Override
public int hashCode() {
return groupId;
}
#Override
public boolean equals(Object o) {
return (o instanceof MyThing
&& (groupId == ((MyThing)o).groupId));
}
}
and then, use a HashSet<MyThing> class to remove the MyThing objects in myArray with duplicated groupId:
myArray = new HashSet<MyThing>(Arrays.asList(myArray)).toArray(new MyThing[0]);