We Keep Coding

Overridden equals is not called

I have a class Reminder that has both hashcode and equals overridden like this:
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((cronExpression == null) ? 0 : cronExpression.hashCode());
result = prime * result + ((subject == null) ? 0 : subject.hashCode());
result = prime * result + timeout;
result = prime * result + ((type == null) ? 0 : type.hashCode());
return result;
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (!(obj instanceof Reminder))
return false;
Reminder other = (Reminder) obj;
if (cronExpression == null) {
if (other.cronExpression != null)
return false;
} else if (!cronExpression.equals(other.cronExpression))
return false;
if (subject == null) {
if (other.subject != null)
return false;
} else if (!subject.equals(other.subject))
return false;
if (timeout != other.timeout)
return false;
if (type == null) {
if (other.type != null)
return false;
} else if (!type.equals(other.type))
return false;
return true;
}
Both overrides were automatically generated using Eclipse. I'm using the Reminder in a HashSet instantiated like this: private Set<Reminder> localReminders = new HashSet<Reminder>();
When updating this set, I'm using localreminders.contains(anotherReminder) and for some reason that I've been trying to figure out for a while now, it does not call the overridden equals method. Even though cronExpression, subject, timeout and type of the reminders compared are the same, contains returns false.
So far I've only come across answers where equalsand/or hashcode were implemented incorrectly or not at all. Any help would be very much appreciated!
Let me know if you need more information like additional code for this!
EDIT: the properties used in hashcodeand equals are all String, except for timeout which is int.
EDIT2: while debugging, I currently have these two reminders in my HashSet:
Reminder [cronExpression=0 10 10 ? * *, subject=, type=OTHER_TYPE, audioPath=/other_type_reminder.mp3, muted=false, future=DelegatingErrorHandlingRunnable for Task#af94b0, timeout=35940]
Reminder [cronExpression=50 53 10 ? * *, subject=sub, type=TYPE, audioPath=/type_reminder.mp3, muted=false, future=DelegatingErrorHandlingRunnable for ReminderTask#f1f373, timeout=35940]
The one that I am checking whether it is contained in my set looks like this:
Reminder [cronExpression=50 53 10 ? * *, subject=sub, type=TYPE, audioPath=/type_reminder.mp3, muted=false, future=null, timeout=35940]
The only difference I can spot here is that in one, the future is null while it is actually set in the other. But since the future property is not included in either hashcode or ´equals`, this should not matter.

As you can see in the implementation of the equals method you call cronExpression.equals(other.cronExpression) and subject.equals(other.subject) and type.equals(other.type). If only one of this is not implemented right then you get wrong result. Please check if all of the properties that you use in this method has correct implementation of equals.
By the way also check the implementation of the methods cronExpression.hashCode(), subject.hashCode() and type.hashCode(). They are used in your hashCode method.
Edit: If as you said cronExpression, subject and type are Strings then it should be easy for you to make main method populate two objects from class Reminder with the same info and test the methods. To be sure where is the problem you can call if(firstReminder.equals(secondReminder)).
From my experiance you can have problems with the strings. For example if one of the string has space at the end is different then the other or similar kind of issue.
Edit 2: Ok, from your input It seems this objects to have the same strings.
Is it possible Reminder class to be extended and you to compare child class object with Reminder object? If this happen in the child class equals and hashcode can be implemented and then the result can be wrong.
Also just be sure can you log the size of each string? This is very strange.
Maybe it is possible you to have hidden character. See this for more information: Is there an invisible character that is not regarded as whitespace?
Good luck!

The Problem may be with your hashcode() method. It should generate a unique code. There are some guidelines to overridde hashcode().Hashcode Best Practice
If hashcode of objects are different then equals() will not called even if they are equal.
Because HashSet first check hashcodes of both objects and if hashcodes are equal then only it will call equals() to check whether both objects are really equal or not.
Read Oracle Javadoc to override hashcode override contract

You need to provide us the import of the Reminder class if you want us to be able to help you.
For your culture and curiosity : java.util.HashSet.contains(Object o), reading the code it points to :
public boolean containsKey(Object key) {
return getNode(hash(key), key) != null;
}
which itself points to :
static final int hash(Object key) {
int h;
return (key == null) ? 0 : (h = key.hashCode()) ^ (h >>> 16);
}
As you can see, the important part of your implementation is Reminder.hashCode().
Regarding your specific issue : As you are probably using quartz for org.quartz.CronExpression, you can see that org.quartz.CronExpression.hashCode() method is not implemented, so it calls it's parent hashCode(), which is Object.hashCode().
From the documentation (JRE 7), you can read :
As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)
So both of similar item with different instance of org.quartz.CronExpression will have different hashCode() result.

How to implement efficient hash cons with java HashSet

I am trying to implement a hash cons in java, comparable to what String.intern does for strings. I.e., I want a class to store all distinct values of a data type T in a set and provide an T intern(T t) method that checks whether t is already in the set. If so, the instance in the set is returned, otherwise t is added to the set and returned. The reason is that the resulting values can be compared using reference equality since two equal values returned from intern will for sure also be the same instance.
Of course, the most obvious candidate data structure for a hash cons is java.util.HashSet<T>. However, it seems that its interface is flawed and does not allow efficient insertion, because there is no method to retrieve an element that is already in the set or insert one if it is not in there.
An algorithm using HashSet would look like this:
class HashCons<T>{
HashSet<T> set = new HashSet<>();
public T intern(T t){
if(set.contains(t)) {
return ???; // <----- PROBLEM
} else {
set.add(t); // <--- Inefficient, second hash lookup
return t;
}
}
As you see, the problem is twofold:
This solution would be inefficient since I would access the hash table twice, once for contains and once for add. But okay, this may not be a too big performance hit since the correct bucket will be in the cache after the contains, so add will not trigger a cache miss and thus be quite fast.
I cannot retrieve an element already in the set (see line flagged PROBLEM). There is just no method to retrieve the element in the set. So it is just not possible to implement this.
Am I missing something here? Or is it really impossible to build a usual hash cons with java.util.HashSet?

I don't think it's possible using HashSet. You could use some kind of Map instead and use your value as key and as value. The java.util.concurrent.ConcurrentMap also happens to posess the quite convenient method
putIfAbsent(K key, V value)
that returns the value if it is already existent. However, I don't know about the performance of this method (compared to checking "manually" on non-concurrent implementations of Map).
Here is how you would do it using a HashMap:
class HashCons<T>{
Map<T,T> map = new HashMap<T,T>();
public T intern(T t){
if (!map.containsKey(t))
map.put(t,t);
return map.get(t);
}
}
I think the reason why it is not possible with HashSet is quite simple: To the set, if contains(t) is fulfilled, it means that the given t also equals one of the t' in the set. There is no reason for being able return it (as you already have it).

Well HashSet is implemented as HashMap wrapper in OpenJDK, so you won't win in memory usage comparing to solution suggested by aRestless.
10-min sketch
class HashCons<T> {
T[] table;
int size;
int sizeLimit;
HashCons(int expectedSize) {
init(Math.max(Integer.highestOneBit(expectedSize * 2) * 2, 16));
}
private void init(int capacity) {
table = (T[]) new Object[capacity];
size = 0;
sizeLimit = (int) (capacity * 2L / 3);
}
T cons(#Nonnull T key) {
int mask = table.length - 1;
int i = key.hashCode() & mask;
do {
if (table[i] == null) break;
if (key.equals(table[i])) return table[i];
i = (i + 1) & mask;
} while (true);
table[i] = key;
if (++size > sizeLimit) rehash();
return key;
}
private void rehash() {
T[] table = this.table;
if (table.length == (1 << 30))
throw new IllegalStateException("HashCons is full");
init(table.length << 1);
for (T key : table) {
if (key != null) cons(key);
}
}
}

ContainsKey of HashMap does not work with a Custom Class as key

I have a custom class MarioState that I want to use in a HashMap. The class represents a possible state in a state space of the Mario game. Below is a simplified version of the class MarioState.
In my HashMap I want to store these states. However, not ever property in the MarioState is something that should be considered when comparing two MarioState's. For example if one MarioState has the stuck property set to true and a distance of 30 and another MarioState also has the stuck property set to true but a different distance value (e.g. 20) then they still should be considered the same.
I know for this to work in my HashMap I have to implement the .equals() and .hashcode() methods, which is what I did (by letting them be automatically generated by the InteliJ IDE).
public class MarioState{
// Tracking the distance Mario has moved.
private int distance;
private int lastDistance;
// To keep track of if Mario is stuck or not.
private int stuckCount;
private boolean stuck;
public MarioState(){
stuckCount = 0;
stuck = false;
distance = 0;
lastDistance = 0;
}
public void update(Environment environment){
// Computing the distance
int tempDistance = environment.getEvaluationInfo().distancePassedPhys;
distance = tempDistance - lastDistance;
lastDistance = tempDistance;
// If Mario hasn't moved for over 25 turns then this means he is stuck.
if(distance == 0){
stuckCount++;
} else {
stuckCount = 0;
stuck = false;
}
if(stuckCount > 25){ stuck = true; }
}
public float calculateReward(){
float reward = 0f;
reward += distance * 2;
if(stuck){ reward += -20; }
return reward;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
MarioState that = (MarioState) o;
if (stuck != that.stuck) return false;
return true;
}
#Override
public int hashCode() {
return (stuck ? 1 : 0);
}
}
The problem is however that when running the code some of the keys are considered different when it shouldn't be according to their .equals() and .hashcode() functions. What can possibly cause this? Did I forget something?
The code used when inserting states in the HashMap (additional information can be provided if necessary):
public float[] getActionsQValues(MarioState state){
if(!table.containsKey(state)) {
float[] initialQvalues = getInitialQvalues(state);
table.put(state, initialQvalues);
return initialQvalues;
}
return table.get(state);
}
A screenshot when I'm in debug mode shows my table containing two keys with different values, but the keys itself are the same (but in the HashMap it is considered different).

Your hash code computation and equality comparison are both based on stuck - but that can change over time.
If you mutate an object after adding it as a key within a hash map, in such a way that the hash code changes, then the key will not be found when you later request it - because the hash code that was stored when the key was first added will no longer be the same as its current hash code.
Wherever possible, try to avoid using mutable objects as keys within a map (even a TreeMap which doesn't use the hash code would have the same problem if you changed the object in a way which would change relative ordering). If you must use mutable objects as keys within a map, you should avoid mutating them after adding them as keys.

Java: Retrieving an element from a HashSet

Why cannot I retrieve an element from a HashSet?
Consider my HashSet containing a list of MyHashObjects with their hashCode() and equals() methods overridden correctly. I was hoping to construct a MyHashObject myself, and set the relevant hash code properties to certain values.
I can query the HashSet to see if there "equivalent" objects in the set using the contains() method. So even though contains() returns true for the two objects, they may not be == true.
How come then there isn’t any get() method similar to how the contains() works?
What is the thinking behind this API decision?

If you know what element you want to retrieve, then you already have the element. The only question for a Set to answer, given an element, is whether it contains() it or not.
If you want to iterator over the elements, just use a Set.iterator().
It sounds like what you're trying to do is designate a canonical element for an equivalence class of elements. You can use a Map<MyObject,MyObject> to do this. See this Stack Overflow question or this one for a discussion.
If you are really determined to find an element that .equals() your original element with the constraint that you must use the HashSet, I think you're stuck with iterating over it and checking equals() yourself. The API doesn't let you grab something by its hash code. So you could do:
MyObject findIfPresent(MyObject source, HashSet<MyObject> set)
{
if (set.contains(source)) {
for (MyObject obj : set) {
if (obj.equals(source))
return obj;
}
}
return null;
}
It is brute-force and O(n) ugly, but if that's what you need to do...

You can use HashMap<MyHashObject, MyHashObject> instead of HashSet<MyHashObject>.
Calling containsKey() on your "reconstructed" MyHashObject will first hashCode() - check the collection, and if a duplicate hashcode is hit, finally equals() - check your "reconstructed" against the original, at which you can retrieve the original using get()
Complexity is O(1) but the downside is you will likely have to override both equals() and hashCode() methods.

It sounds like you're essentially trying to use the hash code as a key in a map (which is what HashSets do behind the scenes). You could just do it explicitly, by declaring HashMap<Integer, MyHashObject>.
There is no get for HashSets because typically the object you would supply to the get method as a parameter is the same object you would get back.

If you know the order of elements in your Set, you can retrieve them by converting the Set to an Array. Something like this:
Set mySet = MyStorageObject.getMyStringSet();
Object[] myArr = mySet.toArray();
String value1 = myArr[0].toString();
String value2 = myArr[1].toString();

The idea that you need to get the reference to the object that is contained inside a Set object is common. It can be archived by 2 ways:
Use HashSet as you wanted, then:
public Object getObjectReference(HashSet<Xobject> set, Xobject obj) {
if (set.contains(obj)) {
for (Xobject o : set) {
if (obj.equals(o))
return o;
}
}
return null;
}
For this approach to work, you need to override both hashCode() and equals(Object o) methods
In the worst scenario we have O(n)
Second approach is to use TreeSet
public Object getObjectReference(TreeSet<Xobject> set, Xobject obj) {
if (set.contains(obj)) {
return set.floor(obj);
}
return null;
}
This approach gives O(log(n)), more efficient.
You don't need to override hashCode for this approach but you have to implement Comparable interface. ( define function compareTo(Object o)).

One of the easiest ways is to convert to Array:
for(int i = 0; i < set.size(); i++) {
System.out.println(set.toArray()[i]);
}

If I know for sure in my application that the object is not used in search in any of the list or hash data structure and not used equals method elsewhere except the one used indirectly in hash data structure while adding. Is it advisable to update the existing object in set in equals method. Refer the below code. If I add the this bean to HashSet, I can do group aggregation on the matching object on key (id). By this way I am able to achieve aggregation functions such as sum, max, min, ... as well. If not advisable, please feel free to share me your thoughts.
public class MyBean {
String id,
name;
double amountSpent;
#Override
public int hashCode() {
return id.hashCode();
}
#Override
public boolean equals(Object obj) {
if(obj!=null && obj instanceof MyBean ) {
MyBean tmpObj = (MyBean) obj;
if(tmpObj.id!=null && tmpObj.id.equals(this.id)) {
tmpObj.amountSpent += this.amountSpent;
return true;
}
}
return false;
}
}

First of all, convert your set to an array. Then, get the item by indexing the array.
Set uniqueItem = new HashSet();
uniqueItem.add("0");
uniqueItem.add("1");
uniqueItem.add("0");
Object[] arrayItem = uniqueItem.toArray();
for(int i = 0; i < uniqueItem.size(); i++) {
System.out.println("Item " + i + " " + arrayItem[i].toString());
}

If you could use List as a data structure to store your data, instead of using Map to store the result in the value of the Map, you can use following snippet and store the result in the same object.
Here is a Node class:
private class Node {
public int row, col, distance;
public Node(int row, int col, int distance) {
this.row = row;
this.col = col;
this.distance = distance;
}
public boolean equals(Object o) {
return (o instanceof Node &&
row == ((Node) o).row &&
col == ((Node) o).col);
}
}
If you store your result in distance variable and the items in the list are checked based on their coordinates, you can use the following to change the distance to a new one with the help of lastIndexOf method as long as you only need to store one element for each data:
List<Node> nodeList;
nodeList = new ArrayList<>(Arrays.asList(new Node(1, 2, 1), new Node(3, 4, 5)));
Node tempNode = new Node(1, 2, 10);
if(nodeList.contains(tempNode))
nodeList.get(nodeList.lastIndexOf(tempNode)).distance += tempNode.distance;
It is basically reimplementing Set whose items can be accessed and changed.

If you want to have a reference to the real object using the same performance as HashSet, I think the best way is to use HashMap.
Example (in Kotlin, but similar in Java) of finding an object, changing some field in it if it exists, or adding it in case it doesn't exist:
val map = HashMap<DbData, DbData>()
val dbData = map[objectToFind]
if(dbData!=null){
++dbData.someIntField
}
else {
map[dbData] = dbData
}

What is the fastest way to compare two sets in Java?

I am trying to optimize a piece of code which compares elements of list.
Eg.
public void compare(Set<Record> firstSet, Set<Record> secondSet){
for(Record firstRecord : firstSet){
for(Record secondRecord : secondSet){
// comparing logic
}
}
}
Please take into account that the number of records in sets will be high.
Thanks
Shekhar

firstSet.equals(secondSet)
It really depends on what you want to do in the comparison logic... ie what happens if you find an element in one set not in the other? Your method has a void return type so I assume you'll do the necessary work in this method.
More fine-grained control if you need it:
if (!firstSet.containsAll(secondSet)) {
// do something if needs be
}
if (!secondSet.containsAll(firstSet)) {
// do something if needs be
}
If you need to get the elements that are in one set and not the other.
EDIT: set.removeAll(otherSet) returns a boolean, not a set. To use removeAll(), you'll have to copy the set then use it.
Set one = new HashSet<>(firstSet);
Set two = new HashSet<>(secondSet);
one.removeAll(secondSet);
two.removeAll(firstSet);
If the contents of one and two are both empty, then you know that the two sets were equal. If not, then you've got the elements that made the sets unequal.
You mentioned that the number of records might be high. If the underlying implementation is a HashSet then the fetching of each record is done in O(1) time, so you can't really get much better than that. TreeSet is O(log n).

If you simply want to know if the sets are equal, the equals method on AbstractSet is implemented roughly as below:
public boolean equals(Object o) {
if (o == this)
return true;
if (!(o instanceof Set))
return false;
Collection c = (Collection) o;
if (c.size() != size())
return false;
return containsAll(c);
}
Note how it optimizes the common cases where:
the two objects are the same
the other object is not a set at all, and
the two sets' sizes are different.
After that, containsAll(...) will return false as soon as it finds an element in the other set that is not also in this set. But if all elements are present in both sets, it will need to test all of them.
The worst case performance therefore occurs when the two sets are equal but not the same objects. That cost is typically O(N) or O(NlogN) depending on the implementation of this.containsAll(c).
And you get close-to-worst case performance if the sets are large and only differ in a tiny percentage of the elements.
UPDATE
If you are willing to invest time in a custom set implementation, there is an approach that can improve the "almost the same" case.
The idea is that you need to pre-calculate and cache a hash for the entire set so that you could get the set's current hashcode value in O(1). Then you can compare the hashcode for the two sets as an acceleration.
How could you implement a hashcode like that? Well if the set hashcode was:
zero for an empty set, and
the XOR of all of the element hashcodes for a non-empty set,
then you could cheaply update the set's cached hashcode each time you added or removed an element. In both cases, you simply XOR the element's hashcode with the current set hashcode.
Of course, this assumes that element hashcodes are stable while the elements are members of sets. It also assumes that the element classes hashcode function gives a good spread. That is because when the two set hashcodes are the same you still have to fall back to the O(N) comparison of all elements.
You could take this idea a bit further ... at least in theory.
WARNING - This is highly speculative. A "thought experiment" if you like.
Suppose that your set element class has a method to return a crypto checksums for the element. Now implement the set's checksums by XORing the checksums returned for the elements.
What does this buy us?
Well, if we assume that nothing underhand is going on, the probability that any two unequal set elements have the same N-bit checksums is 2-N. And the probability 2 unequal sets have the same N-bit checksums is also 2-N. So my idea is that you can implement equals as:
public boolean equals(Object o) {
if (o == this)
return true;
if (!(o instanceof Set))
return false;
Collection c = (Collection) o;
if (c.size() != size())
return false;
return checksums.equals(c.checksums);
}
Under the assumptions above, this will only give you the wrong answer once in 2-N time. If you make N large enough (e.g. 512 bits) the probability of a wrong answer becomes negligible (e.g. roughly 10-150).
The downside is that computing the crypto checksums for elements is very expensive, especially as the number of bits increases. So you really need an effective mechanism for memoizing the checksums. And that could be problematic.
And the other downside is that a non-zero probability of error may be unacceptable no matter how small the probability is. (But if that is the case ... how do you deal with the case where a cosmic ray flips a critical bit? Or if it simultaneously flips the same bit in two instances of a redundant system?)

There is a method in Guava Sets which can help here:
public static <E> boolean equals(Set<? extends E> set1, Set<? extends E> set2){
return Sets.symmetricDifference(set1,set2).isEmpty();
}

There's an O(N) solution for very specific cases where:
the sets are both sorted
both sorted in the same order
The following code assumes that both sets are based on the records comparable. A similar method could be based on on a Comparator.
public class SortedSetComparitor <Foo extends Comparable<Foo>>
implements Comparator<SortedSet<Foo>> {
#Override
public int compare( SortedSet<Foo> arg0, SortedSet<Foo> arg1 ) {
Iterator<Foo> otherRecords = arg1.iterator();
for (Foo thisRecord : arg0) {
// Shorter sets sort first.
if (!otherRecords.hasNext()) return 1;
int comparison = thisRecord.compareTo(otherRecords.next());
if (comparison != 0) return comparison;
}
// Shorter sets sort first
if (otherRecords.hasNext()) return -1;
else return 0;
}
}

You have the following solution from https://www.mkyong.com/java/java-how-to-compare-two-sets/
public static boolean equals(Set<?> set1, Set<?> set2){
if(set1 == null || set2 ==null){
return false;
}
if(set1.size() != set2.size()){
return false;
}
return set1.containsAll(set2);
}
Or if you prefer to use a single return statement:
public static boolean equals(Set<?> set1, Set<?> set2){
return set1 != null
&& set2 != null
&& set1.size() == set2.size()
&& set1.containsAll(set2);
}

If you are using Guava library it's possible to do:
SetView<Record> added = Sets.difference(secondSet, firstSet);
SetView<Record> removed = Sets.difference(firstSet, secondSet);
And then make a conclusion based on these.

I would put the secondSet in a HashMap before the comparison. This way you will reduce the second list's search time to n(1). Like this:
HashMap<Integer,Record> hm = new HashMap<Integer,Record>(secondSet.size());
int i = 0;
for(Record secondRecord : secondSet){
hm.put(i,secondRecord);
i++;
}
for(Record firstRecord : firstSet){
for(int i=0; i<secondSet.size(); i++){
//use hm for comparison
}
}

public boolean equals(Object o) {
if (o == this)
return true;
if (!(o instanceof Set))
return false;
Set<String> a = this;
Set<String> b = o;
Set<String> thedifference_a_b = new HashSet<String>(a);
thedifference_a_b.removeAll(b);
if(thedifference_a_b.isEmpty() == false) return false;
Set<String> thedifference_b_a = new HashSet<String>(b);
thedifference_b_a.removeAll(a);
if(thedifference_b_a.isEmpty() == false) return false;
return true;
}

I think method reference with equals method can be used. We assume that the object type without a shadow of a doubt has its own comparison method. Plain and simple example is here,
Set<String> set = new HashSet<>();
set.addAll(Arrays.asList("leo","bale","hanks"));
Set<String> set2 = new HashSet<>();
set2.addAll(Arrays.asList("hanks","leo","bale"));
Predicate<Set> pred = set::equals;
boolean result = pred.test(set2);
System.out.println(result); // true