We Keep Coding

Java performance : Java logic to find the 3 highest number from huge list [duplicate]

This question already has answers here:
Find the k largest elements in order
(7 answers)
Closed 4 years ago.
This was the question I was asked in an interview for checking the performance knowledge.
Question - I have a list (Arraylist by default and if you wanna change the list then justify) of integers.
There are millions of entries with random int values.
Values can repeat.
From this list I need to find 3 highest unique numbers in below cases.
1) when time is limited (time effective)
2) when memory is limited (memory effective)
I attempted the questions but couldn't get the effective solution.
My solution was to use stream API,
then distinct() to get unique numbers
Sort() to sort the list
And then display top 3 after collecting.
However, they said you don't need to sort.
I thought of using 3 variables to hold top 3 values.
Then I iterate over the list and check if current values in top 3 have higher values or not? If not then I swap.
However here, there are many comparisons and thus at every iteration we have to do these comparisons.
Can anyone suggest me what are some better ways to solve this problem?
Also, I'll be very thankful if anyone can provide some link /description /approaches for such performances related solving.
Edit : output required is top 3 unique values

If you don't have any space limitations, then one option might be to add your list collection to a Java TreeSet:
List<Integer> list = new ArrayList<>();
// populate the above list
TreeSet<Integer> set = new TreeSet<>(list);
set = (TreeSet<Integer>)set.descendingSet();
Each entry in your list will be placed into a red black tree behind TreeSet, and duplicates will be automatically removed. I make a call to TreeSet#descendingSet above, to give us a sorted set which will iterate in descending order by default.
Now all that is needed is to iterate the first three entries:
int count = 0;
Iterator<Integer> iterator = set.iterator();
while(iterator.hasNext() && count < 3) {
System.out.println("Value #" + count + " = " + iterator.next());
++count;
}
As for the memory limited approach, you would likely have to resort to some sort of in place sorting algorithm, which uses either only the original data structure, or perhaps just a bit extra. I have little expertise with such solutions, so I won't attempt anything other than what I just mentioned.

The following method will return the n largest elements of the given list. We iterate over the list and add the elements to a TreeSet which stores its elements in sorted order (with O(log n) insertion). When the number of elements in the set exceeds n, the first element (i.e. the smallest is removed). Additionally, the set does not allow duplicate entries.
public static <T extends Comparable<T>> List<T> highest(List<T> list, int n) {
final TreeSet<T> set = new TreeSet<T>();
for (final T t : list) {
set.add(t);
if (set.size() > n)
set.pollFirst();
}
return new ArrayList<T>(set);
}
Example (for the list [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]):
public static void main(String[] args) {
final List<Integer> list = new ArrayList<Integer>();
for (int i = 0; i < 10; ++i)
list.add(i);
System.out.println(highest(list, 3));
}
Output:
[7, 8, 9]

Print same index position out of 2 ArrayLists

Given two ArrayLists, one containing Strings whereas the other contains Integers.
The next step would be to grab the first position of each ArrayList and print them out side by side next to each other.
Then 2, 3, 4 , 5, You name it.
1st List: public List getDoubleHistory(){...}
2nd : List<StringBuilder> timeAtThatMoment = new ArrayList<>();
I guess that we start with a for loop in which i is the index.

First we have to assume that the lists are the same size, or else we cannot continue to do as you asked.
for (int i = 0; i < getDoubleHistory().size(); i++){
System.out.println(getDoubleHistory.get(i));
System.out.println(timeAtThatMoment.get(i));
}
If the lists are not the same size you will have to add some if statements to make sure you dont get index out of bounds exceptions.
You can read more on this here. https://docs.oracle.com/javase/7/docs/api/java/lang/IndexOutOfBoundsException.html

Why is my algorithm to find the total amount of unique integers in a subarray (contiguous), not working for larger amounts of Integers and subarrays?

My algorithm to find the maximum number of unique integers among all possible contiguous subarrays doesn't work for larger amounts of Integers and subarrays.
For instance, I have to read a total amount of 6 Integers from the console and each subarray has a size of 3.
So, for this kind of input 5 3 5 2 3 2
my program should print 3 and this works fine.
The first subarray stores 5 3 5 so the number of unique Integers is 2.
The second subarray stores 3 5 2 so the number of unique Integers is 3.
The third subarray would also print 3 because it stores 5 2 3 and so on...
But, it seems like my algorithm can't handle a total amount of 100000 Integers with a subarray size of 99877.
Can anyone explain me, what I have done wrong?
FYI: I have to use a Deque implementation like LinkedList or ArrayDeque
for (int i = 0; i < totalAmountOfIntegers; i++) {
int anyIntegerNumber = consoleInput.nextInt();
arrayDequeToStoreAllIntegers.addLast(anyIntegerNumber);
hashSetToStoreUniqueIntegers.add(anyIntegerNumber);
if (arrayDequeToStoreAllIntegers.size() == sizeOfEachArrayDequeAsSubArray) {
if (hashSetToStoreUniqueIntegers.size() > quantityOfUniqueIntegersInSubarray) {
quantityOfUniqueIntegersInSubarray = hashSetToStoreUniqueIntegers.size();
}
int firstNumberInDeque = arrayDequeToStoreAllIntegers.remove();
if (hashSetToStoreUniqueIntegers.size() == sizeOfEachArrayDequeAsSubArray) {
hashSetToStoreUniqueIntegers.remove(firstNumberInDeque);
}
}
}

The answer would be simply the unique integers in the whole array, since the array is the superset of all subarrays, all numbers would be present in it
Just find how many unique element exist

To be honest, i don't understand your algorithm. I don't really get what the variables are referring to (although they seem to be named in a semantic way).
But what about this:
import java.util.HashSet;
import java.util.Set;
public class UniqueIntegers {
public static void main(String[] args) {
UniqueIntegers ui = new UniqueIntegers();
Integer[][] integers = {
{3,5,3,4,6},
{1,6,3,2,4},
{2,3,4},
{3,3,6,9,2}
};
Set<Integer> unique = ui.uniqueIntegers(integers);
System.out.println("Unique Integers: " + unique.size());
System.out.println("Integers: " + unique);
}
private Set<Integer> uniqueIntegers(Integer[][] ints){
Set<Integer> result = new HashSet<Integer>();
for (Integer[] iSub : ints){
for (Integer i : iSub){
result.add(i);
}
}
return result;
}
}
It prints:
Unique Integers: 7
Integers: [1, 2, 3, 4, 5, 6, 9]

After a day of researching, I have found my mistake.
My third IF-Statement is wrong. I am comparing, if the size of my HashSet variable is equal to the maximum size of elements each subarray can hold.
Instead, I should compare, if my int variable firstNumberInDeque, which I remove first from my ArrayDeque variable, contains another int variable with the same value. So if this is true, my HashSet variable remains unchanged.
But, if my ArrayDeque variable doesn't contain another int with the same value of firstNumberInDeque than firstNumberInDeque should be removed from my HashSet variable.
Here is the right code:
int firstNumberInDeque = arrayDequeToStoreAllIntegers.remove();
if (!arrayDequeToStoreAllIntegers.contains(firstNumberInDeque)) {
hashSetToStoreUniqueIntegers.remove(firstNumberInDeque);
}

Counting Int Occurrences in an Array

I am trying to count the occurrences of integers in an array. I was able to get it to work by piecing together some code I found online but I don't really understand why its working. What I have is:
int[] hand = {2, 4, 3, 2, 4};
int[] numOccurence = new int[hand.length];
for (int i = 0; i < hand.length; i++)
numOccurence[hand[i]]++;
for (int i = 1; i < numOccurence.length; i++)
if (numOccurence[i] > 0)
System.out.println("The number " + i + " occurs " + numOccurence[i] + " times.");
The output is:
The number 2 occurs 2 times.
The number 3 occurs 1 times.
The number 4 occurs 2 times.
How is this code counting the number of occurrences properly? I don't see how its accomplishing this. Thank you in advance!

This is only working because you've a good luck. Try making the second element in the hand array as 5 and see what happens. Its because the number present at the current index of hand is taken as the index of array numOccurence. In case of a number greater than or equal to the length of the numOccurence, you'll get the ArrayIndexOutOfBoundsException.
Thereforce, you can better use a Map for this where the key would be the number and the value could be its count.
Something like this:-
Map<Integer, Integer> numOccurence = new HashMap<Integer, Integer>();
for (int i = 0; i < hand.length; i++) {
int cnt = 1;
if (numOccurence.containsKey(hand[i])) {
cnt = numOccurence.get(hand[i]);
cnt++;
}
numOccurence.put(hand[i], cnt);
}

This code does not really work. At least it does for the author's use case but probably not for yours.
Try with {2, 4, 99, 2, 4}; as hand and it will fail.
The author takes the number found in hand as the index of array numOccurence.
numOccurence has the following structure : {nb occ of 0; nb occs of 1;...; nb occs of 4}. Here 99 will be out of bounds.

When you create an array
int[] numOccurence = new int[hand.length];
it is populated by their default values. For primitive int this value is 0.
This will of course only work if hand contains numbers less than or equal to max index (length -1) of the array otherwise it's ArrayIndexOutOfBound for you mister!

Actually it's the same method for creating histogram for picture ;)
You create a table where you will gather the occurrence.
numOccurence[0] will stock the number of 0
numOccurence[1] will stock the number of 1
etc.
That's what is done by this
for (int i = 0; i < hand.length; i++)
numOccurence[hand[i]]++;
it adds 1 to the value in the case corresponding to the number hand[i]
so if you look at this step by step
first he will take hand[0] = 2
so he will put
numOccurence[2] = numOccurence[2] + 1 ;
which is same (but faster to write) as
numOccurence[2]++;

This kind of performing a count is called counting sort.
The advantage of counting sort is it's speed. The disadvantage is the memory requirements when sorting big numbers.
There is a bug in the code:
int[] numOccurence = new int[hand.length];
numOccurence needs to be as long as the highest number in the list (not the number of numbers in the list). Try changing one of the numbers to 15 and you will get an exception.

The code iterates through the given array hand. it takes each value encountered as an index into the array numOccurrence. for each number n in hand, this will happen exactly as often as n occurs in hand, and each time this happens, the nth element of numOccurrence will be incremented.
thus numOccurrence is effectively an array of counters (assuming that the array elements are initialized with 0).
drawbacks of this approach:
the number of counters allocated depends on the magnitude of numbers in your handarray.
if the numbers in your hand array are distributed sparsely, most of the allocated space is never used.
alternative
you could improve the code by sorting hands first. in the sorted array the indexes of all occurrences of a given number are contiguous, so you scan the sorted array once needing a single counter only to compile the frequencies.

First of all the code is wrong. You should set the size of numOccurence array to the max number value from hand array + 1. For example:
int[] hand = {2, 100};
int[] numOccurence[] = new int[101];
(you should obviously find max number programatically)
Now let's take a look at the algorithm.
It takes each number from hand array, treats it as a numOccurence index value and increments number at that index by 1 in hand array. Note that all elements of numOccurence array are 0 by default at the beginning.
int[] hand = {2, 4, 3, 2, 4};
int[] numOccurence = new int[5];
Steps:
i = 0 (nothing happens, because there is no 0 in hand array)
i = 1 (same situation as for 0)
i = 2 (there are two 2 numbers in hand array, so we do operation numOccurence[2] += 1 twice, which in result gives 0 + 1 + 1 = 2. So we got numOccurence[2] = 2)
it continues for all numbers from 0 to max number from hand array (here: 100).

Confusing regarding .get and .set in an ArrayList

(I'm new at Java, coming over from Python ---)
I'm going through a tutorial and they've created a program which counts how many times a number appears in a file, then returns that number. One particular part of the program is somewhat mysterious to me and deals with the ArrayList's .get and .set (methods? functions?). The program goes like this:
// (Scan a file with the numbers, say, 2 2 3 4, and put it into data1 variable.)
// (Make An Empty ArrayList with a bunch of 0's)
Scanner data1 = null;
ArrayList<Integer> count = new ArrayList<Integer>();
Integer idx;
while(data1.hasNextInt()){
idx = data1.nextInt();
System.out.println(idx);
System.out.println(count.get(idx)+1);
count.set(idx,count.get(idx)+1);
}
//Then prints out all the values; the ArrayList contains the number of times the number n occurs in the n-th index.
My question comes at the "while" part. For concrete, let's assume data1 has the numbers 2 2 3 4. It seems that it takes idx = 2, then puts a 1 in count[2], which is reasonable. It then takes idx = 2 again (the next integer in data1) and puts a 2 in count[2], which is also reasonable. At this point, the next number in data1 makes idx = 3, but it occurs at the index 2 in the ArrayList, so it should put a 3 in count[3], which is incorrect.
So, what is .get and .set doing here? Do they pop the elements off of the list when they're done with it? Am I overlooking something?

A .get() will not automagically get elements from a List which does not have that many elements. Note: list indices, like arrays, start at 0.
If you do:
final List<Integer> = new ArrayList<Integer>();
list.get(0);
this is a runtime error (IndexOutOfBoundsException) because your list has no elements.
You have to fill it:
list.add(1); // append an element
list.get(0); // returns element at index 0
list.get(1); // IndexOutOfBoundsException!
The .set() method takes an index and a value at an argument. In a similar vein, you cannot set an element which does not already exist, except at the very end of the list:
// start from an empty list
list.set(1, 32); // IndexOutOfBoundsException!
list.set(0, 32); // OK
Final note: try not to use list[i], bracket indices are used for arrays ;) Note that arrays are not resizabe in Java. Lists (which are an implementation of a Collection), however, are. But you must append to them.
So, what this line does:
count.set(idx, count.get(idx) + 1);
is take the value at index idx, add 1 to it, and sets back this value at the same index.

In your specific case, what you are looking for is a sparse array. In Java we use a HashMap<Integer, Integer> for that purpose. You don't need any initialization with zeros, but you do need to check for null:
final Integer curr = count.get(idx);
count.put(idx, curr == null? 1 : curr+1);

You are currently using an ArrayList, which acts like an array, but makes it easier to expand it - for example, add elements to it.
What you want is the java equivalent of the python dict, a key/value storage, in java, this is called a HashMap<K,V> (docs).
Scanner data1 = null;
HashMap<Integer, Integer> count = new HashMap<Integer, Integer>();
Integer idx;
while(data1.hasNextInt()) {
idx = data1.nextInt();
System.out.println(idx);
Integer g = count.get(idx)+1;
if(g == null) {
g = 0;
}
g++;
System.out.println(g);
count.put(idx, g);
}