We Keep Coding

Sum up List to a limit with streams

How to get the records, which count sum should be in limit. In below example there is Records Object contains recordId and count, i wanted to fetch the records data based on the total sum of count should be less than or equal to my limit condition.
public class Records {
private int recordID;
private int count;
public Records(int recordID, int count) {
this.recordID = recordID;
this.count = count;
}
public int getRecordID() {
return recordID;
}
public void setRecordID(int recordID) {
this.recordID = recordID;
}
public int getCount() {
return count;
}
public void setCount(int count) {
this.count = count;
}
}
public static void main(String[] args) {
final List<Records> recordList = new ArrayList<>();
recordList.add(new Records(100, 10));
recordList.add(new Records(501, 20));
recordList.add(new Records(302, 5));
recordList.add(new Records(405, 2));
recordList.add(new Records(918, 8));
int limit = 35;
}
Expected Result
recordList should have records objects : [100,10], [500,20], [302,5] records

The problems of solving this with Stream API is that you have to keep some information outside of the context of processing and read/update (depend) on it at the same time. These tasks are not suitable for Stream API.
Use a for-loop instead which is suitable and great for this:
int index = 0; // highest index possible
int sum = 0; // sum as a temporary variable
for (int i=0; i<recordList.size(); i++) { // for each Record
sum += recordList.get(i).getCount(); // ... add the 'count' to the 'sum'
if (sum <= limit) { // ... until the sum is below the limit
index = i; // ... move the pivot
} else break; // ... or else stop processing
}
// here you need to get the list from 0 to index+1
// as long as the 2nd parameter of subList(int, int) is exlcusive
List<Record> filteredRecords = recordList.subList(0, index + 1);

This is the only thing I could come up with but it's not as efficient as a regular loop because it runs for each list entry it has. That also causes it to add other values further down. For example if limit was 46, third entry where count is 5 would be skipped but the next entry with count 2 would be still added. Don't know if this is desired behavior for you
AtomicInteger count = new AtomicInteger();
recordList = recordList.stream().filter(r -> {
if(count.get() + r.count <= limit){
count.addAndGet(r.count);
return true;
}
return false;
}).collect(Collectors.toList());

Adding the following toString to your class for printing you can do it as follows:
public String toString() {
return String.format("[%s, %s]", recordID, count);
}
Allocate a List to store the results
initialize the sum
iterate thru the list, summing the count until the
threshhold is reached.
List<Records> results = new ArrayList<>();
int sum = 0;
for (Records rec : recordList) {
// sum the counts
sum += rec.getCount();
if (sum > limit) {
// stop when limit exceeded
break;
}
results.add(rec);
}
results.forEach(System.out::println);
Prints
[100, 10]
[501, 20]
[302, 5]

with java 8 you can do something like:
public static void main(String[] args) {
int limit = 35;
List<Records> recordList = new ArrayList<>();
recordList.add(new Records(100, 10));
recordList.add(new Records(501, 20));
recordList.add(new Records(302, 5));
recordList.add(new Records(405, 2));
recordList.add(new Records(918, 8));
List<Records> limitedResult = recordList.stream().filter(new Predicate<Records>() {
int sum = 0;
#Override
public boolean test(Records records) {
sum=sum+records.getCount();
return sum <= limit;
}
}).collect(Collectors.toList());
//do what do you want with limitedResult
System.out.println(limitedResult);
}
Edit:
Or you can make function which return Predicate which can be reuse as:
//Reusable predicate
public static Predicate<Records> limitRecordPredicate(int limit){
return new Predicate<Records>() {
int sum = 0;
#Override
public boolean test (Records records){
sum = sum + records.getCount();
return sum <= limit;
}
};
}
and then used it like:
List<Records> limitedResult = recordList.stream().filter(limitRecordPredicate(limit)).collect(Collectors.toList());
//do what do you want with limitedResult
System.out.println(limitedResult);
Output:
[Records[recordID=100, count=10], Records[recordID=501, count=20], Records[recordID=302, count=5]]

java multi threads access Hashtable

I tried to use multi threads to access the Hashtable, since Hashtable is thread safe on get. But I cannot get it work.
I thought the sum of local counter should be equal to the size of the Hashtable or the global_counter. But it is not.
Serveral threads get java.util.NoSuchElementException: Hashtable Enumerator error. I think the error is due to the enumeration of Hashtable. Is that so?
TestMain:
public class TestMain {
// MAIN
public static void main(String argv[]) throws InterruptedException
{
Hashtable<Integer, Integer> id2 = new Hashtable<Integer, Integer>();
for (int i = 0; i < 100000; ++i)
id2.put(i, i+1);
int num_threads = Runtime.getRuntime().availableProcessors() - 1;
ExecutorService ExeSvc = Executors.newFixedThreadPool(num_threads);
for (int i = 0; i < num_threads; ++i)
{
ExeSvc.execute(new CalcLink(id2, i));
}
ExeSvc.shutdown();
ExeSvc.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
}
}
CalcLink:
public class CalcLink implements Runnable {
private Hashtable<Integer, Integer> linktable;
private static Enumeration keys;
private static int global_counter;
private int thread_id;
private int total_size;
public CalcLink(Hashtable<Integer, Integer> lt, int id)
{
linktable = lt;
keys = lt.keys();
thread_id = id;
total_size = lt.size();
global_counter = 0;
}
private synchronized void increment()
{
++global_counter;
}
#Override
public void run()
{
int counter = 0;
while (keys.hasMoreElements())
{
++counter;
increment();
Integer key = (Integer)keys.nextElement();
Integer value = linktable.get(key);
}
System.out.println("local counter = " + Integer.toString(counter));
if (thread_id == 1)
System.out.println("global counter = " + Integer.toString(global_counter));
}
}

while (keys.hasMoreElements()) // here you check whether there's an element
{
++counter; // other stuff...
increment(); // other stuff...
Integer key = (Integer)keys.nextElement(); // only here you step
During you are in the other stuff in "this thread" you can enter the other stuff in another thread, thus IMHO you might see higher number in the global counter than what you expect.
This is also the reason you see NoSuchElementException in some of the threads, that entered to the "other stuff" together, but are trying to catch the last element. The later threads won't have the element there when they get to nextElement();

The problem is that this block isn't synchronized :
while (keys.hasMoreElements())
{
++counter;
increment();
Integer key = (Integer)keys.nextElement();
Integer value = linktable.get(key);
}
keys.hasMoreElements() can be evaluated to true in multiple threads when there is still only one element in the Enumeration. In those threads : the first one reaching keys.nextElement() will be fine, but all the others will raise a NoSuchElementException
Try this :
#Override
public void run()
{
int counter = 0;
synchronized (keys){
while (keys.hasMoreElements())
{
++counter;
increment();
Integer key = (Integer)keys.nextElement();
Integer value = linktable.get(key);
}
}
System.out.println("local counter = " + Integer.toString(counter));
if (thread_id == 1)
System.out.println("global counter = " + Integer.toString(global_counter));
}

An naive solution: I just let each thread to process Length / num_threads of records. Only the last thread will process length/num_threads + length%num_threads records.

Java Integer Accumulator optimisation

I need to write Integer Accumulator. Implementation needs to be thread-safe and efficient under high thread contention.
Usage example:
Accumulator accumulator = new AccumulatorSum();
int firstSum = accumulator.accumulate(1, 2, 3);
int secondSum = accumulator.accumulate(4);
int total = accumulator.getTotal();
In this case, the value of firstSum is 6, secondSum is 4 and the value of total is 10.
Calling accumulator.reset() would reset the total value to 0.
Code below is something that I've managed to do. Do you think it is thread-safe, guarantees consistency and efficient enough?
Thanks for any help!
public class AccumulatorSum implements Accumulator{
private ReadWriteLock rwlock = new ReentrantReadWriteLock();
private Integer totalSum = 0;
#Override
public int accumulate(int... values) {
int sum = 0;
for(int v : values){
sum += v;
}
rwlock.writeLock().lock();
try {
totalSum += sum;
}
finally {
rwlock.writeLock().unlock();
}
return sum;
}
#Override
public int getTotal() {
rwlock.readLock().lock();
try {
return totalSum;
}
finally {
rwlock.readLock().unlock();
}
}
#Override
public void reset() {
rwlock.writeLock().lock();
try{
totalSum = 0;
}
finally {
rwlock.writeLock().unlock();
}
}

ArrayIndexOutOfBoundsException at method of concurrency approaches comprise

I want to run some comparison of different approaches for concurrency technique.
But it throws next exceptions:
Warmup
BaseLine : 21246915
============================
Cycles : 50000
Exception in thread "pool-1-thread-3" Exception in thread "pool-1-thread-5" java.lang.ArrayIndexOutOfBoundsException: 100000
at concurrency.BaseLine.accumulate(SynchronizationComparisons.java:89)
at concurrency.Accumulator$Modifier.run(SynchronizationComparisons.java:39)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)
java.lang.ArrayIndexOutOfBoundsException: 100000
at concurrency.BaseLine.accumulate(SynchronizationComparisons.java:89)
at concurrency.Accumulator$Modifier.run(SynchronizationComparisons.java:39)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)
Here is code:
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
import java.util.concurrent.locks.*;
import java.util.*;
import static net.mindview.util.Print.*;
abstract class Accumulator {
public static long cycles = 50000L;
// Number of Modifiers and Readers during each test:
private static final int N = 4;
public static ExecutorService exec = Executors.newFixedThreadPool(N * 2);
private static CyclicBarrier barrier = new CyclicBarrier(N * 2 + 1);
protected volatile int index = 0;
protected volatile long value = 0;
protected long duration = 0;
protected String id = "error";
protected final static int SIZE = 100000;
protected static int[] preLoaded = new int[SIZE];
static {
// Load the array of random numbers:
Random rand = new Random(47);
for (int i = 0; i < SIZE; i++)
preLoaded[i] = rand.nextInt();
}
public abstract void accumulate();
public abstract long read();
private class Modifier implements Runnable {
public void run() {
for (long i = 0; i < cycles; i++)
accumulate();
try {
barrier.await();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
private class Reader implements Runnable {
#SuppressWarnings("unused")
private volatile long value;
public void run() {
for (long i = 0; i < cycles; i++)
value = read();
try {
barrier.await();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
public void timedTest() {
long start = System.nanoTime();
for (int i = 0; i < N; i++) {
exec.execute(new Modifier());
exec.execute(new Reader());
}
try {
barrier.await();
} catch (Exception e) {
throw new RuntimeException(e);
}
duration = System.nanoTime() - start;
printf("%-13s: %13d\n", id, duration);
}
public static void report(Accumulator acc1, Accumulator acc2) {
printf("%-22s: %.2f\n", acc1.id + "/" + acc2.id, (double) acc1.duration / (double) acc2.duration);
}
}
class BaseLine extends Accumulator {
{
id = "BaseLine";
}
public void accumulate() {
value += preLoaded[index++];
if (index >= SIZE)
index = 0;
}
public long read() {
return value;
}
}
class SynchronizedTest extends Accumulator {
{
id = "synchronized";
}
public synchronized void accumulate() {
value += preLoaded[index++];
if (index >= SIZE)
index = 0;
}
public synchronized long read() {
return value;
}
}
class LockTest extends Accumulator {
{
id = "Lock";
}
private Lock lock = new ReentrantLock();
public void accumulate() {
lock.lock();
try {
value += preLoaded[index++];
if (index >= SIZE)
index = 0;
} finally {
lock.unlock();
}
}
public long read() {
lock.lock();
try {
return value;
} finally {
lock.unlock();
}
}
}
class AtomicTest extends Accumulator {
{
id = "Atomic";
}
private AtomicInteger index = new AtomicInteger(0);
private AtomicLong value = new AtomicLong(0);
public void accumulate() {
// Oops! Relying on more than one Atomic at
// a time doesn't work. But it still gives us
// a performance indicator:
int i = index.getAndIncrement();
value.getAndAdd(preLoaded[i]);
if (++i >= SIZE)
index.set(0);
}
public long read() {
return value.get();
}
}
public class SynchronizationComparisons {
static BaseLine baseLine = new BaseLine();
static SynchronizedTest synch = new SynchronizedTest();
static LockTest lock = new LockTest();
static AtomicTest atomic = new AtomicTest();
static void test() {
print("============================");
printf("%-12s : %13d\n", "Cycles", Accumulator.cycles);
baseLine.timedTest();
synch.timedTest();
lock.timedTest();
atomic.timedTest();
Accumulator.report(synch, baseLine);
Accumulator.report(lock, baseLine);
Accumulator.report(atomic, baseLine);
Accumulator.report(synch, lock);
Accumulator.report(synch, atomic);
Accumulator.report(lock, atomic);
}
public static void main(String[] args) {
int iterations = 5; // Default
if (args.length > 0) // Optionally change iterations
iterations = new Integer(args[0]);
// The first time fills the thread pool:
print("Warmup");
baseLine.timedTest();
// Now the initial test doesn't include the cost
// of starting the threads for the first time.
// Produce multiple data points:
for (int i = 0; i < iterations; i++) {
test();
Accumulator.cycles *= 2;
}
Accumulator.exec.shutdown();
}
}
How to solve this trouble?

The array preLoaded is of size 100000. So, the valid index starts from 0 to 99999 since array index starts from 0. You need to swap the statements in method accumulate()
Change this
value += preLoaded[index++]; //index validity is not done
if (index >= SIZE)
index = 0;
to
if (index >= SIZE)
index = 0;
value += preLoaded[index++]; // index validity is done and controlled
This will not make the index go to 100000. It will make it to 0 when it turns 100000 before the index value is accessed.
Note : The above code is vulnerable only in multi-threaded environment. The above code will work fine with single thread.

Change BaseLine class and AtomicTest class:
class BaseLine extends Accumulator {
{
id = "BaseLine";
}
public void accumulate() {
int early = index++; // early add and assign to a temp.
if(early >= SIZE) {
index = 0;
early = 0;
}
value += preLoaded[early];
}
public long read() {
return value;
}
}
class AtomicTest extends Accumulator {
{
id = "Atomic";
}
private AtomicInteger index = new AtomicInteger(0);
private AtomicLong value = new AtomicLong(0);
public void accumulate() {
int early = index.getAndIncrement();
if(early >= SIZE) {
index.set(0);
early = 0;
}
value.getAndAdd(preLoaded[early]);
}
public long read() {
return value.get();
}
}

I suspect that you're running into concurrent executions of BaseLine.accumulate() near the boundary of the preLoaded array.
You've got 4 threads hammering away at an unsynchronized method, which is potentially leading to index being incremented to 100000 by say, Thread 1, and before Thread 1 can set it back to 0, one of Thread 2, 3 or 4 is coming in and attempting to access preLoaded[index++], which fails as index is still 100000.

java programming & finding the mode of an array

i have a task where i need to find the mode of an array. which means i am looking for the int which is most frequent. i have kinda finished that, but the task also says if there are two modes which is the same, i should return the smallest int e.g {1,1,1,2,2,2} should give 1 (like in my file which i use that array and it gives 2)
public class theMode
{
public theMode()
{
int[] testingArray = new int[] {1,1,1,2,2,2,4};
int mode=findMode(testingArray);
System.out.println(mode);
}
public int findMode(int[] testingArray)
{
int modeWeAreLookingFor = 0;
int frequencyOfMode = 0;
for (int i = 0; i < testingArray.length; i++)
{
int currentIndexOfArray = testingArray[i];
int frequencyOfEachInArray = howMany(testingArray,currentIndexOfArray);
if (frequencyOfEachInArray > frequencyOfMode)
{
modeWeAreLookingFor = currentIndexOfArray;
frequencyOfMode = modeWeAreLookingFor;
}
}
return modeWeAreLookingFor;
}
public int howMany(int[] testingArray, int c)
{
int howManyOfThisInt=0;
for(int i=0; i < testingArray.length;i++)
{
if(testingArray[i]==c){
howManyOfThisInt++;
}
}
return howManyOfThisInt;
}
public static void main(String[] args)
{
new theMode();
}
}
as you see my algorithm returns the last found mode or how i should explain it.

I'd approach it differently. Using a map you could use each unique number as the key and then the count as the value. step through the array and for each number found, check the map to see if there is a key with that value. If one is found increment its value by 1, otherwise create a new entry with the value of 1.
Then you can check the value of each map entry to see which has the highest count. If the current key has a higher count than the previous key, then it is the "current" answer. But you have the possibility of keys with similar counts so you need to store each 'winnning' answer.
One way to approach this is to check each map each entry and remove each entry that is less than the current highest count. What you will be left with is a map of all "highest counts". If you map has only one entry, then it's key is the answer, otherwise you will need to compare the set of keys to determine the lowest.

Hint: You're updating ModeWeAreLookingFor when you find a integer with a strictly higher frequency. What if you find an integer that has the same frequency as ModeWeAreLookingFor ?
Extra exercice: In the first iteration of the main loop execution, you compute the frequency of '1'. On the second iteration (and the third, and the fourth), you re-compute this value. You may save some time if you store the result of the first computation. Could be done with a Map.
Java code convention states that method names and variable name should start with a lower case character. You would have a better syntax coloring and code easier to read if you follow this convention.

this might work with a little modification.
http://www.toves.org/books/java/ch19-array/index.html#fig2
if ((count > maxCount) || (count == maxCount && nums[i] < maxValue)) {
maxValue = nums[i];
maxCount = count;
}

since it seems there are no other way, i did a hashmap after all. i am stuck once again in the logics when it comes to comparing frequencys and and the same time picking lowest integer if equal frequencys.
public void theMode()
{
for (Integer number: intAndFrequencyMap.keySet())
{
int key = number;
int value = intAndFrequencyMap.get(number);
System.out.println("the integer: " +key + " exists " + value + " time(s).");
int lowestIntegerOfArray = 0;
int highestFrequencyOfArray = 0;
int theInteger = 0;
int theModeWanted = 0;
if (value > highestFrequencyOfArray)
{
highestFrequencyOfArray = value;
theInteger = number;
}
else if (value == highestFrequencyOfArray)
{
if (number < theInteger)
{
number = theInteger;
}
else if (number > theInteger)
{
}
else if (number == theInteger)
{
number = theInteger;
}
}
}
}

Completed:
import java.util.Arrays;
public class TheMode
{
//Probably not the most effective solution, but works without hashmap
//or any sorting algorithms
public TheMode()
{
int[] testingArray = new int[] {2,3,5,4,2,3,3,3};
int mode = findMode(testingArray);
System.out.println(Arrays.toString(testingArray));
System.out.println("The lowest mode is: " + mode);
int[] test2 = new int[] {3,3,2,2,1};
int mode2=findMode(test2);
System.out.println(Arrays.toString(test2));
System.out.println("The lowest mode is: " +mode2);
int[] test3 = new int[] {4,4,5,5,1};
int mode3 = findMode(test3);
System.out.println(Arrays.toString(test3));
System.out.println(The lowest mode is: " +mode3);
}
public int findMode(int[] testingArray)
{
int modeWeAreLookingFor = 0;
int frequencyOfMode = 0;
for (int i = 0; i < testingArray.length; i++)
{
int currentIndexOfArray = testingArray[i];
int countIntegerInArray = howMany(testingArray, currentIndexOfArray);
if (countIntegerInArray == frequencyOfMode)
{
if (modeWeAreLookingFor > currentIndexOfArray)
{
modeWeAreLookingFor = currentIndexOfArray;
}
}
else if (countIntegerInArray > frequencyOfMode)
{
modeWeAreLookingFor = currentIndexOfArray;
frequencyOfMode = countIntegerInArray;
}
}
return modeWeAreLookingFor;
}
public int howMany(int[] testingArray, int c)
{
int howManyOfThisInt=0;
for(int i=0; i < testingArray.length;i++)
{
if(testingArray[i]==c){
howManyOfThisInt++;
}
}
return howManyOfThisInt;
}
public static void main(String[] args)
{
new TheMode();
}
}

Glad you managed to solve it. As you will now see, there is more than one way to approach a problem. Here's what I meant by using a map
package util;
import java.util.HashMap;
import java.util.Map;
public class MathUtil {
public static void main(String[] args) {
MathUtil app = new MathUtil();
int[] numbers = {1, 1, 1, 2, 2, 2, 3, 4};
System.out.println(app.getMode(numbers));
}
public int getMode(int[] numbers) {
int mode = 0;
Map<Integer, Integer> numberMap = getFrequencyMap(numbers);
int highestCount = 0;
for (int number : numberMap.keySet()) {
int currentCount = numberMap.get(number);
if (currentCount > highestCount) {
highestCount = currentCount;
mode = number;
} else if (currentCount == highestCount && number < mode) {
mode = number;
}
}
return mode;
}
private Map<Integer,Integer> getFrequencyMap(int[] numbers){
Map<Integer, Integer> numberMap = new HashMap<Integer, Integer>();
for (int number : numbers) {
if (numberMap.containsKey(number)) {
int count = numberMap.get(number);
count++;
numberMap.put(number, count);
} else {
numberMap.put(number, 1);
}
}
return numberMap;
}
}