Java 8 - Using ConcurrentHashMap.merge in a class and thread safety - java

While trying out the merge in ConcurrentHashMap, I designed the following class to do the following:
Add a value (double) for a key. If there is an existing Statistic for that key then recalculate the running total (which is the sum of all the values so far seen), the counter is incremented (to take into account the number of hits).
To clarify, let's take an example: we need to hold the response times for a particular method invocation. For each response time, we invoke the "addStatistic" method and pass in a "response time". The response time is accumulated, the counter is incremented (to account for one more invocation event) and the min and max are recalculated and added as a Statistic back for that key.
This is a draft implementation of the functionality that captures, for example, the "response time" of an invocation.
Now the ConcurrentHashMap.merge, is documented as atomic and am taking care to not leak any references, was wondering if there is a thread safety / race condition in this code?
public class SummaryStatistics implements SummaryStatisticsI {
public static class Statistic {
private double min = Double.MAX_VALUE;
private double max = Double.MIN_VALUE;
// the sum of all the values seen so far
private DoubleAccumulator runningValue = new DoubleAccumulator((x, y) -> x + y, 0.0);
private int counter = 0;
protected double getMin() {
return min;
}
protected double getMax() {
return max;
}
protected double getValue() {
return runningValue.get();
}
protected int getCounter() {
return counter;
}
}
private final Map<String, Statistic> statsMap = new ConcurrentHashMap<>();
private Map<String, Statistic> getStatsMap() {
return statsMap;
}
#Override
public void addStatistic(String key, double value) {
Statistic s2 = new Statistic();
s2.runningValue.accumulate(value);
s2.min = value;
s2.max = value;
s2.counter++;
getStatsMap().merge(key, s2, new BiFunction<Statistic, Statistic, Statistic>() {
#Override
public Statistic apply(Statistic oldValue, Statistic s2) {
// only called when there is an existing value for this key
// all of this is atomic
oldValue.max = oldValue.max > s2.max ? oldValue.max : s2.max;
oldValue.min = oldValue.min < s2.min ? oldValue.min : s2.min;
// s2.min == s2.max == s2.runningValue.get()
oldValue.runningValue.accumulate(s2.min);
oldValue.counter++;
return oldValue;
}
});
}
#Override
public Statistic getStatistic(String key) {
return getStatsMap().get(key);
}
}

Related

Synchronized across instances of object

I have an object Rotor which has a goalSpeed and a currentSpeed. Each one tries to change its currentSpeed to match the goalSpeed set. I have 4 of these rotors running ing 4 separate threads. Each one gets assigned a new goalSpeed periodically by a controller.
When I attempt in each Rotor to change its currentSpeed, I cannot ever exceed the sum of all rotor's currentSpeed to exceed X value. sum(currentSpeed(Rotor1) + ... + currentSpeed(Rotor2)) !> X.
Here is my issue: when I check wether I can increase the current speed of a Rotor, I make an if statement on the sum of speeds condition. However, it is possible that right after this check, since each rotor is a separate thread that another one changes its value. Therefore my check in the other thread is not valid anymore. How can I make sure that while I'm in the setNewSpeed() method of one rotor, no other rotor will change its current speed?
class Rotor implements Runnable {
private int id;
private int goalSpeed;
private int currentSpeed;
private Controller controller;
private int Y;
private int failedAttempts;
private int successAttempts;
private int maxSpeed;
public int getSuccessAttempts() {
return successAttempts;
}
public void setSuccessAttempts(int successAttempts) {
this.successAttempts = successAttempts;
}
public int getMaxSpeed() {
return maxSpeed;
}
public void setMaxSpeed(int maxSpeed) {
this.maxSpeed = maxSpeed;
}
public int getFailedAttempts() {
return failedAttempts;
}
public Rotor(Controller c, int Y, int id){
this.controller = c;
this.Y = Y;
this.id = id;
this.currentSpeed = 0;
this.failedAttempts = 0;
this.goalSpeed = 0;
this.maxSpeed = 0;
this.successAttempts = 0;
}
synchronized public void setGoalSpeed(int s){
this.goalSpeed = s;
}
public int getCurrentSpeed(){
return currentSpeed;
}
synchronized private void setNewSpeed(){
int currentDrain = 0;
for(Rotor r : controller.getRotors()){
currentDrain = currentDrain + r.getCurrentSpeed();
}
if((currentDrain + (goalSpeed - currentSpeed)) > 20){
//we cannot increase by total amount because drain too high
System.out.println("failed");
this.failedAttempts++;
currentSpeed = currentSpeed + (20 - currentDrain);
System.out.println("currentSpeed:" + currentSpeed);
} else {
System.out.println("success");
successAttempts++;
currentSpeed = goalSpeed;
}
// System.out.println("goalSpeed:" + goalSpeed);
// System.out.println("currentDrain:" + currentDrain);
}
public void run() {
try {
while(true){
setNewSpeed();
if(currentSpeed > maxSpeed){
maxSpeed = currentSpeed;
}
Thread.sleep(Y);
}
} catch (InterruptedException e) {
System.out.println("Rotor " + id + ": checks=" + (int)(successAttempts + failedAttempts) + ", success rate=" + successAttempts + ", failedAttempts=" + failedAttempts + ", max=" + maxSpeed);
}
}
}
Synchronize on a lock that's shared between all the rotors. Now each of them is synchronizing on their own lock (i.e. this), so even though the method is synchronized, it can be called on different objects at the same time.
The most simple way to synchronize them all is to use a static synchronized method.
That said, using an explicit lock object shared among the instances is probably a better approach.
1) You should not write
synchronized private void setNewSpeed() and synchronized public void setGoalSpeed(int s)
but private synchronized void setNewSpeed() and public synchronized void setGoalSpeed(int s) if you want to respect conventions and standard.
2) You declare two synchronized methods in your Rotor Runnable class but it makes no sense because in the synchronized methods you don't manipulate data shared between the threads.
3) You have multiple ways to address your problem.
A flexible solution consists of using a artificial object shared between the threads and performing the lock on this object when you call the setNewSpeed() method. It allows each tread to wait for the lock to be removed before entering in setNewSpeed().
Here is the idea to implement the solution :
Before instantiating the Rotor, create the shared object in this way :
Object lockObject = new Object();
change public Rotor(Controller c, int Y, int id) to public Rotor(Controller c, int Y, int id, Object lockObject)
invoke the constructor of Rotor by adding the same lockObject instance for all Rotors which you want synchronize between them the speed change.
Store the lockObject as an instance field of the Rotor in the constructor body.
In Rotor use the lockObject to make the synchronization in this way :
sample code :
private void setNewSpeed(){
synchronized(lockObject){
... your actual processing
}
}

Return Comparator's compare method using a double?

So I have a class to compare the rating of a film. It implements the comparator class as seen below:
public class FilmComparator implements Comparator<Film> {
private Map<Film, List<Rating>> ratings;
public FilmComparator(Map<Film, List<Rating>> ratings) {
this.ratings = ratings;
}
#Override
public int compare(Film o1, Film o2) {
double average1 = average(o1);
double average2 = average(o2);
return average2 - average1; // I cant do this because it need to return an int
}
private double average(Film f) {
int sum = 0;
for (Rating r : ratings.get(f)) {
sum += r.getValue();
}
return sum / ratings.get(f).size();
}
}
As you can see, the average might not always be an integer. I am wondering how I would be able to have a more accurate compare. For example, I am having issues when the average returns 3.6 for one object but 3.0 for the other. To the compare method, the are the same but I need to show a difference. Is this possible?
Simple, let Double do the work for you. Do
return Double.compare(average1, average2); // or swap if desired

some elements processed more than once, some not at all

I have a fairly straightforward task: I have a list of strings each of which is processed and a score is produced. The string and its score then get added to a map:
public class My1Thread
{
final private static List<String> ids = Arrays.asList("id1","id2","id3","id4","id5");
private static HashMap<String,Double> result = null;
private Double computeResult(String id)
{
Double res = 0.0;
// do stuff to compute result
return res;
}
public static void main(String[] args)
{
result = new HashMap<String,Double>();
for (String id: ids)
{
result.put(id,computeResult(id));
}
}
}
Since scores of any two strings are independent of each other, this seems to be a perfect case to use multithreading. However, I am getting unexpected results, which is probably a typical result for a multithreading newbie.
Here's a m/t version of the above:
public class MyMultiThread
{
final private static int nWorkers = 3; // number of threads
final private static List<String> ids = Arrays.asList("id1","id2","id3","id4","id5");
private static int curIndex = 0; // indexing pointing to position in ids currently being processed
private static HashMap<String,Double> result = null;
public static class Worker implements Runnable {
private int id;
public Worker(int id) {
this.id = id;
}
synchronized void setCounter(final int counter)
{
curIndex = counter;
}
synchronized int getCounter()
{
return curIndex;
}
synchronized void addToResult(final String id, final Double score)
{
result.put(id,score);
}
#Override
public void run()
{
try {
while (true)
{
int index = getCounter();
if (index >= ids.size())
{
// exit thread
return;
}
String id = ids.get(index);
setCounter(index+1);
System.out.print(String.format("Thread %d: processing %s from pos %d\n", id, id, curIndex-1));
Double score = ... // compute score here
addToResult(id,score);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
public static void main(String[] args)
{
result = new HashMap<String,ArrayList<Pair<Document,Double>>>();
for (int i = 0; i < nWorkers; i++) {
Thread worker = new Thread(new MyMultiThread.Worker(i));
worker.start();
}
}
}
According to the output produced by System.out.print, this code appears to be processing some elements of ids more than once while not processing others at all. What am I doing wrong here?
Your while(true) loop inside the thread starts at the index specified in the constructor, and then increment it by one, and then the loop starts again. So thread 0 does index 0, then index 1, etc.. Thread 1 does index 1, then index 2, etc... So index 2 will be done 3 times.
I would use a synchronized linked list for ids, and have each thread take and remove the first element of the list, until the list is empty. Use LinkedList.removeFirst().
Also the result hash map also needs to be synchronized, since multiple threads may write to it at the same time.
The problem is that the map is being modified concurrently in multiple threads, so some updates are getting lost.
You declared the methods that modify the map as synchronized, but note that they are synchronized on multiple worker objects: not on a single object, which would provide the locking you are after.
I'd recommend using ConcurrentHashMap and getting rid of all the synchronized declarations.
Some of your synchronization is too narrow - for example, this bit here:
int index = getCounter();
if (index >= ids.size())
{
// exit thread
return;
}
String id = ids.get(index);
setCounter(index+1);
What happens if thread A reads the counter, thread B reads the counter, then thread A updates the counter?
A: int index = getCounter(); // returns 3
B: int index = getCounter(); // returns 3
...
A: setCounter(index + 1); // sets it to 4
B: setCounter(index + 1); // Uh-oh, sets it to 4 as well, we lost an update!
In this case, when you read a variable, then write to it based on the value you read, both the read and the write need to be within the same synchronization block. Declaring getCounter and setCounter as synchronized is not enough.
Simply use Java 8 Stream API :
Map<String, Double> map = ids.parallelStream().collect(Collectors.toConcurrentMap(id -> id, id -> computeScore(id)));
...
Double computeScore(String id) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return ThreadLocalRandom.current().nextDouble(100);
}
Here's a nice tutorial.

How to get length of path using java 8 streams

I have List<Vector3D> , where Vector3D is a coordinate. I want to find sum of all distance between Vector3D elements of list. I want to find it using java 8 streams. I try to use reduce but it cant help me.
UPD:
Class Vector3D has method double distance(Vector3D) witch find distance between two positions. e.g. i have list with (1,0,0) (2,0,0) (3,0,0). As a result i want to find length of this path. It is 3.
If we are using java 7 or lower we have to do:
public static double calcPathLength(List<Vector3D> path){
double length = 0d;
for (int i=0; i< path.size()-1; i++){
length += path.get(i).distance(path.get(i+1));
}
return length;
}
The operation you are performing is called Mutable reduction.
Pshemo’s answer shows how you can implement such operation ad-hoc by providing the three necessary functions. However, when all three functions are implemented by a dedicated class it might be useful to implement these functions inside a class implementing Collector for easier reuse:
public class Distance implements Collector<Vector3D, Distance.Helper, Double> {
public static final Distance COLLECTOR = new Distance();
static final class Helper {
private double sum = 0;
private Vector3D first = null, previous = null;
}
public Set<Characteristics> characteristics() {
return Collections.emptySet();
}
public Supplier<Helper> supplier() {
return Helper::new;
}
public BiConsumer<Helper, Vector3D> accumulator() {
return (helper,vector3d)-> {
if (helper.previous != null)
helper.sum += vector3d.distance(helper.previous);
else helper.first = vector3d;
helper.previous = vector3d;
};
}
public BinaryOperator<Helper> combiner() {
return (h1,h2)-> {
h2.sum += h1.sum;
if(h1.previous!=null && h2.first!=null) {
h2.sum += h1.previous.distance(h2.first);
h2.first=h1.first;
}
return h2;
};
}
public Function<Helper, Double> finisher() {
return helper -> helper.sum;
}
}
You will recognize the three function from the ad-hoc version. New is a fourth function, finisher which allows to specify how the final result can be extracted from the mutable container so we don’t need the getSum() call.
The use case simplifies to:
List<Vector3D> list;
//…
double distance=list.stream().collect(Distance.COLLECTOR);
One of the options would be creating some helper class which would remember previously used vector and based on it calculate difference between it and current vector. This class could look like
class DistanceHelper {
private double sum = 0;
private Vector3D first = null;
private Vector3D last = null;
public void add(Vector3D vector3d) {
if (first == null)
first = vector3d;
if (last != null)
sum += vector3d.distance(last);
last = vector3d;
}
public void combine(DistanceHelper otherHelper) {
//add distance of path from current thread with distance of path
//from other thread
sum += otherHelper.sum;
//also add distance between paths handled by separate threads like
// when path of Thread1 is A->B and Thread2 is C->D then we need to
// include path from `B` to `C`
if (this.last!=null && otherHelper.first!=null)
sum += this.last.distance(otherHelper.first);
this.last = otherHelper.last;
}
public double getSum() {
return sum;
}
}
and you can use it for example with combine instead of reduce like
double sum = list
.stream()//or parallelStream()
.collect(DistanceHelper::new, DistanceHelper::add,
DistanceHelper::combine).getSum();

Providing a field accessor method as an argument

G'day comrades. I have a problem.
I have two methods which are completely duplicate apart from that they are accessing a different field. I cannot pass the field value as a parameter because accessing takes place inside a loop (simplified example):
public final class Thing {
ImmutableList<Box> boxes;
public int getNumberOfApples() {
int total = 0;
for (Box box : boxes) {
total += box.getApplesCount();
}
return total;
}
public int getNumberOfPears() {
int total = 0;
for (Box box : boxes) {
total += box.getPearsCount();
}
return total;
}
}
I could put my fruit into a map and passed the field name as a parameter but it looks dirty and I am quite happy with the current class composition. So the question is - how do I refactor my code to have a single method of type:
public int getNumberOfFruit(SomethingMagic) {
moreMagic;
return total;
}
Cheerio.
Well, you could have something like:
public interface Function<In, Out> {
Out apply(In input);
}
Then:
public int getCount(Function<Box, Integer> projection) {
int total = 0;
for (Box box : boxes) {
total += projection(box);
}
return total;
}
For the moment constructing that projection will be ugly, e.g.
int pears = thing.getCount(new Function<Box, Integer>() {
#Override public Integer apply(Box input) {
return box.getPearsCount();
}
});
but in Java 8, it'll be much simpler with lambda expressions:
int pears = thing.getCount(box -> box.getPearsCount());
Note that Raffaele Rossi's answer is slightly more specific than mine by making the interface non-generic. This means it can be more efficient, as there's no need to box the counts - but less reusable, of course. Which approach you decide to use is a judgement call which will largely depend on your actual use case.
I would suggest a functional approach. Define a function to pass into your getNumberOfFruit which will retrieve the number of the correct fruit. Something like:
public interface FruitCounter {
int apply(Box box);
}
public int getNumberOfFruit(FruitCounter counter) {
int total = 0;
for (Box box : boxes) {
total += counter.apply(box);
}
return total;
}
Then pass the proper implementation to count apples or pears:
int numOfApples theThing.getNumberOfFruit(new FruitCounter() {
#Override
public int apply(Box box) {
return box.getApplesCount();
});
int numOfPears theThing.getNumberOfFruit(new FruitCounter() {
#Override
public int apply(Box box) {
return box.getPearsCount();
});

Categories