Exclude overlapping intervals - java

I have two lists of intervals. I would like to remove all times from list1 that already exists in list2.
Example:
List1:
[(0,10),(15,20)]
List2:
[(2,3),(5,6)]
Output:
[(0,2),(3,5),(6,10),(15,20)]
Any hints?
Tried to remove one interval at the time but it seems like I need to take a different approach:
public List<Interval> removeOneTime(Interval interval, Interval remove){
List<Interval> removed = new LinkedList<Interval>();
Interval overlap = interval.getOverlap(remove);
if(overlap.getLength() > 0){
List<Interval> rms = interval.remove(overlap);
removed.addAll(rms);
}
return removed;
}

I would approach this problem with a sweep line algorithm. The start and end points of the intervals are events, that are put in a priority queue. You just move from left to right, stop at every event, and update the current status according to that event.
I made a small implementation, in which I use the following Interval class, just for simplicity:
public class Interval {
public int start, end;
public Interval(int start, int end) {
this.start = start;
this.end = end;
}
public String toString() {
return "(" + start + "," + end + ")";
}
}
The event points mentioned earlier are represented by the following class:
public class AnnotatedPoint implements Comparable<AnnotatedPoint> {
public int value;
public PointType type;
public AnnotatedPoint(int value, PointType type) {
this.value = value;
this.type = type;
}
#Override
public int compareTo(AnnotatedPoint other) {
if (other.value == this.value) {
return this.type.ordinal() < other.type.ordinal() ? -1 : 1;
} else {
return this.value < other.value ? -1 : 1;
}
}
// the order is important here: if multiple events happen at the same point,
// this is the order in which you want to deal with them
public enum PointType {
End, GapEnd, GapStart, Start
}
}
Now, what remains is building the queue and doing the sweep, as shown in the code below
public class Test {
public static void main(String[] args) {
List<Interval> interval = Arrays.asList(new Interval(0, 10), new Interval(15, 20));
List<Interval> remove = Arrays.asList(new Interval(2, 3), new Interval(5, 6));
List<AnnotatedPoint> queue = initQueue(interval, remove);
List<Interval> result = doSweep(queue);
// print result
for (Interval i : result) {
System.out.println(i);
}
}
private static List<AnnotatedPoint> initQueue(List<Interval> interval, List<Interval> remove) {
// annotate all points and put them in a list
List<AnnotatedPoint> queue = new ArrayList<>();
for (Interval i : interval) {
queue.add(new AnnotatedPoint(i.start, PointType.Start));
queue.add(new AnnotatedPoint(i.end, PointType.End));
}
for (Interval i : remove) {
queue.add(new AnnotatedPoint(i.start, PointType.GapStart));
queue.add(new AnnotatedPoint(i.end, PointType.GapEnd));
}
// sort the queue
Collections.sort(queue);
return queue;
}
private static List<Interval> doSweep(List<AnnotatedPoint> queue) {
List<Interval> result = new ArrayList<>();
// iterate over the queue
boolean isInterval = false; // isInterval: #Start seen > #End seen
boolean isGap = false; // isGap: #GapStart seen > #GapEnd seen
int intervalStart = 0;
for (AnnotatedPoint point : queue) {
switch (point.type) {
case Start:
if (!isGap) {
intervalStart = point.value;
}
isInterval = true;
break;
case End:
if (!isGap) {
result.add(new Interval(intervalStart, point.value));
}
isInterval = false;
break;
case GapStart:
if (isInterval) {
result.add(new Interval(intervalStart, point.value));
}
isGap = true;
break;
case GapEnd:
if (isInterval) {
intervalStart = point.value;
}
isGap = false;
break;
}
}
return result;
}
}
This results in:
(0,2)
(3,5)
(6,10)
(15,20)

You probably want to use an interval tree - this will quickly tell you if an interval overlaps with any of the intervals in the tree.
Once you have a set of overlapping intervals the task should be fairly easy (interval1 is from list1, interval2 is the overlapping interval from list2 / the interval tree): if interval1 contains interval2, then you have two new intervals (interval1min, interval2min), (interval2max, interval1max); if interval1 does not contain interval2, then you only have one new interval (interval1min, interval2min) or (interval2max, interval1max)

Related

Java Detecting a cyclic directed Graph

I am currently trying to write a procedure to check whether a directed graph is cyclic or not. I am not sure what i did wrong (it may be well possible that I did everything wrong, so please StackOverflow, show me my stupidity!). I'd be thankful for any kind of help as I've come to the point where I don't know what could be the problem.
The input is an adjacency list such as:
0: 2 4
1: 2 4
2: 3 4
3: 4
4: 0 1 2 3
(0 directs to 2 and 4; 1 directs to 2 and 4 and so on...)
The idea is that I check whether the node I am checking is 'grey' (partially explored) or not. If it is, it must be a back edge and thus a cyclic graph. Black edges are always explored or cross-edges, so this shouldn't trigger a cyclic message. I am aiming to do depth first search
If A-->B and B-->A, this should not trigger a message about cyclic (but A--> B, B-->C, C-->A should).
hasCycle calls hasCycleInSubgraph which calls itself recursively through the Adjency List of the Graph.
class qs {
private ArrayList<Integer>[] adjList;
private Stack<Integer> stack;
private ArrayList<Integer> whiteHat;
private ArrayList<Integer> greyHat;
private ArrayList<Integer> blackHat;
public qs(ArrayList<Integer>[] graph) {
this.adjList = graph;
this.stack = new Stack();
this.whiteHat = new ArrayList<Integer>();
this.greyHat = new ArrayList<Integer>();
this.blackHat = new ArrayList<Integer>();
for (Integer h = 0; h < adjList.length; h++) {
whiteHat.add(h);
}
}
public boolean hasCycle() {
for (Integer i = 0; i < adjList.length; i++) {
// System.out.print("Local root is: ");
// System.out.println(i);
whiteHat.remove(i);
greyHat.add(i);
if (hasCycleInSubgraph(i) == true) {
return true;
}
greyHat.remove(i);
blackHat.add(i);
}
return false;
}
public boolean hasCycleInSubgraph(Integer inp) {
if (blackHat.contains(inp)) {
return false;
}
for (Integer branch : adjList[inp]) {
// System.out.print("Adj is: ");
// System.out.println(branch);
if ( greyHat.contains(branch) && !inp.equals(branch) ) {
return true;
}
whiteHat.remove(branch);
greyHat.add(branch);
if ( hasCycleInSubgraph(branch) == true ) {
return true;
}
greyHat.remove(branch);
blackHat.add(branch);
}
return false;
}
}
You are over-complicating it: a cycle can be detected via a depth-first search: from any given node, walk to each of the connected nodes; if you arrive back at an already-visited node, you've got a cycle.
class qs {
private final ArrayList<Integer>[] graph;
qs(ArrayList<Integer>[] graph) {
this.graph = graph;
}
boolean hasCycle() {
List<Integer> visited = new ArrayList<>();
for (int i = 0; i < graph.length; ++i) {
if (hasCycle(i, visited)) {
return true;
}
}
}
private boolean hasCycle(int node, List<Integer> visited) {
if (visited.contains(node)) {
return true;
}
visited.add(node);
for (Integer nextNode : graph[node]) {
if (hasCycle(nextNode, visited)) {
return true;
}
}
visited.remove(visited.length() - 1);
return false;
}
}
If you want to detect cycles longer than a given length, just check the depth of the recursion:
if (visited.contains(node) && visited.size() > 2) {
Note that this does not require any state to be kept, aside from what is in the stack. Relying upon mutable state makes the code thread-unsafe (e.g. that two threads calling hasCycle at the same time would interfer with each other), and so should be avoided - even if you don't expect the code to be used in a multi-threaded way now, it avoids problems down the line.

Java iterate through priority queue

I'm trying to do a while loop that will iterate through a java priority queue and read in top of the queue's date. One it has the date value, looks through the rest of the queue to see if this date is being used elsewhere, if so add these elements temporarily to their own queue so I can call a different comparator method to sort them out.
public JobRequest closestDeadlineJob(int freeCPUS) {
// find top job to determine if other jobs for date need to be considered
JobRequest nextJob = scheduledJobs.peek(); // return top most job
// what is it's date?
Date currentDate = nextJob.getConvertedDeadlineDate();
JobPriorityQueue schedulerPriorityQueue = new JobPriorityQueue();
schedulerPriorityQueue.addJob( nextJob );
while(true) {
}
// this is the item at the top of the PRIORTY JOB queue to return
// remove that item from scheduledJobs
// return null; // replace with to the one you want to return
}
what I have so far, as you can see not very much
PriorityQueue does not provide you a sorted iteration order. The only guarantee of a PriorityQueue is that extraction methods (peek/poll/remove) will return a smallest element in the set according to your Comparator. If you need a sorted iteration order - use TreeSet/TreeMap instead.
import java.util.Date;
import java.util.Comparator;
import java.util.PriorityQueue;
class Job implements Runnable{
Priority priority;
Date dateOccurance;
public Job(Priority priority, Date occurance){
this.priority = priority;
this.dateOccurance = occurance;
}
public void run(){
//Job execution
System.out.println("executed");
}
}
enum Priority {
High,
Medium,
Low
}
class JobComparator implements Comparator<Job> {
#Override
public int compare(Job j1, Job j2) {
if(j1.priority.ordinal() > j2.priority.ordinal()) {
return 1;
} else if (j1.priority == j2.priority) {
if(j1.dateOccurance.after(j2.dateOccurance)) {
return 1;
} else if (j1.dateOccurance.before(j2.dateOccurance)) {
return -1;
} else {
return 0;
}
}
return -1;
}
}
public class PriorityQueueTest {
public static void main(String[] args) throws InterruptedException {
Date d = new Date();
Job job1 = new Job(Priority.High, d);
Job job2 = new Job(Priority.High, d);
Job job3 = new Job(Priority.Medium, d);
Job job4 = new Job(Priority.Low, d);
Thread.sleep(2000);
Date l = new Date();
Job job5 = new Job(Priority.Low, l);
Comparator<Job> jComp = new JobComparator();
PriorityQueue<Job> queue =
new PriorityQueue<Job>(10, jComp);
queue.add(job4);
queue.add(job3);
queue.add(job1);
queue.add(job2);
queue.add(job5);
while (queue.size() != 0)
{
Job j = queue.remove();
System.out.println(j.priority +" "+j.dateOccurance);
}
}
}

Hashtable key within integer interval

I don't know if this is possible but i'm trying to make an Hashtable of where Interval is a class with 2 integer / long values, a start and an end and i wanted to make something like this:
Hashtable<Interval, WhateverObject> test = new Hashtable<Interval, WhateverObject>();
test.put(new Interval(100, 200), new WhateverObject());
test.get(new Interval(150, 150)) // returns the new WhateverObject i created above because 150 is betwwen 100 and 200
test.get(new Interval(250, 250)) // doesn't find the value because there is no key that contains 250 in it's interval
So basically what i want is that a key between a range of values in an Interval object give the correspondent WhateverObject. I know i have to override equals() and hashcode() in the interval object, the main problem i think is to somehow have all the values between 100 and 200 (in this specific example) to give the same hash.
Any ideias if this is possible?
Thanks
No need to reinvent the wheel, use a NavigableMap. Example Code:
final NavigableMap<Integer, String> map = new TreeMap<Integer, String>();
map.put(0, "Cry Baby");
map.put(6, "School Time");
map.put(16, "Got a car yet?");
map.put(21, "Tequila anyone?");
map.put(45, "Time to buy a corvette");
System.out.println(map.floorEntry(3).getValue());
System.out.println(map.floorEntry(10).getValue());
System.out.println(map.floorEntry(18).getValue());
Output:
Cry Baby
School Time
Got a car yet?
You could use an IntervalTree. Here's one I made earlier.
public class IntervalTree<T extends IntervalTree.Interval> {
// My intervals.
private final List<T> intervals;
// My center value. All my intervals contain this center.
private final long center;
// My interval range.
private final long lBound;
private final long uBound;
// My left tree. All intervals that end below my center.
private final IntervalTree<T> left;
// My right tree. All intervals that start above my center.
private final IntervalTree<T> right;
public IntervalTree(List<T> intervals) {
if (intervals == null) {
throw new NullPointerException();
}
// Initially, my root contains all intervals.
this.intervals = intervals;
// Find my center.
center = findCenter();
/*
* Builds lefts out of all intervals that end below my center.
* Builds rights out of all intervals that start above my center.
* What remains contains all the intervals that contain my center.
*/
// Lefts contains all intervals that end below my center point.
final List<T> lefts = new ArrayList<T>();
// Rights contains all intervals that start above my center point.
final List<T> rights = new ArrayList<T>();
long uB = Long.MIN_VALUE;
long lB = Long.MAX_VALUE;
for (T i : intervals) {
long start = i.getStart();
long end = i.getEnd();
if (end < center) {
lefts.add(i);
} else if (start > center) {
rights.add(i);
} else {
// One of mine.
lB = Math.min(lB, start);
uB = Math.max(uB, end);
}
}
// Remove all those not mine.
intervals.removeAll(lefts);
intervals.removeAll(rights);
uBound = uB;
lBound = lB;
// Build the subtrees.
left = lefts.size() > 0 ? new IntervalTree<T>(lefts) : null;
right = rights.size() > 0 ? new IntervalTree<T>(rights) : null;
// Build my ascending and descending arrays.
/** #todo Build my ascending and descending arrays. */
}
/*
* Returns a list of all intervals containing the point.
*/
List<T> query(long point) {
// Check my range.
if (point >= lBound) {
if (point <= uBound) {
// In my range but remember, there may also be contributors from left or right.
List<T> found = new ArrayList<T>();
// Gather all intersecting ones.
// Could be made faster (perhaps) by holding two sorted lists by start and end.
for (T i : intervals) {
if (i.getStart() <= point && point <= i.getEnd()) {
found.add(i);
}
}
// Gather others.
if (point < center && left != null) {
found.addAll(left.query(point));
}
if (point > center && right != null) {
found.addAll(right.query(point));
}
return found;
} else {
// To right.
return right != null ? right.query(point) : Collections.<T>emptyList();
}
} else {
// To left.
return left != null ? left.query(point) : Collections.<T>emptyList();
}
}
private long findCenter() {
//return average();
return median();
}
protected long median() {
// Choose the median of all centers. Could choose just ends etc or anything.
long[] points = new long[intervals.size()];
int x = 0;
for (T i : intervals) {
// Take the mid point.
points[x++] = (i.getStart() + i.getEnd()) / 2;
}
Arrays.sort(points);
return points[points.length / 2];
}
/*
* What an interval looks like.
*/
public interface Interval {
public long getStart();
public long getEnd();
}
/*
* A simple implemementation of an interval.
*/
public static class SimpleInterval implements Interval {
private final long start;
private final long end;
public SimpleInterval(long start, long end) {
this.start = start;
this.end = end;
}
public long getStart() {
return start;
}
public long getEnd() {
return end;
}
#Override
public String toString() {
return "{" + start + "," + end + "}";
}
}
}
A naive HashTable is the wrong solution here. Overriding the equals() method doesn't do you any good because the HashTable compares a key entry by the hash code first, NOT the equals() method. The equals() method is only checked AFTER the hash code is matched.
It's easy to make a hash function on your interval object, but it's much more difficult to make one that would yield the same hashcode for all possible intervals that would be within another interval. Overriding the get() method (such as here https://stackoverflow.com/a/11189075/1261844) for a HashTable completely negates the advantages of a HashTable, which is very fast lookup times. At the point where you are scanning through each member of a HashTable, then you know you are using the HashTable incorrectly.
I'd say that Using java map for range searches and https://stackoverflow.com/a/11189080/1261844 are better solutions, but a HashTable is simply not the way to go about this.
I think implementing a specialized get-method would be much easier.
The new method can be part of a map-wrapper-class.
The key-class: (interval is [lower;upper[ )
public class Interval {
private int upper;
private int lower;
public Interval(int upper, int lower) {
this.upper = upper;
this.lower = lower;
}
public boolean contains(int i) {
return i < upper && i >= lower;
}
#Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final Interval other = (Interval) obj;
if (this.upper != other.upper) {
return false;
}
if (this.lower != other.lower) {
return false;
}
return true;
}
#Override
public int hashCode() {
int hash = 5;
hash = 61 * hash + this.upper;
hash = 61 * hash + this.lower;
return hash;
}
}
The Map-class:
public class IntervalMap<T> extends HashMap<Interval, T> {
public T get(int key) {
for (Interval iv : keySet()) {
if (iv.contains(key)) {
return super.get(iv);
}
}
return null;
}
}
This is just an example and can surely be optimized, and there are a few flaws as well:
For Example if Intervals overlap, there's no garantee to know which Interval will be used for lookup and Intervals are not garanteed to not overlap!
OldCurmudgeon's solution works perfectly for me, but is very slow to initialise (took 20 mins for 70k entries).
If you know your incoming list of items is already ordered (ascending) and has only non overlapping intervals, you can make it initialise in milliseconds by adding and using the following constructor:
public IntervalTree(List<T> intervals, boolean constructorFlagToIndicateOrderedNonOverlappingIntervals) {
if (intervals == null) throw new NullPointerException();
int centerPoint = intervals.size() / 2;
T centerInterval = intervals.get(centerPoint);
this.intervals = new ArrayList<T>();
this.intervals.add(centerInterval);
this.uBound = centerInterval.getEnd();
this.lBound = centerInterval.getStart();
this.center = (this.uBound + this.lBound) / 2;
List<T> toTheLeft = centerPoint < 1 ? Collections.<T>emptyList() : intervals.subList(0, centerPoint);
this.left = toTheLeft.isEmpty() ? null : new IntervalTree<T>(toTheLeft, true);
List<T> toTheRight = centerPoint >= intervals.size() ? Collections.<T>emptyList() : intervals.subList(centerPoint+1, intervals.size());
this.right = toTheRight.isEmpty() ? null : new IntervalTree<T>(toTheRight, true);
}
This depends on your hashCode implementation. You may have two Objects with the same hashCode value.
Please use eclipse to generate a hashCode method for your class (no point to re-invent the wheel
For Hastable or HashMap to work as expected it's not only a equal hashcode, but also the equals method must return true. What you are requesting is that Interval(x, y).equals(Interval(m, n)) for m, n within x,y. As this must be true for any overlapping living instance of Interval, the class has to record all of them and needs to implement what you are trying to achieve, indeed.
So in short the answer is no.
The Google guava library is planning to offer a RangeSet and Map: guava RangeSet
For reasonable small ranges an easy approach would be to specialize HashMap by putting and getting the indivual values of the intervals.

Thread-safe circular buffer in Java

Consider a few web server instances running in parallel. Each server holds a reference to a single shared "Status keeper", whose role is keeping the last N requests from all servers.
For example (N=3):
Server a: "Request id = ABCD" Status keeper=["ABCD"]
Server b: "Request id = XYZZ" Status keeper=["ABCD", "XYZZ"]
Server c: "Request id = 1234" Status keeper=["ABCD", "XYZZ", "1234"]
Server b: "Request id = FOO" Status keeper=["XYZZ", "1234", "FOO"]
Server a: "Request id = BAR" Status keeper=["1234", "FOO", "BAR"]
At any point in time, the "Status keeper" might be called from a monitoring application that reads these last N requests for an SLA report.
What's the best way to implement this producer-consumer scenario in Java, giving the web servers higher priority than the SLA report?
CircularFifoBuffer seems to be the appropriate data structure to hold the requests, but I'm not sure what's the optimal way to implement efficient concurrency.
Buffer fifo = BufferUtils.synchronizedBuffer(new CircularFifoBuffer());
Here's a lock-free ring buffer implementation. It implements a fixed-size buffer - there is no FIFO functionality. I would suggest you store a Collection of requests for each server instead. That way your report can do the filtering rather than getting your data structure to filter.
/**
* Container
* ---------
*
* A lock-free container that offers a close-to O(1) add/remove performance.
*
*/
public class Container<T> implements Iterable<T> {
// The capacity of the container.
final int capacity;
// The list.
AtomicReference<Node<T>> head = new AtomicReference<Node<T>>();
// TESTING {
AtomicLong totalAdded = new AtomicLong(0);
AtomicLong totalFreed = new AtomicLong(0);
AtomicLong totalSkipped = new AtomicLong(0);
private void resetStats() {
totalAdded.set(0);
totalFreed.set(0);
totalSkipped.set(0);
}
// TESTING }
// Constructor
public Container(int capacity) {
this.capacity = capacity;
// Construct the list.
Node<T> h = new Node<T>();
Node<T> it = h;
// One created, now add (capacity - 1) more
for (int i = 0; i < capacity - 1; i++) {
// Add it.
it.next = new Node<T>();
// Step on to it.
it = it.next;
}
// Make it a ring.
it.next = h;
// Install it.
head.set(h);
}
// Empty ... NOT thread safe.
public void clear() {
Node<T> it = head.get();
for (int i = 0; i < capacity; i++) {
// Trash the element
it.element = null;
// Mark it free.
it.free.set(true);
it = it.next;
}
// Clear stats.
resetStats();
}
// Add a new one.
public Node<T> add(T element) {
// Get a free node and attach the element.
totalAdded.incrementAndGet();
return getFree().attach(element);
}
// Find the next free element and mark it not free.
private Node<T> getFree() {
Node<T> freeNode = head.get();
int skipped = 0;
// Stop when we hit the end of the list
// ... or we successfully transit a node from free to not-free.
while (skipped < capacity && !freeNode.free.compareAndSet(true, false)) {
skipped += 1;
freeNode = freeNode.next;
}
// Keep count of skipped.
totalSkipped.addAndGet(skipped);
if (skipped < capacity) {
// Put the head as next.
// Doesn't matter if it fails. That would just mean someone else was doing the same.
head.set(freeNode.next);
} else {
// We hit the end! No more free nodes.
throw new IllegalStateException("Capacity exhausted.");
}
return freeNode;
}
// Mark it free.
public void remove(Node<T> it, T element) {
totalFreed.incrementAndGet();
// Remove the element first.
it.detach(element);
// Mark it as free.
if (!it.free.compareAndSet(false, true)) {
throw new IllegalStateException("Freeing a freed node.");
}
}
// The Node class. It is static so needs the <T> repeated.
public static class Node<T> {
// The element in the node.
private T element;
// Are we free?
private AtomicBoolean free = new AtomicBoolean(true);
// The next reference in whatever list I am in.
private Node<T> next;
// Construct a node of the list
private Node() {
// Start empty.
element = null;
}
// Attach the element.
public Node<T> attach(T element) {
// Sanity check.
if (this.element == null) {
this.element = element;
} else {
throw new IllegalArgumentException("There is already an element attached.");
}
// Useful for chaining.
return this;
}
// Detach the element.
public Node<T> detach(T element) {
// Sanity check.
if (this.element == element) {
this.element = null;
} else {
throw new IllegalArgumentException("Removal of wrong element.");
}
// Useful for chaining.
return this;
}
public T get () {
return element;
}
#Override
public String toString() {
return element != null ? element.toString() : "null";
}
}
// Provides an iterator across all items in the container.
public Iterator<T> iterator() {
return new UsedNodesIterator<T>(this);
}
// Iterates across used nodes.
private static class UsedNodesIterator<T> implements Iterator<T> {
// Where next to look for the next used node.
Node<T> it;
int limit = 0;
T next = null;
public UsedNodesIterator(Container<T> c) {
// Snapshot the head node at this time.
it = c.head.get();
limit = c.capacity;
}
public boolean hasNext() {
// Made into a `while` loop to fix issue reported by #Nim in code review
while (next == null && limit > 0) {
// Scan to the next non-free node.
while (limit > 0 && it.free.get() == true) {
it = it.next;
// Step down 1.
limit -= 1;
}
if (limit != 0) {
next = it.element;
}
}
return next != null;
}
public T next() {
T n = null;
if ( hasNext () ) {
// Give it to them.
n = next;
next = null;
// Step forward.
it = it.next;
limit -= 1;
} else {
// Not there!!
throw new NoSuchElementException ();
}
return n;
}
public void remove() {
throw new UnsupportedOperationException("Not supported.");
}
}
#Override
public String toString() {
StringBuilder s = new StringBuilder();
Separator comma = new Separator(",");
// Keep counts too.
int usedCount = 0;
int freeCount = 0;
// I will iterate the list myself as I want to count free nodes too.
Node<T> it = head.get();
int count = 0;
s.append("[");
// Scan to the end.
while (count < capacity) {
// Is it in-use?
if (it.free.get() == false) {
// Grab its element.
T e = it.element;
// Is it null?
if (e != null) {
// Good element.
s.append(comma.sep()).append(e.toString());
// Count them.
usedCount += 1;
} else {
// Probably became free while I was traversing.
// Because the element is detached before the entry is marked free.
freeCount += 1;
}
} else {
// Free one.
freeCount += 1;
}
// Next
it = it.next;
count += 1;
}
// Decorate with counts "]used+free".
s.append("]").append(usedCount).append("+").append(freeCount);
if (usedCount + freeCount != capacity) {
// Perhaps something was added/freed while we were iterating.
s.append("?");
}
return s.toString();
}
}
Note that this is close to O1 put and get. A Separator just emits "" first time around and then its parameter from then on.
Edit: Added test methods.
// ***** Following only needed for testing. *****
private static boolean Debug = false;
private final static String logName = "Container.log";
private final static NamedFileOutput log = new NamedFileOutput("C:\\Junk\\");
private static synchronized void log(boolean toStdoutToo, String s) {
if (Debug) {
if (toStdoutToo) {
System.out.println(s);
}
log(s);
}
}
private static synchronized void log(String s) {
if (Debug) {
try {
log.writeLn(logName, s);
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
static volatile boolean testing = true;
// Tester object to exercise the container.
static class Tester<T> implements Runnable {
// My name.
T me;
// The container I am testing.
Container<T> c;
public Tester(Container<T> container, T name) {
c = container;
me = name;
}
private void pause() {
try {
Thread.sleep(0);
} catch (InterruptedException ex) {
testing = false;
}
}
public void run() {
// Spin on add/remove until stopped.
while (testing) {
// Add it.
Node<T> n = c.add(me);
log("Added " + me + ": " + c.toString());
pause();
// Remove it.
c.remove(n, me);
log("Removed " + me + ": " + c.toString());
pause();
}
}
}
static final String[] strings = {
"One", "Two", "Three", "Four", "Five",
"Six", "Seven", "Eight", "Nine", "Ten"
};
static final int TEST_THREADS = Math.min(10, strings.length);
public static void main(String[] args) throws InterruptedException {
Debug = true;
log.delete(logName);
Container<String> c = new Container<String>(10);
// Simple add/remove
log(true, "Simple test");
Node<String> it = c.add(strings[0]);
log("Added " + c.toString());
c.remove(it, strings[0]);
log("Removed " + c.toString());
// Capacity test.
log(true, "Capacity test");
ArrayList<Node<String>> nodes = new ArrayList<Node<String>>(strings.length);
// Fill it.
for (int i = 0; i < strings.length; i++) {
nodes.add(i, c.add(strings[i]));
log("Added " + strings[i] + " " + c.toString());
}
// Add one more.
try {
c.add("Wafer thin mint!");
} catch (IllegalStateException ise) {
log("Full!");
}
c.clear();
log("Empty: " + c.toString());
// Iterate test.
log(true, "Iterator test");
for (int i = 0; i < strings.length; i++) {
nodes.add(i, c.add(strings[i]));
}
StringBuilder all = new StringBuilder ();
Separator sep = new Separator(",");
for (String s : c) {
all.append(sep.sep()).append(s);
}
log("All: "+all);
for (int i = 0; i < strings.length; i++) {
c.remove(nodes.get(i), strings[i]);
}
sep.reset();
all.setLength(0);
for (String s : c) {
all.append(sep.sep()).append(s);
}
log("None: " + all.toString());
// Multiple add/remove
log(true, "Multi test");
for (int i = 0; i < strings.length; i++) {
nodes.add(i, c.add(strings[i]));
log("Added " + strings[i] + " " + c.toString());
}
log("Filled " + c.toString());
for (int i = 0; i < strings.length - 1; i++) {
c.remove(nodes.get(i), strings[i]);
log("Removed " + strings[i] + " " + c.toString());
}
c.remove(nodes.get(strings.length - 1), strings[strings.length - 1]);
log("Empty " + c.toString());
// Multi-threaded add/remove
log(true, "Threads test");
c.clear();
for (int i = 0; i < TEST_THREADS; i++) {
Thread t = new Thread(new Tester<String>(c, strings[i]));
t.setName("Tester " + strings[i]);
log("Starting " + t.getName());
t.start();
}
// Wait for 10 seconds.
long stop = System.currentTimeMillis() + 10 * 1000;
while (System.currentTimeMillis() < stop) {
Thread.sleep(100);
}
// Stop the testers.
testing = false;
// Wait some more.
Thread.sleep(1 * 100);
// Get stats.
double added = c.totalAdded.doubleValue();
double skipped = c.totalSkipped.doubleValue();
//double freed = c.freed.doubleValue();
log(true, "Stats: added=" + c.totalAdded + ",freed=" + c.totalFreed + ",skipped=" + c.totalSkipped + ",O(" + ((added + skipped) / added) + ")");
}
Maybe you want to look at Disruptor - Concurrent Programming Framework.
Find a paper describing the alternatives, design and also a performance comparement to java.util.concurrent.ArrayBlockingQueue here: pdf
Consider to read the first three articles from BlogsAndArticles
If the library is too much, stick to java.util.concurrent.ArrayBlockingQueue
I would have a look at ArrayDeque, or for a more concurrent implementation have a look at the Disruptor library which is one of the most sophisticated/complex ring buffer in Java.
An alternative is to use an unbounded queue which is more concurrent as the producer never needs to wait for the consumer. Java Chronicle
Unless your needs justify the complexity, an ArrayDeque may be all you need.
Also have a look at java.util.concurrent.
Blocking queues will block until there is something to consume or (optionally) space to produce:
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/BlockingQueue.html
Concurrent linked queue is non-blocking and uses a slick algorithm that allows a producer and consumer to be active concurrently:
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html
Hazelcast's Queue offers almost everything you ask for, but doesn't support circularity. But from your description I am not sure if you actually need it.
If it were me, I would use the CircularFIFOBuffer as you indicated, and synchronize around the buffer when writing (add). When the monitoring application wants to read the buffer, synchronize on the buffer, and then copy or clone it to use for reporting.
This suggestion is predicated on the assumption that latency is minimal to copy/clone the buffer to a new object. If there are large number of elements, and copying time is slow, then this is not a good idea.
Pseudo-Code example:
public void writeRequest(String requestID) {
synchronized(buffer) {
buffer.add(requestID);
}
}
public Collection<String> getRequests() {
synchronized(buffer) {
return buffer.clone();
}
}
Since you specifically ask to give writers (that is web servers) higher priority than the reader (that is monitoring), I would suggest the following design.
Web servers add request information to a concurrent queue which is read by a dedicated thread, which adds requests to a thread-local (therefore non-synchronized) queue that overwrites the oldest element, like EvictingQueue or CircularFifoQueue.
This same thread checks a flag which indicates if a report has been requested after every request processed, and if positive, produces a report from all elements present in the thread-local queue.

how to implement multithreaded Breadth-first search in java?

I've done the breadth-first search in a normal way.
now I'm trying to do it in a multithreaded way.
i have one queue which is shared between the threads.
i use synchronize(LockObject) when i remove a node from the queue ( FIFI queue )
so what I'm trying to do is that.
when i thread finds a solution all the other threads will stop immediately.
i assume you are traversing a tree for your BFS.
create a thread pool.
for each unexplored children in the node, retrieve a thread from the thread pool (perhaps using a Semaphore). mark the child node as 'explored' and explore the node's children in a BFS manner. when you have found a solution or done exploring all the nodes, release the semaphore.
^ i've never done this before so i might have missed out something.
Assuming you want to do this iteratively (see note at the bottom why there may be better closed solutions), this is not a great problem for exercising multi threading. The problem is that multithreading is great if you don't depend on previous results, but here you want the minimum amount of coins.
As you point out, a breadth first solution guarantees that once you reach the desired amount, you won't have any further solutions with less coins in a single threaded environment. However, in a multithreaded environment, once you start calculating a solution, you cannot guarantee that it will finish before some other solution. Let's imagine for the value 21: it can be a 20c coin and a 1c or four 5c coins and a 1c; if both are calculating simultaneously, you cannot guarantee that the first (and correct) solution will finish first. In practice, it is unlikely the situation will happen, but when you work with multithreading you want the solution to work in theory, because multithreads always fail in the demonstration, no matter if they should not have failed until the death heat of the universe.
Now you have 2 possible solutions: one is to introduce choke points at the beginning of each level; you don't start that level until the previous level is finished. The other is once you reach a solution continue doing all the calculations with a lower level than the current result (which means you cannot purge the others). Probably with all the synchronization needed you get better performance by going single threaded, but let's go on.
For the first solution, the natural form is to iterate increasing the level. You can use the solution provided by happymeal, with a Semaphore. An alternative is to use the new classes provided by java.
CoinSet getCoinSet(int desiredAmount) throws InterruptedException {
// Use whatever number of threads you prefer or another member of Executors.
final ExecutorService executor = Executors.newFixedThreadPool(10);
ResultContainer container = new ResultContainer();
container.getNext().add(new Producer(desiredAmount, new CoinSet(), container));
while (container.getResult() == null) {
executor.invokeAll(container.setNext(new Vector<Producer>()));
}
return container.getResult();
}
public class Producer implements Callable<CoinSet> {
private final int desiredAmount;
private final CoinSet data;
private final ResultContainer container;
public Producer(int desiredAmount, CoinSet data, ResultContainer container) {
this.desiredAmount = desiredAmount;
this.data = data;
this.container = container;
}
public CoinSet call() {
if (data.getSum() == desiredAmount) {
container.setResult(data);
return data;
} else {
Collection<CoinSet> nextSets = data.addCoins();
for (CoinSet nextSet : nextSets) {
container.getNext().add(new Producer(desiredAmount, nextSet, container));
}
return null;
}
}
}
// Probably it is better to split this class, but you then need to pass too many parameters
// The only really needed part is to create a wrapper around getNext, since invokeAll is
// undefined if you modify the list of tasks.
public class ResultContainer {
// I use Vector because it is synchronized.
private Vector<Producer> next = new Vector<Producer>();
private CoinSet result = null;
// Note I return the existing value.
public Vector<Producer> setNext(Vector<Producer> newValue) {
Vector<Producer> current = next;
next = newValue;
return current;
}
public Vector<Producer> getNext() {
return next;
}
public synchronized void setResult(CoinSet newValue) {
result = newValue;
}
public synchronized CoinSet getResult() {
return result;
}
}
This still has the problem that existing tasks are executed; however, it is simple to fix that; pass the thread executor into each Producer (or the container). Then, when you find a result, call executor.shutdownNow. The threads that are executing won't be interrupted, but the operation in each thread is trivial so it will finish fast; the runnables that have not started won't start.
The second option means you have to let all the current tasks finish, unless you keep track of how many tasks you have run at each level. You no longer need to keep track of the levels, though, and you don't need the while cycle. Instead, you just call
executor.submit(new Producer(new CoinSet(), desiredAmount, container)).get();
And then, the call method is pretty similar (assume you have executor in the Producer):
public CoinSet call() {
if (container.getResult() != null && data.getCount() < container.getResult().getCount()) {
if (data.getSum() == desiredAmount)) {
container.setResult(data);
return data;
} else {
Collection<CoinSet> nextSets = data.addCoins();
for (CoinSet nextSet : nextSets) {
executor.submit(new Producer(desiredAmount, nextSet, container));
}
return null;
}
}
}
and you also have to modify container.setResult, since you cannot depend that between the if and setting the value it has not been set by some other threads (threads are really annoying, aren't they?)
public synchronized void setResult(CoinSet newValue) {
if (newValue.getCount() < result.getCount()) {
result = newValue;
}
}
In all previous answers, CoinSet.getSum() returns the sum of the coins in the set, CoinSet.getCount() returns the number of coins, and CoinSet.addCoins() returns a Collection of CoinSet in which each element is the current CoinSet plus one coin of each possible different value
Note: For the problem of the coins with the values 1, 5, 10 and 20, the simplest solution is take the amount and divide it by the largest coin. Then take the modulus of that and use the next largest value and so on. That is the minimum amount of coins you are going to need. This rule applies (AFAICT) when the following property if true: if for all consecutive pairs of coin values (i.e. in this case, 1-5, 5-10, 10-20) you can reach any int multiple of the lower element in the pair with with a smaller number of coins using the larger element and whatever coins are necessary. You only need to prove it to the min common multiple of both elements in the pair (after that it repeats itself)
I gather from your comment on happymeal's anwer that you are trying to find how to reach a specific amount of money by adding coins of 1c, 5c, 10c and 20c.
Since each coin denomination divides the denomination of the next bigger coin, this can be solved in constant time as follows:
int[] coinCount(int amount) {
int[] coinValue = {20, 10, 5, 1};
int[] coinCount = new int[coinValue.length];
for (int i = 0; i < coinValue.length; i++) {
coinCount[i] = amount / coinValue[i];
amount -= coinCount[i] * coinValue[i];
}
return coinCount;
}
Take home message: Try to optimize your algorithm before resorting to multithreading, because algorithmic improvements can yield much greater improvements.
I've successfully implemented it.
what i did is that i took all the nodes in the first level, let's say 4 nodes.
then i had 2 threads. each one takes 2 nodes and generate their children. whenever a node finds a solution it has to report the level that it found the solution in and limit the searching level so other threads don't exceed the level.
only the reporting method should be synchronized.
i did the code for the coins change problem. this is my code for others to use
Main Class (CoinsProblemBFS.java)
package coinsproblembfs;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.Queue;
import java.util.Scanner;
/**
*
* #author Kassem M. Bagher
*/
public class CoinsProblemBFS
{
private static List<Item> MoneyList = new ArrayList<Item>();
private static Queue<Item> q = new LinkedList<Item>();
private static LinkedList<Item> tmpQ;
public static Object lockLevelLimitation = new Object();
public static int searchLevelLimit = 1000;
public static Item lastFoundNode = null;
private static int numberOfThreads = 2;
private static void InitializeQueu(Item Root)
{
for (int x = 0; x < MoneyList.size(); x++)
{
Item t = new Item();
t.value = MoneyList.get(x).value;
t.Totalvalue = MoneyList.get(x).Totalvalue;
t.Title = MoneyList.get(x).Title;
t.parent = Root;
t.level = 1;
q.add(t);
}
}
private static int[] calculateQueueLimit(int numberOfItems, int numberOfThreads)
{
int total = 0;
int[] queueLimit = new int[numberOfThreads];
for (int x = 0; x < numberOfItems; x++)
{
if (total < numberOfItems)
{
queueLimit[x % numberOfThreads] += 1;
total++;
}
else
{
break;
}
}
return queueLimit;
}
private static void initializeMoneyList(int numberOfItems, Item Root)
{
for (int x = 0; x < numberOfItems; x++)
{
Scanner input = new Scanner(System.in);
Item t = new Item();
System.out.print("Enter the Title and Value for item " + (x + 1) + ": ");
String tmp = input.nextLine();
t.Title = tmp.split(" ")[0];
t.value = Double.parseDouble(tmp.split(" ")[1]);
t.Totalvalue = t.value;
t.parent = Root;
MoneyList.add(t);
}
}
private static void printPath(Item item)
{
System.out.println("\nSolution Found in Thread:" + item.winnerThreadName + "\nExecution Time: " + item.searchTime + " ms, " + (item.searchTime / 1000) + " s");
while (item != null)
{
for (Item listItem : MoneyList)
{
if (listItem.Title.equals(item.Title))
{
listItem.counter++;
}
}
item = item.parent;
}
for (Item listItem : MoneyList)
{
System.out.println(listItem.Title + " x " + listItem.counter);
}
}
public static void main(String[] args) throws InterruptedException
{
Item Root = new Item();
Root.Title = "Root Node";
Scanner input = new Scanner(System.in);
System.out.print("Number of Items: ");
int numberOfItems = input.nextInt();
input.nextLine();
initializeMoneyList(numberOfItems, Root);
System.out.print("Enter the Amount of Money: ");
double searchValue = input.nextDouble();
int searchLimit = (int) Math.ceil((searchValue / MoneyList.get(MoneyList.size() - 1).value));
System.out.print("Number of Threads (Muste be less than the number of items): ");
numberOfThreads = input.nextInt();
if (numberOfThreads > numberOfItems)
{
System.exit(1);
}
InitializeQueu(Root);
int[] queueLimit = calculateQueueLimit(numberOfItems, numberOfThreads);
List<Thread> threadList = new ArrayList<Thread>();
for (int x = 0; x < numberOfThreads; x++)
{
tmpQ = new LinkedList<Item>();
for (int y = 0; y < queueLimit[x]; y++)
{
tmpQ.add(q.remove());
}
BFS tmpThreadObject = new BFS(MoneyList, searchValue, tmpQ);
Thread t = new Thread(tmpThreadObject);
t.setName((x + 1) + "");
threadList.add(t);
}
for (Thread t : threadList)
{
t.start();
}
boolean finish = false;
while (!finish)
{
Thread.sleep(250);
for (Thread t : threadList)
{
if (t.isAlive())
{
finish = false;
break;
}
else
{
finish = true;
}
}
}
printPath(lastFoundNode);
}
}
Item Class (Item.java)
package coinsproblembfs;
/**
*
* #author Kassem
*/
public class Item
{
String Title = "";
double value = 0;
int level = 0;
double Totalvalue = 0;
int counter = 0;
Item parent = null;
long searchTime = 0;
String winnerThreadName="";
}
Threads Class (BFS.java)
package coinsproblembfs;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
/**
*
* #author Kassem M. Bagher
*/
public class BFS implements Runnable
{
private LinkedList<Item> q;
private List<Item> MoneyList;
private double searchValue = 0;
private long start = 0, end = 0;
public BFS(List<Item> monyList, double searchValue, LinkedList<Item> queue)
{
q = new LinkedList<Item>();
MoneyList = new ArrayList<Item>();
this.searchValue = searchValue;
for (int x = 0; x < queue.size(); x++)
{
q.addLast(queue.get(x));
}
for (int x = 0; x < monyList.size(); x++)
{
MoneyList.add(monyList.get(x));
}
}
private synchronized void printPath(Item item)
{
while (item != null)
{
for (Item listItem : MoneyList)
{
if (listItem.Title.equals(item.Title))
{
listItem.counter++;
}
}
item = item.parent;
}
for (Item listItem : MoneyList)
{
System.out.println(listItem.Title + " x " + listItem.counter);
}
}
private void addChildren(Item node, LinkedList<Item> q, boolean initialized)
{
for (int x = 0; x < MoneyList.size(); x++)
{
Item t = new Item();
t.value = MoneyList.get(x).value;
if (initialized)
{
t.Totalvalue = 0;
t.level = 0;
}
else
{
t.parent = node;
t.Totalvalue = MoneyList.get(x).Totalvalue;
if (t.parent == null)
{
t.level = 0;
}
else
{
t.level = t.parent.level + 1;
}
}
t.Title = MoneyList.get(x).Title;
q.addLast(t);
}
}
#Override
public void run()
{
start = System.currentTimeMillis();
try
{
while (!q.isEmpty())
{
Item node = null;
node = (Item) q.removeFirst();
node.Totalvalue = node.value + node.parent.Totalvalue;
if (node.level < CoinsProblemBFS.searchLevelLimit)
{
if (node.Totalvalue == searchValue)
{
synchronized (CoinsProblemBFS.lockLevelLimitation)
{
CoinsProblemBFS.searchLevelLimit = node.level;
CoinsProblemBFS.lastFoundNode = node;
end = System.currentTimeMillis();
CoinsProblemBFS.lastFoundNode.searchTime = (end - start);
CoinsProblemBFS.lastFoundNode.winnerThreadName=Thread.currentThread().getName();
}
}
else
{
if (node.level + 1 < CoinsProblemBFS.searchLevelLimit)
{
addChildren(node, q, false);
}
}
}
}
} catch (Exception e)
{
e.printStackTrace();
}
}
}
Sample Input:
Number of Items: 4
Enter the Title and Value for item 1: one 1
Enter the Title and Value for item 2: five 5
Enter the Title and Value for item 3: ten 10
Enter the Title and Value for item 4: twenty 20
Enter the Amount of Money: 150
Number of Threads (Muste be less than the number of items): 2

Categories