Find missing date ranges in timeline using java - java

I have a custom java sync that fetch data by date range thoght SOAP service running on tomcat.
Ex:
getDataByDateRange(startDate,endDate)
getDataByDateRange('2016-01-01 10:00:00.00000','2016-01-01 11:00:00.00000')
I want to write a control program to check if any range has been missed by any kind of runtime or server error.
How can I find the missing date ranges?
Thanks.
Visually Example:
TimeLine : [------------------------------------------------------------------]
Processed Dates: [----1---][---2----]---[-3-][--4---]---[----5---][---6--]-----------
Missing Dates : -------------------[-1-]-----------[-2-]----------------[-----3----]
TimeLine:
1: '2016-01-01 10:00:00.00000','2016-02-01 09:00:00.00000'
Processed Dates:
1: '2016-01-01 10:00:00.00000','2016-01-01 11:00:00.00000'
2: '2016-01-01 11:00:00.00000','2016-01-01 12:00:00.00000'
3: '2016-01-01 13:00:00.00000','2016-01-01 13:30:00.00000'
4: '2016-01-01 13:30:00.00000','2016-01-01 14:30:00.00000'
5: '2016-01-01 15:30:00.00000','2016-01-01 16:30:00.00000'
6: '2016-01-01 16:30:00.00000','2016-01-01 17:00:00.00000'
Missing Dates:
1: '2016-01-01 12:00:00.00000','2016-01-01 13:00:00.00000'
2: '2016-01-01 14:30:00.00000','2016-01-01 15:30:00.00000'
3: '2016-01-01 17:00:00.00000','2016-01-02 09:00:00.00000'

According to your comment I post my previous comment as answer. This solution uses my library Time4J (including the range-module):
// prepare parser
ChronoFormatter<PlainTimestamp> f =
ChronoFormatter.ofTimestampPattern( // five decimal digits
"uuuu-MM-dd HH:mm:ss.SSSSS", PatternType.CLDR, Locale.ROOT);
// parse input to intervals - here the overall time window
TimestampInterval timeline =
TimestampInterval.between(
f.parse("2016-01-01 10:00:00.00000"),
f.parse("2016-02-01 09:00:00.00000"));
// for more flexibility - consider a for-each-loop
TimestampInterval i1 =
TimestampInterval.between(f.parse("2016-01-01 10:00:00.00000"), f.parse("2016-01-01 11:00:00.00000"));
TimestampInterval i2 =
TimestampInterval.between(f.parse("2016-01-01 11:00:00.00000"), f.parse("2016-01-01 12:00:00.00000"));
TimestampInterval i3 =
TimestampInterval.between(f.parse("2016-01-01 13:00:00.00000"), f.parse("2016-01-01 13:30:00.00000"));
TimestampInterval i4 =
TimestampInterval.between(f.parse("2016-01-01 13:30:00.00000"), f.parse("2016-01-01 14:30:00.00000"));
TimestampInterval i5 =
TimestampInterval.between(f.parse("2016-01-01 15:30:00.00000"), f.parse("2016-01-01 16:30:00.00000"));
TimestampInterval i6 =
TimestampInterval.between(f.parse("2016-01-01 16:30:00.00000"), f.parse("2016-01-01 17:00:00.00000"));
// apply interval arithmetic
IntervalCollection<PlainTimestamp> icoll =
IntervalCollection.onTimestampAxis().plus(Arrays.asList(i1, i2, i3, i4, i5, i6));
List<ChronoInterval<PlainTimestamp>> missed = icoll.withComplement(timeline).getIntervals();
// result
System.out.println(missed);
// [[2016-01-01T12/2016-01-01T13), [2016-01-01T14:30/2016-01-01T15:30), [2016-01-01T17/2016-02-01T09)]
The core of the whole interval arithmetic is just done by the code fragment icoll.withComplement(timeline). The rest is only about creation of intervals. By applying a for-each-loop you can surely minimize again the count of lines in presented code.
The output is based on the canonical description of the intervals implicitly using toString(), for example: [2016-01-01T12/2016-01-01T13) The square bracket denotes a closed boundary while the round bracket to the right end denotes an open boundary. So we have here the standard case of half-open timestamp intervals (without timezone). While other interval types are possible I have chosen that type because it corresponds to the type of your input strings.
If you plan to combine this solution with Joda-Time in other parts of your app then keep in mind that a) there is not yet any special bridge between both libraries available and b) the conversion looses microsecond precision (Joda-Time only supports milliseconds) and c) Time4J has much more power than Joda-Time (for almost everything). Anyway, you can do this as conversion (important if you don't want to do the effort of bigger rewriting of your app):
ChronoInterval<PlainTimestamp> missed0 = missed.get(0);
PlainTimestamp tsp = missed0.getStart().getTemporal();
LocalDateTime ldt = // joda-equivalent
new LocalDateTime(
tsp.getYear(), tsp.getMonth(), tsp.getDayOfMonth(),
tsp.getHour(), tsp.getMinute(), tsp.getSecond(), tsp.get(PlainTime.MILLI_OF_SECOND));
System.out.println(ldt); // 2016-01-01T10:00:00.000
About a Joda-only solution:
Joda-Time does only support instant intervals, not timestamp intervals without timezone. However, you could simulate that missing interval type by hardwiring the timezone to UTC (using fixed offset).
Another problem is missing support for five decimal digits. You can circumvent it by this hack:
DateTime start =
DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS")
.withZoneUTC()
.parseDateTime("2016-01-01 16:30:00.00000".substring(0, 23));
System.out.println(start); // 2016-01-01T16:30:00.000Z
DateTime end = ...;
Interval interval = new Interval(start, end);
The other more critical element of a solution is almost missing - interval arithmetic. You have to sort the intervals first by start instant (and then by end instant). After sorting, you can iterate over all intervals such that you find the gaps. The best thing Joda-Time can do for you here is giving you methods like isBefore(anotherInstant) etc. which you can use in your own solution. But it gets pretty much bloated.

Given that the frequency of date ranges is one hour, you can start with range start date, iterate till range end date and write a method that checks for an entry with dates. You can use DateUtils to add hour to date, as shown in the below pseudo code:
Date startDate = startDate;
Date endDate = endDate;
while (startDate.before(endDate){
if(!exists(startDate, DateUtils.addHours(startDate, 1), entries)){
//Add into missing entries
}
startDate = DateUtils.addHours(startDate, 1);
}

I posted my IntervalTree a while ago - it seems to work well with this kind of problem.
See the minimise method for what you are looking for.
/**
* Title: IntervlTree
*
* Description: Implements a static Interval Tree. i.e. adding and removal are not possible.
*
* This implementation uses longs to bound the intervals but could just as easily use doubles or any other linear value.
*
* #author OldCurmudgeon
* #version 1.0
* #param <T> - The Intervals to work with.
*/
public class IntervalTree<T extends IntervalTree.Interval> {
// My intervals.
private final List<T> intervals;
// My center value. All my intervals contain this center.
private final long center;
// My interval range.
private final long lBound;
private final long uBound;
// My left tree. All intervals that end below my center.
private final IntervalTree<T> left;
// My right tree. All intervals that start above my center.
private final IntervalTree<T> right;
public IntervalTree(List<T> intervals) {
if (intervals == null) {
throw new NullPointerException();
}
// Initially, my root contains all intervals.
this.intervals = intervals;
// Find my center.
center = findCenter();
/*
* Builds lefts out of all intervals that end below my center.
* Builds rights out of all intervals that start above my center.
* What remains contains all the intervals that contain my center.
*/
// Lefts contains all intervals that end below my center point.
final List<T> lefts = new ArrayList<>();
// Rights contains all intervals that start above my center point.
final List<T> rights = new ArrayList<>();
long uB = Long.MIN_VALUE;
long lB = Long.MAX_VALUE;
for (T i : intervals) {
long start = i.getStart();
long end = i.getEnd();
if (end < center) {
lefts.add(i);
} else if (start > center) {
rights.add(i);
} else {
// One of mine.
lB = Math.min(lB, start);
uB = Math.max(uB, end);
}
}
// Remove all those not mine.
intervals.removeAll(lefts);
intervals.removeAll(rights);
uBound = uB;
lBound = lB;
// Build the subtrees.
left = lefts.size() > 0 ? new IntervalTree<>(lefts) : null;
right = rights.size() > 0 ? new IntervalTree<>(rights) : null;
// Build my ascending and descending arrays.
/**
* #todo Build my ascending and descending arrays.
*/
}
/*
* Returns a list of all intervals containing the point.
*/
List<T> query(long point) {
// Check my range.
if (point >= lBound) {
if (point <= uBound) {
// Gather all intersecting ones.
List<T> found = intervals
.stream()
.filter((i) -> (i.getStart() <= point && point <= i.getEnd()))
.collect(Collectors.toList());
// Gather others.
if (point < center && left != null) {
found.addAll(left.query(point));
}
if (point > center && right != null) {
found.addAll(right.query(point));
}
return found;
} else {
// To right.
return right != null ? right.query(point) : Collections.<T>emptyList();
}
} else {
// To left.
return left != null ? left.query(point) : Collections.<T>emptyList();
}
}
/**
* Blends the two lists together.
*
* If the ends touch then make them one.
*
* #param a
* #param b
* #return
*/
static List<Interval> blend(List<Interval> a, List<Interval> b) {
// Either empty - return the other.
if (a.isEmpty()) {
return b;
}
if (b.isEmpty()) {
return a;
}
// Where does a end and b start.
Interval aEnd = a.get(a.size() - 1);
Interval bStart = b.get(0);
ArrayList<Interval> blended = new ArrayList<>();
// Do they meet/cross?
if (aEnd.getEnd() >= bStart.getStart() - 1) {
// Yes! merge them.
// Remove the last.
blended.addAll(a.subList(0, a.size() - 1));
// Add a combined one.
blended.add(new SimpleInterval(aEnd.getStart(), bStart.getEnd()));
// Add all but the first.
blended.addAll(b.subList(1, b.size()));
} else {
// Just join them.
blended.addAll(a);
blended.addAll(b);
}
return blended;
}
static List<Interval> blend(List<Interval> a, List<Interval> b, List<Interval>... more) {
List<Interval> blended = blend(a, b);
for (List<Interval> l : more) {
blended = blend(blended, l);
}
return blended;
}
List<Interval> minimise() {
// Calculate min of left and right.
List<Interval> minLeft = left != null ? left.minimise() : Collections.EMPTY_LIST;
List<Interval> minRight = right != null ? right.minimise() : Collections.EMPTY_LIST;
// My contribution.
long meLeft = minLeft.isEmpty() ? lBound : Math.max(lBound, minLeft.get(minLeft.size() - 1).getEnd());
long meRight = minRight.isEmpty() ? uBound : Math.min(uBound, minRight.get(0).getEnd());
return blend(minLeft,
Collections.singletonList(new SimpleInterval(meLeft, meRight)),
minRight);
}
private long findCenter() {
//return average();
return median();
}
protected long median() {
if (intervals.isEmpty()) {
return 0;
}
// Choose the median of all centers. Could choose just ends etc or anything.
long[] points = new long[intervals.size()];
int x = 0;
for (T i : intervals) {
// Take the mid point.
points[x++] = (i.getStart() + i.getEnd()) / 2;
}
Arrays.sort(points);
return points[points.length / 2];
}
/*
* What an interval looks like.
*/
public interface Interval {
public long getStart();
public long getEnd();
}
/*
* A simple implemementation of an interval.
*/
public static class SimpleInterval implements Interval {
private final long start;
private final long end;
public SimpleInterval(long start, long end) {
this.start = start;
this.end = end;
}
#Override
public long getStart() {
return start;
}
#Override
public long getEnd() {
return end;
}
#Override
public String toString() {
return "{" + start + "," + end + "}";
}
}
/**
* Not called by App, so you will have to call this directly.
*
* #param args
*/
public static void main(String[] args) {
/**
* #todo Needs MUCH more rigorous testing.
*/
// Test data.
long[][] data = {
{1, 4}, {2, 5}, {5, 7}, {10, 11}, {13, 20}, {19, 21},};
List<Interval> intervals = new ArrayList<>();
for (long[] pair : data) {
intervals.add(new SimpleInterval(pair[0], pair[1]));
}
// Build it.
IntervalTree<Interval> test = new IntervalTree<>(intervals);
// Test it.
System.out.println("Normal test: ---");
for (long i = 0; i < 10; i++) {
List<Interval> intersects = test.query(i);
System.out.println("Point " + i + " intersects:");
intersects.stream().forEach((t) -> {
System.out.println(t.toString());
});
}
// Check minimise.
List<Interval> min = test.minimise();
System.out.println("Minimise test: ---");
System.out.println(min);
// Check for empty list.
intervals.clear();
test = new IntervalTree<>(intervals);
// Test it.
System.out.println("Empty test: ---");
for (long i = 0; i < 10; i++) {
List<Interval> intersects = test.query(i);
System.out.println("Point " + i + " intersects:");
intersects.stream().forEach((t) -> {
System.out.println(t.toString());
});
}
}
}
This gets close to what you are looking for. Here's some code to minimise your ranges into just 3.
static String[][] dates = {{"2016-01-01 10:00:00.00000", "2016-01-01 11:00:00.00000"}, {"2016-01-01 11:00:00.00000", "2016-01-01 12:00:00.00000"}, {"2016-01-01 13:00:00.00000", "2016-01-01 13:30:00.00000"}, {"2016-01-01 13:30:00.00000", "2016-01-01 14:30:00.00000"}, {"2016-01-01 15:30:00.00000", "2016-01-01 16:30:00.00000"}, {"2016-01-01 16:30:00.00000", "2016-01-01 17:00:00.00000"}};
static List<IntervalTree.SimpleInterval> ranges = new ArrayList<>();
static final DateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.S");
static {
for (String[] pair : dates) {
try {
ranges.add(new IntervalTree.SimpleInterval(df.parse(pair[0]).getTime(), df.parse(pair[1]).getTime()));
} catch (ParseException ex) {
Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
public void test() {
IntervalTree tree = new IntervalTree<>(ranges);
List<IntervalTree.Interval> min = tree.minimise();
//System.out.println("min->" + min);
for (IntervalTree.Interval i : min) {
System.out.println(df.format(new Date(i.getStart())) + " - " + df.format(new Date(i.getEnd())));
}
}
which prints
2016-01-01 10:00:00.0 - 2016-01-01 12:00:00.0
2016-01-01 13:00:00.0 - 2016-01-01 14:30:00.0
2016-01-01 15:30:00.0 - 2016-01-01 17:00:00.0
which is all of your Processed Dates joined into three date ranges.

Related

Java Stream API: Change values by criteria

There is Java class:
public class Item {
private String dateModified;
private Integer color;
}
where dateModified in format "hh:mm:ss",
and ArrayList<Item> list which contains 10 elements.
So i want check my list and:
if now() - dateModified > 1 min , then change color to 1
if now() - dateModified > 5 min , then change color to 2
if now() - dateModified > 10 min, then change color to 3
How to implements it with Java Stream API?
UPDATE:
I implemented my task in such code below. It works as expected, but it seems huge and non-elegance.
I forget to say that list should be mutable.
list.stream()
.map(c -> {
if (compareTime(c.getDateModified(), 600)) {
c.setColor(3);
} else if (compareTime(c.getDateModified(), 300)) {
c.setColor(2);
} else if (compareTime(c.getDateModified(), 60)) {
c.setColor(1);
}
return c;
}).collect(Collectors.toList());
private boolean compareTime(String dateModified, Integer delta) {
boolean result = false;
LocalDateTime now = LocalDateTime.now();
int hour = now.getHour();
int minute = now.getMinute();
int second = now.getSecond();
Integer secondsDateModified = Integer.parseInt(dateModified.substring(0, 2)) * 3600 +
Integer.parseInt(dateModified.substring(3, 5)) * 60 +
Integer.parseInt(dateModified.substring(6, 8)) ;
Integer secondsNow = hour * 3600 + minute * 60 + second ;
Integer delta1 = secondsNow - secondsDateModified;
if ((delta1) > delta) {
result = true;
}
return result;
}
Any suggestions to improve the code are appreciated.
Instead of storing a String as the time, store a LocalTime object. Also, instead of mutating the original item, return the item or a new item with the new color.
public static class Item {
private final LocalTime dateModified;
private final Integer color;
public Item(LocalTime dateModified, Integer color) {
this.dateModified = dateModified;
this.color = color;
}
public Item withColor(int color) {
return new Item(dateModified, color);
}
public LocalTime getDateModified() {
return dateModified;
}
public Integer getColor() {
return color;
}
}
Example
public static void main(String[] args) {
List<Item> items = new ArrayList<>(Arrays.asList(
new Item(LocalTime.parse("10:30:00"), 0),
new Item(LocalTime.parse("10:30:01"), 255)));
LocalTime now = LocalTime.now();
List<Item> modified = items.stream().map(item -> {
long minutes = Duration.between(item.dateModified, LocalTime.now())
.toMinutes();
return minutes >= 1 ?
item.withColor(minutes >= 10 ? 3 : minutes >= 5 ? 2 : 1) : item;
}).collect(Collectors.toList());
}
What about to use separate streams for update each required range of items:
public static void updateColor(List<Item> items) {
final LocalTime now = LocalTime.now();
final Function<Item, Long> getDurationInMinutes = item -> Duration.between(LocalTime.parse(item.dateModified), now).toMinutes()
final Predicate<Item> betweenOneAndFive = item -> {
long duration = getDurationInMinutes.apply(item);
return duration > ONE && duration <= FIVE;
};
final Predicate<Item> betweenFiveAndTen = item -> {
long duration = getDurationInMinutes.apply(item);
return duration > FIVE && duration <= TEN;
};
final Predicate<Item> greaterThanTen = item -> getDurationInMinutes.apply(item) > TEN;
items.stream().filter(betweenOneAndFive).forEach(item -> item.color = 1);
items.stream().filter(betweenFiveAndTen).forEach(item -> item.color = 2);
items.stream().filter(greaterThanTen).forEach(item -> item.color = 3);
}
It is a matter of proper mapping function from difference of minutes to numbers.
items.forEach(item -> item.setColor(((int) Math.floor(differenceInMinutes(item.getDateModified(), now) + 5)) / 5));
Note, that the differenceInMinutes method would return the difference in floating point arithmetic.
The steps taken are:
Find the difference in minutes from the date of the items with now.
Cast the result to an int which will work like Math.floor.
Add 5 to the result.
Divide by 5.
So, for example 1.3 minutes would result in (1+5)/5 which is 1.
9.8 minutes would result in (9+5)/5 which is 2.
And so on.
First, as Jason, said do not mutate yours items inside streams, make copies. (What is the danger of side effects in Java 8 Streams?).
You will need intermediates operations :
long getElapseTimeSinceModification(Item item) {
return ChronoUnit.MINUTES.between(LocalTime.parse(item.dateModified), LocalDate.now());
}
Optional<Integer> getNewColor(long elapseTimeSinceModification) {
if (elapseTimeSinceModification > 10) {
return Optional.of(3);
} else if (elapseTimeSinceModification > 5) {
return Optional.of(2);
} else if (elapseTimeSinceModification > 1) {
return Optional.of(1);
}
return Optional.empty();
}
Item withNewColor(int newColor, Item item) {
Item newTtem = new Item();
newTtem.color = newColor;
newTtem.dateModified = item.dateModified;
return newTtem;
}
and then you can apply them to your stream :
List<Item> itemsWithNewColor = items.stream()
.map(item -> getNewColor(getElapseTimeSinceModification(item))
.map(newColor -> withNewColor(newColor , item))
.orElse(i))
.collect(Collectors.toList());

How can I count the number of events in a given interval?

I need to know how frequency different events occur. For example how many HTTP requests have occurred in the last 15 minutes. Because there can be a large count of events (millions) this must be use a limited amount of memory.
It there any util class in Java that can do this?
How can I implement this self in Java?
Theoretical usage code can look like:
FrequencyCounter counter = new FrequencyCounter( 15, TimeUnit.Minutes );
...
counter.add();
...
int count = counter.getCount();
Edit: It must be a real time value which can changed thousand times the minute and will be query thousands times the minute. That a database or file based solution are not possible.
Here is my implementation of such a counter. The memory usage with the default precision is fewer as 100 bytes. The memory usage is independent of the event count.
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
/**
* A counter that counts events within the past time interval. All events that occurred before this interval will be
* removed from the counter.
*/
public class FrequencyCounter {
private final long monitoringInterval;
private final int[] details;
private final AtomicInteger currentCount = new AtomicInteger();
private long startInterval;
private int total;
/**
* Create a new instance of the counter for the given interval.
*
* #param interval the time to monitor/count the events.
* #param unit the time unit of the {#code interval} argument
*/
FrequencyCounter( long interval, TimeUnit unit ) {
this( interval, unit, 16 );
}
/**
* Create a new instance of the counter for the given interval.
*
* #param interval the time to monitor/count the events.
* #param unit the time unit of the {#code interval} argument
* #param precision the count of time slices for the for the measurement
*/
FrequencyCounter( long interval, TimeUnit unit, int precision ) {
monitoringInterval = unit.toMillis( interval );
if( monitoringInterval <= 0 ) {
throw new IllegalArgumentException( "Interval mus be a positive value:" + interval );
}
details = new int[precision];
startInterval = System.currentTimeMillis() - monitoringInterval;
}
/**
* Count a single event.
*/
public void increment() {
checkInterval( System.currentTimeMillis() );
currentCount.incrementAndGet();
}
/**
* Get the current value of the counter.
*
* #return the counter value
*/
public int getCount() {
long currentTime = System.currentTimeMillis();
checkInterval( currentTime );
long diff = currentTime - startInterval - monitoringInterval;
double partFactor = (diff * details.length / (double)monitoringInterval);
int part = (int)(details[0] * partFactor);
return total + currentCount.get() - part;
}
/**
* Check the interval of the detail counters and move the interval if needed.
*
* #param time the current time
*/
private void checkInterval( final long time ) {
if( (time - startInterval - monitoringInterval) > monitoringInterval / details.length ) {
synchronized( details ) {
long detailInterval = monitoringInterval / details.length;
while( (time - startInterval - monitoringInterval) > detailInterval ) {
int currentValue = currentCount.getAndSet( 0 );
if( (total | currentValue) == 0 ) {
// for the case that the counter was not used for a long time
startInterval = time - monitoringInterval;
return;
}
int size = details.length - 1;
total += currentValue - details[0];
System.arraycopy( details, 1, details, 0, size );
details[size] = currentValue;
startInterval += detailInterval;
}
}
}
}
}
The best way I can think to implement this is using another "time counting" thread.
If you're concerned about the amount of memory, you can add a threshold for the size of eventsCounter (Integer.MAX_VALUE seems like the natural choice).
Here's an example for an implementation, that is also thread-safe:
public class FrequencyCounter {
private AtomicInteger eventsCounter = new AtomicInteger(0);
private int timeCounter;
private boolean active;
public FrequencyCounter(int timeInSeconds) {
timeCounter = timeInSeconds;
active = true;
}
// Call this method whenever an interesting event occurs
public int add() {
if(active) {
int current;
do {
current = eventsCounter.get();
} while (eventsCounter.compareAndSet(current, current + 1));
return current + 1;
}
else return -1;
}
// Get current number of events
public int getCount() {
return eventsCounter.get();
}
// Start the FrequencyCounter
public void run() {
Thread timer = new Thread(() -> {
while(timeCounter > 0) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
timeCounter --;
}
active = false;
});
timer.start();
}
}
How about a scheduled executor service.
class TimedValue{
int startValue;
int finishedValue;
TimedValue(int start){
startValue = start;
}
}
List<TimedValue> intervals = new CopyOnWriteArrayList<>();
//then when starting a measurement.
TimeValue value = new TimedValue();
//set the start value.
Callable<TimedValue> callable = ()->{
//performs the task.
value.setValueAtFinish(getCount());
return value;
}
ScheduledExecutorService executor = Executors.newScheduledThreadPool(2);
ScheduledFuture<TimedValue> future = executor.schedule(
callable,
TimeUnit.MINUTES,
15);
executor.schedule(()->itervals.add(
future.get(),
TimeUnit.MINUTES,
future.getDelay(TimeUnit.MINUTES
);
This is a bit of a complicated method.
I would probably just have a List<LoggedValues> and accumulate values in that list at a fixed rate. Then it could be inspected whenever you want to know an intervals.

How to generate random number but not repetitive based on today's date in java?

I want to generate a unique random index everyday to show "word of the day" from N number of words in a list.
Until every words are indexed from a list, I dont't want the same index to be repeated. For example, I have N words in a list; the index each day should be different for N days.
If you really can't be bothered to keep a record of the numbers you've already used you could use a rather nice mechanism known as a Linear Feedback Shift Register or LFSR. This Generates a random (but predictable, if you know it's an LFSR) sequence of numbers spanning all numbers of n bits.
Just choose n to be greater than your 'N' and throw away any numbers too big.
/**
* Linear feedback shift register
*
* Taps can be found at: See http://www.xilinx.com/support/documentation/application_notes/xapp052.pdf See http://mathoverflow.net/questions/46961/how-are-taps-proven-to-work-for-lfsrs/46983#46983 See
* http://www.newwaveinstruments.com/resources/articles/m_sequence_linear_feedback_shift_register_lfsr.htm See http://www.yikes.com/~ptolemy/lfsr_web/index.htm See
* http://seanerikoconnor.freeservers.com/Mathematics/AbstractAlgebra/PrimitivePolynomials/overview.html
*
* #author OldCurmudgeon
*/
public class LFSR implements Iterable<BigInteger> {
// Bit pattern for taps.
private final BigInteger taps;
// Where to start (and end).
private final BigInteger start;
// The poly must be primitive to span the full sequence.
public LFSR(BigInteger primitivePoly, BigInteger start) {
// Where to start from (and stop).
this.start = start.equals(BigInteger.ZERO) ? BigInteger.ONE : start;
// Knock off the 2^0 coefficient of the polynomial for the TAP.
this.taps = primitivePoly.shiftRight(1);
}
#Override
public Iterator<BigInteger> iterator() {
return new LFSRIterator(start);
}
private class LFSRIterator implements Iterator<BigInteger> {
// The last one we returned.
private BigInteger last = null;
// The next one to return.
private BigInteger next = null;
public LFSRIterator(BigInteger start) {
// Do not return the seed.
last = start;
}
#Override
public boolean hasNext() {
if (next == null) {
/*
* Uses the Galois form.
*
* Shift last right one.
*
* If the bit shifted out was a 1 - xor with the tap mask.
*/
boolean shiftedOutA1 = last.testBit(0);
// Shift right.
next = last.shiftRight(1);
if (shiftedOutA1) {
// Tap!
next = next.xor(taps);
}
// Never give them `start` again.
if (next.equals(start)) {
// Could set a finished flag here too.
next = null;
}
}
return next != null;
}
#Override
public BigInteger next() {
// Remember this one.
last = hasNext() ? next : null;
// Don't deliver it again.
next = null;
return last;
}
#Override
public void remove() {
throw new UnsupportedOperationException("Not supported.");
}
#Override
public String toString() {
return LFSR.this.toString()
+ "[" + (last != null ? last.toString(16) : "")
+ "-" + (next != null ? next.toString(16) : "") + "]";
}
}
#Override
public String toString() {
return "(" + taps.toString(32) + ")-" + start.toString(32);
}
public static void main(String args[]) {
try {
new LFSRTest().test();
} catch (Throwable t) {
t.printStackTrace(System.err);
}
}
}
class LFSRTest {
public void test(int[] tap, int base) {
System.out.println("Test: " + Arrays.toString(tap));
// Build the BigInteger.
BigInteger primitive = BigInteger.ZERO;
for (int bit : tap) {
primitive = primitive.or(BigInteger.ONE.shiftLeft(bit));
}
// Stop at 100.
int count = 100;
LFSR lfsr = new LFSR(primitive, BigInteger.ONE);
for (BigInteger b : lfsr) {
if (count-- > 0) {
System.out.println(b.toString(base));
} else {
break;
}
}
}
public void test() {
// Just 6 bits.
int[] tap7 = {6, 5, 0};
test(tap7, 10);
// An example 48-bit tap.
int[] tap48 = {48, 46, 45, 44, 42, 40, 36, 34, 33, 32, 29, 27, 26, 20, 17, 16, 12, 11, 10, 5, 3, 1, 0};
test(tap48, 32);
}
}
As you can see, the efficiency is very good - just a few boolean ops per iteration. You can therefore just iterate N times to get the number you want. Choose the number of bits to achieve at least the number of days you want.
Here are some steps to get you started:
1) Generate a random number
2) Check this random number against an array/hashmap/list/whatever.
3) If it doesn't exist, add it in
4) Find 'word of the day' using this number.
5) Repeat these steps
If the random number does exist, then simply generate another one. You would then repeat these steps everyday until the size of the 'already used numbers' array matches the length of the 'words of the day' array. However, this process is not very efficicent and I wouldn't necessarily use it, it is just here to get you thinking.
Some possibly better ideas:
If you never want it to be the same, instead of 'randomly generating' a number. Why not just iterate through an array to begin with and increase it every day?
You could also just generate a random number, find the word of the day and then delete it from your list of random words and repeat this process until the list is empty, ensuring you alter the boundaries of your random numbers each time. Then when it is empty, just repopulate it.
Create an array of these possible numbers and shuffle it.
Every day use the next index, starting at 0.

Exclude overlapping intervals

I have two lists of intervals. I would like to remove all times from list1 that already exists in list2.
Example:
List1:
[(0,10),(15,20)]
List2:
[(2,3),(5,6)]
Output:
[(0,2),(3,5),(6,10),(15,20)]
Any hints?
Tried to remove one interval at the time but it seems like I need to take a different approach:
public List<Interval> removeOneTime(Interval interval, Interval remove){
List<Interval> removed = new LinkedList<Interval>();
Interval overlap = interval.getOverlap(remove);
if(overlap.getLength() > 0){
List<Interval> rms = interval.remove(overlap);
removed.addAll(rms);
}
return removed;
}
I would approach this problem with a sweep line algorithm. The start and end points of the intervals are events, that are put in a priority queue. You just move from left to right, stop at every event, and update the current status according to that event.
I made a small implementation, in which I use the following Interval class, just for simplicity:
public class Interval {
public int start, end;
public Interval(int start, int end) {
this.start = start;
this.end = end;
}
public String toString() {
return "(" + start + "," + end + ")";
}
}
The event points mentioned earlier are represented by the following class:
public class AnnotatedPoint implements Comparable<AnnotatedPoint> {
public int value;
public PointType type;
public AnnotatedPoint(int value, PointType type) {
this.value = value;
this.type = type;
}
#Override
public int compareTo(AnnotatedPoint other) {
if (other.value == this.value) {
return this.type.ordinal() < other.type.ordinal() ? -1 : 1;
} else {
return this.value < other.value ? -1 : 1;
}
}
// the order is important here: if multiple events happen at the same point,
// this is the order in which you want to deal with them
public enum PointType {
End, GapEnd, GapStart, Start
}
}
Now, what remains is building the queue and doing the sweep, as shown in the code below
public class Test {
public static void main(String[] args) {
List<Interval> interval = Arrays.asList(new Interval(0, 10), new Interval(15, 20));
List<Interval> remove = Arrays.asList(new Interval(2, 3), new Interval(5, 6));
List<AnnotatedPoint> queue = initQueue(interval, remove);
List<Interval> result = doSweep(queue);
// print result
for (Interval i : result) {
System.out.println(i);
}
}
private static List<AnnotatedPoint> initQueue(List<Interval> interval, List<Interval> remove) {
// annotate all points and put them in a list
List<AnnotatedPoint> queue = new ArrayList<>();
for (Interval i : interval) {
queue.add(new AnnotatedPoint(i.start, PointType.Start));
queue.add(new AnnotatedPoint(i.end, PointType.End));
}
for (Interval i : remove) {
queue.add(new AnnotatedPoint(i.start, PointType.GapStart));
queue.add(new AnnotatedPoint(i.end, PointType.GapEnd));
}
// sort the queue
Collections.sort(queue);
return queue;
}
private static List<Interval> doSweep(List<AnnotatedPoint> queue) {
List<Interval> result = new ArrayList<>();
// iterate over the queue
boolean isInterval = false; // isInterval: #Start seen > #End seen
boolean isGap = false; // isGap: #GapStart seen > #GapEnd seen
int intervalStart = 0;
for (AnnotatedPoint point : queue) {
switch (point.type) {
case Start:
if (!isGap) {
intervalStart = point.value;
}
isInterval = true;
break;
case End:
if (!isGap) {
result.add(new Interval(intervalStart, point.value));
}
isInterval = false;
break;
case GapStart:
if (isInterval) {
result.add(new Interval(intervalStart, point.value));
}
isGap = true;
break;
case GapEnd:
if (isInterval) {
intervalStart = point.value;
}
isGap = false;
break;
}
}
return result;
}
}
This results in:
(0,2)
(3,5)
(6,10)
(15,20)
You probably want to use an interval tree - this will quickly tell you if an interval overlaps with any of the intervals in the tree.
Once you have a set of overlapping intervals the task should be fairly easy (interval1 is from list1, interval2 is the overlapping interval from list2 / the interval tree): if interval1 contains interval2, then you have two new intervals (interval1min, interval2min), (interval2max, interval1max); if interval1 does not contain interval2, then you only have one new interval (interval1min, interval2min) or (interval2max, interval1max)

Hashtable key within integer interval

I don't know if this is possible but i'm trying to make an Hashtable of where Interval is a class with 2 integer / long values, a start and an end and i wanted to make something like this:
Hashtable<Interval, WhateverObject> test = new Hashtable<Interval, WhateverObject>();
test.put(new Interval(100, 200), new WhateverObject());
test.get(new Interval(150, 150)) // returns the new WhateverObject i created above because 150 is betwwen 100 and 200
test.get(new Interval(250, 250)) // doesn't find the value because there is no key that contains 250 in it's interval
So basically what i want is that a key between a range of values in an Interval object give the correspondent WhateverObject. I know i have to override equals() and hashcode() in the interval object, the main problem i think is to somehow have all the values between 100 and 200 (in this specific example) to give the same hash.
Any ideias if this is possible?
Thanks
No need to reinvent the wheel, use a NavigableMap. Example Code:
final NavigableMap<Integer, String> map = new TreeMap<Integer, String>();
map.put(0, "Cry Baby");
map.put(6, "School Time");
map.put(16, "Got a car yet?");
map.put(21, "Tequila anyone?");
map.put(45, "Time to buy a corvette");
System.out.println(map.floorEntry(3).getValue());
System.out.println(map.floorEntry(10).getValue());
System.out.println(map.floorEntry(18).getValue());
Output:
Cry Baby
School Time
Got a car yet?
You could use an IntervalTree. Here's one I made earlier.
public class IntervalTree<T extends IntervalTree.Interval> {
// My intervals.
private final List<T> intervals;
// My center value. All my intervals contain this center.
private final long center;
// My interval range.
private final long lBound;
private final long uBound;
// My left tree. All intervals that end below my center.
private final IntervalTree<T> left;
// My right tree. All intervals that start above my center.
private final IntervalTree<T> right;
public IntervalTree(List<T> intervals) {
if (intervals == null) {
throw new NullPointerException();
}
// Initially, my root contains all intervals.
this.intervals = intervals;
// Find my center.
center = findCenter();
/*
* Builds lefts out of all intervals that end below my center.
* Builds rights out of all intervals that start above my center.
* What remains contains all the intervals that contain my center.
*/
// Lefts contains all intervals that end below my center point.
final List<T> lefts = new ArrayList<T>();
// Rights contains all intervals that start above my center point.
final List<T> rights = new ArrayList<T>();
long uB = Long.MIN_VALUE;
long lB = Long.MAX_VALUE;
for (T i : intervals) {
long start = i.getStart();
long end = i.getEnd();
if (end < center) {
lefts.add(i);
} else if (start > center) {
rights.add(i);
} else {
// One of mine.
lB = Math.min(lB, start);
uB = Math.max(uB, end);
}
}
// Remove all those not mine.
intervals.removeAll(lefts);
intervals.removeAll(rights);
uBound = uB;
lBound = lB;
// Build the subtrees.
left = lefts.size() > 0 ? new IntervalTree<T>(lefts) : null;
right = rights.size() > 0 ? new IntervalTree<T>(rights) : null;
// Build my ascending and descending arrays.
/** #todo Build my ascending and descending arrays. */
}
/*
* Returns a list of all intervals containing the point.
*/
List<T> query(long point) {
// Check my range.
if (point >= lBound) {
if (point <= uBound) {
// In my range but remember, there may also be contributors from left or right.
List<T> found = new ArrayList<T>();
// Gather all intersecting ones.
// Could be made faster (perhaps) by holding two sorted lists by start and end.
for (T i : intervals) {
if (i.getStart() <= point && point <= i.getEnd()) {
found.add(i);
}
}
// Gather others.
if (point < center && left != null) {
found.addAll(left.query(point));
}
if (point > center && right != null) {
found.addAll(right.query(point));
}
return found;
} else {
// To right.
return right != null ? right.query(point) : Collections.<T>emptyList();
}
} else {
// To left.
return left != null ? left.query(point) : Collections.<T>emptyList();
}
}
private long findCenter() {
//return average();
return median();
}
protected long median() {
// Choose the median of all centers. Could choose just ends etc or anything.
long[] points = new long[intervals.size()];
int x = 0;
for (T i : intervals) {
// Take the mid point.
points[x++] = (i.getStart() + i.getEnd()) / 2;
}
Arrays.sort(points);
return points[points.length / 2];
}
/*
* What an interval looks like.
*/
public interface Interval {
public long getStart();
public long getEnd();
}
/*
* A simple implemementation of an interval.
*/
public static class SimpleInterval implements Interval {
private final long start;
private final long end;
public SimpleInterval(long start, long end) {
this.start = start;
this.end = end;
}
public long getStart() {
return start;
}
public long getEnd() {
return end;
}
#Override
public String toString() {
return "{" + start + "," + end + "}";
}
}
}
A naive HashTable is the wrong solution here. Overriding the equals() method doesn't do you any good because the HashTable compares a key entry by the hash code first, NOT the equals() method. The equals() method is only checked AFTER the hash code is matched.
It's easy to make a hash function on your interval object, but it's much more difficult to make one that would yield the same hashcode for all possible intervals that would be within another interval. Overriding the get() method (such as here https://stackoverflow.com/a/11189075/1261844) for a HashTable completely negates the advantages of a HashTable, which is very fast lookup times. At the point where you are scanning through each member of a HashTable, then you know you are using the HashTable incorrectly.
I'd say that Using java map for range searches and https://stackoverflow.com/a/11189080/1261844 are better solutions, but a HashTable is simply not the way to go about this.
I think implementing a specialized get-method would be much easier.
The new method can be part of a map-wrapper-class.
The key-class: (interval is [lower;upper[ )
public class Interval {
private int upper;
private int lower;
public Interval(int upper, int lower) {
this.upper = upper;
this.lower = lower;
}
public boolean contains(int i) {
return i < upper && i >= lower;
}
#Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final Interval other = (Interval) obj;
if (this.upper != other.upper) {
return false;
}
if (this.lower != other.lower) {
return false;
}
return true;
}
#Override
public int hashCode() {
int hash = 5;
hash = 61 * hash + this.upper;
hash = 61 * hash + this.lower;
return hash;
}
}
The Map-class:
public class IntervalMap<T> extends HashMap<Interval, T> {
public T get(int key) {
for (Interval iv : keySet()) {
if (iv.contains(key)) {
return super.get(iv);
}
}
return null;
}
}
This is just an example and can surely be optimized, and there are a few flaws as well:
For Example if Intervals overlap, there's no garantee to know which Interval will be used for lookup and Intervals are not garanteed to not overlap!
OldCurmudgeon's solution works perfectly for me, but is very slow to initialise (took 20 mins for 70k entries).
If you know your incoming list of items is already ordered (ascending) and has only non overlapping intervals, you can make it initialise in milliseconds by adding and using the following constructor:
public IntervalTree(List<T> intervals, boolean constructorFlagToIndicateOrderedNonOverlappingIntervals) {
if (intervals == null) throw new NullPointerException();
int centerPoint = intervals.size() / 2;
T centerInterval = intervals.get(centerPoint);
this.intervals = new ArrayList<T>();
this.intervals.add(centerInterval);
this.uBound = centerInterval.getEnd();
this.lBound = centerInterval.getStart();
this.center = (this.uBound + this.lBound) / 2;
List<T> toTheLeft = centerPoint < 1 ? Collections.<T>emptyList() : intervals.subList(0, centerPoint);
this.left = toTheLeft.isEmpty() ? null : new IntervalTree<T>(toTheLeft, true);
List<T> toTheRight = centerPoint >= intervals.size() ? Collections.<T>emptyList() : intervals.subList(centerPoint+1, intervals.size());
this.right = toTheRight.isEmpty() ? null : new IntervalTree<T>(toTheRight, true);
}
This depends on your hashCode implementation. You may have two Objects with the same hashCode value.
Please use eclipse to generate a hashCode method for your class (no point to re-invent the wheel
For Hastable or HashMap to work as expected it's not only a equal hashcode, but also the equals method must return true. What you are requesting is that Interval(x, y).equals(Interval(m, n)) for m, n within x,y. As this must be true for any overlapping living instance of Interval, the class has to record all of them and needs to implement what you are trying to achieve, indeed.
So in short the answer is no.
The Google guava library is planning to offer a RangeSet and Map: guava RangeSet
For reasonable small ranges an easy approach would be to specialize HashMap by putting and getting the indivual values of the intervals.

Categories