We Keep Coding

Binary search on a sorted list of E(startTime,endTime) to find all E's matched by a given time range (t1,t2)

I have Event objects as follows,
public class Event {
String name;
int startTime; // minutes since midnight, e.g 4:15am = 255
int endTime; // minutes since midnight, e.g.6:30am = 390
// + getters/setters
}
They are sorted by startTime ASC,
events.sort(Comparator.comparing(Event::getStartTime));
Events can overlap in any way.
I need to obtain a List of all Events matching (incl. partially) a particular range t1,t2 (also ints for minutes since midnight).
List<Event> eventsMatching = findMatching(t1, t2); // e.g. between 200,205
I don't want to go through the whole list and check e.getStartTime() <= t1 && e.getEndTime() >= t2. Since the list is sorted, I should be able to use Collections.binarySearch() in some way. But normally, a binary search finds the exact object you're looking for: int position = Collections.binarySearch(events, key). Is there some way to find matching ranges quickly using a binary search?

You need to check for all events that meet e.startTime <= t1.
record Event(String name, int startTime, int endTime) {}
List<Event> list = Arrays.asList(
new Event("a", 2, 3), new Event("b", 3, 4),
new Event("c", 0, 1), new Event("d", 4, 5));
list.sort(Comparator.comparing(Event::startTime));
System.out.println("sorted: " + list);
int t1 = 2, t2 = 3;
List<Event> filtered = list.stream()
.takeWhile(e -> e.startTime() <= t1)
.peek(e -> System.out.println("checked: " + e))
.filter(e -> e.endTime() >= t2)
.toList();
System.out.println("filtered: " + filtered);
output:
sorted: [Event[name=c, startTime=0, endTime=1], Event[name=a, startTime=2, endTime=3], Event[name=b, startTime=3, endTime=4], Event[name=d, startTime=4, endTime=5]]
checked: Event[name=c, startTime=0, endTime=1]
checked: Event[name=a, startTime=2, endTime=3]
filtered: [Event[name=a, startTime=2, endTime=3]]

Binary search will not help you much, because you're not searching for a single equality-based match, but rather a range of results that can be ordered, but not in a way that's much help to quickly finding matches.
Unless you're dealing with a great many range elements (1000's), a linear (ie O(n)) process would work OK.
To speed things up, sort by start date beforehand, so your iteration over the list would be able to exit early when you encounter an element whose start date is after the target.

You should browse the list and stop when an item is not in range. In terms of complexity, it's the best you can do.

With Java 8 streams, How to find items with specific conditions within a specific period

I need help to find out how to find something in java which is currently being done in sql. I need to find specific data in a list within a specific time duration using streams
scenario:
I have a Bet Object with String CustId, Instant timestamp, Double betAmount
I need to find all customers which passed the 100.00 limit per 24 hour period, how would i do it in Java 8?
The method would be
public List<String> findLimits(List<Bet> bets){
...
}
sample data :
note: to be parsed in List<Bet>
A00001 2019-01-15T02:01:10 43.00
A00001 2019-01-15T04:00:00 13.00
A00001 2019-01-15T04:01:15 50.00
B00034 2019-01-15T05:00:00 15.00
A00001 2019-01-15T06:56:20 5.00
B00034 2019-01-15T06:57:00 20.00
C00004 2019-01-15T07:01:00 90.00
C00004 2019-01-15T07:11:00 30.00
B00034 2019-01-17T01:00:00 90.00
expected result:
["A00001","C00004"] (List<String>)
note: the list would contain all bets with diff cust id and chronological timestamps
The sliding 24 hour period and grouping the customers combined is a tricky one that i'm trying to solve.

You can map dates with the sum of bet amounts. Then filter them.
public List<String> findLimits(List<Bet> bets) {
return bets.stream()
.collect(Collectors.toMap(
b -> b.getCustId() + LocalDate.ofInstant(b.getTimestamp(), ZoneOffset.UTC).toString(),
Bet::getAmount,
(o1, o2) -> o1 + o2))
.entrySet().stream()
.filter(e -> e.getValue() > 100.0)
.map(e -> e.getKey().substring(0, e.getKey().length() - LocalDate.EPOCH.toString().length()))
.collect(Collectors.toList());
}

Firstly you can group data by customer ID and then analyze sums in 24h periods. As you mentioned records are sorted by date ascending, so findLimits method can look like below:
class Bet {
String ID;
LocalDateTime dateTime;
BigDecimal value;
}
public List<String> findLimits(List<Bet> bets) {
BigDecimal sumLimit = new BigDecimal(100);
Map<String, List<Bet>> map = new HashMap<String, List<Bet>>();
List<String> result = new ArrayList<String>();
for (Bet bet : bets) {
if (map.get(bet.ID) == null)
map.put(bet.ID, new ArrayList<Bet>());
map.get(bet.ID).add(bet);
}
for (String ID : map.keySet()) {
List<Bet> betListForCustomer = map.get(ID);
boolean customerExceededLimit = false;
for (int i = 0; i < betListForCustomer.size(); i++) {
LocalDateTime endOfPeriod = betListForCustomer.get(i).dateTime.plusDays(1); //calculating end of 24h period current data
BigDecimal sum = new BigDecimal(0);
for (int j = i; j < betListForCustomer.size() //move start period to next dateTime
&& endOfPeriod.isAfter(betListForCustomer.get(j).dateTime); j++) { //analyzing to the last date in 24h period or end data set
sum = sum.add(betListForCustomer.get(j).value);
}
if (sum.compareTo(sumLimit) >= 0) { //sum >= 100
customerExceededLimit = true;
break; //there is no need to analyze this customer, limit exceeded
}
}
if (customerExceededLimit) {
result.add(ID);
}
}
return result;
}

MongoDB aggregate variation to match on days difference match

What could be variation of this aggregation to get the result or items where the difference of the days = (a number) and the $acceptedDate is greater than $liveDate - x days
AggregationOperation redact = new AggregationOperation() {
#Override
public DBObject toDBObject(AggregationOperationContext aggregationOperationContext) {
Map<String, Object> map = new LinkedHashMap<>();
map.put("if", BasicDBObject.parse("{'$gte':[{'$subtract':[{'$ifNull':['$acceptedDate',
{'$date':" + System.currentTimeMillis() + "}]},'$lastVisit' **here minus x days **]},1296000000]}}")); //and the difference is 10 days
map.put("then", "$$KEEP");
map.put("else", "$$PRUNE");
return new BasicDBObject("$redact", new BasicDBObject("$cond", map));
};
Aggregation aggregation = Aggregation.newAggregation(redact);
List<FactoryAcceptance> results = mongoTemplate.aggregate(aggregation, FactoryAcceptance.class, FactoryAcceptance.class).getMappedResults();
Basically a parameter is being passed as a variation for this where they want to match the difference in days (say show me records where the difference is 5 days), but in this case lastVisit should be minus 5 days.

You can use below query.
BasicDBObject.parse("
{'$eq':[
{'$subtract':[
{'$ifNull':['$acceptedDate',{'$date':" + System.currentTimeMillis() + "}]},
{'$subtract':['$lastVisit', 216000000]} // 5 days
]},
432000000 // 10 days
]}"
);

JAVA & Joda Time API: compare intervals, detect overlapping and generate new intervals

I am working on a project that confuses me really bad right now.
Given is a List<TimeInterval> list that contains elements of the class TimeInterval, which looks like this:
public class TimeInterval {
private static final Instant CONSTANT = new Instant(0);
private final LocalDate validFrom;
private final LocalDate validTo;
public TimeInterval(LocalDate validFrom, LocalDate validTo) {
this.validFrom = validFrom;
this.validTo = validTo;
}
public boolean isValid() {
try {
return toInterval() != null;
}
catch (IllegalArgumentException e) {
return false;
}
}
public boolean overlapsWith(TimeInterval timeInterval) {
return this.toInterval().overlaps(timeInterval.toInterval());
}
private Interval toInterval() throws IllegalArgumentException {
return new Interval(validFrom.toDateTime(CONSTANT), validTo.toDateTime(CONSTANT));
}
The intervals are generated using the following:
TimeInterval tI = new TimeInterval(ld_dateValidFrom, ld_dateValidTo);
The intervals within the list may overlap:
|--------------------|
|-------------------|
This should result in:
|-------||-----------||------|
It should NOT result in:
|--------|-----------|-------|
Generally speaking in numbers:
I1: 2014-01-01 - 2014-01-30
I2: 2014-01-07 - 2014-01-15
That should result in:
I1: 2014-01-01 - 2014-01-06
I2: 2014-01-07 - 2014-01-15
I3: 2014-01-16 - 2014-01-30
I'm using JODA Time API but since I'm using for the first time, I actually don't really have a clue how to solve my problem. I already had a look at the method overlap() / overlapWith() but I still don't get it.
Your help is much appreciated!
UPDATE
I found something similar to my problem >here< but that doesn't help me for now.
I tried it over and over again, and even though it worked for the first intervals I tested, it doesn't actually work the way I wanted it to.
Here are the intervals I have been given:
2014-10-20 ---> 2014-10-26
2014-10-27 ---> 2014-11-02
2014-11-03 ---> 2014-11-09
2014-11-10 ---> 2014-11-16
2014-11-17 ---> 9999-12-31
This is the function I am using to generate the new intervals:
private List<Interval> cleanIntervalList(List<Interval> sourceList) {
TreeMap<DateTime, Integer> endPoints = new TreeMap<DateTime, Integer>();
// Fill the treeMap from the TimeInterval list. For each start point,
// increment the value in the map, and for each end point, decrement it.
for (Interval interval : sourceList) {
DateTime start = interval.getStart();
if (endPoints.containsKey(start)) {
endPoints.put(start, endPoints.get(start)+1);
}
else {
endPoints.put(start, 1);
}
DateTime end = interval.getEnd();
if (endPoints.containsKey(end)) {
endPoints.put(end, endPoints.get(start)-1);
}
else {
endPoints.put(end, 1);
}
}
System.out.println(endPoints);
int curr = 0;
DateTime currStart = null;
// Iterate over the (sorted) map. Note that the first iteration is used
// merely to initialize curr and currStart to meaningful values, as no
// interval precedes the first point.
List<Interval> targetList = new LinkedList<Interval>();
for (Entry<DateTime, Integer> e : endPoints.entrySet()) {
if (curr > 0) {
if (e.getKey().equals(endPoints.lastEntry().getKey())){
targetList.add(new Interval(currStart, e.getKey()));
}
else {
targetList.add(new Interval(currStart, e.getKey().minusDays(1)));
}
}
curr += e.getValue();
currStart = e.getKey();
}
System.out.println(targetList);
return targetList;
}
This is what the output actually looks like:
2014-10-20 ---> 2014-10-25
2014-10-26 ---> 2014-10-26
2014-10-27 ---> 2014-11-01
2014-11-02 ---> 2014-11-02
2014-11-03 ---> 2014-11-08
2014-11-09 ---> 2014-11-09
2014-11-10 ---> 2014-11-15
2014-11-16 ---> 2014-11-16
2014-11-17 ---> 9999-12-31
And this is what the output SHOULD look like:
2014-10-20 ---> 2014-10-26
2014-10-27 ---> 2014-11-02
2014-11-03 ---> 2014-11-09
2014-11-10 ---> 2014-11-16
2014-11-17 ---> 9999-12-31
Since there is no overlap in the original intervals, I don't get why it produces stuff like
2014-10-26 ---> 2014-10-26
2014-11-02 ---> 2014-11-02
2014-11-09 ---> 2014-11-09
etc
I've been trying to fix this all day long and I'm still not getting there :( Any more help is much appreciated!

Half-Open
I suggest you reconsider the terms of your goal. Joda-Time wisely uses the "Half-Open" approach to defining a span of time. The beginning is inclusive while the ending is exclusive. For example, a week starts an the beginning of the first day and runs up to, but not including, the first moment of the next week. Half-open proves to be quite helpful and natural way to handle spans of time, as discussed in other answers.
Using this Half-Open approach for your example, you do indeed want this result:
|--------|-----------|-------|
I1: 2014-01-01 - 2014-01-07
I2: 2014-01-07 - 2014-01-16
I3: 2014-01-16 - 2014-01-30
Search StackOverflow for "half-open" to find discussion and examples, such as this answer of mine.
Joda-Time Interval
Joda-Time has an excellent Interval class to represent a span of time defined by a pair of endpoints on the timeline. That Interval class offers overlap, overlaps (sic), abuts, and gap methods. Note in particular the overlap method that generates a new Interval when comparing two others; that may be key to your solution.
But unfortunately, that class only works with DateTime objects and not LocalDate (date-only, no time-of-day or time zone). Perhaps that lack of support for LocalDate is why you or your team invented that TimeInterval class. But I suggest rather that using that custom class, consider using DateTime objects with Joda-Time's classes. I'm not 100% certain that is better than rolling your own date-only interval class (I've been tempted to do that), but my gut tells me so.
To focus on days rather than day+time, on your DateTime objects call the withTimeAtStartOfDay method to adjust the time portion to the first moment of the day. That first moment is usually 00:00:00.000 but not necessarily due to Daylight Saving Time (DST) and possibly other anomalies. Just be careful and consistent with the time zone; perhaps use UTC throughout.
Here is some example code in Joda-Time 2.5 using the values suggested in the Question. In these particular lines, the call to withTimeAtStartOfDay may be unnecessary as Joda-Time defaults to first moment of day when no day-of-time is provided. But I suggest using those calls to withTimeAtStartOfDay as it makes your code self-documenting as to your intent. And it makes all your day-focused use of DateTime code consistent.
Interval i1 = new Interval( new DateTime( "2014-01-01", DateTimeZone.UTC ).withTimeAtStartOfDay(), new DateTime( "2014-01-30", DateTimeZone.UTC ).withTimeAtStartOfDay() );
Interval i2 = new Interval( new DateTime( "2014-01-07", DateTimeZone.UTC ).withTimeAtStartOfDay(), new DateTime( "2014-01-15", DateTimeZone.UTC ).withTimeAtStartOfDay() );
From there, apply the logic suggested in the other answers.

Here is a suggested algorithm, based on the answer you have already found. First, you need to sort all the end points of the intervals.
TreeMap<LocalDate,Integer> endPoints = new TreeMap<LocalDate,Integer>();
This map's keys - which are sorted since this is a TreeMap - will be the LocalDate objects at the start and end of your intervals. They are mapped to a number that represents the number of end points at this date subtracted from the number of start points at this date.
Now traverse your list of TimeIntervals. For each one, for the start point, check whether it is already in the map. If so, add one to the Integer. If not, add it to the map with the value of 1.
For the end point of the same interval, if it exists in the map, subtract 1 from the Integer. If not, create it with the value of -1.
Once you finished filling endPoints, create a new list for the "broken up" intervals you will create.
List<TimeInterval> newList = new ArrayList<TimeInterval>();
Now start iterating over endPoints. If you had at least one interval in the original list, you'll have at least two points in endPoints. You take the first, and keep the key (LocalDate) in a variable currStart, and its associated Integer in another variable (curr or something).
Loop starting from the second element until the end. At each iteration:
If curr > 0, create a new TimeInterval starting at currStart and ending at the current key date. Add it to newList.
Add the Integer value to curr.
Assign the key as your next currStart.
And so on until the end.
What happens here is this: ordering the dates makes sure you have no overlaps. Each new interval is guaranteed not to overlap with any new one since they have exclusive and sorted end points. The trick here is to find the spaces in the timeline which are not covered by any intervals at all. Those empty spaces are characterized by the fact that your curr is zero, as it means that all the intervals that started before the current point in time have also ended. All the other "spaces" between the end points are covered by at least one interval so there should be a corresponding new interval in your newList.
Here is an implementation, but please notice that I did not use Joda Time (I don't have it installed at the moment, and there is no particular feature here that requires it). I created my own rudimentary TimeInterval class:
public class TimeInterval {
private final Date validFrom;
private final Date validTo;
public TimeInterval(Date validFrom, Date validTo) {
this.validFrom = validFrom;
this.validTo = validTo;
}
public Date getStart() {
return validFrom;
}
public Date getEnd() {
return validTo;
}
#Override
public String toString() {
return "[" + validFrom + " - " + validTo + "]";
}
}
The important thing is to add the accessor methods for the start and end to be able to perform the algorithm as I wrote it. In reality, you should probably use Joda's Interval or implement their ReadableInterval if you want to use their extended features.
Now for the method itself. For this to work with yours you'll have to change all Date to LocalDate:
public static List<TimeInterval> breakOverlappingIntervals( List<TimeInterval> sourceList ) {
TreeMap<Date,Integer> endPoints = new TreeMap<>();
// Fill the treeMap from the TimeInterval list. For each start point, increment
// the value in the map, and for each end point, decrement it.
for ( TimeInterval interval : sourceList ) {
Date start = interval.getStart();
if ( endPoints.containsKey(start)) {
endPoints.put(start, endPoints.get(start) + 1);
} else {
endPoints.put(start, 1);
}
Date end = interval.getEnd();
if ( endPoints.containsKey(end)) {
endPoints.put(end, endPoints.get(start) - 1);
} else {
endPoints.put(end, -1);
}
}
int curr = 0;
Date currStart = null;
// Iterate over the (sorted) map. Note that the first iteration is used
// merely to initialize curr and currStart to meaningful values, as no
// interval precedes the first point.
List<TimeInterval> targetList = new ArrayList<>();
for ( Map.Entry<Date,Integer> e : endPoints.entrySet() ) {
if ( curr > 0 ) {
targetList.add(new TimeInterval(currStart, e.getKey()));
}
curr += e.getValue();
currStart = e.getKey();
}
return targetList;
}
(Note that it would probably be more efficient to use a mutable Integer-like object rather than Integer here, but I opted for clarity).

I'm not fully up to speed on Joda; I'll need to read up on that if you want an overlap-specific solution.
However, this is possible using only the dates. This is mostly pseudocode, but should bring the point across. I've also added notation so you can tell what the intervals look like. There's also some confusion for me as to whether I should be adding 1 or subtracting 1 for an overlap, so I erred on the side of caution by pointing outward from the overlap (-1 for start, +1 for end).
TimeInterval a, b; //a and b are our two starting intervals
TimeInterval c = null;; //in case we have a third interval
if(a.start > b.start) { //move the earliest interval to a, latest to b, if necessary
c = a;
a = b;
b = c;
c = null;
}
if(b.start > a.start && b.start < a.end) { //case where b starts in the a interval
if(b.end > a.end) { //b ends after a |AA||AB||BB|
c = new TimeInterval(a.end + 1, b.end);//we need time interval c
b.end = a.end;
a.end = b.start - 1;
}
else if (b.end < a.end) { //b ends before a |AA||AB||AA|
c = new TimeInterval(b.end + 1, a.end);//we need time interval c
a.end = b.start - 1;
}
else { //b and a end at the same time, we don't need c |AA||AB|
c = null;
a.end = b.start - 1;
}
}
else if(a.start == b.start) { //case where b starts same time as a
if(b.end > a.end) { //b ends after a |AB||B|
b.start = a.end + 1;
a.end = a.end;
}
else if(b.end < a.end) { //b ends before a |AB||A|
b.start = b.end + 1;
b.end = a.end;
a.end = b.start;
}
else { //b and a are the same |AB|
b = null;
}
}
else {
//no overlap
}

Find holes in Joda-Time intervals

I have list of Joda-Time intervals
List<Interval> intervals = new ArrayList<Interval>();
and another Joda-Time interval (search time interval), like on the picture below.
I need to write Java function that finds the holes in time and returns List<Interval> with the red intervals.

Building up on fge's response - the following version actually handles both cases (when the big interval is larger than the extremes of the intervals being searched over + the case when the big interval is in fact smaller ... or smaller on one side)
you can see the full code along with the tests at https://github.com/erfangc/JodaTimeGapFinder.git
public class DateTimeGapFinder {
/**
* Finds gaps on the time line between a list of existing {#link Interval}
* and a search {#link Interval}
*
* #param existingIntervals
* #param searchInterval
* #return The list of gaps
*/
public List<Interval> findGaps(List<Interval> existingIntervals, Interval searchInterval) {
List<Interval> gaps = new ArrayList<Interval>();
DateTime searchStart = searchInterval.getStart();
DateTime searchEnd = searchInterval.getEnd();
if (hasNoOverlap(existingIntervals, searchInterval, searchStart, searchEnd)) {
gaps.add(searchInterval);
return gaps;
}
// create a sub-list that excludes interval which does not overlap with
// searchInterval
List<Interval> subExistingList = removeNoneOverlappingIntervals(existingIntervals, searchInterval);
DateTime subEarliestStart = subExistingList.get(0).getStart();
DateTime subLatestStop = subExistingList.get(subExistingList.size() - 1).getEnd();
// in case the searchInterval is wider than the union of the existing
// include searchInterval.start => earliestExisting.start
if (searchStart.isBefore(subEarliestStart)) {
gaps.add(new Interval(searchStart, subEarliestStart));
}
// get all the gaps in the existing list
gaps.addAll(getExistingIntervalGaps(subExistingList));
// include latestExisting.stop => searchInterval.stop
if (searchEnd.isAfter(subLatestStop)) {
gaps.add(new Interval(subLatestStop, searchEnd));
}
return gaps;
}
private List<Interval> getExistingIntervalGaps(List<Interval> existingList) {
List<Interval> gaps = new ArrayList<Interval>();
Interval current = existingList.get(0);
for (int i = 1; i < existingList.size(); i++) {
Interval next = existingList.get(i);
Interval gap = current.gap(next);
if (gap != null)
gaps.add(gap);
current = next;
}
return gaps;
}
private List<Interval> removeNoneOverlappingIntervals(List<Interval> existingIntervals, Interval searchInterval) {
List<Interval> subExistingList = new ArrayList<Interval>();
for (Interval interval : existingIntervals) {
if (interval.overlaps(searchInterval)) {
subExistingList.add(interval);
}
}
return subExistingList;
}
private boolean hasNoOverlap(List<Interval> existingIntervals, Interval searchInterval, DateTime searchStart, DateTime searchEnd) {
DateTime earliestStart = existingIntervals.get(0).getStart();
DateTime latestStop = existingIntervals.get(existingIntervals.size() - 1).getEnd();
// return the entire search interval if it does not overlap with
// existing at all
if (searchEnd.isBefore(earliestStart) || searchStart.isAfter(latestStop)) {
return true;
}
return false;
}
}

A quick look at the Interval API gives this (UNTESTED):
// SUPPOSED: the big interval is "bigInterval"; the list is "intervals"
// Intervals returned
List<Interval> ret = new ArrayList<>();
Interval gap, current, next;
// First, compute the gaps between the elements in the list
current = intervals.get(0);
for (int i = 1; i < intervals.size(); i++) {
next = intervals.get(i);
gap = current.gap(next);
if (gap != null)
ret.add(gap);
current = next;
}
// Now, compute the time difference between the starting time of the first interval
// and the starting time of the "big" interval; add it at the beginning
ReadableInstant start, end;
start = bigInterval.getStart();
end = intervals.get(0).getStart();
if (start.isBefore(end))
ret.add(0, new Interval(start, end));
//
// finally, append the time difference between the ending time of the last interval
// and the ending time of the "big" interval
// next still contains the last interval
start = next.getEnd();
end = bigInterval.getEnd();
if (start.isBefore(end))
ret.add(new Interval(start, end));
return ret;

The answer by fge seems to be correct, though I've not run that untested code.
The term "gap" seems to be a more common term for what you are calling "holes".
See this answer by Katja Christiansen, which makes good use of the gap method on the Interval class.
Interval gapInterval = interval_X.gap( interval_Y );
// … Test for null to see whether or a gap exists.
If there is a non-zero duration between them, you get a new Interval object returned. If the intervals overlap or abut, then null is returned. Note that the Interval class also offers the methods overlap and abuts if you are interested in those particular conditions.
Of course your collection of Interval objects must be sorted for this to work.