We Keep Coding

How can a task be scheduled in Java to occur at midnight, even if a leap second has just occurred?

First, definition of "occur at midnight" is that when task is run, new DateTime() or similar will show 00:00:00 or later for the time portion when converted to a human readable format. Important point is that it must not show 23:59:59 of the previous day.
A common way to achieve this would be to calculate how many milliseconds are between now and the desired point in time, and then use a ScheduledExecutorService to execute the task at the correct time. However, when a leap second is inserted this will result in the task running a second early (or a few milliseconds early depending on how the leap second is 'smeared' and when you scheduled the task):
Runnable task = ...
long numberOfMillisUntilMidnight = ...
ScheduledExecutorService executor = ...
// task runs too early when leap seconds are inserted
executor.schedule(task, numberOfMillisUntilMidnight, TimeUnit.MILLISECONDS);
The reason is that executor.schedule() is based on System.nanoTime() which obviously ignores the leap seconds. I guess what I need is some scheduler based on "run at this time" rather than "run after this amount of time".
For those who are interested, the reason the task must run at midnight related to the fact that all events in my system must be categorized according to which day they occurred on, and in so far as is possible, this needs to be in sync with another system. Of course it would be better if the other system stamped each event with what day it is, but we are not there.

I guess what I need is some scheduler based on "run at this time" rather than "run after this amount of time"
That would be the all-singing, all-dancing solution. But:
First, definition of "occur at midnight" is that when task is run, new DateTime() or similar will show 00:00:00 or later for the time portion...Important point is that it must not show 23:59:59 of the previous day.
(my emphasis)
The simple way is always add a second, or even two. So it'd be 00:00:01 (00:00:00 or later) in the common case, and 00:00:00 (not 23:59:59) in the leap second case.

Based on the resulting discussions it seems clear that, in general, it is unwise to rely on your scheduler to run a task at the "correct" time if "wall time" is important to you. This is also true when running daily tasks at the same "wall time" across daylight savings shifts, although unlike the leap second case, the daylight savings case is well supported by existing tools (by Quartz for example).
Instead I think the best approach for such "wall time sensitive" processes is that when the task is run, check the system clock at that point. If your schedule was inaccurate for whatever reason (leap seconds are not the only time your system clock is adjusted relative to the elapsed time measured by System.nanoTime()) and the time has not yet been reached, then do nothing and reschedule the task for the correct time. This approach would also work for schedules that respond to daylight savings changes but as mentioned above this is already supported by common tools.
This approach was inspired by the comment by Jonathon Reinhart above. Rescheduling rather than sleeping though seems better.

Assuming that your concrete ScheduledExecutorService-implementation relies on System.nanoTime() (as you said) and taking into account your requirement/configuration that the initial delay parameter of the method schedule(...) counts the elapsed milliseconds until next midnight including a possible leap second,
I suggest you to use a leap-second-aware solution. An example using my library Time4J shows how to calculate the delay parameter:
Moment now = SystemClock.currentMoment(); // should not be called during a leap second
Moment nextMidnight =
now.with(
PlainTime.COMPONENT.setToNext(PlainTime.midnightAtStartOfDay()).in(
Timezone.ofSystem().with(
GapResolver.NEXT_VALID_TIME.and(OverlapResolver.EARLIER_OFFSET)
)
)
);
long delayInMilliseconds = SI.NANOSECONDS.between(now, nextMidnight) / 1_000_000;
This code will also choose the earliest valid local time after midnight in case of daylight-saving-change (standard-Java would rather push the time forward by the size of the gap possibly resulting in a local time later than first valid time). For most zones, this is only relevant if choosing an arbitrary start time after midnight (dependent on business requirements).
What so ever, you should also care about connecting your systems to the same NTP-clock. Either you rely on OS-NTP-configuration, or you can use a Java-based clock connecting to NTP (Time4J offers here a solution, too).
If your chosen clock is just doing arbitrary jumps (i.e. if someone has manually adjusted it or in case of bad NTP-configuration) then rescheduling the task after having checked the local walltime again is probably safer. However, I still think that calculating the delay parameter by Time4J-code above is a good idea because the chance to match midnight is higher than just to run the task and rechecking the local time.
You could also combine both approaches: exact calculation of delay AND check/reschedule.

How to start batch jobs using DataMinder scheduler?

We need to run a batch job each night at 03:00. We have a few processes each built with some tasks.
The idea is to let the processes run each night and download some data files and update other systems.
What I don't understand is how to implement a scheduler (DMScheduler)
that will start a process every night at a specific time.
There exits general information about plugin development but nothing
specific about schedulers as far as I can see.
Can someone explain how DataMinder schedulers work and how to implement one as above?
Perhaps some implementation examples?

The 2 scheduler methods bellow work with "machine" time (as opposed to "human" time):
getNextRunTimeInMilliseconds(long nowInMilliseconds)
setLastStartedTimeInMilliseconds(long lastStartedTimeInMilliseconds)
Please see discussion about "machine" and "human" time at https://docs.oracle.com/javase/tutorial/datetime/iso/instant.html
Basically "machine" time is a time stamp in time without any relation to dates, time zones.
"Human" time is a specific date and time in a specific time zone.
This means that in this case you need to transform between the "machine" time and "human" time.
Calls to method getNextRunTimeInMilliseconds(nowInMilliseconds) may be called multiple times by DataMinder and is just a check
to see if next run time has changed because, for example, plugin/scheduler parameters may have changed.
When a process has been started and finished running the method
setLastStartedTimeInMilliseconds(lastStartedTimeInMilliseconds)
is called with the last time the process was started and run.
Let's take an example:
Here are some date/times and corresponding time stamps:
2016-08-06 03:00:00 = 1470445200000
2016-08-07 03:00:00 = 1470531600000
Assume the process was last run "6:th of August 2016, 03:00 CET". This corresponds to time stamp (as Java long) 1470445200000.
When
setLastStartedTimeInMilliseconds(1470445200000)
is called with above value you need to transfer it to "human" time
"6:th of August 2016, 03:00 CET" and then figure out what the next run time should be.
In this case the next runtime in "human" time would be the day after, that is "7:th of August 2016, 03:00 CET" and this corresponds to
timestamp 1470531600000.
The
getNextRunTimeInMilliseconds(nowInMilliseconds)
should return the new value
of 1470531600000 which tells DataMinder to run the processes on "7:th of August 2016, 03:00 CET". And when the process
has run a call will be made to
setLastStartedTimeInMilliseconds(lastStartedTimeInMilliseconds)
and the scheduler must again figure
out what the next run time should be based on the received lastStartedTimeInMilliseconds.
Hope it make things more clear.

java Calendar, Date, and Time management for a multi-timezone application

I am designing a scheduling web app.
I expect events to be added by users in several different timezones and locales. The challenge is correct presentation of these events.
So, as an example:
if a user is in EST timezone and is looking at a webinar event that was added by another user in PST, I want to convert the actual PST time of event into a local time for viewer. So, if an event is scheduled for 2 PM PST, then it should shown as 5 PM EST.
I also want to be careful that performance does not take a hit if there are thousands of events which may require a conversion from actual event time to the viewer's local time.
All thoughts and comments are appreciated.
TIA

In general, scheduling future events is a complex subject. You have to make a distinction in the context of what is going to be scheduled:
Does the event occur at a specific universal instant in time? If so, you should record the event time in terms of UTC.
For example, a task that runs every 24 hours would be scheduled by UTC time and not by the local time. It might start off at some local midnight, but as daylight saving time changes take effect it could be running at 23:00 or 01:00 by the local clock.
However, if the event is scheduled by human beings, it is likely to be in terms of a local time, so you should record it that way.
For example, a meeting that occurs at 08:00 Eastern Time will always occur at that local time. In the winter, that would be 13:00 UTC, and in the summer it would be at 12:00 UTC.
So in this context, you cannot record the scheduled start time in terms of UTC. This is a very common mistake, as there is a ton of advice on the Internet that says "always store using UTC", which would be wrong in this scenario.
Instead, you should store two values - the local time such as 08:00 and its IANA time zone identifier, such as America/New_York. You may also need to store a recurrence pattern or specific date depending on how the event is scheduled.
Consider using Joda Time instead of Java's Calendar or Date classess. It will save you from many headaches. Make sure you read the Joda Time documentation and understand how it works.
Joda Time has all of the functions you will need for converting between one time zone and another - which I believe was the primary concern of your question.
Be sure to have a procedure in place for updating the time zone data regularly. Updates are pushed out multiple times a year, as the governments of the world make changes to the legal definitions of their time zones. You cannot just deploy it once and forget about it.
Also be sure you understand that conversion from local time to a specific UTC moment is not a perfect function due to daylight saving time. If an event is scheduled during an invalid or ambiguous local time, you should have a strategy for detecting and dealing with that in your application. You might just apply some assumptions, or you might want to go out of your way to ask the user what to do.
For example, if I schedule an event at 2:00 AM Eastern Time every day, then on March 10th 2013, that time does not exist. Should the event occur at 3:00 AM? Or should it not occur at all?
Another example, if I schedule an event at 1:00 AM Eastern Time every day, then on November 3rd, 2013, that time occurs twice. Should the event happen at the first (daylight time) instance? Or at the second (standard time) instance? Or both? Should I assume one or the other, or should I ask the user which they mean?
Only you can decide what to do, as it is your application. But ignoring the problem will likely lead to errors.
Once an event has passed, you can record it in UTC if you wish, or record it with the full local date time and offset. Either are acceptable. This works just fine for singular past events, just not for recurring future ones.

Check if a time with matching units has past since the last check

I have a system that needs to trigger at certain intervals
If it is given the following:
minutes:25, seconds:10
It should then trigger every hour, 25 minutes and 10 seconds past the hour. For example, 7:25:10, 8:25:10, 9:25:10, and so on.
This would be simple enough, and I already have that code working. The problem is that it only works if the system checks at least once a second, otherwise it will miss a trigger.
So how can I check if the trigger units would have matched the current time units since the last check? The time between checks will vary greatly.
The trigger units are stored like this:
// units that must match the current time <Calendar Unit, Trigger>
private Map<Integer, Integer> triggerUnits;

First move to enum instead of map. The enum can contain hour, second, day, whatever. The good new is that such enum already exists in JDK and is called TimeUnit. Each member of this enum knows to translate the time to other units.
So, parse string hour:5. Translate the 5 hours to 3600*5*1000 milliseconds. This value will be used in your system. You have several possibilities but I'd suggest you to start from java.util.Timer. Each task can be scheduled in the timer. It uses one single thread for execution of all tasks.
If you want you can use use Executors framework:
ScheduledExecutorService service = Executors.newSingleThreadScheduledExecutor();
service.schedule(...);
The schedule method here directly works with time units. You even do not have to translate from one unit to another.

Each time you trigger the system, store the next time that the code needs to be triggered. During the checks, simply check if the current time has passed this "next time", and if so, then trigger the system.

Not precisely an answer for what you've asked, but I'd suggest using cron4j: the Predictor sounds like it is almost exactly what you need. Translate your spec into cron format, and then pass it to the Predictor and it will actually tell you the absolute time of the next execution. You can just schedule your check to happen at that moment in time or use a ScheduledExecutor to fire the job.
Might be a bit more complicated in some ways (and its an additional dependency), but completely eliminates the timing risk.

Java Calendar scheduling events

In my application i want to schedule the start of processing. In two ways first on a preset date or say like every Monday. In the case of a single date i can watch the time on a tread and start processing but when two are mixed i can't come up with a good solution.
My initial thought was when application boots up i can schedule events to calendar and check if there is job to do every min or so, that would work for both single date and every week case turns out i can not use the calendar that way.
What is a good way to solve this?

Quartz is an open-source job scheduling component written in Java, you might want to check that out.
Its features range from simple timers to full-blown CRON expressions, and it's used extensively by the JBoss AS.

Look at java.util.Timer. It allows you to schedule tasks for execution at a specified time on a background thread, and it support recurring events.