how do you map java localdatetime to avro - java

Im looking at understanding how we map localdatetime in java to avro.
is correct to use
{
"name": "example",
"type": ["null", {"type": "int", "logicalType": "timestamp-millis"}],
"default": null
}
How do i set this on my avro object..
LocalDateTime localDateTime = LocalDateTime.now();
long epochMillis = localDateTime.toInstant(ZoneOffset.UTC).toEpochMilli();
should avro not be defined as a Long?

int can't store epoch millis; you need long.
Compare these values:
1,674,123,456,789 // current epoch millis (approx)
2,147,483,647 // largest int value
9,223,372,036,854,775,807 // largest long value
Define the avro field as long.

As per ealier comments i ended up using the below which gave me what i wanted
"type": ["null", {"type": "long", "logicalType": "local-timestamp-millis"}],

Related

Get time period ( hour / half-hour / quarter hour ) of day

Is there any way to download every hour and another period of the day?
I mean exactly the result in the form of a list.
[
"00:00", // start of the day e.q 14.03
"01:00",
"02:00",
....
"23:00",
"00:00 // end of the day e.q 14.03
]
and
[
"00:00", // start of the day e.q 14.03
"00:30",
"01:00"
"01:30"
....
"00:00 // end of the day e.q 14.03
]
and
[
"00:00", // start of the day e.q 14.03
"00:15",
"00:30"
"00:45"
"01:00"
....
"00:00 // end of the day e.q 14.03
]
14.03 means March 14 for the sake of the example.
Of course it is possible to add this manually, but it would not be a particularly elegant solution. Is it possible to do it without explicitly declaring constants as values of each hour ?
The best solution would be a function without using loops and if else constructions.
I have not yet been able to find an implementation of such a solution. I myself also spend another hour on this without success. I need it to implement a functionality that creates a Map or a list of pairs from a list of such e.q hours:
[
"00:00 - 01:00",
"01:00 - 02:00",
"02:00 - 03:00 "
//.......
"23:00 - 00:00"
]
Has anyone had occasion to implement such a problem and would be able to help?
You could write a function to generate a Sequence<LocalTime> like this:
fun generateTimesOfDay(interval: java.time.Duration) = sequence {
if (interval > java.time.Duration.ofDays(1)) {
repeat(2) { yield(LocalTime.MIDNIGHT) }
return#sequence
}
var time = LocalTime.MIDNIGHT
while (true) {
yield(time)
val newTime = time + interval
if (newTime > time) {
time = newTime
} else {
break // passed or reached midnight
}
}
yield(LocalTime.MIDNIGHT) // Midnight the next day to end last period
}
Then you have all the LocalTimes you can use with toString() or with .format() and some DateTimeFormatter to format them how you like. You can use zipWithNext() to create the time ranges.
val x = generateTimesOfDay(java.time.Duration.ofMinutes(15))
println(x.toList())
println(
x.zipWithNext { a, b -> "$a - $b" }.toList()
)
Note, I'm using fully qualified java.time.Duration to avoid conflicts with the Kotlin standard library Duration class.

Compare time difference in hour from util Date

I want to compare time difference in hours. Based on current time and time I get from database.
DateFormat sdf = new SimpleDateFormat(“yyyy-MM-dd HH:mm:ss”);
Date date = new Date();
Logger.info(“current time is”,sdf.format(date));
// gives date in 2019-11-06 17:03:54
// dB gives following record
Date successDate = loader.getLastSuccess();
// gives date in 2019-10-31T:56:08.066+0000
Both formats are different how to get the time difference any suggestion experts
You can use the java-8 date API Duration to get the duration between both the dates
long hours = Duration.between(date1.toInstant(), date2.toInstant()).toHours();
Note : It can return negative value also here
the number of hours in the duration, may be negative
public int getHours() on util.Date is deprecated, so convert them to Instant and use Duration.between and also i will suggest to use java-8 Date API instead of older version Date
If you want difference in hours as double, you can do this;
Date your_date = loader.getLastSuccess();
Date currentDate = new Date();
double hourdifference = (currentDate.getTime() - your_date.getTime()) / 3600000.0;
You can get long or int, just change 3600000.0 to 3600000, and make the variable int or long

java 8 - ZonedDateTime not equal another ZonedDateTime

I created two ZonedDateTime objects and I think they are should be equal:
public static void main(String[] args) {
ZoneId zid = ZoneId.of("America/New_York");
ZoneOffset offset = ZoneOffset.from(LocalDateTime.now().atZone(zid));
ZonedDateTime zdt0 = ZonedDateTime.of(2014, 8, 24, 21, 10, 1, 777000002, offset);
ZonedDateTime zdt1 = ZonedDateTime.of(2014, 8, 24, 21, 10, 1, 777000002, zid);
boolean equals = Objects.equals(zdt0, zdt1);
System.out.println("equals: " + equals);
}
In debugger I see that class of member of ZonedDateTime zone in first case is java.time.ZoneOffset and in second java.time.ZoneRegion and this is makes ZonedDateTime objects not equal. This is confusing...
Any ideas?
You are checking for object equality which evaluates to false as these objects are not equivalent. One is bound to a ZoneId, the other to a ZoneOffset. If you want to check whether they represent the same time, you can use the not very intuitively named method isEqual.
E.g.:
ZoneId zid = ZoneId.of("America/New_York");
ZoneOffset offset = ZoneOffset.from(LocalDateTime.now().atZone(zid));
ZonedDateTime zdt0 = ZonedDateTime.of(2014, 8, 24, 21, 10, 1, 777000002, offset);
ZonedDateTime zdt1 = ZonedDateTime.of(2014, 8, 24, 21, 10, 1, 777000002, zid);
System.out.println("isEqual:" + zdt0.isEqual(zdt1));
System.out.println("equals: " + zdt0.equals(zdt1));
prints:
isEqual:true
equals: false
Btw, note that you don’t need to use Objects.equals(a,b) for two objects you already know to be non-null. You can invoke a.equals(b) directly.
This played hell on me for hours too when using Jackson to serialize / deserialize instances of ZonedDateTime and then compare them against each other for equality to verify that my code was working correctly. I don't fully understand the implications but all I've learned is to use isEqual instead of equals. But this throws a big wrench in testing plans as most assertion utilities will just call the standard .equals().
Here's what I finally came up with after struggling for quite some time:
#Test
public void zonedDateTimeCorrectlyRestoresItself() {
// construct a new instance of ZonedDateTime
ZonedDateTime now = ZonedDateTime.now(ZoneId.of("Z"));
// offset = {ZoneOffset#3820} "Z"
// zone = {ZoneOffset#3820} "Z"
String starting = now.toString();
// restore an instance of ZonedDateTime from String
ZonedDateTime restored = ZonedDateTime.parse(starting);
// offset = {ZoneOffset#3820} "Z"
// zone = {ZoneOffset#3820} "Z"
assertThat(now).isEqualTo(restored); // ALWAYS succeeds
System.out.println("test");
}
#Test
public void jacksonIncorrectlyRestoresZonedDateTime() throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.findAndRegisterModules();
// construct a new instance of ZonedDateTime
ZonedDateTime now = ZonedDateTime.now(ZoneId.of("Z"));
// offset = {ZoneOffset#3820} "Z"
// zone = {ZoneOffset#3820} "Z"
String converted = objectMapper.writeValueAsString(now);
// restore an instance of ZonedDateTime from String
ZonedDateTime restored = objectMapper.readValue(converted, ZonedDateTime.class);
// offset = {ZoneOffset#3820} "Z"
// zone = {ZoneOffset#3821} "UTC"
assertThat(now).isEqualTo(restored); // NEVER succeeds
System.out.println("break point me");
}
The equals() method on ZonedDateTime requires that all component parts of the object are equal. Since a ZoneOffset is not equal to a ZoneRegion (even though both are subclasses of ZoneId), the method returns false. Read about VALJOs to understand more as to why value types are compared in this way.
The isEqual method only compares the instant on the time-line, which may or may not be what you want. You can also use the timeLineOrder() method to compare two ZoneDateTime only using the time-line.
This caused us pain for a while too. The ZonedDateTime values actually represent identical times, however their zone "types" are different, which is why they are not equal.
The above answers are correct, however I thought I'd add a visual that might further help:
In the ZonedDateTime code, we find, which shows the zone comparison:

How to extract epoch from LocalDate and LocalDateTime?

How do I extract the epoch value to Long from instances of LocalDateTime or LocalDate? I've tried
the following, but it gives me other results:
LocalDateTime time = LocalDateTime.parse("04.02.2014 19:51:01", DateTimeFormatter.ofPattern("dd.MM.yyyy HH:mm:ss"));
System.out.println(time.getLong(ChronoField.SECOND_OF_DAY)); // gives 71461
System.out.println(time.getLong(ChronoField.EPOCH_DAY)); // gives 16105
What I want is simply the value 1391539861 for the local datetime "04.02.2014 19:51:01".
My timezone is Europe/Oslo UTC+1 with daylight saving time.
The classes LocalDate and LocalDateTime do not contain information about the timezone or time offset, and seconds since epoch would be ambigious without this information. However, the objects have several methods to convert them into date/time objects with timezones by passing a ZoneId instance.
LocalDate
LocalDate date = ...;
ZoneId zoneId = ZoneId.systemDefault(); // or: ZoneId.of("Europe/Oslo");
long epoch = date.atStartOfDay(zoneId).toEpochSecond();
LocalDateTime
LocalDateTime time = ...;
ZoneId zoneId = ZoneId.systemDefault(); // or: ZoneId.of("Europe/Oslo");
long epoch = time.atZone(zoneId).toEpochSecond();
'Millis since unix epoch' represents an instant, so you should use the Instant class:
private long toEpochMilli(LocalDateTime localDateTime)
{
return localDateTime.atZone(ZoneId.systemDefault())
.toInstant().toEpochMilli();
}
The conversion you need requires the offset from UTC/Greewich, or a time-zone.
If you have an offset, there is a dedicated method on LocalDateTime for this task:
long epochSec = localDateTime.toEpochSecond(zoneOffset);
If you only have a ZoneId then you can obtain the ZoneOffset from the ZoneId:
ZoneOffset zoneOffset = ZoneId.of("Europe/Oslo").getRules().getOffset(ldt);
But you may find conversion via ZonedDateTime simpler:
long epochSec = ldt.atZone(zoneId).toEpochSecond();
Look at this method to see which fields are supported. You will find for LocalDateTime:
•NANO_OF_SECOND
•NANO_OF_DAY
•MICRO_OF_SECOND
•MICRO_OF_DAY
•MILLI_OF_SECOND
•MILLI_OF_DAY
•SECOND_OF_MINUTE
•SECOND_OF_DAY
•MINUTE_OF_HOUR
•MINUTE_OF_DAY
•HOUR_OF_AMPM
•CLOCK_HOUR_OF_AMPM
•HOUR_OF_DAY
•CLOCK_HOUR_OF_DAY
•AMPM_OF_DAY
•DAY_OF_WEEK
•ALIGNED_DAY_OF_WEEK_IN_MONTH
•ALIGNED_DAY_OF_WEEK_IN_YEAR
•DAY_OF_MONTH
•DAY_OF_YEAR
•EPOCH_DAY
•ALIGNED_WEEK_OF_MONTH
•ALIGNED_WEEK_OF_YEAR
•MONTH_OF_YEAR
•PROLEPTIC_MONTH
•YEAR_OF_ERA
•YEAR
•ERA
The field INSTANT_SECONDS is - of course - not supported because a LocalDateTime cannot refer to any absolute (global) timestamp. But what is helpful is the field EPOCH_DAY which counts the elapsed days since 1970-01-01. Similar thoughts are valid for the type LocalDate (with even less supported fields).
If you intend to get the non-existing millis-since-unix-epoch field you also need the timezone for converting from a local to a global type. This conversion can be done much simpler, see other SO-posts.
Coming back to your question and the numbers in your code:
The result 1605 is correct
=> (2014 - 1970) * 365 + 11 (leap days) + 31 (in january 2014) + 3 (in february 2014)
The result 71461 is also correct => 19 * 3600 + 51 * 60 + 1
16105L * 86400 + 71461 = 1391543461 seconds since 1970-01-01T00:00:00 (attention, no timezone)
Then you can subtract the timezone offset (watch out for possible multiplication by 1000 if in milliseconds).
UPDATE after given timezone info:
local time = 1391543461 secs
offset = 3600 secs (Europe/Oslo, winter time in february)
utc = 1391543461 - 3600 = 1391539861
As JSR-310-code with two equivalent approaches:
long secondsSinceUnixEpoch1 =
LocalDateTime.of(2014, 2, 4, 19, 51, 1).atZone(ZoneId.of("Europe/Oslo")).toEpochSecond();
long secondsSinceUnixEpoch2 =
LocalDate
.of(2014, 2, 4)
.atTime(19, 51, 1)
.atZone(ZoneId.of("Europe/Oslo"))
.toEpochSecond();
This is one way without using time a zone:
LocalDateTime now = LocalDateTime.now();
long epoch = (now.getLong(ChronoField.EPOCH_DAY) * 86400000) + now.getLong(ChronoField.MILLI_OF_DAY);
Convert from human readable date to epoch:
long epoch = new java.text.SimpleDateFormat("MM/dd/yyyyHH:mm:ss").parse("01/01/1970 01:00:00").getTime() / 1000;
Convert from epoch to human readable date:
String date = new java.text.SimpleDateFormat("MM/dd/yyyyHH:mm:ss").format(new java.util.Date (epoch*1000));
For other language converter:
https://www.epochconverter.com
Extracting two good answers found in comments on this post,
If you only care about UTC (Coordinated Universal Time), there is
final long epoch = localDateTime.toEpochSecond(ZoneOffset.UTC);
or where you already know the timezone
final long epoch = localDateTime.toEpochSecond(ZoneId.systemDefault());
More information from https://docs.oracle.com/javase/8/docs/api/java/time/chrono/ChronoLocalDateTime.html#toEpochSecond-java.time.ZoneOffset-
toEpochSecond
default long toEpochSecond(ZoneOffset offset)
Converts this date-time to the number of seconds from the epoch of
1970-01-01T00:00:00Z.
This combines this local date-time and the specified offset to
calculate the epoch-second value, which is the number of elapsed
seconds from 1970-01-01T00:00:00Z. Instants on the time-line after the
epoch are positive, earlier are negative.
This default implementation calculates from the epoch-day of the date
and the second-of-day of the time.
Parameters:
offset - the offset to use for the conversion, not null
Returns:
the number of seconds from the epoch of 1970-01-01T00:00:00Z

Time and Calendar

How do I add/subtract two time objects. I have two time objects (arrival and departure) in format of "yyyy/MMM/dd HH:mm:ss". I need to print the difference between departure and arrival time. I am generating time ad below:
public String getTime() {
Calendar currentDate = Calendar.getInstance();
SimpleDateFormat formatter = new SimpleDateFormat("yyyy/MMM/dd HH:mm:ss");
return formatter.format(currentDate.getTime());
}
Can I get time in mills and than format it when I needed to print ?
Take a look at Joda Time library.
You can easily subtract and add DateTime and find out interval easily :
// interval from start to end
DateTime start = new DateTime(2004, 12, 25, 0, 0, 0, 0);
DateTime end = new DateTime(2005, 1, 1, 0, 0, 0, 0);
Interval interval = new Interval(start, end);
something like this.....
public long getTimeDiff() throws Exception {
String arrival = "2011/Nov/10 13:15:24";
String departure = "2011/Jan/10 13:15:24";
SimpleDateFormat formatter = new SimpleDateFormat("yyyy/MMM/dd HH:mm:ss");
java.util.Date date1 = formatter.parse(arrival);
java.util.Date date2 = formatter.parse(departure);
return date2.getTime() - date1.getTime();
}
Convert them to date and then to long and subtract, that would give the time difference in milli seconds,
Date d1 = DateFormat.parse(time1);
Date d2 = DateFormat.parse(time2);
long diffInMilliSeconds = d1.getTime()-d2.getTime();
You can get time in milliseconds for both calendars using getTime method. When you can convert the result of subtraction to measure units that you need. If you're going to work with time/duration seriously when take a look at Joda library
Upd. You should call getTime twice. First object being returned is Date, when you call getTime on Date you get long value.
I would convert the two time/Date objects in milliseconds. Then i would subtract them (we are dealing with longs).
Then i would create a Date object from the resulting long value. After that you can construct a Calendar with Calendar.setDate(Date).
Regards!
Yes, start with your Dates and use getTime() to convert to milliseconds (or getTimeInMillis() for your Calendars). That give you long values you can subtract. That's the easy part.
Then you can convert these milliseconds into a readable format yourself. But it probably makes sense to use a packaged library to do it.
Some folks like the Joda library for these types of date calculations. I find Commons Lang is fantastic. It provides DateUtils which is useful if you find you want to perform calculations like rounding or truncating your dates to the nearest minute or hour etc. The part that will be most useful to you is the DurationFormatUtils class which gives you functions like formatDurationHMS to format into nice Hour:Minute:Second display and formatDurationWords to get text (fancy!) or other similar functions to easily format your milliseconds into a nicely human-readable format.

Categories