We Keep Coding

How to converter a TIMESTAMP_MICROS_STRINGIFIER Parquet using Java [duplicate]

There has been changes in Java Date & Time API Since Java 9.
LocalDateTime now has microseconds precision.
Java 9 has a fresh implementation of java.time.Clock capable of capturing the current moment in resolution finer than milliseconds (three digits of decimal fraction).
We get the time in microseconds from our backend service.
System.currentTimeMillis > 1565245051795 > 2019-08-08T06:17:31.795
Service.getTime > 1565245051795306 > 2019-08-08T06:17:31.795306
In order to construct a LocalDateTime to be used in our application, we do
long timeMicros = service.getTime();
long timeMillis = timeMicros / 1000;
LocalDateTime ldt = Instant.ofEpochMilli(timeMillis).atZone(ZoneId.systemDefault()).toLocalDateTime();
For querying the service we need time microseconds again, then we do
long timeMillis = dateTime.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli();
long timeMicros = timeMillis * 1000;
The problem is we do not get back the time microseconds precision.
Is it possible to create an Instant with microsecond precision?
We are now using Java 11. I noticed this change when one of our JUnit tests failed because of the increased microsecond precision.
For the JUnit test I found a workaround:
private static final LocalDateTime START = LocalDateTime.now().truncatedTo(ChronoUnit.MILLIS);
I'm not sure if this is a workaround or an actual solution, but adding the last three microseconds digits from the timestamp seems to work.
long micros = 306L; //TODO get the last three digits from the timeMicros
ldt.plus(micros, ChronoUnit.MICROS));

long timeMicros = 1_565_245_051_795_306L;
Instant i = Instant.EPOCH.plus(timeMicros, ChronoUnit.MICROS);
System.out.println(i);
Output is:
2019-08-08T06:17:31.795306Z
Edit: Rather than dividing and multiplying to convert microseconds to milliseconds and/or seconds I preferred to use the built-in support for microseconds. Also when explicitly adding them to the epoch feels a little hand-held.
You already know how to convert Instant to LocalDateTime, you’ve shown it in the question, so I am not repeating that.
Edit:
Do you have a solution to get the timeMicros back from the Instant?
There are a couple of options. This way the calculation is not so complicated, so I might do:
long microsBack = TimeUnit.SECONDS.toMicros(i.getEpochSecond())
+ TimeUnit.NANOSECONDS.toMicros(i.getNano());
System.out.println(microsBack);
1565245051795306
To be more in style with the first conversion you may prefer the slightly shorter:
long microsBack = ChronoUnit.MICROS.between(Instant.EPOCH, i);
Edit: Possibly nit-picking, but also to avoid anyone misunderstanding: LocalDateTime has had nanosecond precision always. Only the now method had millisecond precision on Java 8. I read somewhere that from Java 9 the precision varies with the platform, but you are right, microsecond precision seems typical.

Converting mm:ss to only seconds (java)

Part of my current project is to convert mm:ss to seconds...but the user has the option to enter x:xx or xx:xx.
For example, if someone wanted to enter one minute and thirty seconds into the program, they have the option to write it as either "01:30" or "1:30". And the output of both would be 90 seconds.
This is my current code.
System.out.print("Time (mm:ss): ")
String time = scan.nextLine();
int min = Integer.parseInt(time.substring(0, time.indexOf(':'))) * 60;
int sec = Integer.parseInt(time.substring(3, time.length()));
int duration = (min + sec);
System.out.println("Seconds: " + duration)
It works whenever I enter xx:xx, but fails when I enter x:xx.
I am not sure how to only read the characters after ":" . If I start the substring at ":" (I have it at 3 now), it can't convert to int because it reads the ":".
I have looked all over Google and my textbook, but have not found anything. I assume I am just using the wrong technique. The code needs to stay within the parameters of basic beginner String methods. Thank you!

This answer probably does not stay within the parameters of basic beginner String methods as requested. I think it will be useful for other readers of your question who don’t have the same limitation.
java.time.Duration
The Duration class is the class to use for an amount of time like 1 minute 30 seconds. Unfortunately, the Duration class can only parse strings in ISO 8601 format (link below), but the string conversion isn’t hard with a regular expression. And fortunately in ISO 8601 format leading zeroes don’t matter. The Duration class is part of java.time, the modern Java date and time API.
ISO 8601 format for a duration is like PT01M30S. Think of it as a period of time of 01 minute 30 seconds. If the format feels unusual at first, it is straightforward. So let’s convert to it. The following method accepts your user’s format, converts and returns a Duration.
public static Duration parseDuration(String durStr) {
String isoString = durStr.replaceFirst("^(\\d{1,2}):(\\d{2})$", "PT$1M$2S");
return Duration.parse(isoString);
}
Duration has a toSeconds method for converting to seconds. So let’s try the whole thing out:
System.out.println(parseDuration("01:30").toSeconds());
System.out.println(parseDuration("1:30").toSeconds());
Output is the expected:
90
90
Consider whether you need to convert to seconds at all, though. Keeping the Duration objects as they are will probably make your code more self-explanatory.
Links
Oracle tutorial: Date Time explaining how to use java.time.
Wikipedia article: ISO 8601

int sec = Integer.parseInt(time.substring(time.indexOf(':') + 1), time.length()));
Thanks guys!

Here I have simplified your code.
String time = "1:30";
String[] timeUnit = time.split(":");
int totalSeconds = 60 * Integer.parseInt(timeUnit[0]) + Integer.parseInt(timeUnit[1]);
System.out.println("Seconds "+totalSeconds);

What does 'PT' prefix stand for in Duration?

I am trying to use the Duration class instead of long.
It has superior literal syntax. I like its flexibility, though it looks weird.
"PT10S" means 10 seconds, what is the problem to accept "10 seconds"?!
Okay never mind.
I am just curious why PT prefix has been chosen (not "DU" e.g.) and why any prefix is better here rather than nothing?

As can be found on the page Jesper linked to (ISO-8601 - Data elements and interchange formats – Information interchange – Representation of dates and times)
P is the duration designator (for period) placed at the start of the duration representation.
Y is the year designator that follows the value for the number of years.
M is the month designator that follows the value for the number of months.
W is the week designator that follows the value for the number of weeks.
D is the day designator that follows the value for the number of days.
T is the time designator that precedes the time components of the representation.
So P means 'Period' and because there are no date-components it only has a 'Time'.
You could interpret this as 'Period of Time'
The 'why' this was chosen, you have to ask the ISO members that wrote the standard, but my guess is that it is easier to parse. (short and unambigious)
The details for the time component are:
H is the hour designator that follows the value for the number of hours.
M is the minute designator that follows the value for the number of minutes.
S is the second designator that follows the value for the number of seconds.
The value of PT20S then parses to:
Period
Time
20
Seconds
So, a duration of 20 seconds.
More examples can be found in the javadoc: https://docs.oracle.com/javase/8/docs/api/java/time/Duration.html#parse-java.lang.CharSequence-

Java has taken a subset of the ISO 8601 standard format for a duration. So the “why” is why the standard was written the way it is, and it’s a guessing game. My go is:
P for period was chosen so that you can distinguish a duration from a date and/or time. Especially since a period may also be written in the same format as a local date-time, for example P0003-06-04T12:30:05 for 3 years 6 months 4 days 12 hours 30 minutes 5 seconds, the P can be necessary to distinguish. The P also gives a little but quick and convenient bit of validation in case you happen to pass a completely different string in a place where a duration was expected. And yes, PT10S looks weird, but once you get accustomed to it, you recognize it immediately as a duration, which can be practical.
T for time between the date part and the time part was chosen for two reasons:
For consistency with date-time strings that have T in the same place, for example 2018-07-04T15:00 for July 4, 2018 at 15:00 hours.
To disambiguate the otherwise ambiguous M for either months or minutes: P3M unambiguously means 3 months while PT3M means 3 minutes.

Actually if go on Duration API developed in Java since 1.8, they have gone with standard ISO 8601:
with java doc as below :
/**
* Applies an ISO 8601 Duration to a {#link ZonedDateTime}.
*
* <p>Since the JDK defined different types for the different parts of a Duration
* specification, this utility method is needed when a full Duration is to be applied to a
* {#link ZonedDateTime}. See {#link Period} and {#link Duration}.
*
* <p>All date-based parts of a Duration specification (Year, Month, Day or Week) are parsed
* using {#link Period#parse(CharSequence)} and added to the time. The remaining parts (Hour,
* Minute, Second) are parsed using {#link Duration#parse(CharSequence)} and added to the time.
*
* #param time A zoned date time to apply the offset to
* #param offset The offset in ISO 8601 Duration format
* #return A zoned date time with the offset applied
*/
public static ZonedDateTime addOffset(ZonedDateTime time, String offset) { }
Obtains a Duration from a text string of pattern: PnDTnHnMn.nS, where
nD = number of days,
nH = number of hours,
nM = number of minutes,
n.nS = number of seconds, the decimal point may be either a dot or a comma.
T = must be used before the part consisting of nH, nM, n.nS
Example of implementation with java as
import java.time.Duration;
public class ParseExample {
public static void main(String... args) {
parse("PT20S");//T must be at the beginning to time part
parse("P2D");//2 day
parse("-P2D");//minus 2 days
parse("P-2DT-20S");//S for seconds
parse("PT20H");//H for hours
parse("PT220H");
parse("PT20M");//M for minutes
parse("PT20.3S");//second can be in fraction like 20.3
parse("P4DT12H20M20.3S");
parse("P-4DT-12H-20M-20.3S");
parse("-P4DT12H20M20.3S");
}
private static void parse(String pattern) {
Duration d = Duration.parse(pattern);
System.out.println("Pattern: %s => %s%n", pattern, d);
}
}

Java Unparsable date

I have a string with the format: String dateString = "2014-03-17T20:05:49.2300963Z"
Trying this:
SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'kk:mm:ss.SSSX");
Date date = df.parse(dateString);
Results in an Unparsable date exceptioon.
The docs: http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html indicate that X is used with ISO 8601 when a single letter is used for the TimeZone.
EDIT
Re-reading the docs, I've switched up the SimpleDateFormat a little:
SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
dateString = dateString.replace("Z", "");
I take out the Z because I know the timezone, use H instead of k and add a couple more S for giggles.
Now the time is parsing, but incorrectly. Date is accurate, Time seems to be random.
EDIT 2
The problem is that java only allows millisecond accuracy, so 2300963 is being interpreted as 2300 seconds and 963 milliseconds. I'll need to format my string a little differently to get this to work.
EDIT 3
Turns out you can't have a fractional part of a second in Java. It has to be truncated to milliseconds. I ended up using a type made available to me by my database, but the general solution is to truncate the fractional part of the second to millisecond. I'll post example code of how to do that as an answer.

You need to provide as many S as you have in your date String. In this case, 7
SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'kk:mm:ss.SSSSSSSX");
This is required because, otherwise, the DateFormat doesn't know where the milliseconds end and where the time zone starts.
Note also, that
2300963
as a millisecond value means 2300 seconds and 963 milliseconds. Why do you have it that way? Why aren't those seconds part of the value in their corresponding position? When the DateFormat parses it, they will be added.

This works: SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'kk:mm:ss.SSSSSSS'Z'");
It's safer to specify exactly how much precision you expect (say, for milliseconds in this case). It's odd to have 7 digits but if all your dates look like this, put 7 S.
The X will parse a timezone of the sort -0800. So your string would have to look like 2014-03-17T20:05:49.2300963-0800 (or something similar). Treat the Z as a literal, like T.
EDIT: Relevant to your partial seconds issue.

How to truncate the fractional seconds to milliseconds (because Java can't handle fractional seconds):
public String truncate(String dateString){
int numberOfDigits = dateString.substring(dateString.indexOf("."), dateString.length() - 1).length();
String justMilliSecondsDate = null;
if (numberOfDigits == 3) {
justMicrosDate = dateString;
}
else if (numberOfDigits > 3) {
justMilliSecondsDate = dateString.substring(0, dateString.length() - numberOfDigits + 3);
}
else {
justMilliSecondsDate = dateString;
for (int i = numberOfDigits ; i < 3 ; i++) justMilliSecondsDate += "0";
}
return justMilliSecondsDate;
}

tl;dr
Instant.parse( "2014-03-17T20:05:49.2300963Z" )
java.time
You are using the troublesome old date-time classes, now legacy, supplanted by the java.time classes.
Resolution
MillisecondsThe old date-time classes were limited to milliseconds resolution, for up to three digits of decimal fraction.
MicrosecondsYour input has six digits of fractional second for microseconds.
NanosecondsThe java.time classes have a resolution of nanoseconds for up to nine digits of fractional seconds. More than enough for your microseconds.
ISO 8601
The ISO 8601 standard defines string formats to represent date-time values. You input complies with ISO 8601.
The java.time classes use ISO 8601 formats by default when parsing and generating strings. So no need to specify a formatting pattern.
Instant instant =
Instant.parse( "2014-03-17T20:05:49.2300963Z" );

Formatting time intervals with localization in Java

We have intervals (elapsed time between two oracle timestamps) stored in our database as seconds and we format them at front end with Java.
What we would to achieve on the reports is a format of the form "HH:MM" or "HH:MM:SS", with the time separator ":" localized as it happens with dates and time information, i.e '.' for Italian and ':' for English.
Unfortunately the date-related formatting classes, like SimpleDateFormat, do not work** because we can expect durations above the 24 hours.
We don't want to employ 3rdy party library as well.
Do you know how we can solve this problem?
TIA

If you want hours of more than 24 you can print this separately.
int hour = time / 3600000;
String duration = time + new SimpleDateFormat(":mm:ss").format(new Date(time));
To support other locales you could do this more complicated example.
int hour = time / 3600000;
String duration = hour
+ DateFormat.getDateInstance(DateFormat.MEDIUM, locale)
.format(new Date(time % 3600000).substring(1);
What this will do is use the locale specific format for the last digit of the hour + mins + secs and prepend the additional digits of the hours. Note: this will not work for negative times.

You can construct a date object, say "2000-01-01", then add your seconds to it. Then you can use this to extract the localized version of the time:
DateFormat format = DateFormat.getTimeInstance(DateFormat.MEDIUM, locale);
String formatted_time_part = format.format(some_date_object);
Finally you still need to add elapsed days! I'm affraid that localization of long intervals (days, months, years, centuries) has no corresponding API in Java but I might be wrong. So you will have to figure out that for yourself.