I'm unsure how YAML "timestamps" are supposed to be represented. From the YAML 1.2 specification, it would seem that YAML just assumes that if you have a value in string format that looks like an ISO 8601 date, then it is parsed as a date unless you say differently. Here are a couple of examples from the spec:
date: 2002-12-14
not-date: !!str 2002-04-28
The Timestamp Language-Independent Type for YAML™ Version 1.1 working draft (from 2005, currently 15 years ago!) seems to indicate that a special tag in the form tag:yaml.org,2002:timestamp should be used. It also indicates a shorthand of !!timestamp.
In Java using SnakeYAML Engine v2.0 (org.snakeyaml:snakeyaml-engine:2.0) I tried parsing the 2002-12-14 form, and got a string as the parsed value, not any sort of date object. I see that the SnakeYAML Engine repository has an example using the !!timestamp approach (e.g. !!timestamp 2020-03-24T12:34:00.333), but this is a recent change and I'm sure if this support has been released yet.
I tried both the form fooBar: !!timestamp 2020-03-24 and also fooBar: !!timestamp 2020-03-24T12:34:00.333, but SnakeYAML Engine reported:
could not determine a constructor for the tag tag:yaml.org,2002:timestamp
So what is the official way to represent a date (specifically a local date with YYYY-MM-DD) in YAML, and is the correct approach reflected in the latest YAML specification? Does SnakeYAML Engine support the official YAML date approach?
From the YAML 1.2 specification, it would seem that YAML just assumes that if you have a value in string format that looks like an ISO 8601 date, then it is parsed as a date unless you say differently.
No. The YAML spec gives three schemas (Failsafe, JSON and Core) that should be supported; none of them includes a timestamp type. However, a scalar looking like a timestamp can be parsed as such if a schema is used that supports it. The spec only tells you that if you want to ensure that a scalar is not loaded as timestamp, prefix it with !!str.
So what is the official way to represent a date (specifically a local date with YYYY-MM-DD) in YAML.
The !!timestamp definition you linked is closest to what an official way would be. However, the tag repository containing it is not part of the spec and implementations are not required to support it. Furthermore, it is defined for outdated YAML 1.1.
This means that SnakeYAML isn't required to support timestamps at all. You can see in the example you give that timestamp support is not included; the example implements loading timestamps itself. You can modify that code to use with the normal public interface:
class TimestampConstructor extends Constructor {
public static final Pattern TIMESTAMP = Pattern
.compile("^(?:[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]|[0-9][0-9][0-9][0-9]-[0-9][0-9]?-[0-9][0-9]?(?:[Tt]|[ \t]+)[0-9][0-9]?:[0-9][0-9]:[0-9][0-9](?:\\.[0-9]*)?(?:[ \t]*(?:Z|[-+][0-9][0-9]?(?::[0-9][0-9])?))?)$");
public static final Tag TAG = new Tag(Tag.PREFIX + "timestamp");
public TimestampConstructor() {
this.yamlConstructors.put(TAG, new ConstructTimestamp());
}
private class ConstructTimestamp extends AbstractConstruct {
public Object construct(Node node) {
String val = (String) constructScalar(node);
return LocalDateTime.parse(val);
}
}
}
Then, use it like this:
Yaml yaml = new Yaml(new TimestampConstructor());
yaml.addImplicitResolver(TimestampConstructor.TAG,
TimestampConstructor.PATTERN, "0123456789");
Related
Some of our POJOs contain fields from java.time API (LocalDate, LocalDateTime). When our pipelines are processing them we can see following information in the logs:
org.apache.flink.api.java.typeutils.TypeExtractor - Class class java.time.LocalDate cannot be used as a POJO type because not all fields are valid POJO fields, and must be processed as GenericType. Please read the Flink documentation on "Data Types & Serialization" for details of the effect on performance.
As I understand, LocalDate can't be classified as POJO, so instead of using POJO serializer flink falls back to Kryo, which is less efficient. However, since 1.9.0 version flink has dedicated serializers for java.time classes (for example LocalDateSerializer), so I would expect that these serializers would do the job here allowing POJO serializer to be used for our classes. Isn't that the case? If yes, is there any performance hit? If no, what is the optimal solution for such case?
In the project we use Flink 1.11 with Java 1.8.
Due to backwards compatibility, even if a new serializer is being introduced in Flink, it can't be used automatically. However, you can tell Flink to use that for your POJO like this (if you are starting without a previous savepoint using Kryo there):
#TypeInfo(MyClassTypeInfoFactory.class)
public class MyClass {
public int id;
public LocalDate date;
// ...
}
public class MyClassTypeInfoFactory extends TypeInfoFactory<MyClass> {
#Override
public TypeInformation<MyClass> createTypeInfo(
Type t, Map<String, TypeInformation<?>> genericParameters) {
return Types.POJO(MyClass.class, new HashMap<String, TypeInformation<?>>() { {
put("id", Types.INT);
put("date", Types.LOCAL_DATE);
// ...
} } );
}
}
You'll have to provide types for all your POJO's fields as shown, but there are a lot of helpers in the Types class that you can use. Also, by using a TypeInfoFactory like this, you don't have to worry about all the places Flink uses this type - it will always derive the given type information.
If you need to convert an old savepoint to use the new serializer, you may also want to look at Flink's State Processor API.
I am having some issue with date logic which I've isolated to Jackson, the JSON serializer.
In the database and in a debug point in the application, dates are correct and everything is written using default timezone. However, in serialization 4 hours are being added. I found this could be remedied by telling Jackson specifically to use EST (it was defaulting to UTC). As below:
#JsonFormat(shape= JsonFormat.Shape.STRING, pattern="yyyy-MM-dd HH:mm:ss.SSSZ", timezone="America/New_York")
private Date startDate;
However, the problem is that only local is using EST and the server will be using UTC. I need Jackson to use system defaults.
Luckily, I found this similar question which is backed up by the documentation. New solution:
#JsonFormat(shape= JsonFormat.Shape.STRING, pattern="yyyy-MM-dd HH:mm:ss.SSSZ", timezone=JsonFormat.DEFAULT_TIMEZONE)
private Date startDate;
However, it doesn't work! I tried also timezone='DEFAULT_TIMEZONE' and a variety of other things but in all cases the api output still seems to be 4 hours ahead of the number in the database.
Other things I have tried:
logging out JsonFormat.DEFAULT_TIMEZONE returns ##default.
logging TimeZone.getDefault().getDisplayName() returns Eastern Standard Time.
Jackson version is 2.9.
Any suggestions?
Solved my own question. Here is what I found:
JsonFormat.DEFAULT_TIMEZONE is NOT the system default, as the documentation and SO answer suggest, but actually defaults to UTC.
org.springframework.http.converter.json.Jackson2ObjectMapperBuilder
/**
* Override the default {#link TimeZone} to use for formatting.
* Default value used is UTC (NOT local timezone).
* #since 4.1.5
*/
public Jackson2ObjectMapperBuilder timeZone(TimeZone timeZone) {
com.fasterxml.jackson.annotation.JsonFormat
/**
* Value that indicates that default {#link java.util.TimeZone}
* (from deserialization or serialization context) should be used:
* annotation does not define value to use.
*<p>
* NOTE: default here does NOT mean JVM defaults but Jackson databindings
* default, usually UTC, but may be changed on <code>ObjectMapper</code>.
*/
public final static String DEFAULT_TIMEZONE = "##default";
Solution:
#Autowired
com.fasterxml.jackson.databind.ObjectMapper objectMapper;
and objectMapper.setTimeZone(TimeZone.getDefault()) in a config class, like so:
package path.to.config;
import ...
#Configuration
public class JacksonConfiguration {
#Autowired
public JacksonConfiguration(ObjectMapper objectMapper){
objectMapper.setTimeZone(TimeZone.getDefault());
}
}
This should set the Jackson ObjectMapper to use system default instead of Jackson default (UTC).
I have an Angular WebApp who retrives data from a backend Java server via http get.
Example:
public getStateForUser(): Observable<UserRegistrationState> {
return this.http.get<any>(Environment.getServerAddress() + '/' +
UserRegistrationStateService.SERVICE_URL + '/getStateForUser', {
params: {
sessionIdentityStr: this.authGuard.getSession().sessionIdentityStr
}
});
}
export class UserRegistrationState {
registerDate: Date;
emailVerified: boolean;
unlocked: boolean;
}
This works fine for all fields. Except the Dates. The Dates are not parsed correctly, so I want to add a custom parser where I can tell him how he gets the date value from the json date string.
Of course now I can add a custom Parser for the whole UserRegistrationState class, but I have Dates on multiple classes and I do not want to create a Parser for every class, because the other properties(except the Dates) are parsed fine.
So my question: Is there a way to add a custom parser for all Date fields?
P.S. This is an example JSON date string, automatically parsed from a Java LocalDateTime:
"registerDate":{"dayOfMonth":17,"dayOfWeek":"WEDNESDAY","dayOfYear":290,"month":"OCTOBER","monthValue":10,"year":2018,"hour":21,"minute":21,"nano":370000000,"second":16,"chronology":{"id":"ISO","calendarType":"iso8601"}}
This unfriendly date format is probably caused by backend. I never had any problems with dates from any API in Angular.
You can create getters/setters to encapsulate custom parsing logic for some field in your class.
In my opinion would be better to fix backend side and move from classes to interfaces in Angular side. Interfaces are better for plain DTO models in TypeScript.
I have an XML file in format described below. I'm currently using JAXB to unmarshall the XML elements (event) to Java objects (Event). All properties work fine, except one...
As you can see in the XML, the date element stores only the date and the time stores only the time of the event. I would like to combine these two XML elements into one LocalDataTime field named time, with appropriate getters and setters.
XML:
...
<event>
...
<date>2014-02-19</date>
<time>2000-01-01T14:17:00Z</time>
...
</event>
...
Desired Java Object:
public class Event {
...
// Returns a combination of the date and time stored in the XML
public LocalDateTime getDateTime() {
return dateTime;
}
...
}
My first thought was to use a XMLAdapter but it seems that this only allows me to map 1 XML element to 1 Java object.
After that I tried to implement this by overriding the setters setTime and setDate. These setters would each just change the time or date of the stored time.. But I wasn't able to get it working and it also seemed a quite ugly way to do this.
I already read the following JAXB convert multiple attributes to one type while unmarshalling. But for this project, I would rather not use an external library like MOXy.
Does anyone know how to do this using solely JAXB in a clean way?
You could define in your Event object lifecycle methods:
void afterUnmarshal(Unmarshaller unm, Object parent) {
localDateTime = ....
};
boolean beforeMarshal(Marshaller mar) {
date = localDateTime.toDate();
....
};
to construct the LocalDateTime property after unmarshalling (using the date and time values), and prepare date/time fields before marshalling using the current LocaLDateTime value.
You would still need the time/date fields to match the xml and the localDateTime field must be XmlTransient. So it is not so different from the set/getter approach but probably more "clean".
I just switched to FlexJson and I am already having a problem.
The documentation at http://flexjson.sourceforge.net/, chapter Controlling JSON naming with #JSON states :
When an object is serialized the names of the properties on JSON
objects are determined from the name of the Java property. This means
the names of the getter methods or the field names are used. However,
sometimes the JSON output doesn't match the names used by the Java
object. You can control this using the #JSON annotation. For example:
public class Species {
#JSON(jsonName = "genus")
private String type;
private String name;
#JSON(jsonName="species")
public String getName() {
return name;
}
}
Except it doesn't work. And then they say :
Defining a #JSON.jsonName is used in both serialization and
deserialization.
Now when I have a look into the javadocs at http://flexjson.sourceforge.net/javadoc/index.html, I can see there are 4 optional elements belonging to the #JSON annotation, those are
include name objectFactory transformer
None of it is jsonName, like in the example.
So how do I get this annotation to work, so I can have different java and json names?
How can I define this annotation element or make use of the predefined name
To clarify, I can annotate #JSON and autocomplete recommends #JSON(include), but then include cannot be resolved...
I am using FlexJson 2.1, and I imported flexjson.JSON;
Btw I am aware of this answer https://stackoverflow.com/a/8879616/2001247, but it's not what I want. I want to use annotations.
You need to use FlexJson 3.2
The problem is, latest jar accessible via developer site have 2.1 version number.
But java doc corresponds to version 3.2.
You can find FlexJson 3.2 jar in maven repository.