Related
How can I configure JPA/Hibernate to store a date/time in the database as UTC (GMT) time zone? Consider this annotated JPA entity:
public class Event {
#Id
public int id;
#Temporal(TemporalType.TIMESTAMP)
public java.util.Date date;
}
If the date is 2008-Feb-03 9:30am Pacific Standard Time (PST), then I want the UTC time of 2008-Feb-03 5:30pm stored in the database. Likewise, when the date is retrieved from the database, I want it interpreted as UTC. So in this case 530pm is 530pm UTC. When it's displayed it will be formatted as 9:30am PST.
Since Hibernate 5.2, you can now force the UTC time zone by adding the following configuration property into the properties.xml JPA configuration file:
<property name="hibernate.jdbc.time_zone" value="UTC"/>
If you're using Spring Boot, then add this property to your application.properties file:
spring.jpa.properties.hibernate.jdbc.time_zone=UTC
To the best of my knowledge, you need to put your entire Java app in UTC timezone (so that Hibernate will store dates in UTC), and you'll need to convert to whatever timezone desired when you display stuff (at least we do it this way).
At startup, we do:
TimeZone.setDefault(TimeZone.getTimeZone("Etc/UTC"));
And set the desired timezone to the DateFormat:
fmt.setTimeZone(TimeZone.getTimeZone("Europe/Budapest"))
Hibernate is ignorant of time zone stuff in Dates (because there isn't any), but it's actually the JDBC layer that's causing problems. ResultSet.getTimestamp and PreparedStatement.setTimestamp both say in their docs that they transform dates to/from the current JVM timezone by default when reading and writing from/to the database.
I came up with a solution to this in Hibernate 3.5 by subclassing org.hibernate.type.TimestampType that forces these JDBC methods to use UTC instead of the local time zone:
public class UtcTimestampType extends TimestampType {
private static final long serialVersionUID = 8088663383676984635L;
private static final TimeZone UTC = TimeZone.getTimeZone("UTC");
#Override
public Object get(ResultSet rs, String name) throws SQLException {
return rs.getTimestamp(name, Calendar.getInstance(UTC));
}
#Override
public void set(PreparedStatement st, Object value, int index) throws SQLException {
Timestamp ts;
if(value instanceof Timestamp) {
ts = (Timestamp) value;
} else {
ts = new Timestamp(((java.util.Date) value).getTime());
}
st.setTimestamp(index, ts, Calendar.getInstance(UTC));
}
}
The same thing should be done to fix TimeType and DateType if you use those types. The downside is you'll have to manually specify that these types are to be used instead of the defaults on every Date field in your POJOs (and also breaks pure JPA compatibility), unless someone knows of a more general override method.
UPDATE: Hibernate 3.6 has changed the types API. In 3.6, I wrote a class UtcTimestampTypeDescriptor to implement this.
public class UtcTimestampTypeDescriptor extends TimestampTypeDescriptor {
public static final UtcTimestampTypeDescriptor INSTANCE = new UtcTimestampTypeDescriptor();
private static final TimeZone UTC = TimeZone.getTimeZone("UTC");
public <X> ValueBinder<X> getBinder(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicBinder<X>( javaTypeDescriptor, this ) {
#Override
protected void doBind(PreparedStatement st, X value, int index, WrapperOptions options) throws SQLException {
st.setTimestamp( index, javaTypeDescriptor.unwrap( value, Timestamp.class, options ), Calendar.getInstance(UTC) );
}
};
}
public <X> ValueExtractor<X> getExtractor(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicExtractor<X>( javaTypeDescriptor, this ) {
#Override
protected X doExtract(ResultSet rs, String name, WrapperOptions options) throws SQLException {
return javaTypeDescriptor.wrap( rs.getTimestamp( name, Calendar.getInstance(UTC) ), options );
}
};
}
}
Now when the app starts, if you set TimestampTypeDescriptor.INSTANCE to an instance of UtcTimestampTypeDescriptor, all timestamps will be stored and treated as being in UTC without having to change the annotations on POJOs. [I haven't tested this yet]
With Spring Boot JPA, use the below code in your application.properties file and obviously you can modify timezone to your choice
spring.jpa.properties.hibernate.jdbc.time_zone = UTC
Then in your Entity class file,
#Column
private LocalDateTime created;
Adding an answer that's completely based on and indebted to divestoclimb with a hint from Shaun Stone. Just wanted to spell it out in detail since it's a common problem and the solution is a bit confusing.
This is using Hibernate 4.1.4.Final, though I suspect anything after 3.6 will work.
First, create divestoclimb's UtcTimestampTypeDescriptor
public class UtcTimestampTypeDescriptor extends TimestampTypeDescriptor {
public static final UtcTimestampTypeDescriptor INSTANCE = new UtcTimestampTypeDescriptor();
private static final TimeZone UTC = TimeZone.getTimeZone("UTC");
public <X> ValueBinder<X> getBinder(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicBinder<X>( javaTypeDescriptor, this ) {
#Override
protected void doBind(PreparedStatement st, X value, int index, WrapperOptions options) throws SQLException {
st.setTimestamp( index, javaTypeDescriptor.unwrap( value, Timestamp.class, options ), Calendar.getInstance(UTC) );
}
};
}
public <X> ValueExtractor<X> getExtractor(final JavaTypeDescriptor<X> javaTypeDescriptor) {
return new BasicExtractor<X>( javaTypeDescriptor, this ) {
#Override
protected X doExtract(ResultSet rs, String name, WrapperOptions options) throws SQLException {
return javaTypeDescriptor.wrap( rs.getTimestamp( name, Calendar.getInstance(UTC) ), options );
}
};
}
}
Then create UtcTimestampType, which uses UtcTimestampTypeDescriptor instead of TimestampTypeDescriptor as the SqlTypeDescriptor in the super constructor call but otherwise delegates everything to TimestampType:
public class UtcTimestampType
extends AbstractSingleColumnStandardBasicType<Date>
implements VersionType<Date>, LiteralType<Date> {
public static final UtcTimestampType INSTANCE = new UtcTimestampType();
public UtcTimestampType() {
super( UtcTimestampTypeDescriptor.INSTANCE, JdbcTimestampTypeDescriptor.INSTANCE );
}
public String getName() {
return TimestampType.INSTANCE.getName();
}
#Override
public String[] getRegistrationKeys() {
return TimestampType.INSTANCE.getRegistrationKeys();
}
public Date next(Date current, SessionImplementor session) {
return TimestampType.INSTANCE.next(current, session);
}
public Date seed(SessionImplementor session) {
return TimestampType.INSTANCE.seed(session);
}
public Comparator<Date> getComparator() {
return TimestampType.INSTANCE.getComparator();
}
public String objectToSQLString(Date value, Dialect dialect) throws Exception {
return TimestampType.INSTANCE.objectToSQLString(value, dialect);
}
public Date fromStringValue(String xml) throws HibernateException {
return TimestampType.INSTANCE.fromStringValue(xml);
}
}
Finally, when you initialize your Hibernate configuration, register UtcTimestampType as a type override:
configuration.registerTypeOverride(new UtcTimestampType());
Now timestamps shouldn't be concerned with the JVM's time zone on their way to and from the database. HTH.
You would think this common problem would be taken care of by Hibernate. But its not! There are a few "hacks" to get it right.
The one I use is to store the Date as a Long in the database. So I am always working with milliseconds after 1/1/70. I then have getters and setters on my Class that return/accept only Dates. So the API remains the same. The down side is that I have longs in the database. SO with SQL I can pretty much only do <,>,= comparisons -- not fancy date operators.
Another approach is to user a custom mapping type as described here:
http://www.hibernate.org/100.html
I think the correct way to deal with this is to use a Calendar instead of a Date though. With the Calendar you can set the TimeZone before persisting.
NOTE: Silly stackoverflow won't let me comment, so here is a response to david a.
If you create this object in Chicago:
new Date(0);
Hibernate persists it as "12/31/1969 18:00:00". Dates should be devoid of timezone, so I'm not sure why the adjustment would be made.
There are several timezones in operation here:
Java's Date classes (util and sql), which have implicit timezones
of UTC
The timezone your JVM is running in, and
the default timezone of your database server.
All of these can be different. Hibernate/JPA has a severe design deficiency in that a user cannot easily ensure that timezone information is preserved in the database server (which allows reconstruction of correct times and dates in the JVM).
Without the ability to (easily) store timezone using JPA/Hibernate then information is lost and once information is lost it becomes expensive to construct it (if at all possible).
I would argue that it is better to always store timezone information (should be the default) and users should then have the optional ability to optimize the timezone away (although it only really affects display, there is still an implicit timezone in any date).
Sorry, this post doesn't provide a work-around (that's been answered elsewhere) but it is a rationalization of why always storing timezone information around is important. Unfortunately it seems many Computer Scientists and programming practitioners argue against the need for timezones simply because they don't appreciate the "loss of information" perspective and how that makes things like internationalization very difficult - which is very important these days with web sites accessible by clients and people in your organization as they move around the world.
Please take a look at my project on Sourceforge which has user types for standard SQL Date and Time types as well as JSR 310 and Joda Time. All of the types try to address the offsetting issue. See http://sourceforge.net/projects/usertype/
EDIT: In response to Derek Mahar's question attached to this comment:
"Chris, do your user types work with Hibernate 3 or greater? – Derek Mahar Nov 7 '10 at 12:30"
Yes these types support Hibernate 3.x versions including Hibernate 3.6.
Date is not in any time zone (it is a millisecond office from a defined moment in time same for everyone), but underlying (R)DBs generally store timestamps in political format (year, month, day, hour, minute, second, ...) that is time-zone sensitive.
To be serious, Hibernate MUST be allow being told within some form of mapping that the DB date is in such-and-such timezone so that when it loads or stores it it does not assume its own...
I encountered just the same problem when I wanted to store the dates in the DB as UTC and avoid using varchar and explicit String <-> java.util.Date conversions, or setting my whole Java app in the UTC time zone (because this could lead to another unexpected issues, if the JVM is shared across many applications).
So, there is an open source project DbAssist, which allows you to easily fix the read/write as UTC date from the database. Since you are using JPA Annotations to map the fields in the entity, all you have to do is to include the following dependency to your Maven pom file:
<dependency>
<groupId>com.montrosesoftware</groupId>
<artifactId>DbAssist-5.2.2</artifactId>
<version>1.0-RELEASE</version>
</dependency>
Then you apply the fix (for Hibernate + Spring Boot example) by adding #EnableAutoConfiguration annotation before the Spring application class. For other setups installation instructions and more use examples, just refer to the project's github.
The good thing is that you don't have to modify the entities at all; you can leave their java.util.Date fields as they are.
5.2.2 has to correspond to the Hibernate version you are using. I am not sure, which version you are using in your project, but the full list of provided fixes is available on the wiki page of the project's github. The reason why the fix is different for various Hibernate versions is because Hibernate creators changed the API a couple of times between the releases.
Internally, the fix uses hints from divestoclimb, Shane and a few other sources in order to create a custom UtcDateType. Then it maps the standard java.util.Date with the custom UtcDateType which handles all the necessary time zone handling.
The mapping of the types is achieved using #Typedef annotation in the provided package-info.java file.
#TypeDef(name = "UtcDateType", defaultForType = Date.class, typeClass = UtcDateType.class),
package com.montrosesoftware.dbassist.types;
You can find an article here which explains why such a time shift occurs at all and what are the approaches to solve it.
Hibernate does not allow for specifying time zones by annotation or any other means. If you use Calendar instead of date, you can implement a workaround using HIbernate property AccessType and implementing the mapping yourself. The more advanced solution is to implement a custom UserType to map your Date or Calendar. Both solutions are explained in my blog post here: http://www.joobik.com/2010/11/mapping-dates-and-time-zones-with.html
Here is the complete checklist for storing dates in your database in a proper way:
first, make sure your OS timezone is st properly: either set TZ env variable in your Dockerfile or start your Java application with -Duser.timezone=“UTC”.
As mentioned earlier: also set the timezone for JPA with hibernate.jdbc.time_zone = UTC
Use OffsetDateTime for storing dates as UTC in your entities. Don’t use Date, Calendar & ZonedDateTime !
I'm having a strange issue with a little servlet which uses Jersey and Gson for the JSON serialization/deserialization. I actually copy-pasted the basic Gson provider written for Jersey, like this one: http://eclipsesource.com/blogs/2012/11/02/integrating-gson-into-a-jax-rs-based-application/ and everything seemed to work fine, until I tried to deserialize a Date (in the standard ISO 8601 format), which always gets mapped into my POJO as null.
My first try was to register a deserializer type adapter before returning the gsonBuilder instance, like that:
import java.util.Date;
...
gsonBuilder.registerTypeAdapter(Date.class,
new JsonDeserializer<Date>() {
#Override
public Date deserialize(JsonElement json, Type type,
JsonDeserializationContext arg2) throws JsonParseException {
try {
System.out.println(json);
return (new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSXX")).parse(json.getAsString());
} catch (ParseException e) {
return null;
}
}
});
This didn't work, and nothing is printed out when I send the POST request. I tried to use the setDateFormat method on the gsonBuilder instance before returning it, but this didn't change anything.
I thought there were some others classes implementing the MessageBodyWriter and MessageBodyReader overriding my own implementation, so I tried to delete my own implementation and Jersey complained that it wasn't able to deserialize the JSON (so there are no other providers, i guess).
I tried to set breakpoints in the readFrom method in my MessageBodyReader but the request is actually deserialized without suspending the execution.
I should mention that my class contains different fields too, some strings and one date: the string are always deserialized correctly.
I tried sending different dates, starting with 2016-06-23T00:00:00.000+0200 (which should be formatted with the date format string I used in the code above), and getting to the simple 2016-06-17 by removing one part at the time, and it never worked.
I cleaned my maven project, recompiled it and it didn't work.
I thought it could have been Jetty not loading the correct classes, so i deployed the same code into a Tomcat 8 server, and the result was the same.
My last try was to write another parallel MessageBodyReader but instead of making it generic for the Object type, I made a specific java.util.Date deserializer, and still the readFrom method seems not to be called.
I seriously don't know what I could try now, do you have any idea?
Thanks in advance.
The reason of the error is here...
try {
System.out.println(json);
return (new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSXX")).parse(json.getAsString());
} catch (ParseException e) {
return null;
}
to be more specific here:
"yyyy-MM-dd'T'HH:mm:ss.SSSXX"
java Date and SimpleParser can hold at the most only 3 places for the milliseconds, and there is no wildcard .SSSXX in the string used for the SimpleDateFormat, so your parser is throwing an Exception, that you are catching but returning a Date referenced to null,
Ergo:
Gson is handling a null referenced object,
I'm currently struggling with Java JSON deserialization with Jackson in the following way:
I want to process and deserialize a JSON response I get back from a webservice and convert the response into POJOs with the help of Jackson. This works fine most of the time, as long as the response I'm getting contains JSON attributes in the correct format.
As however the webservice and the delivered data is out of my control, I can not rely on the data always being in the correct format.
Let me give you an example:
In my POJO, there's a java.util.Date field, and the JSON response contains a property holding a datetime string. Jackson will try to parse the string and convert it into a Date. If the date format matches the ObjectMapper's dateformat (ObjectMapper.setDateFormat(...)), everything is fine. If the format is different, I get an InvalidFormatException.
The problem now is, the dateformat sent from the service can differ. I can get dates formatted like 2014-11-02T00:00:00Z, but I can also get dates formatted like 2014-11 (identifying just a single month instead of an entire datetime).
I know, I can write a custom deserializer which could take care of this exact case and handle datestrings with different date formats correctly. But as this would only fix issues with Dates, but not with potential other datatypes, I'm searching for a more general approach.
(What happens e.g. if I expect a Double and receive an alphanumerical string?)
What I would like is to have the possibility to ignore all cases in which an InvalidFormatException happens and define a default value (like null) to the respective POJO field.
And it would be really valuable for me, if despite an invalid dateformat being returned or any other InvalidFormatException happening, the rest of the JSON would still be deserialized into the POJO.
Is this in any way possible with Jackson?
Thank you for reading my question till the end and I would be grateful for any pointers in the right direction.
Not sure if this is best practice, I have little experience with Jackson.
You can add a DeserializationProblemHandler to the ObjectMapper to specify what happens when the deserializer encounters a weird String or weird number.
In your case you could set the handler such that when encountering an unrecognized format, instead of throwing an InvalidFormatException, it just returns null:
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.addHandler(new DeserializationProblemHandler() {
#Override
public Object handleWeirdStringValue(DeserializationContext ctxt, Class<?> targetType, String valueToConvert, String failureMsg) throws IOException {
return null;
}
#Override
public Object handleWeirdNumberValue(DeserializationContext ctxt, Class<?> targetType, Number valueToConvert, String failureMsg) throws IOException {
return null;
}
});
I have a backend system which we use a third-party Java API to access from our own applications. I can access the system as a normal user along with other users, but I do not have godly powers over it.
Hence to simplify testing I would like to run a real session and record the API calls, and persist them (preferably as editable code), so we can do dry test runs later with API calls just returning the corresponding response from the recording session - and this is the important part - without needing to talk to the above mentioned backend system.
So if my application contains line on the form:
Object b = callBackend(a);
I would like the framework to first capture that callBackend() returned b given the argument a, and then when I do the dry run at any later time say "hey, given a this call should return b". The values of a and b will be the same (if not, we will rerun the recording step).
I can override the class providing the API so all the method calls to capture will go through my code (i.e. byte code instrumentation to alter behavior of classes outside my control is not necessary).
What framework should I look into to do this?
EDIT: Please note that bounty hunters should provide actual code demonstrating the behavior I look for.
Actually You can build such framework or template, by using proxy pattern. Here I explain, how you can do it using dynamic proxy pattern. The idea is to,
Write a proxy manager to get recorder and replayer proxies of API on demand!
Write a wrapper class to store your collected information and also implement hashCode and equals method of that wrapper class for efficient lookup from Map like data structure.
And finally use recorder proxy to record and replayer proxy for replaying purpose.
How recorder works:
invokes the real API
collects the invocation information
persists data in expected persistence context
How replayer works:
Collect the method information (method name, parameters)
If collected information matches with previously recorded information then return the previously collected return value.
If returned value does not match, persist the collected information (As you wanted).
Now, lets look at the implementation. If your API is MyApi like bellow:
public interface MyApi {
public String getMySpouse(String myName);
public int getMyAge(String myName);
...
}
Now we will, record and replay the invocation of public String getMySpouse(String myName). To do that we can use a class to store the invocation information like bellow:
public class RecordedInformation {
private String methodName;
private Object[] args;
private Object returnValue;
public String getMethodName() {
return methodName;
}
public void setMethodName(String methodName) {
this.methodName = methodName;
}
public Object[] getArgs() {
return args;
}
public void setArgs(Object[] args) {
this.args = args;
}
public Object getReturnValue() {
return returnType;
}
public void setReturnValue(Object returnValue) {
this.returnValue = returnValue;
}
#Override
public int hashCode() {
return super.hashCode(); //change your implementation as you like!
}
#Override
public boolean equals(Object obj) {
return super.equals(obj); //change your implementation as you like!
}
}
Now Here comes the main part, The RecordReplyManager. This RecordReplyManager gives you proxy object of your API , depending on your need of recording or replaying.
public class RecordReplyManager implements java.lang.reflect.InvocationHandler {
private Object objOfApi;
private boolean isForRecording;
public static Object newInstance(Object obj, boolean isForRecording) {
return java.lang.reflect.Proxy.newProxyInstance(
obj.getClass().getClassLoader(),
obj.getClass().getInterfaces(),
new RecordReplyManager(obj, isForRecording));
}
private RecordReplyManager(Object obj, boolean isForRecording) {
this.objOfApi = obj;
this.isForRecording = isForRecording;
}
#Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
Object result;
if (isForRecording) {
try {
System.out.println("recording...");
System.out.println("method name: " + method.getName());
System.out.print("method arguments:");
for (Object arg : args) {
System.out.print(" " + arg);
}
System.out.println();
result = method.invoke(objOfApi, args);
System.out.println("result: " + result);
RecordedInformation recordedInformation = new RecordedInformation();
recordedInformation.setMethodName(method.getName());
recordedInformation.setArgs(args);
recordedInformation.setReturnValue(result);
//persist your information
} catch (InvocationTargetException e) {
throw e.getTargetException();
} catch (Exception e) {
throw new RuntimeException("unexpected invocation exception: " +
e.getMessage());
} finally {
// do nothing
}
return result;
} else {
try {
System.out.println("replying...");
System.out.println("method name: " + method.getName());
System.out.print("method arguments:");
for (Object arg : args) {
System.out.print(" " + arg);
}
RecordedInformation recordedInformation = new RecordedInformation();
recordedInformation.setMethodName(method.getName());
recordedInformation.setArgs(args);
//if your invocation information (this RecordedInformation) is found in the previously collected map, then return the returnValue from that RecordedInformation.
//if corresponding RecordedInformation does not exists then invoke the real method (like in recording step) and wrap the collected information into RecordedInformation and persist it as you like!
} catch (InvocationTargetException e) {
throw e.getTargetException();
} catch (Exception e) {
throw new RuntimeException("unexpected invocation exception: " +
e.getMessage());
} finally {
// do nothing
}
return result;
}
}
}
If you want to record the method invocation, all you need is getting an API proxy like bellow:
MyApi realApi = new RealApi(); // using new or whatever way get your service implementation (API implementation)
MyApi myApiWithRecorder = (MyApi) RecordReplyManager.newInstance(realApi, true); // true for recording
myApiWithRecorder.getMySpouse("richard"); // to record getMySpouse
myApiWithRecorder.getMyAge("parker"); // to record getMyAge
...
And to replay all you need:
MyApi realApi = new RealApi(); // using new or whatever way get your service implementation (API implementation)
MyApi myApiWithReplayer = (MyApi) RecordReplyManager.newInstance(realApi, false); // false for replaying
myApiWithReplayer.getMySpouse("richard"); // to replay getMySpouse
myApiWithRecorder.getMyAge("parker"); // to replay getMyAge
...
And You are Done!
Edit:
The basic steps of recorder and replayers can be done in above mentioned way. Now its upto you, that how you want to use or perform those steps. You can do what ever you want and whatever you like in the recorder and replayer code blocks and just choose your implementation!
I should prefix this by saying I share some of the concerns in Yves Martin's answer: that such a system may prove frustrating to work with and ultimately less helpful than it would seem at first blush.
That said, from a technical standpoint, this is an interesting problem, and I couldn't not take a go at it. I put together a gist to log method calls in a fairly general way. The CallLoggingProxy class defined there allows usage such as the following.
Calendar original = CallLoggingProxy.create(Calendar.class, Calendar.getInstance());
original.getTimeInMillis(); // 1368311282470
CallLoggingProxy.ReplayInfo replayInfo = CallLoggingProxy.getReplayInfo(original);
// Persist the replay info to disk, serialize to a DB, whatever floats your boat.
// Come back and load it up later...
Calendar replay = CallLoggingProxy.replay(Calendar.class, replayInfo);
replay.getTimeInMillis(); // 1368311282470
You could imagine wrapping your API object with CallLoggingProxy.create prior to passing it into your testing methods, capturing the data afterwards, and persisting it using whatever your favorite serialization system happens to be. Later, when you want to run your tests, you can load the data back up, create a new instance based on the data with CallLoggingProxy.replay, and passing that into your methods instead.
The CallLoggingProxy is written using Javassist, as Java's native Proxy is limited to working against interfaces. This should cover the general use case, but there are a few limitations to keep in mind:
Classes declared final can't be proxied by this method. (Not easily fixable; this is a system limitation)
The gist assumes the same input to a method will always produce the same output. (More easily fixable; the ReplayInfo would need to keep track of sequences of calls for each input instead of single input/output pairs.)
The gist is not even remotely threadsafe (Fairly easily fixable; just requires a little thought and effort)
Obviously the gist is simply a proof of concept, so it's also not been very thoroughly tested, but I believe the general principle is sound. It's also possible there's a more fully baked framework out there to achieve this sort of goal, but if such a thing does exist, I'm not aware of it.
If you do decide to continue with the replay approach, then hopefully this will be enough to give you a possible direction to work in.
I had the same needs some months ago for non-regression testing when planning a heavy technical refactoring of a large application and... I have found nothing available as a framework.
In fact, replaying may be particularly difficult and may only work in a specific context - no (or few) application with a standard complexity can be really considered as stateless. It is a common problem when testing persistence code with a relational database. To be relevant, the complete system initial state must be restored and each replay step must impact the global state the same way. It becomes a challenge when a system state is distributed into pieces like databases, files, memory... Let's guess what happens if a timestamp taken from a system's clock is used somewhere !
So a more pratical option is to only record... and then do a clever comparison for subsequent runs.
Depending of the number of runs you plan, a human-driven session on the application may be enough, or you have to investing in an automated scenario in a robot playing with your application user interface.
First to record: you can use dynamic proxy interface or aspect programming to intercept method call and to capture state before and after invocation. It may mean: dump concerned database tables, copy some files, serialize Java objects in text format like XML.
Then compare this reference capture with a new run. This comparison should be tuned to exclude any irrelevant elements from each piece of state, like row identifiers, timestamps, file names... to only compare data where your backend's added value shines.
Finally nothing really standard, and often a few specific scripts and codes may be enough to achieve the aim: detect as much errors as possible and try to prevent non-expected side-effects.
This can be done with AOP, aspect oriented programming. It allows to intercept method calls by byte code manipulation. Do a bit of search for examples.
In one case this can do recording, in the other replaying.
Pointers: wikipedia, AspectJ, Spring AOP.
Unfortunately one moves a bit outside the java syntax, and a simple example can better be sought elsewhere. With explanation.
Maybe combined with unit tests / some mocking test framework for offline testing with recorded data.
you could look into 'Mockito'
Example:
//You can mock concrete classes, not only interfaces
LinkedList mockedList = mock(LinkedList.class);
//stubbing
when(mockedList.get(0)).thenReturn("first");
when(mockedList.get(1)).thenThrow(new RuntimeException());
//following prints "first"
System.out.println(mockedList.get(0));
//following throws runtime exception
System.out.println(mockedList.get(1));
//following prints "null" because get(999) was not stubbed
System.out.println(mockedList.get(999));
after you could replay each test more times and it will return data that you put in.
// pseudocode
class LogMethod {
List<String> parameters;
String method;
addCallTo(String method, List<String> params):
this.method = method;
parameters = params;
}
}
Have a list of LogMethods and call new LogMethod().addCallTo() before every call in your test method.
The idea of playing back the API calls sounds like a use case for the event sourcing pattern. Martin Fowler has a good article on it here. This is a nice pattern that records events as a sequence of objects which are then stored, you can then replay the sequence of events as required.
There is an implementation of this pattern using Akka called Eventsourced, which may help you build the type of system you require.
I had a similar problem some years ago. None of the above solutions would have worked for methods that are not pure functions (side effect free). The major task is in my opinion:
how to extract a snapshot of the recorded object(s) (not only restricted to objects implementing Serializable)
how to generate test code of a serialized representation in a readable way (not only restricted to beans, primitives and collections)
So I had to go my own way - with testrecorder.
For example, given:
ResultObject b = callBackend(a);
...
ResultObject callBackend(SourceObject source) {
...
}
you will only have to annotate the method like this:
#Recorded
ResultObject callBackend(SourceObject source) {
...
}
and start your application (the one that should be recorded) with the testrecorder agent. Testrecorder will manage all tasks for you, such as:
serializing arguments, results, state of this, exceptions (complete object graph!)
finding a readable representation for object construction and object matching
generating a test from the serialized data
you can extend recordings to global variables, input and output with annotations
An example for the test will look like this:
void testCallBackend() {
//arrange
SourceObject sourceObject1 = new SourceObject();
sourceObject1.setState(...); // testrecorder can use setters but is not limited to them
... // setting up backend
... // setting up globals, mocking inputs
//act
ResultObject resultObject1 = backend.callBackend(sourceObject1);
//assert
assertThat(resultObject, new GenericMatcher() {
... // property matchers
}.matching(ResultObject.class));
... // assertions on backend and sourceObject1 for potential side effects
... // assertions on outputs and globals
}
If I understood you question correctly, you should try db4o.
You will store the objects with db4o and restore later to mock and JUnit tests.
I have some dates I have to represent in an XML file in a format different than the one JAXB uses by default.
So, I've written some static methods to print and parse the required format:
public static String toDateTime(XMLGregorianCalendar d) {
return new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
.format(d.toGregorianCalendar().getTime());
}
public static XMLGregorianCalendar parseDateTime(String s)
throws DatatypeConfigurationException, ParseException {
GregorianCalendar cal = new GregorianCalendar();
cal.setTime(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
.parse(s));
return DatatypeFactory.newInstance().newXMLGregorianCalendar(cal);
}
And I've written a .xjb file specifying to use these parse and print methods:
<jxb:bindings
node="//xs:element[#name='date']">
<jxb:javaType name="javax.xml.datatype.XMLGregorianCalendar"
parseMethod="MyClass.parseDateTime"
printMethod="MyClass.toDateTime"
/>
</jxb:bindings>
(The XML schema specifies the element as an xsd:dateTime).
The xjc tool is generating an adapter that calls my methods:
public XMLGregorianCalendar unmarshal(String value) {
return (MyClass.parseDateTime(value));
}
public String marshal(XMLGregorianCalendar value) {
return (MyClass.toDateTime(value));
}
My problem is that the generated adapter won't compile. The unmarshal method is declared not to throw exceptions, but MyClass.parseDateTime does. It seems perfectly reasonable that it should, since the date might be invalid. And indeed, the unmarshal method of java.xml.bind.annotation.adapters.XmlAdapter that the generated adapter overrides declares that it throws Exception.
I couldn't find any way to declare in the xjb file which exceptions the parseMethod throws. It occurred to me that maybe I should create my own adapter class, but I couldn't find any way to declare in the xjb that my custom adapter should be used. I would very much like to be able to generate the classes from the xsd and xjb with xjc and not have to modify them afterwards.
What is the best way to resolve this situation?
You should handle both checked exceptions in your code. For an invalid date-time, you have to decide whether to leave it as null (and maybe emit a warning), or blow up and kill the whole parsing attempt (in that case, throw a RuntimeException wrapping the original ParseException).
In the case of DatatypeConfigurationException, this is one of the JDK APIs that is badly designed, throwing checked exceptions when there's nothing your code can do. Just rethrow it as RuntimeException.