We Keep Coding

How can I lazy load specific elements of a JSON file efficiently with GSON?

I have a JSON file that is marshalled into custom object using GSON.
All works fine.
An example of the structure I have:
public class Domain {
private String id;
private String name;
//other fields
private ArrayList<Structures> structs;
private ArrayList<Buildings> buildings;
private ArrayList<CustomObject> objects;
// some more lists and fields
}
So basically I create a builder and parse the json
Gson gson = new GsonBuilder().create();
gson.fromJson(jsonString, Domain.class);
This works absolutely fine. Parsing is done and I get my data.
Problem:
I don't really need to have various fields of the Domain class populated from the start because e.g. I may have the Domain class with a lot of elements in the e.g. list for structs but I might not really need to access them.
What I need to do is some kind of pattern for lazy loading.
I.e. I would like to not load some parts of class during the json parsing and only load them when needed.
Question:
If I understand correctly the way to skip fields from being parsed is by making them transient.
But then if at some later time I need to access e.g. the structs how would I actually load them at that point? I think that reloading/reparsing all the json again is suboptimal.
What is a standard/good approach for this?

This is a really lengthy topic. There are many approaches to this and all of them are usually a lot more complicated. The easiest one, if you really value something very simple for me was so far not using gson, but for example something like JSONObject and then populate the object myself. using this you could easily split this up into multiple steps. The problem that now arises is, that you never know, what exactly is already loaded - or more - what is maybe loaded, but just not filled as a field.
Lazy loading using automatic conversions like gson is unfortunately always gonna involve unnecessary object creation too, so question then is if its not less pain just to do it yourself from the beginning.
if it has to be gson, you could have different objects for different stages. read them in through json and then apply to your main object.
a favourable version of that is probably to split up the object into different components (or aspects or whatever you want to call it) that match the different loading stages. Different possibilities but lets just pick one of them:
class Domain {
private String id;
private DomainStructs domainStructs;
}
class DomainStructs {
private ArrayList<Structures> structs;
}
Now you need a new Object in this version of doing this. This means the overall size of the model is slightly (but not much really) bigger and you should probably match together things that are necessary together anyway - so not load every field separate, but this way you can leave out parts and easily add them later by populating them from Gson like 2 steps here:
Gson gson = new GsonBuilder().create();
Domain domain = gson.fromJson(jsonString, Domain.class); // first time
domain.structs = gson.fromJson(jsonString, DomainStructs.class); // now adding
I am not saying this is the best idea ever, but it fulfills your idea while still using gson.
I would though consider splitting up the Data already - so not storing the strings, but holding the data in different components in this case if it is possible. Basically you want a domainJsonString and a domainStructsJsonString if you get what i mean. stored in a way so you can easily retrieve them separately.
I hope this helps you to move a bit forward

Correct way of finding what was modified by a post in an spring-mvc controller?

It is a rather general question, but I will give a stripped down example. Say I have a Web CRUD application that manages simple entities stored in a database, nothing but classic : JSP view, RequestMapping annotated controller, transactional service layer and DAO.
On an update, I need to know the previous values of my fields, because a business rule asks a for a test involving the old and new values.
So I am searching for a best practice on that use case.
I thing that spring code is way more extensively tested and more robust than my own, and I would like to do it the spring way as much as possible.
Here is what I have tried :
1/ load an empty object in controller and manage the update in service :
Data.java:
class Data {
int id; // primary key
String name;
// ... other fields, getters, and setters omitted for brevity
}
DataController
...
#RequestMapping("/data/edit/{id}", method=RequestMethod.GET)
public String edit(#PathVariable("id") int id, Model model) {
model.setAttribute("data", service.getData(id);
return "/data/edit";
}
#RequestMapping("/data/edit/{id}", method=RequestMethod.POST)
public String update(#PathVariable("id") int id, #ModelAttribute Data data, BindingResult result) {
// binding result tests omitted ..
service.update(id, data)
return "redirect:/data/show";
}
DataService
#Transactional
public void update(int id, Data form) {
Data data = dataDao.find(id);
// ok I have old values in data and new values in form -> do tests stuff ...
// and MANUALLY copy fields from form to data
data.setName(form.getName);
...
}
It works fine, but in real case, if I have many domain objects and many fields in each, it is quite easy to forget one ... when spring WebDataBinder has done it including validation in the controller without I have to write any single thing other than #ModelAttribute !
2/ I tried to preload the Data from the database by declaring a Converter
DataConverter
public class DataConverter<String, Data> {
Data convert(String strid) {
return dataService.getId(Integer.valueOf(strid));
}
}
Absolutely magic ! The data if fully initialized from database and fields present in form are properly updated. But ... no way to get the previous values ...
So my question is : what could be the way to use spring DataBinder magic and to have access to previous values of my domain objects ?

You have already found the possible choices so i will just add some ideas here ;)
I will start with your option of using a empty bean and copying the values over to a loaded instance:
As you have shown in your example it's an easy approach. It's quite easily adaptable to create a generalized solution.
You do not need to copy the properties manually! Take a look at the 'BeanWrapperImpl' class. This spring object allows you to copy properties and is in fact the one used by Spring itself to achieve it's magic. It's used by the 'ParameterResolvers' for example.
So copying properties is the easy part. Clone the loaded object, fill the loaded object and compare them somehow.
If you have one service or just several this is the way to go.
In my case we needed this feature on each entity. Using Hibernate we have the issue that an entity might not only change inside a specific service call, but theoretically all over the place..
So I decided to create a 'MappedSuperClass' which all entities need to extend. This entity has a 'PostLoad' event listener which clones the entity in a transient field directly after loading. (This works if you don't have to load thousands of entities in a request.) Then you need also the 'PostPersist' and 'PostUpdate' listeners to clone the new state again as you probably don't reload the entity before another modification.
To facilitate the controller mapping I have implemented a 'StringToEntityConverter' doing exactly what you did, just generalized to support any entity type.
Finding the changes in a generalized approach will involve quite a bit of reflection. It's not that hard and I don't have the code available right now, but you can also use the 'BeanWrapper' for that:
Create a wrapper for both objects. Get all 'PropertyDescriptors' and compare the results. The hardest part is to find out when to stop. Compare only the first level or do you need deep comparison?
One other solution could also be to rely on Hibernate Envers. This would work if you do not need the changes during the same transaction. As Envers tracks the changes during a flush and creates a 'Revision' you can "simply" fetch twp revisions and compare them.
In all scenarios you will have to write a comparison code. I'm not aware of a library but probably there is something around in the java world :)
Hope that helps a bit.

Create a entity mapper with cyclic dependencies

I've a problem with a project i'm working on. I need to create a mapper that convert an entity object to it's VO (Value Object) form, i need to do that to limit dependencies between my persistence layer and my business layer but i've some cyclic dependencies that i can't resolve, look at this example :
I want to use a Competence object in my business layer so i use the mapper to convert the entity into a VO that is usable by my layer.
class Competence {
private Domain dom;
}
class Domain {
private List<Competence> lComp;
}
But when i convert a Competence object, i need to convert a Domain too and when i convert a Domain i need to convert the Competence list etc etc... And i don't really want to convert half of my database :/
I've thinking about converting half of the object or avoid some object to stop the cycle but it's dangerous and this is not really a solution for my problem :/
Haved you a solution to solve this ?
Thanks in advance ! :)

One standard mechanism to deal with this kind of problem is to recode to something like:
class Competence {
}
class Domain {
}
class CompetenceDomain {
Competence competence;
Domain domain;
}
Map<Domain,List<Competence>> domainsWithCompetence = ...;
Map<Competence,Domain> competencesInDomain = ...;
This breaks the relationships of the objects out from the objects themselves into a separate mechanism.

There are a couple of ways around this. First, you have to decide which side you want to convert. Let's assume you want to convert Competences and not convert Domains.
Use an ID to reference the Domain inside the Competence.
Use a stub Domain object that only contains the ID and a minimal set of fields (not including Competences) that you need.
Hopefully I've correctly understood your question.

List objects - best practice

So there is a new guy that has started where I work. I'm quite a junior programmer myself, but I've been developing for a bit longer, so I'm using my "gut feeling" to guide us in developing a project. The blind leading the blind.
A relatively small thing just came up, and I would like to know what is the best way to create this for future reference and why.
Essentially there is a basic XML file with details for files (structure isn't really relevant). He went about querying this XML file and then storing all retrieved files by creating several lists, something like so:
List<Integer> fileId = new List<Integer>;
List<String> title = new List<String>;
And then you would create a method which would query against these Lists looking for the ID.
I pictured a method would be created to query for a file out of the XML file without storing/setting anything, like so:
public Form getFile(Integer id) {
Form ret = new Form();
//Query XML, set to ret...
return ret;
}
I wanted to use value objects, since that's how I'm used to working. So suggested and settled for this in the end:
List<Form> forms = new List<Form>;
So now, we have 2 methods, 1 to populate the 'forms' variable, and then 1 to query it and return the Form... still seems very strange to me.
Also, instead of:
Form tempForm = new Form();
tempForm.id = 1;
tempForm.title = "Foo";
He prefers to do:
Form tempForm = new Form(id, title);
Purely because it's in 1 line and looks tidier. Later down the line though, I don't think using a value object like this is the best way to go.
Maybe I am worrying and thinking about stuff to much as opposed to getting on with development, but any advice on this would be great.

On your second style question:
One of the reasons to use a constructor is that you can then make your Form object immutable as in:
public class Form {
private final String id;
private final String title;
public Form(String id, String title) {
this.id = id; this.title = title;
}
public String getTitle() { return title; }
public String getId() { return id; }
}
This helps avoid concurrency issues.

I'm not sure I understand your question properly, but at the basis, it sounds like a performance question. ie: is it worth reading in an entire XML file, and restructuring it such that it is faster and easier to query, or is it better to scan the xml file every time and query against it. That's a question that only you can answer. As usual, it's the space-speed tradeoff that you have to evaluate.
If your XML file is huge and would require significant amount of memory to cache and you only query against in sporadically, then perhaps your solution is better. If it is small and speed is critical, then caching it is a good idea.
All that being said, there are several different libraries that you can use to speed up the processing in different ways. You can look at using XQuery and/or XPath (see How to read XML using XPath in Java), JAXB, SAX, etc. Each technology has its own advantages and disadvantages.
Hopefully that will give you a little more background that you can discuss with each other.

Interresting question! There are however several questions in one. Let me answer each one of them separately.
Let me first lay down the definition of a value type as found on domaindrivendesign.org
Definition: A Value Object is an object that describes some
characteristic or attribute but carries no concept of identity.
For example a file path is a string, but it also has some restrictions on the format of the string and some operations. Here it would be a good idea to create a value object. Note also that a path carries no notation of identity. That is, two path objects representing the same path would be considered equal.
Now to the actual question, I strongly recommend your way of coding - Creating a class for data that belong together. In your first example id and title are only related by an index into two separate lists.
It's better to use this form
Form tempForm = new Form(id, title);
That way the Form class can be immutable which will give you great readability benefits and also performance gains. Also the fields of the class are encapsulated.
Now to the last thing you thought was strange - Having two methods, one for creating the list and one for querying against it.
Here I would actually create a new class, containing only those two methods instead of having them say in a static class. I would call it a FormCollection. You guys can probably come up with some smarter name since you have more context. Spend at most five minutes figuring out a meaningful name.
You could also refactor your code further to for example take the xml file path or stream as a constructor argument and then have a single method for querying aginst it on id. Like so:
class FormCollection
{
public FormCollection(String xmlFilePath) { ... }
public Form getById(int id) { ... }
}
This is probably a good interface to the rest of your application, since it easy and to the point. Also it's easy to test.

Overlaying an existing object graph?

I have a database that contains an object graph. However, during use of the objects contained in the database I need to add additional functionality.
I cannot change the classes of the database, so I figure my only option is to make wrapper classes composed of their database equivalent objects and create forwarding methods along with methods and fields for the additional functionality. For instance:
public class Foo() {
private DBFoo databaseFoo;
// a bunch of forwarding methods to databaseFoo
// some methods for additional functionality
}
I'm struggling with a good pattern to construct my wrapper objects on top of the pre-existing database object graph. In particular how to reconstruct my graph of objects to mirror the graph in the database (with all its various references to other objects). Does anyone have experience with this sort of problem?

I can see two problems here:
a). Given some dbClassA, which references dbClassB, and collections of dbClassC, with dbClassB and dbClassC each themselves being arbitrarily complex: what design of proxy is appropriate? If dbClassA has methos
dbClassB getB() { ... }
we can quite easily see a mapping to
myClassB getB() { return new myClassB( theDbClassB.getB() ); }
or some such. But exactly what do we do with the collections?
List<dbClassC> getAllCs() { }
what does that become?
I figure that this is a solvable problem, just a matter of choosing some rules.
I suspect that you're more concerned about:
b). Creating all those proxy classes. Apply the chosen rule to many, many classes. One possible approach is to use code generator capabilities, for example JET in Eclipse. I've seen very good use of this capability for similar problems.