We Keep Coding

JPA Spring Data entity to be used outside of transaction

I have a Spring Boot application with a service that returns a Spring Data entity that is exposed to a controller. The problem is that I know it's not a good idea to use entities outside of DB transactions, so what would be the best practices?
Consider the following service:
#Transactional
public MyData getMyData(Long id) {
return myDataRepository.findById(id);
}
where MyData is a database #Entity and myDataRepository is a JpaRepository
This service method is called from a controller class, that sends this object in JSON format to a client that calls this method.
#RequestMapping("/")
public ResponseEntity<?> getMyData(#RequestParam Long id) {
return myService.getMyData(id);
}
If I expose MyData to a controller, then it will be exposed outside of a transaction and might cause all kind of hibernate errors. What are the best practices for these scenarios? Should I convert entity to POJO in side the service and return MyDataPOJO instead of MyData in MyService?

Using entities outside of transactions does not necessarily lead to problems; it may actually have valid use cases. However, there's quite a few variables at play and once you let them out of your sight things may and will go south. Consider the following scenarios:
Your entity doesn't have any relationships to other entities or those relationships are pretty shallow and eagerly fetched. You retrieve that entity from repository, detach it from persistence unit (implicitly or explicitly) and pass to controller. Controller does not attempt to modify the entity; it only serializes it into JSON - totally safe.
Same as above but controller modifies the entity before serializing it into JSON - again, totally safe (just don't expect those changes to be reflected in DB)
Same as above, but you've forgotten to detach the entity from PU - ouch, if controller changes the entity you may either see it reflected in DB or get transaction closed exception; both most likely being unintended consequences.
Same as above, but some of entity's relationships are lazy. Again, you may or may not get any exceptions depending on whether these lazy properties are being accessed or not.
And there are so many more combinations of intentional and unintentional design choices...
As you may see, things can get out of control very quickly. Especially so when your model has to evolve: before long you're going to find yourself fiddling with JSON views, #JsonIgnore, entity projections and so on. Thus the rule of thumb: although it may seem tempting to cut some corners and expose your entities to external layers, it's rarely a good idea. Properly designed solution always has a clear separation of concerns between layers:
Persistence layer never exposes more methods or entities than required by business logic. More over, the same table(s) can and should be mapped into several different entities depending on the use cases they participate in.
Business logic layer (btw this is your API, not the REST services! see below) never leaks any details from persistence layer. Its methods clearly define use cases from the problem domain.
Presentation layer only translates API provided by business logic into one or another form suitable for client and never implements additional use cases. Keep in mind that REST controllers, SOAP services etc logically are all part of presentation layer, not business logic.
So yeah, the short answer is: persistence entities should not be exposed to external layers. One common technique is to use DTOs instead; besides, DTO objects provide additional abstraction layer in case you need to change your entities but leave API intact or vice versa. If at some point your DTOs happen to closely resemble your entities, there are Java bean mapping frameworks like Dozer, Orika, MapStruct, JMapper, ModelMapper etc that help to eliminate the boilerplate code.
Try googling "hexagonal architecture". This is a very interesting concept for designing cleanly separated layers. Here's one of the articles on this subject https://blog.octo.com/en/hexagonal-architecture-three-principles-and-an-implementation-example/; it uses C# examples but they're pretty simple.

You should never leak the internal model to outside resources (in your case - the #RestController). The "POJO" you mentioned is typically called a DTO (Data Transfer Object). The DTO can be defined as an interface on the Service-side and implemented on the Controller-side. The Service would then - as you described - transform the internal model into an instance of the DTO, achieving looser coupling between the Controler and the Service.
By defining the DTO-interface on the service-side, you have the additional benefit that you can optimize your persistence-acces by only fetching the data specified in the corresponding DTO-interface. There is, for example, no need to fetch the friends of a User if the #Controller does not specifically requests them, thus you do not need to perform the additional JOIN in the database (provided you use a database).

Why do we need a bean to bean mapper like dozer in a web application

In simple terms, why do we need 'a bean to bean mapping service' (like Dozer) in a web-application.
Suppose I'm working on a web-service.
I'm receiving an XML in request.
I fetch the the values from XML elements.
Perform the required operation on the fetched values.
Prepare the response XML.
Send the response XML as response
Why should I add one more steps of mapping XML elements to own custom elements.
I'm not able to convince myself, probably because I'm not able to think of a better situation/reason.
Please suggest, with example if possible.

It helps to reduce coupling between the presentation (i.e. the XML schema) and the business logic. For example in case of schema changes you don't have to touch the business logic, just the mapping between the objects.
In simple cases it might not be worth the additional complexity. But if the objects are used extensively in the business logic component you should consider it.

Just as a quick answer, the case you described is not the only one :).
Suppose you are working with an internal library providing some POJO / entity / other beans. You want to abstract from the internal representation (for a reason or anohter), you then want to map those bean to yours. It works :
for ejb client, or somehting like that,
when you don't want to expose internal entities (Business Object vs Presentation object) (see #Henry's reply)
you have beans that don't inherit from the same parent (and can't for any reason, even leacy) and you want to tarnsfert value from on to another
There are plenty of (other) reasons :)
As an advice see also orika
and this post :
any tool for java object to object mapping?

Short answer for me as henry said it helps reduce coupling between what you expose or consume and your core data model.
It is one way build Hexagonal Architecture. You can freely modify your core model without impacting the exposed model. In hexagonal architecture, it is used to expose only a small relevant part of the core model.
It is also a very goog way to handle services and model versionning since multiple versions can be mapped to the core model.
When working with XML services I tend to build contract first application so, I first write the XMLSchema then generate Jaxbeans and I realy don't want my business code to be polluted by JAxb annotations.
If you know that your exposed model will always be the same and that your application does not fall in the previously mentionned cases, so you realy don't need to use DTO.
Last, I would recommend using a framework with strong compile time checking like Selma instead of Dozer or Orika because they are evaluating the mapping only at runtime which is weak typing and sensible to refactoring.

How to merge input from a web service to a JPA entity

I'm trying to figure out the best way to use JPA in the context of a restful web service. The input comes in as JSON and I can use Jackson/JAX-RS to convert that to a POJO. This gets passed to a service where I need to somehow merge into a JPA entity.
These are the options I've found so far with pros and cons.
1. JPA merge()
The first thing I tried was probably the simplest. The GET action returns the JPA entity which is easily serialized into JSON. On the update the object is passed back is JSON which can be used to populate a detached entity. This can be saved to the DB using the JPA merge() method.
Pros
Simple architecture with less code duplication (i.e. no DTO's)
Cons
As far as I can tell this only works if you pass the whole model around. If you try to hide certain fields, like the maybe the password on a User entity, then the merge thinks you're trying to set these fields to null in the DB. Not good!
2. DTO's using JPA find() and dozer
Next I thought I'd look at using data transfer objects. Apparently an anti-pattern but worth a look. The service now creates a DTO instance based on the entity and it is this DTO that is serialized to JSON. The update then gets the entity from the DB using a find() method and the values need to be copied across from the DTO to the entity. I tried automating this mapping using the dozer framework.
Pros
You don't have to return the entire model. If you have certain fields you don't want to be updated you can leave them off the DTO and they can't be copied to the entity by mistake. Using dozer means you don't have to manually copy attributes from dto to entity and vice versa.
Cons
It feels like repeating yourself when writing the DTO's. Somehow you have to map between entities and DTO's. I tried to automate this with dozer but it was a bit disappointing. It was nulling out things it shouldn't have been and to get full control you have to write xml.
3. DTO's using manual merge
A third way would be to abandon dozer and just copy the properties across from the DTO to the entity in the service. Everybody seems to say anti-pattern but it's pretty much how every non-trivial application that I've seen in the past has worked.
Summary
It seems to be a decision between keeping things simple for the developer but not having control over the input/output or making a more robust web service but having to use an anti-pattern in the process...
Have I missed anything? Perhaps there's an elusive alternative?

Using JPA merge looks the simplest, cleanest and with very less effort but as correctly discovered creates problems with detached entity attributes set to null.
Another problem which turned out to be big in one of my experiences was that if you rely on JPA merge operation you must be using Cascade feature as well.
For simple and less nested relation this works reasonably well, but for deeply nested domain objects and lots of relations, this becomes a big impact on performance. The reason being that the ORM tool (Hibernate in my experience) upfront caches the SQL to load the merge entity ( 'merge path' in Hibernate parlance) and if the nesting is too deep with Cascade mappings the joins in the SQL becomes too big. Marking realtions Lazy does not help here as the merge path is determined by the Cascades in relations. This problem becomes apparent slowly as your model evolves. Plus the prospect of angry DBA waving a huge join query on our face prompted us to do something different :-)
There is an interesting issue related to Hibernate regarding Merging of Lazy relations still unresolved (actually rejected but the discussion is very enjoyable to read) in Hibernate JIRA.
We then moved towards the DTO approach where we refrained from using merge and relied on doing it manually. Yes it was tedious and required the knowledge of
what state is actally coming from the detached entity, but to us it was worth. This way we do not touch the Lazy relations and attributes not meant to change. and set only what is required. The automatic state detection of Hibernate does the rest on transaction commit.

This is approach I am using:
suppress serialization of certain fields with XmlTransient annotation
when updating the record from the client, get the entity from the database and use ModelMapper with custom property mapping to copy the updated values without changing the fields that are not in the JSON representation.
For example:
public class User {
#Id
private long id;
private String email;
#XmlTransient
private String password;
...
}
public class UserService {
...
public User updateUser(User dto) {
User entity = em.find(User.class, dto.getId());
ModelMapper modelMapper = new ModelMapper();
modelMapper.addMappings(new UserMap());
modelMapper.map(userDto, user);
return user;
}
}
public class UserMap extends PropertyMap<User, User> {
protected void configure() {
skip().setPassword(null);
}
}
BeanUtils is an alternative to ModelMapper.
It would be nice if these libraries could recognize the XmlTransient annotation so the programmer can avoid creating the custom property map.

What is a good strategy for converting jpa entities into restful resources

Restful resources do not always have a one-to-one mapping with your jpa entities. As I see it there are a few problems that I am trying to figure out how to handle:
When a resource has information that is populated and saved by more than one entity.
When an entity has more information in it that you want to send down as a resource. I could just use Jackson's #JsonIgnore but I would still have issue 1, 3 and 4.
When an entity (like an aggregate root) has nested entities and you want to include part of its nested entities but only to a certain level of nesting as your resource.
When you want to exclude once piece of an entity when its part of one parent entity but exclude a separate piece when its part of a different parent entity.
Blasted circular references (I got this mostly working with JSOG using Jackson's #JsonIdentityInfo)
Possible solutions:
The only way I could think of that would handle all of these issues would be to create a whole bunch of "resource" classes that would have constructors that took the needed entities to construct the resource and put necessary getters and setters for that resource on it. Is that overkill?
To solve 2, 3, 4 , and 5 I could just do some pre and post processing on the actual entity before sending it to Jackson to serialize or deserialize my pojo into JSON, but that doesn't address issue 1.
These are all problems I would think others would have come across and I am curious what solutions other people of come up with. (I am currently using JPA 2, Spring MVC, Jackson, and Spring-Data but open to other technologies)

With a combination of JAX_RS 1.1 and Jackson/GSON you can expose JPA entities directly as REST resources, but you will run into a myriad of problems.
DTOs i.e. projections onto the JPA entities are the way to go. It would allow you to separate the resource representation concerns of REST from the transactional concerns of JPA. You get to explicitly define the nature of the representations. You can control the amount of data that appears in the representation, including the depth of the object graph to be traversed, if you design your DTOs/projections carefully. You may need to create multiple DTOs/projections for the same JPA entity for the different resources in which the entity may need to be represented differently.
Besides, in my experience using annotations like #JsonIgnore and #JsonIdentityInfo on JPA entities doesnt exactly lend to more usable resource representations. You may eventually run into trouble when merging the objects back into the persistence context (because of ignored properties), or your clients may be unable to consume the resource representations, since object references as a scheme may not be understood. Most JavaScript clients will usually have trouble consuming object references produced by the #JsonidentityInfo annotation, due to the lack of standardization here.
There are other additional aspects that would be possible through DTOs/projections. JPA #EmbeddedIds do not fit naturally into REST resource representations. Some advocate using the JAX-RS #MatrixParam annotation to identify the resource uniquely in the resource URIs, but this does not work out of the box for most clients. Matrix parameters are after all only a design note, and not a standard (yet). With a DTO/projection, you can serve out the resource representation against a computed Id (could be a combination of the constituent keys).
Note: I currently work on the JBoss Forge plugin for REST where some or all of these issues exist and would be fixed in some future release via the generation of DTOs.

I agree with the other answers that DTOs are the way to go. They solve many problems:
Separation of layers and clean code. One day you may need to expose the data model using a different format (eg. XML) or interface (eg. non web-service based). Keeping all configuration (such as #JsonIgnore, #JsonidentityInfo) for each interface/format in domain model would make is really messy. DTOs separate the concerns. They can contain all the configuration required by your external interface (web-service) without involving changes in domain model, which can stay web-service and format agnostic.
Security - you easily control what is exposed to the client and what the client is allowed to modify.
Performance - you easily control what is sent to the client.
Issues such as (circular) entity references, lazily-loaded collections are also resolved explicitly and knowingly by you on converting to DTO.

Given your constraints, there looks to be no other solution than Data Transfer Objects - yes, it's occurring frequently enough that people named this pattern...

If you application is completely CRUDish then the way to go is definitely Spring Data REST in which you absolutely do not need DTOs. If it's more complicated than that you will be safer with DTOs securing the application layer. But do not attempt to encapsulate DTOs inside the controller layer. They belong to a service layer cause the mapping is also the part of logic (what you let in the application and what you let out of it). This way the application layer stays hermetic. Of course in most cases it can be the mix of those two.

what is a good pattern for converting between hibernate entities and data transfer objects?

I have had similar questions and concerns as to how to convert between Hibernate entities and data transfer objects to be returned by a web service as are discussed in this question:
Is using data transfer objects in ejb3 considered best practice
One of the factors mentioned here is that if the domain model changes, a set of DTOs will protect consumers in the case of a web service.
Even though it seems like it will add a substantial amount of code to my project, this reasoning seems sound.
Is there a good design pattern that I can use to convert a Hibernate entity (which implements an interface) to a DTO that implements the same interface?
So assuming both of the following implement 'Book', I would need to convert a BookEntity.class to a BookDTO.class so that I can let JAXB serialize and return.
Again, this whole prospect seems dubious to me, but if there are good patterns out there for helping to deal with this conversion, I would love to get some insight.
Is there perhaps some interesting way to convert via reflection? Or a 'builder' pattern that I'm not thinking of?
Should I just ignore the DTO pattern and pass entities around?

Should I just ignore the DTO pattern
and pass entities around?
My preference is usually "yes". I don't like the idea of parallel hierarchies created just for the sake of architectural or layer purity.
The original reason for the DTO pattern was excessive chattiness in EJB 1.0 and 2.0 apps when passing entity EJBs to the view tier. The solution was to put the entity bean state into a DTO.
Another reason that's usually given for creating DTOs is to prohibit modification by the view layer. DTOs are immutable objects in that case, with no behavior. They do nothing but ferry data to the view layer.
I would argue that DTO is a Core J2EE pattern that's become an anti-pattern.
I realize that some people would disagree. I'm simply offering my opinion. It's not the only way to do it, nor necessarily the "right" way. It's my preference.

There needs to be a contrarian view amongst all the jolly kicking of the DTO.
tl;dr - It is sometimes still useful.
The advantage of the DTO is that you don't have to add a zillion annotations to your domain classes.
You start with #Entity. Not so bad. But then you need JAXB so you add #XMLElement etc - and then you need JSON so you add things like #JsonManagedReference for Jackson to do the right thing with relationships then you add etc. etc. etc. ad infinitum.
Pretty soon your POJO ain't so plain any more. Read about "domain driven design" sometime.
In addition you can "filter" some properties that you don't want the view to know about.

We should not forget that entity objects are not easy to handle when they are in managed state. This makes their passing to GUI forms problematic. To be more precise, child objects are handled eagerly. This cannot be done out of session, cousing exceptions. So, they either have to be evicted (detached) from the entity manager of they have to be converted to appropriate DTOs. Unless of cource there is a pattern, which I am not aware of, that I would be very glad to know.

For quickly create a "look-alike" DTO, without a bunch of duplicate get/set code, you can use BeanUtils.copyProperties. That function help you quickly copy the data from DAO to DTO class. Just remember that there are more than one common libraries support BeanUtils.copyProperties, but their syntax are not the same.

I know this is an old question, but thought I would add an answer offering a framework to help in case someone else is tackling this problem.
Our project has JAXB annotated POJOs that are separate from the JPA annotated POJOs. Our team was debating how best to move data between the two objects (actually data structures).
Here is an option for people to consider:
We found and are experimenting with Dozer which handles (1) same name, (2) XML mapping and (3) custom conversions as ways to copy data between two POJOs.
It has been very easy to use so far.