We Keep Coding

Microservices Restful API - DTOs or not?

REST API - DTOs or not?
I would like to re-ask this question in Microservices' context. Here is the quote from original question.
I am currently creating a REST-API for a project and have been reading
article upon article about best practices. Many seem to be against
DTOs and simply just expose the domain model, while others seem to
think DTOs (or User Models or whatever you want to call it) are bad
practice. Personally, I thought that this article made a lot of sense.
However, I also understand the drawbacks of DTOs with all the extra
mapping code, domain models that might be 100% identical to their
DTO-counterpart and so on.
Now, My question
I am more aligned towards using one Object through all the layers of my application (In other words, just expose Domain Object rather than creating DTO and manually copying over each fields). And the differences in my Rest contract vs domain object can be addressed using Jackson annotations like #JsonIgnore or #JsonProperty(access = Access.WRITE_ONLY) or #JsonView etc). Or if there is one or two fields that needs a transformation which cannot be done using Jackson Annotation, then I will write custom logic to handle just that (Trust me, I haven't come across this scenario not even once in my 5+ years long journey in Rest services)
I would like to know if I am missing any real bad effects for not copying the Domain to DTO

I would vote for using DTOs and here is why:
Different requests (events) and your DB entities. Often it happens that your requests/responses different from what you have in the domain model. Especially it makes sense in microservice architecture, where you have a lot of events coming from other microservices. For instance, you have Order entity, but the event you get from another microservice is OrderItemAdded. Even if half of the events (or requests) are the same as entities it still does make sense to have a DTOs for all of them in order to avoid a mess.
Coupling between DB schema and API you expose. When using entities you basically expose how you model your DB in a particular microservice. In MySQL you probably would want to have your entities to have relations, they will be pretty massive in terms of composition. In other types of DBs, you would have flat entities without lots of inner objects. This means that if you use entities to expose your API and want to change your DB from let's say MySQL to Cassandra - you'll need to change your API as well which is obviously a bad thing to have.
Consumer Driven Contracts. Probably this is related to the previous bullet, but DTOs makes it easier to make sure that communication between microservices is not broken whilst their evolution. Because contracts and DB are not coupled this is just easier to test.
Aggregation. Sometimes you need to return more than you have in one single DB entity. In this case, your DTO will be just an aggregator.
Performance. Microservices implies a lot of data transferring over the network, which may cost you issues with performance. If clients of your microservice need less data than you store in DB - you should provide them less data. Again - just make a DTO and your network load will be decreased.
Forget about LazyInitializationException. DTOs doesn't have any lazy loading and proxying as opposed to domain entities managed by your ORM.
DTO layer is not that hard to support with right tools. Usually, there is a problem when mapping entities to DTOs and backwards - you need to set right fields manually each time you want to make a conversion. It's easy to forget about setting the mapping when adding new fields to the entity and to the DTO, but fortunately, there are a lot of tools that can do this task for you. For instance, we used to have MapStruct on our project - it can generate conversion for you automatically and in compile time.

The Pros of Just exposing Domain Objects
The less code you write, the less bugs you produce.
despite of having extensive (arguable) test cases in our code base, I have came across bugs due to missed/wrong copying of fields from domain to DTO or viceversa.
Maintainability - Less boiler plate code.
If I have to add a new attribute, I don't have to add in Domain, DTO, Mapper and the testcases, of course. Don't tell me that this can be achieved using a reflection beanCopy utils, it defeats the whole purpose.
Lombok, Groovy, Kotlin I know, but it will save me only getter setter headache.
DRY
Performance
I know this falls under the category of "premature performance optimization is the root of all evil". But still this will save some CPU cycles for not having to create (and later garbage collect) one more Object (at the very least) per request
Cons
DTOs will give you more flexibility in the long run
If only I ever need that flexibility. At least, whatever I came across so far are CRUD operations over http which I can manage using couple of #JsonIgnores. Or if there is one or two fields that needs a transformation which cannot be done using Jackson Annotation, As I said earlier, I can write custom logic to handle just that.
Domain Objects getting bloated with Annotations.
This is a valid concern. If I use JPA or MyBatis as my persistent framework, domain object might have those annotations, then there will be Jackson annotations too. In my case, this is not much applicable though, I am using Spring boot and I can get away by using application-wide properties like mybatis.configuration.map-underscore-to-camel-case: true , spring.jackson.property-naming-strategy: SNAKE_CASE
Short story, at least in my case, cons doesn't outweigh the pros, so it doesn't make any sense to repeat myself by having a new POJO as DTO. Less code, less chances of bugs. So, going ahead with exposing the Domain object and not having a separate "view" object.
Disclaimer: This may or may not be applicable in your use case. This observation is per my usecase (basically a CRUD api having 15ish endpoints)

The decision is a much simpler one in case you use CQRS because:
for the write side you use Commands that are already DTOs; Aggregates - the rich behavior objects in your domain layer - are not exposed/queried so there is no problem there.
for the read side, because you use a thin layer, the objects fetched from the persistence should be already DTOs. There should be no mapping problem because you can have a readmodel for every use case. In worst case you can use something like GraphQL to select only the fields you need.
If you do not split the read from write then the decision is harder because there are tradeoffs in both solutions.

what is a good pattern for converting between hibernate entities and data transfer objects?

I have had similar questions and concerns as to how to convert between Hibernate entities and data transfer objects to be returned by a web service as are discussed in this question:
Is using data transfer objects in ejb3 considered best practice
One of the factors mentioned here is that if the domain model changes, a set of DTOs will protect consumers in the case of a web service.
Even though it seems like it will add a substantial amount of code to my project, this reasoning seems sound.
Is there a good design pattern that I can use to convert a Hibernate entity (which implements an interface) to a DTO that implements the same interface?
So assuming both of the following implement 'Book', I would need to convert a BookEntity.class to a BookDTO.class so that I can let JAXB serialize and return.
Again, this whole prospect seems dubious to me, but if there are good patterns out there for helping to deal with this conversion, I would love to get some insight.
Is there perhaps some interesting way to convert via reflection? Or a 'builder' pattern that I'm not thinking of?
Should I just ignore the DTO pattern and pass entities around?

Should I just ignore the DTO pattern
and pass entities around?
My preference is usually "yes". I don't like the idea of parallel hierarchies created just for the sake of architectural or layer purity.
The original reason for the DTO pattern was excessive chattiness in EJB 1.0 and 2.0 apps when passing entity EJBs to the view tier. The solution was to put the entity bean state into a DTO.
Another reason that's usually given for creating DTOs is to prohibit modification by the view layer. DTOs are immutable objects in that case, with no behavior. They do nothing but ferry data to the view layer.
I would argue that DTO is a Core J2EE pattern that's become an anti-pattern.
I realize that some people would disagree. I'm simply offering my opinion. It's not the only way to do it, nor necessarily the "right" way. It's my preference.

There needs to be a contrarian view amongst all the jolly kicking of the DTO.
tl;dr - It is sometimes still useful.
The advantage of the DTO is that you don't have to add a zillion annotations to your domain classes.
You start with #Entity. Not so bad. But then you need JAXB so you add #XMLElement etc - and then you need JSON so you add things like #JsonManagedReference for Jackson to do the right thing with relationships then you add etc. etc. etc. ad infinitum.
Pretty soon your POJO ain't so plain any more. Read about "domain driven design" sometime.
In addition you can "filter" some properties that you don't want the view to know about.

We should not forget that entity objects are not easy to handle when they are in managed state. This makes their passing to GUI forms problematic. To be more precise, child objects are handled eagerly. This cannot be done out of session, cousing exceptions. So, they either have to be evicted (detached) from the entity manager of they have to be converted to appropriate DTOs. Unless of cource there is a pattern, which I am not aware of, that I would be very glad to know.

For quickly create a "look-alike" DTO, without a bunch of duplicate get/set code, you can use BeanUtils.copyProperties. That function help you quickly copy the data from DAO to DTO class. Just remember that there are more than one common libraries support BeanUtils.copyProperties, but their syntax are not the same.

I know this is an old question, but thought I would add an answer offering a framework to help in case someone else is tackling this problem.
Our project has JAXB annotated POJOs that are separate from the JPA annotated POJOs. Our team was debating how best to move data between the two objects (actually data structures).
Here is an option for people to consider:
We found and are experimenting with Dozer which handles (1) same name, (2) XML mapping and (3) custom conversions as ways to copy data between two POJOs.
It has been very easy to use so far.

How large should be Data Transfer Objects?

As I understand, Data Transfer Objects are used for different purposes, so let's bound the scope with the view layer in Java (JSF) -based web-applications (i.e. there are usually some entity-objects mapped on a DB, which can be also used in a Business-Logic layer, and some transfer objects used in a presentation layer).
So, I have some misunderstanding about how should well-designed DTOs look. Should I keep them as small as possible? Or should I try to pass with them as much information as possible and design them in such way that only some (different in different use-cases) part of the DTO fields is initialized at a time?
Should I consider using some OO principles (at least inheritance and composition) when designing DTOs or they should be as simple as only a number of primitive-type fields with their accessors?

DTOs, if at all different from the domain objects/entities, should be as big as needed - you should transfer exactly all data that you need.

DTOs in any language should be pretty light weight. Whether or not to use inheritance is a question only you could answer - it really depends on the business needs. Otherwise the DTO itself should include the basic get/set properties.
Generally these objects are pretty light weight, however it really depends on the data / properties you need. If your DTO has 1 property vs 50 properties, if you need 50 so be it. When it comes time to passing data to functions / methods the DTO saves your life from having to add all those extra parameters. You're essentially just passing one object.

DTOs should be as lightweight as possible, distinct from the business objects, and limited in scope (for example package level objects).
I say they should be separate from the business objects, contrary to Bozho's statement "if at all different from the domain objects", because a DTO will often need setters that users of the business object should not use.
I have, for example, a Person object and a PersonDTO... the DTO needs a setter for the person's name (first, last, etc.) but that is retrieved from an external data source and my application is not allowed to change it, so my business object "Person" shouldn't have a setter.

Spring MVC: should service layer be returning operation specific DTO's?

In my Spring MVC application I am using DTO in the presentation layer in order to encapsulate the domain model in the service layer. The DTO's are being used as the spring form backing objects.
hence my services look something like this:
userService.storeUser(NewUserRequestDTO req);
The service layer will translate DTO -> Domain object and do the rest of the work.
Now my problem is that when I want to retrieve a DTO from the service to perform say an Update or Display I can't seem to find a better way to do it then to have multiple methods for the lookup that return different DTO's like...
EditUserRequestDTO userService.loadUserForEdit(int id);
DisplayUserDTO userService.loadUserForDisplay(int id);
but something does not feel right about this approach. Perhaps the service should not return things like EditUserRequestDTO and the controller should be responsible of assembling a requestDTO from a dedicated form object and vice versa.
The reason do have separate DTO's is that DisplayUserDTO is strongly typed to be read only and also there are many properties of user that are entities from a lookup table in the db (like city and state) so the DisplayUserDTO would have the string description of the properties while the EditUserRequestDTO will have the id's that will back the select drop down lists in the forms.
What do you think?
thanks

I like the stripped down display objects. It's more efficient than building the whole domain object just to display a few fields of it. I have used a similar pattern with one difference. Instead of using an edit version of a DTO, I just used the domain object in the view. It significantly reduced the work of copying data back and forth between objects. I haven't decided if I want to do that now, since I'm using the annotations for JPA and the Bean Validation Framework and mixing the annotations looks messy. But I'm not fond of using DTOs for the sole purpose of keeping domain objects out of the MVC layer. It seems like a lot of work for not much benefit. Also, it might be useful to read Fowler's take on anemic objects. It may not apply exactly, but it's worth thinking about.
1st Edit: reply to below comment.
Yes, I like to use the actual domain objects for all the pages that operate on a single object at a time: edit, view, create, etc.
You said you are taking an existing object and copying the fields you need into a DTO and then passing the DTO as part of the model to your templating engine for a view page (or vice-versa for a create). What does that buy you? The ref to the DTO doesn't weigh any less than the ref to the full domain object, and you have all the extra attribute copying to do. There's no rule that says your templating engine has to use every method on your object.
I would use a small partial domain object if it improves efficiency (no relationship graphs to build), especially for the results of a search. But if the object already exists don't worry about how big or complex it is when you are sticking it in the model to render a page. It doesn't move the object around in memory. It doesn't cause the templating engine stress. It just accesses the methods it needs and ignores the rest.
2nd edit:
Good point. There are situations where you would want a limited set of properties available to the view (ie. different front-end and back-end developers). I should read more carefully before replying. If I were going to do what you want I would probably put separate methods on User (or whatever class) of the form forEdit() and forDisplay(). That way you could just get User from the service layer and tell User to give you the use limited copies of itself. I think maybe that's what I was reaching for with the anemic objects comment.

You should use a DTO and never an ORM in the MVC layer! There are a number of really good questions already asked on this, such as the following: Why should I isolate my domain entities from my presentation layer?
But to add to that question, you should separate them to help prevent the ORM being bound on a post as the potential is there for someone to add an extra field and cause all kinds of mayhem requiring unnecessary extra validation.

Why are data transfer objects (DTOs) an anti-pattern?

I've recently overheard people saying that data transfer objects (DTOs) are an anti-pattern.
Why? What are the alternatives?

Some projects have all data twice. Once as domain objects, and once as data transfer objects.
This duplication has a huge cost, so the architecture needs to get a huge benefit from this separation to be worth it.

DTOs are not an anti-pattern. When you're sending some data across the wire (say, to an web page in an Ajax call), you want to be sure that you conserve bandwidth by only sending data that the destination will use. Also, often it is convenient for the presentation layer to have the data in a slightly different format than a native business object.
I know this is a Java-oriented question, but in .NET languages anonymous types, serialization, and LINQ allow DTOs to be constructed on-the-fly, which reduces the setup and overhead of using them.

"DTO an AntiPattern in EJB 3.0" (original link currently offline) says:
The heavy weight nature of Entity
Beans in EJB specifications prior to
EJB 3.0, resulted in the usage of
design patterns like Data Transfer
Objects (DTO). DTOs became the
lightweight objects (which should have
been the entity beans themselves in
the first place), used for sending the
data across the tiers... now EJB 3.0
spec makes the Entity bean model same
as Plain old Java object (POJO). With
this new POJO model, you will no
longer need to create a DTO for each
entity or for a set of entities... If
you want to send the EJB 3.0 entities
across the tier make them just
implement java.io.Serialiazable

OO purists would say that DTO is anti-pattern because objects become data table representations instead of real domain objects.

I don't think DTOs are an anti-pattern per se, but there are antipatterns associated with the use of DTOs. Bill Dudney refers to DTO explosion as an example:
http://www.softwaresummit.com/2003/speakers/DudneyJ2EEAntiPatterns.pdf
There are also a number of abuses of DTOs mentioned here:
http://anirudhvyas.com/root/2008/04/19/abuses-of-dto-pattern-in-java-world/
They originated because of three tier systems (typically using EJB as technology) as a means to pass data between tiers. Most modern day Java systems based on frameworks such as Spring take a alternative simplified view using POJOs as domain objects (often annotated with JPA etc...) in a single tier... The use of DTOs here is unnecessary.

Some consider DTOs an anti-pattern due to their possible abuses. They're often used when they shouldn't be/don't need to be.
This article vaguely describes some abuses.

The question should not be "why", but "when".
Definitively it's anti-pattern when only result of using it is higher cost - run-time or maintenance. I worked on projects having hundreds of DTOs identical to database entity classes. Each time you wanted to add a single field you ad to add id like four times - to DTO, to entity, to conversion from DTO to domain classes or entities, the inverse conversion, ... You forgot some of the places and data got inconsistent.
It's not anti-pattern when you really need different representation of domain classes - flatter, richer, ...
Personally I start with a domain class and pass it around, with proper checks at the right places. I can annotate and/or add some "helper" classes to make mappings to database, to serialization formats like JSON or XML ... I can always split a class to two if I feel the need.
It's about your point of view - I prefer to look at a domain object as a single object playing various roles, instead of multiple objects created from each other. If the only role an object is to transport data, then it's DTO.

If you're building a distributed system, then DTOs are certainly not an anti pattern. Not everyone will develop in that sense, but if you have a (for example) Open Social app all running off JavaScript.
It will post a load of data to your API. This is then deserialized into some form of object, typically a DTO/Request object. This can then be validated to ensure the data entered is correct before being converted into a model object.
In my opinion, it's seen as an anti-pattern because it's mis-used. If you're not build a distributed system, chances are you don't need them.

DTO becomes a necessity and not an ANTI-PATTERN when you have all your domain objects load associated objects EAGERly.
If you don't make DTOs, you will have unnecessary transferred objects from your business layer to your client/web layer.
To limit overhead for this case, rather transfer DTOs.

I think the people mean it could be an anti-pattern if you implement all remote objects as DTOs. A DTO is merely just a set of attributes and if you have big objects you would always transfer all the attributes even if you do not need or use them. In the latter case prefer using a Proxy pattern.

The intention of a Data Transfer Object is to store data from different sources and then transfer it into a database (or Remote Facade) at once.
However, the DTO pattern violates the Single Responsibility Principle, since the DTO not only stores data, but also transfers it from or to the database/facade.
The need to separate data objects from business objects is not an antipattern, since it is probably required to separate the database layer anyway.
Instead of DTOs you should use the Aggregate and Repository Patterns, which separates the collection of objects (Aggregate) and the data transfer (Repository).
To transfer a group of objects you can use the Unit Of Work pattern, that holds a set of Repositories and a transaction context; in order to transfer each object in the aggregate separately within the transaction.