We Keep Coding

Microservices Restful API - DTOs or not?

REST API - DTOs or not?
I would like to re-ask this question in Microservices' context. Here is the quote from original question.
I am currently creating a REST-API for a project and have been reading
article upon article about best practices. Many seem to be against
DTOs and simply just expose the domain model, while others seem to
think DTOs (or User Models or whatever you want to call it) are bad
practice. Personally, I thought that this article made a lot of sense.
However, I also understand the drawbacks of DTOs with all the extra
mapping code, domain models that might be 100% identical to their
DTO-counterpart and so on.
Now, My question
I am more aligned towards using one Object through all the layers of my application (In other words, just expose Domain Object rather than creating DTO and manually copying over each fields). And the differences in my Rest contract vs domain object can be addressed using Jackson annotations like #JsonIgnore or #JsonProperty(access = Access.WRITE_ONLY) or #JsonView etc). Or if there is one or two fields that needs a transformation which cannot be done using Jackson Annotation, then I will write custom logic to handle just that (Trust me, I haven't come across this scenario not even once in my 5+ years long journey in Rest services)
I would like to know if I am missing any real bad effects for not copying the Domain to DTO

I would vote for using DTOs and here is why:
Different requests (events) and your DB entities. Often it happens that your requests/responses different from what you have in the domain model. Especially it makes sense in microservice architecture, where you have a lot of events coming from other microservices. For instance, you have Order entity, but the event you get from another microservice is OrderItemAdded. Even if half of the events (or requests) are the same as entities it still does make sense to have a DTOs for all of them in order to avoid a mess.
Coupling between DB schema and API you expose. When using entities you basically expose how you model your DB in a particular microservice. In MySQL you probably would want to have your entities to have relations, they will be pretty massive in terms of composition. In other types of DBs, you would have flat entities without lots of inner objects. This means that if you use entities to expose your API and want to change your DB from let's say MySQL to Cassandra - you'll need to change your API as well which is obviously a bad thing to have.
Consumer Driven Contracts. Probably this is related to the previous bullet, but DTOs makes it easier to make sure that communication between microservices is not broken whilst their evolution. Because contracts and DB are not coupled this is just easier to test.
Aggregation. Sometimes you need to return more than you have in one single DB entity. In this case, your DTO will be just an aggregator.
Performance. Microservices implies a lot of data transferring over the network, which may cost you issues with performance. If clients of your microservice need less data than you store in DB - you should provide them less data. Again - just make a DTO and your network load will be decreased.
Forget about LazyInitializationException. DTOs doesn't have any lazy loading and proxying as opposed to domain entities managed by your ORM.
DTO layer is not that hard to support with right tools. Usually, there is a problem when mapping entities to DTOs and backwards - you need to set right fields manually each time you want to make a conversion. It's easy to forget about setting the mapping when adding new fields to the entity and to the DTO, but fortunately, there are a lot of tools that can do this task for you. For instance, we used to have MapStruct on our project - it can generate conversion for you automatically and in compile time.

The Pros of Just exposing Domain Objects
The less code you write, the less bugs you produce.
despite of having extensive (arguable) test cases in our code base, I have came across bugs due to missed/wrong copying of fields from domain to DTO or viceversa.
Maintainability - Less boiler plate code.
If I have to add a new attribute, I don't have to add in Domain, DTO, Mapper and the testcases, of course. Don't tell me that this can be achieved using a reflection beanCopy utils, it defeats the whole purpose.
Lombok, Groovy, Kotlin I know, but it will save me only getter setter headache.
DRY
Performance
I know this falls under the category of "premature performance optimization is the root of all evil". But still this will save some CPU cycles for not having to create (and later garbage collect) one more Object (at the very least) per request
Cons
DTOs will give you more flexibility in the long run
If only I ever need that flexibility. At least, whatever I came across so far are CRUD operations over http which I can manage using couple of #JsonIgnores. Or if there is one or two fields that needs a transformation which cannot be done using Jackson Annotation, As I said earlier, I can write custom logic to handle just that.
Domain Objects getting bloated with Annotations.
This is a valid concern. If I use JPA or MyBatis as my persistent framework, domain object might have those annotations, then there will be Jackson annotations too. In my case, this is not much applicable though, I am using Spring boot and I can get away by using application-wide properties like mybatis.configuration.map-underscore-to-camel-case: true , spring.jackson.property-naming-strategy: SNAKE_CASE
Short story, at least in my case, cons doesn't outweigh the pros, so it doesn't make any sense to repeat myself by having a new POJO as DTO. Less code, less chances of bugs. So, going ahead with exposing the Domain object and not having a separate "view" object.
Disclaimer: This may or may not be applicable in your use case. This observation is per my usecase (basically a CRUD api having 15ish endpoints)

The decision is a much simpler one in case you use CQRS because:
for the write side you use Commands that are already DTOs; Aggregates - the rich behavior objects in your domain layer - are not exposed/queried so there is no problem there.
for the read side, because you use a thin layer, the objects fetched from the persistence should be already DTOs. There should be no mapping problem because you can have a readmodel for every use case. In worst case you can use something like GraphQL to select only the fields you need.
If you do not split the read from write then the decision is harder because there are tradeoffs in both solutions.

How to design layer architecture for web application?

I have design issue when implement 1 simple web application.
I use struts2 web controller, spring’s IOC and Hibernate as persist layer.
Because this web application is very simple at begging. So I only have 2 layers:
1 DAO layer which used to access database. Almost every table have related DAO.
2 Action layer. User struts2.
I am satisfy with this architecture because can quickly implement my web application.
As project become bigger, I found the action layer become big and complex, and very hard to re-use.
I try to create service layer, to solve complex business logic is good, but my application still have a lot of simply logic. E.g: Load 1 object, save 1 object, and get collection by some condition and display it to webpage. If give each simple DB access method have corresponding service method. Still cost a lot of effort. How can solve this issue?
And I think, if service layer existing, direct call DAO layer still not good design for my application.
Is any good solution for this kind of small web application?

When planing the different layers in a web application it is good practice to explicitly protect attributes and associations in your model from being manipulated without providing an identity context.
This is something that should neither be done in the DAO layer nor in the Controller. You should wrap your DAO layer in a service layer and have the controller only talk to the services not the DAO directly.
Protecting your model against unwanted manipulation means that you for instance adapt the amount of information passed in a data structure between Controller and Service to the actual operation that you want to perform.
For instance: adding or removing an element from a collection is an explicit operation in the service, it does not happen implicitly by manipulating a collection as a member of a DAO object and passing that DAO back into the service.
Instead your service may look like this:
+ getAllCompanies(): CompanyType[*]
+ getAllEmployeesOfCompany(c: CompanyType) : EmployeeType[*]
+ addEmployeeToCompany(e: EmployeeType, c: CompanyType)
+ removeEmployeeFromCompany(e: EmployeeType, c: CompanyType)
The additional benefit of such an architecture is that the service layer serves as boundary for your transactions. Using the methods of your controller as boundary for your transactions is in fact a very bad habbit. You could even call it an anti-pattern. Why? Because for instance it would mean that when your client hangs up it would roll back your transaction. That is clearly unwanted in 99% of the cases.

As #mwhs commented, Apache Isis provides plenty of guidance on layering your app. To figure out whether it fits your requirements, you could run through this tutorial I presented at a recent conference.

DAO and Service layer design

I am developing web application with Java EE 6. In order to minimize calls to database will it be a good idea to have classes:
Data access class (DAO) will call only basic methods getAllClients, getAllProducts, getAllOrders, delete, update methods - CRUD methods.
Service class which will call CRUD methods but in addition filter methods e.g. findClientByName, findProuctByType, findProductByYear, findOrderFullyPaid/NotPaid etc... which will be based on basic DAO methods.
Thank you

In my experience (albeit, limited) DAO classes tend to have all the possible database operations which the application is allowed to perform. So in your case, it will have methods such as getAllClients() and getClientByName(String name), etc.
Getting all the users in your DAO and iterating all over them until you find the one you need will result in unneeded waste of computational time and memory consumption.
If you want to reduce the amount of times that your database is hit you could, maybe, implement some caching mechanism. An ORM framework such as Hibernate should be able to provide what you need as shown here.
EDIT:
As per your comment question, no, your service will not be made redundant. What one does is to usually use a Service layer to expose the DAO functionalities. This will, basically, not make the DAO visible from the from front end of your application. It usually also allows for extra methods, such as, for instance, public String getUserFormatted(String userName). This will make use of the getUserByName function offered by the DAO but provide some extra functionality.
The Service layer will also make itself useful should there be a change in specification and you now also need a web service to interface with your application. Having a service layer in between will allow the web service to query the DAO through the Service layer.
So basically, the DAO layer will still worry about the database stuff (CRUD Operations) while the service will adapt the data returned by the DAO without exposing the DAO.

It's hard to say without more information, but I think it's probably a good idea to leverage your database more than with just CRUD operations. Databases are good at searching, provided you configure them correctly, so IMHO it's a good idea to let your database handle the searching in your find methods for you. This means that your find methods would probably go in your DAOs...
It's good to think about/be aware of the implications of DB access on performance, but don't go overboard. Also, your approach implies that since your services are going to be doing the filtering, you are going to load a large amount of DB data into your application, which is a bad idea. The bottom line is you should use your RDBMS as it is intended to be used, and worry about performance due to over-access when you can show its a problem. I doubt you will run into that scenario.

I would say that you're better off having your DAO be more fine grained than you've specified.
I'd suggest putting findClientByName, findProuctByType, findProductByYear, findOrderFullyPaid/NotPaid on your DAO as well in some way because your database will most likely be better at filtering and sorting data than your in memory code.
Imagine you have 10 years of data and you call findProductsByYear on your service class and it then calls getAllProducts and then throws away 9 years of data in memory. You're far better off getting your database to only return you the year you are interested in.

Yes, this is the right way to do it.
The service will own the transactions. You should write these as POJOs; that way you can expose them as SOAO or REST web services, EJBs, or anything else that you want later on.

What pattern fits between a façade and a DAO?

I'm in the process of designing part of my companies architecture for its Java EE web applications. I'm pretty clear on the reasons to use a façade and one or more DAOs. The problem I have is this:
There will be some logic that definitely belongs in the integration tier because it's all about keeping the data model consistent. Except the logic goes beyond simply maintaining referential integrity and other 'raw' persistence tasks which will be handled by JPA and Hibernate. I don't class this as business logic because it's separate to any business function. However, my understanding is that a DAO should only implement the logic required to access and persist objects to the data source.
My conclusion is that I need a 'business object'-like pattern which is appropriate for the integration tier. I've looked around and the closest thing I have found (yet still not quite right to my mind) is the Sun Transfer Object Assembler pattern.
Either there's a gap in my understanding of Java EE or there is a pattern out there that will fit.

maybe a mediator is what you want:
Define an object that encapsulates how a set of objects interact. Mediator promotes loose coupling by keeping objects from referring to each other explicitly, and it lets you vary their interaction independently.
then you can use a DaoMediator in order to coordinate two or more DAOs

It sounds to me like you may be missing a controller, and consequently may need the MVC pattern. The controller will look after the DAOs and present a consistent view (don't think in terms of GUIs, burt rather some client-facing interface). When modifications are made via this view, then the controller coordinates the changes to the model via the DAO. I suspect that your facade objects may be the view in this scenario.
Having said this, I wouldn't worry too much about identifying particular patterns. You often find that taking into account all your requirements and separating concerns where applicable, that you end up implementing a particular pattern and only identify it as such after the fact.

Have you considered using aggregates from Domain Driven Design?
I'm a student of DDD myself and it seems the business logic you're trying to design could be accomplished by richer POJO-like domain models. You'd have each
domain object to be responsible for its aggregate objects, and also including any logic concerning that business concept; that said, your integration layer would coordinate those rich objects but would refrain from having real logic per se (i.e several conditional logic).
Perhaps the pattern you're trying to find is actually a step into richer domain objects?

I think it's a DataMapper (or Adapter) pattern between your DAL and your Business Layer, but without a more concrete understanding i couldn't be sure.

What is driving the requirement for consistency between the DAOs? If there is some business assumption that is dictating the relationship. For example, you might have an invoice type that when it is 'Capital' then we have to make sure several other objects are in the right state or have the right set of values. This is definitely outside the realm of the data-layer.
I wouldn't try to hard to find the perfect pattern for this case. You need some sort of coordinating class though, a mediator or controller of some sort.

Domain driven design and transactions in Spring environment

I used to design my application around anemic domain model, so I had many repository object, which were injected to the big, fat, transaction-aware service layer. This pattern is called Transaction script. It's not considered a good practice since it leads to the procedural code, so I wanted to move forward to the domain driven design.
After reading couple of articles on the web, listening to the Chris Richardson's talk on Parleys and reading DDD chapters of the POJOs in Action, I think I got the big picture.
Problem is, that I don't know, how to organize transactions in my application. Chis Richardson in his book states:
The presentation tier handles HTTP requests from the user’s browser by calling
the domain model either directly or indirectly via a façade, which as I
described in the previous chapter is either a POJO or an EJB.
Good so far, but Srini Penchikala on InfoQ article states:
Some developers prefer managing the transactions in the DAO classes which is a poor design. This results in too fine-grained transaction control which doesn't give the flexibility of managing the use cases where the transactions span multiple domain objects. Service classes should handle transactions; this way even if the transaction spans multiple domain objects, the service class can manage the transaction since in most of the use cases the Service class handles the control flow.
Ok, so if I understand this correctly, repository classes should not be transactional, service layer (which is now much thinner) is transactional (as it used to be in Transaction script pattern). But what if domain objects are called by presentation layer directly? Does it mean, that my domain object should have transactional behavior? And how to implement it in Spring or EJB environment?
This seems kind of weird to me, so I'd be happy if somebody would clarify that. Thank you.

My personal take on applying DDD with Spring and Hibernate, so far, is to have a stateless transactional service layer and access the domain objects through that. So the way I'm doing it the domain model does not know about transactions at all, that is handled entirely by the services.
There is an example application you might find helpful to take a look at. It looks like Eric Evans was involved in creating it.

See this extremely useful blog-post. It explains how to achieve smooth DDD while not loosing Spring's and JPA's capabilities. It is centered around the #Configurable annotation.
My opinion on these matters is a bit non-popular. Anemic data model is actually not wrong. Instead of having one object with data+operations, you have two objects - one with data and one with operations. You can view them as one object - i.e. satisfying DDD, but for the sake of easier use they are physically separated. Logically they are the same.
Yes, this breaks encapsulation, but it doesn't make you use some 'magic' (aop + java agent) in order to achieve your goals.
As for the transactions - there is something called Transaction propagation. Spring supports it with #Transactional(propagation=Propagation.REQUIRED).
See this, point 9.5.7. In case you want your transactions to span multiple methods (of multiple objects) you can change the propagation attribute accordingly.
You can also use #Transactional in your service layer, where appropriate, but this might introduce a lot of boilerplace service-classes in cases when you want to use simple, single-step operations like "save".

I think one easy way to get started with DDD and Spring and have good examples of how to deal with transactions is by looking at one of the sample apps that get shipped with Spring Roo. Roo churns out code that follows the DDD principles. It relies heavily on AspectJ. I suspect it was implemented building on the ideas put forward (back in 2006) by one of SpringSource's heavyweights, Ramnivas Laddad, in this talk.

Unnfortunately I haven't make myself familiar with Spring but I will give feedback about your DDD understanding. Reading your description of transaction it is obvious that you haven't understand DDD especially Aggregate, Aggregate Root, and Repository concept.
In DDD transaction can't span across aggregates so one Aggregate will always have one transaction.