We are developing a java program which has lots of independent components(let's say A, B, C, D). We use Hibernate framework for database mapping. Some of them working on the same JVM process and some of them are working the different JVM process. All of these components have REST endpoints and has own entity map file. Each of these components talks to each other over the TCP. There is some business logic which required a few of these components work together. And the problem starts from here. Let's say component A is called. A start a new transaction and change some data on the database then calls the component B. When component B accepted the request it opens a new transaction and changes some information on the database and commit. After finishing its work return the result to the component A. Component A start again doing some operations but failed and rollback all changes what it has done and returns the response to the back. But the changes which component B did still there. This is a little illustration of our program. What we need what is the best approaches to handle this?
Somehow we share the transaction between all the components. So when the component gets request first check is there any active transaction for this request.
The second option is using the one component to all entity and all others use this. When the request comes we open a transaction, all components do their works and when we return the response we check if there is no any exception we commit otherwise rollback.
Is there any other way? Or what are the best practices using transaction between multiple components architecture system? What are the pros and cons using a single component to keep all entity information at one point? Any post, blogs, documentation, tutorial or book are acceptable. Or if it is ok you can share your experiments.
You are looking to implement 'Distributed Transactions'.
I recommend Spring if you have a Java application as it supports distributed transactions:
https://www.javaworld.com/article/2077963/open-source-tools/distributed-transactions-in-spring--with-and-without-xa.html
Related
We are currently using axon framework with hikaricp as data source pooling system. We are facing pool exhausting issues from time to time and we have a theory:
To update our read models we use the command bus to send UpdateEntityViewCommand.
As the command bus starts a transaction using the primary transaction manager (the write one) it acquires a connection from the write pool
On the handler, we open an inner transaction using a connection from the read pool, thus blocking the outer one.
This seems to exhaust the pools under some conditions. The question is: should we stop using the command bus to update our read models? Is appropiate to have two buses( one for write and one for read?
Thanks in advance
Axon's intent with the separation of Commands, Events, and Queries, is in general to support CQRS within your system. This means you would introduce dedicated Command Models and Query Models, which respectively only receive Command and Query messages.
The Query Models are typically also referred to as Projections or Read Models.
It is this combination of ideas Axon is normally used for which makes your question a bit vague. Or, perchance, you are using it differently than normally intended.
That Axon supports CQRS through this approach, does not necessitate that you do CQRS within your set up as well, by the way.
At any note, it would be good too (as also stated in the comments):
Include code snippets of the message handlers when these prove useful
Give some deeper insight in your configuration
Whether you aim to use the CQRS pattern in your application, yes or no
Completely separate from this, I can give some guidance on how Axon deals with transactions. Every message inside an Axon application will start a so-called "Unit of Work". It is the UnitOfWork (UoW for short) that will start a transaction. This means that no matter if you use commands, events or queries, Axon will have started that transaction for you already.
Taking this a step further, this also means that whatever you do inside a message handling function (thus the #CommandHandler, #EventHandler and #QueryHandler annotated methods) will always already have an active transaction running. That for example defines that you do not have to include your own transaction management inside a message handling function, with for example the #Transactional annotation.
Concluding, I am guessing you might be mixing some concepts. This can obviously happen, so no worries there; that's what SO is for. Next to this though, you thus do not have to start your own transaction inside any message handling function, as you already have an active transaction at that point in time.
I have a code in my business layer that updates data on database and also in a rest service.
The question is that if it doesn't fail data must be save in both places and, in other hand, if it fails it must to rollback in database and send another requisition to rest api.
So, what I'm looking for is a way to use transaction management of EJB to also orchestrait calls to api. When in commit time, send a set requisition to api and, when in rollback time, send delete requisition to api.
In fact I need to maintain consistency and make both places syncronous.
I have read about UserTransactions and managedbeans but I don't have a clue about what is the best way to do that.
You can use regular distributed transactions, depending on your infrastructure and participants. This might be possible e.g. if all participants are EJBs and the data stores are capable to handle distributed transactions.
This won't work with loosely coupled componentes, and your setup looks like this.
I do not recommend to create your own distributed transaction protocol. Regarding the edge and corner cases, you will probably not end up with consistent data in the end.
I would suggest to think about using event sourcing and eventually consistency for things like that. For example, you could emit an event (command) for writing data. If your "rollback" is needed, you can emit an event (command) to delete the date written before. After all events are processed, the data is consistent.
Some interesting links might be:
Martin Fowler - Event Sourcing
Martin Fowler - CQRS
Apache Kafka
I'm writing an application (a CMS) using JPA/Hibernate, and in a single UI i have multiple components that might show the same entity (each one will show only a portion). I also have multiple UIs, in multiple sessions for multiple users.
Some of these components can also edit the entity, and all others component should always show the up-to-date entity.
A gross way would be a periodic refresh, but this is laggy and heavy, so i come up with a synchronization mechanism for jpa.
Using an interceptor (the hibernate one, since the jpa one is dumb) i can listen all the transactions, all the new/updated/removed entity and send notifications to every component interested in.
I also can catch derived transactions, meaning that if a component, responding to a notification, modify in any way the entity (opening a new transaction) i can resend the notification (only a delta).
This is becouse a component may modify the entity in a such way that another component may need to modify again it. (just a stupid example: a component set the birth date, and another re-calculate the age)
ps. The notifications are dispatched only after the transaction commit. This is mainly becouse
The section 3.5 of JPA specification states:
“In general, the lifecycle method of a portable application should not invoke EntityManager or Query operations, access other entity instances, or modify relationships within the same persistence context. A lifecycle callback method may modify the non-relationship state of the entity on which it is invoked.”
so the listeners would be useless if notified inside the transaction.
And (to grin and bear it) to group up modified entities and notify them all together.
This notification mechanism is growing complex, and i was wondering:
Why jpa has not such mechanism?
I'm inventing something strange?
How "real applications" solves this problem?
Describe please a typical lifecycle of a Hibernate object (that maps to a db table) in a web app.
Suppose, you create a new instance of an object and persist in the db.
But during the app lifetime you'll be working on a detached object and finally
you need to update it in the database, for example on exit.
How does it look like with hibernate and spring?
p.s. Can transactions and sessions live between servlet transitions? So that we opened 1 session and use it in all servlets without a need to reopen it?
I'll try to give a descriptive example.
Suppose, when the app starts, the log record is created. this can be done at once,
Log log = new Log(...) and then something like save(log) -- log corresponds to a table LOG.
then, as the application processes user inputs and keeps going, new data is being accumulated.
and after the second step we could add something to a log object, a collection for example:
// now we have a tracking of what user chosen: Set thisUserChoice,
// so we can update the persistent object, we have new data now !
// log.userChoices = thisUserChoice.
Here occurs the nature of my question. How are we supposed to deal with it, if we want to
update the database whenever new data is gotten from a user?
In a relational model we can work with a row id, so we could get this record and update some other data of the row.
In Hibernate we are also able to load a object by its id.
But is IT THE WAY TO GO? IS ANYTHING BETTER?
You could do everything in a single session. But that's like doing everything in a single class. It could make sense from a beginner's point of view, but nobody does it like that in practice.
In a web app, you can normally expect to have several threads running at once, each dealing with a different user. Each thread would typically have a separate session, and the session would only have managed instances of the objects that were actually needed by that user. It's not that you can completely ignore concurrency in your own code, but it's useful to have hibernate's help. If you were to do everything with one session, you would have to do all the concurrency management yourself.
Hibernate can also manage the concurrency if you have multiple application servers talking to a single database. The separate JVMs can't possibly share the same session in this case...
The lifecycle is described in the hibernate documentation (which I'm sure you've seen).
Whenever a request comes from the web client to the server, the first thing you should do is load the relevant objects (see section 10.3) so that you have persistent, not detached entities to deal with. Then, you do whatever operations are required. When the session closes (ie. when the server returns the response to the client), it will write any updates to the database. Or, if your operation involves creating new entities, you'll have to create transient ones (with new) and then call persist() or save() (see section 10.2). That will result in a managed entity -- you can make more changes to it, and hibernate will record those changes when the session closes.
I try to avoid using detached objects. But if I have to (perhaps they're stored in the user's session), then whenever they might need to be saved to the database, you'll have to use update() (see section 10.6). This converts it into a managed object, and so the session will save any changes to the database when it's closed.
Spring makes it very easy to generate a new session for each request. You would normally tell Spring to create a sessionFactory, and then every request will be given its own session. Search for "spring hibernate tutorial" and you'll find several examples.
http://scbcd.blogspot.com/2007/01/hibernate-persistence-lifecycle.html This explains transient, persistent objects.
Also have a look at the Lifecycle interface to know what hibernate does (and it provides hooks at all stages for user to do something)
I'm asking this question given my chosen development frameworks of JPA (Hibernate implementation of), Spring, and <insert MVC framework here - Struts 1, Struts 2, Spring MVC, Stripes...>.
I've been thinking a bit about relationships in my entity layer - for example I have an order entity that has many order lines. I've set up my app so that it eagerly loads the order lines for every order. Do you think this is a lazy way to get around the lazy initialization problems that I would come across if I was to set the fetch strategy to false?
The way I see it, I have the following alternatives when retrieving entities and their associations:
Use the Open Session In View pattern to create the session on each request and commit the transaction before returning the response.
Implement a DTO (Data Transfer Object) layer such that every DAO query I execute returns the correctly initialized DTO for my purposes. I don't really like this option much because in my experience I've found that it creates a lot of boilerplate copying code and becomes messy to maintain.
Don't map any associations in JPA so that every query I execute returns only the entities I'm interested in - this will probably require me to have DTOs anyway and will be a pain to maintain and I think defeats the purpose of having an ORM in the first place.
Eagerly fetch all (or most associations) - in the example above, always fetch all order lines when I retrieve an order.
So my question is, when and under what circumstances would you use which of these options? Do you always stick with one way of doing it?
I would ask a colleague but I think that if I even mentioned the term 'Open Session in View' I would be greeted with blank stares :( What I'm really looking for here is some advice from a senior or very experienced developer.
Thanks guys!
Open Session in View has some problems.
For example, if the transaction fails, you might know it too late at commit time, once you are nearly done rendering your page (possibly the response already commited, so you can't change the page !) ... If you had know that error before, you would have followed a different flow and ended up rendering a different page...
Other example, reading data on-demand might turn to many "N+1 select" problems, that kill your performance.
Many projects use the following path:
Maintain transactions at the business layer ; load at that point everything you are supposed to need.
Presentation layer runs the risk of LazyExceptions : each is considered a programming error, caught during tests, and corrected by loading more data in the business layer (you have the opportunity to do it efficiently, avoiding "N+1 select" problems).
To avoid creating extra classes for DTOs, you can load the data inside the entity objects themselves. This is the whole point of the POJO approach (uses by modern data-access layers, and even integration technologies like Spring).
I've successfully solved all my lazy initialization problems with Open Session In View -pattern (ie. the Spring implementation). The technologies I used were the exact same as you have.
Using this pattern allows me to fully map the entity relationships and not worry about fetching child entities in the dao. Mostly. In 90% of the cases the pattern solves the lazy initialization needs in the view. In some cases you'll have to "manually" initialize relationships. These cases were rare and always involved very very complex mappings in my case.
When using Open Entity Manager In View pattern it's important to define the entity relationships and especially propagation and transactional settings correctly. If these are not configured properly, there will be errors related to closed sessions when some entity is lazily initialized in the view and it fails due to the session having been closed already.
I definately would go with option 1. Option 2 might be needed sometimes, but I see absolutely no reason to use option 3. Option 4 is also a no no. Eagerly fetching everything kills the performance of any view that needs to list just a few properties of some parent entities (orders in tis case).
N+1 Selects
During development there will be N+1 selects as a result of initializing some relationships in the view. But this is not a reason to discard the pattern. Just fix these problems as they arise and before delivering the code to production. It's as easy to fix these problems with OEMIV pattern as it's with any other pattern: add the proper dao or service methods, fix the controller to call a different finder method, maybe add a view to the database etc.
I have successfully used the Open-Session-in-View pattern on a project. However, I recently read in "Spring In Practice" of an interesting potential problem with non-repeatable reads if you manage your transactions at a lower layer while keeping the Hibernate session open in the view layer.
We managed most of our transactions in the service layer, but kept the hibernate session open in the view layer. This meant that lazy reads in the view were resulting in separate read transactions.
We managed our transactions in our service layer to minimize transaction duration. For instance, some of our service calls resulted in both a database transaction and a web service call to an external service. We did not want our transaction to be open while waiting for a web service call to respond.
As our system never went into production, I am not sure if there were any real problems with it, but I suspect that there was the potential for the view to attempt to lazily load an object that has been deleted by someone else.
There are some benefits of DTO approach though. You have to think beforehand what information you need. In some cases this will prevent you from generating n+1 select statements. It helps also to see where to use eager fetching and/or optimized views.
I'll also throw my weight behind the Open-Session-in-View pattern, having been in the exact same boat before.
I work with Stripes without spring, and have created a manual filter before that tends to work well. Coding transaction logic on the backend turns messy really quick as you've mentioned. Eagerly fetching everything becomes TERRIBLE as you map more and more objects to each other.
One thing I want to add that you may not have come across is Stripersist and Stripernate - Stripersist being the more JPA flavor - auto-hydration filters that take a lot of the work off your shoulders.
With Stripersist you can say things like /appContextRoot/actions/view/3 and it will auto-hydrate the JPA Entity on the ActionBean with id of 3 before the event is executed.
Stripersist is in the stripes-stuff package on sourceforge. I now use this for all new projects, as it's clean and easily supports multiple datasources if necessary.
Does the Order and Order Lines compose a high volume of data? Do they take part in online processes where real-time response is required? If so, you might consider not using eager fetching - it does make a huge diference in performance. If the amount of data is small, there is no problem in eager fetching.
About using DTOs, it might be a viable implementation.
If your business layer is used internally by your own application (i.e a small web app and its business logic) it'd probably be best to use your own entities in your view with open session in view pattern since it's simpler.
If your entities are used by many applications (i.e a backend application providing a service in your corporation) it'd be interesting to use DTOs since you would not expose your model to your clients. Exposing it could mean you would have a harder time refactoring your model since it could mean breaking contracts with your clients. A DTO would make that easier since you have another layer of
abstraction. This can be a bit strange since EJB3 would theorically eliminate the need of DTOs.