Summarize
Goal
I have an application that is written in Java using the Spring framework. There is a service that is being used as the handler for grabbing and releasing locks in the database (InnoDB). My goal is to be able to log the grabbing and releasing of the locks to create a lock history. For each lock interaction, I would like to know not only the name of the lock involved, but also where this request is coming from in the code (if possible, class name, method name, and line number).
My expected database entry will look something like this:
id
lock_name
clazz
method
line
lock_date
unlock_date
unlock_type
0
tb_member
MemberTools
createMember
123
2021-12-23 10:16:00
2021-12-23 10:16:01
COMMIT
1
tb_member
MemberTools
editMember
234
2021-12-23 10:16:01
2021-12-23 10:16:02
COMMIT
I would like to know if there is an easy way to obtain this given that I am using the Spring framework.
Describe
So far, I have tried two things:
Forcing the caller to pass a reference to itself or its current StackTraceElement (using Thread.currentThread().getStackTrace()[1]). This is not only extremely repetitive, but it also is prone to human error, as a developer might not realize that they need to pass in some reference to themselves.
Inside of the lock service, use the getStackTrace method and walk through the elements to find the "correct" one. This is made very hard by Spring and the fact that before a call actually reaches the inside of a class with the #Service annotation, the call stack is muddled by numbers of calls between proxies and generated classes and such. Unless there is a deterministic way to find the number of calls in between the Service and the caller, then this doesn't seem like a good way either.
I have referenced this stack overflow question while working, but these do not take into account the usage of the Spring framework.
Show
A reproducible example will look something like this. First, the structure:
root\
LockService.java
getLock()
MemberTools.java
createMember()
LockService.java:
#Service
public class LockService {
#Transactional
public Lock getLock(String key) {
Lock searchLock = new Lock();
searchLock.setKey(key);
lockMapper.getLock(searchLock);
LockHistory lockHistory = new LockHistory();
// Fill out lockHistory object...
lockMapper.markAsLocked(lockHistory);
attachTransactionCompletedListener(lockHistory);
}
private void attachTransactionCompletedListener(LockHistory lockHistory) {
/* Attach a listener onto the current spring transaction so that we
* can update the database entry when the transaction finishes and
* the lock is released.
*/
}
}
MemberTools.java:
public class MemberTools {
#Autowired
LockService lockService;
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void createMember() {
lockService.getLock("tb_member");
/* Do create member stuff...
* When this returns, the lock will be released
* (either from COMMIT, ROLLBACK, or UNKNOWN Spring error)
*/
}
}
By the time the getLock() method is reached, the stack trace is muddled with many calls that Spring inserts (proxies, reflections, etc.). Putting a breakpoint in this function and examining Thread.currentThread().getStackTrace() will show this.
Related
Our application calls many external API's which take a session token of the current user as input. So what we currently do is in a controller, get the session token for the user and pass it into a service which in turn might call another service or some API client. To give an idea, we end up with something like this (example is .NET but something similar is I think possible in Java)
public IActionResult DoSomething(string something)
{
this.someService.DoSomethingForUser(this.HttpContext.SessionToken, something);
return View();
}
And then we have
public class SomeService
{
private readonly IApiClient apiClient;
public SomeService(IApiClient apiClient)
{
this.apiClient = apiClient;
}
public void DoSomethingForUser(string sessionToken, something)
{
this.apiClient.DoSomethingForUser(sessionToken, something);
}
}
It can also happen that in SomeService another service is injected which in turn calls the IApiClient instead of SomeService calling IApiClient directly, basically adding another "layer".
We had a discussion with the team if it isn't better to instead of passing the session token, inject it using DI so you get something like this:
public IActionResult DoSomething(string something)
{
this.someService.DoSomethingForUser(something);
return View();
}
And then we have
public class SomeService
{
private readonly IUserService userService;
private readonly IApiClient apiClient;
public SomeService(IUserService userService, IApiClient apiClient)
{
this.userService = userService;
this.apiClient = apiClient;
}
public void DoSomethingForUser(string something)
{
this.apiClient.DoSomethingForUser(userService.SessionToken, something);
}
}
The IUserService would have an IHttpContextAccessor injected:
public class UserService : IUserService
{
private readonly IHttpContextAccessor httpContextAccessor;
public UserService(IHttpContextAccessor httpContextAccessor)
{
this.httpContextAccessor = httpContextAccessor;
}
public string SessionToken => httpContextAccessor.HttpContext.SessionToken;
}
The benefits of this pattern are I think pretty clear. Especially with many services, it keeps the code "cleaner" and you end up with less boilerplate code to pass a token around.
Still, I don't like it. To me the downsides of this pattern are more important than its benefit:
I like that passing the token in the methods is concise. It is clear that the service needs some sort of authentication token for it to function. I'm not sure if you can call it a side effect but the fact that a session token is magically injected three layers deep is impossible to tell just by reading the code
Unit testing is a bit more tedious if you have to Mock the IUserService
You run into problems when calling this in another thread, e.g. calling SomeService from another thread. Although these problems can be mitigated by injecting another concrete type of IUserService which gets the token from some place else, it feels like a chore.
To me it strongly feels like an anti pattern but apart from the arguments above it is mostly a feeling. There was a lot of discussion and not everybody was convinced that it was a bad idea. Therefor, my question is, is it an anti pattern or is it perfectly valid? What are some strong arguments for and against it, hopefully so there can be not much debate that this pattern is indeed, either perfectly valid or something to avoid.
I would say the main point is to enable your desired separation of concerns. I think it is a good question if expressed in those terms. As Kit says, different people may prefer different solutions.
REQUEST SCOPED OBJECTS
These occur quite naturally in APIs. Consider the following example, where a UI calls an Orders API, then the Orders API forwards the JWT to an upstream Billing API. A unique Request ID is also sent, in case the flow experiences a temporary problem. If the flow is retried, the Request ID can be used by APIs to prevent data duplication. Yet business logic should not need to know about either the Request ID or the JWT.
BUSINESS LOGIC CLASS DESIGN
I would start by designing my logic classes with my desired inputs, then work out the DI later. In my example the OrderService class might use claims to get the user identity and also for authorization. But I would not want it to know about HTTP level concerns:
public class OrderService
{
private readonly IBillingApiClient billingClient;
public OrderService(IBillingApiClient billingClient, ClaimsPrincipal user)
{
this.billingClient = billingClient;
}
public async void CreateOrder(OrderInput data)
{
this.Authorize();
var order = this.CreateOrder(data);
await this.billingClient.CreateInvoice(order);
}
}
DI SETUP
To enable my preferred business logic, I would write a little DI plumbing, so that I could inject request scoped dependencies in my preferred way. First, when the app starts, I would create a small middleware class. This will run early in the HTTP request pipeline:
private void ConfigureApiMiddleware(IApplicationBuilder api)
{
api.UseMiddleware<ClientContextMiddleware>();
}
In the middleware class I would then create a ClientContext object from runtime data. The OrderService class will run later, after next() is called:
public class ClientContextMiddleware
{
public async Task Invoke(HttpContext context)
{
var jwt = readJwt(context.Request);
var requestId = readRequestId(context.Request);
var holder = context.RequestServices.GetService<ClientContextHolder>();
holder.ClientContext = new ClientContext(jwt, requestIO);
await this.next(context);
}
}
In my DI composition at application startup I would express that the API client should be created when it is first referenced. In the HTTP request pipeline, the OrderService request scoped object will be constructed after the middleware has run. The below lambda will then be invoked:
private void RegisterDependencies(IServiceCollection services)
{
this.services.AddScoped<IApiClient>(
ctx =>
{
var holder = ctx.GetService<ClientContextHolder>();
return new ApiClient(holder.context);
});
this.services.AddScoped<ClientContextHolder>();
}
The holder object is just due to a technology limitation. The MS stack does not allow you to create new request scoped injectable objects at runtime, so you have to update an existing one. In a previous .NET tech stack, the concept of child container per request was made available to developers, so the holder object was not needed.
ASYNC AWAIT
Request scoped objects are stored against the HTTP request object, which is the correct behaviour when using async await. The current thread ID may switch, eg from 4 to 6 after the call to the Billing API.
If the OrderService class has a transient scope, it could get recreated when the flow resumes on thread 6. If this is the case, then resolution will continue to work.
SUMMARY
Designing inputs first, then writing some support code if needed is a good approach I think, and it is also useful to know the DI techniques. Personally I think natural request scoped objects that need to be created at runtime should be usable in DI. Some people may prefer a different approach though.
See in dotnet the area that I am an expert is not an anti standard on the contrary it is the model that many adopt but it is not a model that I would follow for the following reasons
it is not clear where is the token for those who read and use it being an anti clean code
you load important information in a place that is frequently accessed by the framework in the case of .netCore
your classes will reference a large property carrying a lot of unnecessary information when you could have created a more clean model that costs less memory and allocation time, I'm saying this because the HttpAcessor carries all the information relevant to your request
As I would take care of readability (clean code) and improve my performance
I would make a middleware or filter in my flow mvc where I would do the authentication part and create a class like:
public class TokenAuthenciationValues
{
public string TokenClient { get; set; }
public string TokenValue { get; set; }
}
Of course my method is an example but in my middleware I would implement it by loading its token values after calling the necessary apis (of course this model needs an interface and it needs to be configured as .AddScoped() in the case of .net)
That way I would use it in my methods only instantiating my ITokenAuthenciationValues in the constructor and I would have clear and clean information loaded in memory during the entire request
If it is necessary in the middle of the request to change the token any class can access it and change its value
I would have less memory allocated unused in my classes since the IHttpAcessor contract the ITokenAuthenciationValues only has relevant information
Hope this helps
We have an application with three databases. Two of them are only very seldomly updated. We tried JPA to create transactions around it and it worked for the databases, but grails then did not work on different places (gsp related I am told). This was tried quite a while ago (and not by me).
Due to delivery pressure we needed a solution that at least works for us, so I created a new aspect for the methods changing data in multiple databases. I got this to work, it is a fairly simple approach.
In the aspect we request to start a transaction for each data source, by calling getTransaction(TransactionDefinition def) with the propagation set to REQUIRES_NEW. We then proceed and finally rollback or commit depending on the outcome of the call.
However, one test flow failed. This is the scenario where the code requests a rollback by calling TransactionAspectSupport.currentTransactionStatus().setRollbackOnly(). Of the three TransactionStatusses obtained initially, none actually returns isRollbackOnly() with true. However calling TransactionAspectSupport.currentTransationStatus().isRollbackOnly() does return true. So this seems to point to a different transaction status.
I have not been able to figure out how to make this work, other than checking this additional status. I could not find a way to change the currentTransactionStatus to the one of created TransactionStatus. Looking at the TransactionTemplate implementation, I seem to do things correctly (it also just calls getTransaction() on the datasource).
The code calling the decorated method has specified #Transactional(propagation=Propagation.NOT_SUPPORTED), so I expected no currentTransactionStatus, but one is there.
However, if it is not there the proxied code will not be able to request a rollback the standard way, which I want to be able to fix.
So the question is, how to start a transaction correctly from an Aspect so that the currentTransactionStatus is set correctly or how to set the currentTransactionStatus to what I think is the correct one.
Regards,
Wim Veldhuis.
I finally figured it out.
#Transactional leads to a different code path, where eventually TransactionAspectSupport.invokeWithinTransaction is invoked. This method will set up the current transaction correctly.
So in order to make my approach working, I needed to derive from TransactionAspectSupport, do a number of cast operations so I could get to the correct values for the invokeWithinTransaction call, and within the guarded function block use getTransaction(def) to obtain txns for the OTHER databases. I have choose the most important database to be the one used for invoke...
To make it work I had also to provide a TransactionAttributeSource, that returned my default transaction attributes.That one is stored into the TransactionAspectSupport base class during initialization.
#Around("#annotation(framework.db.MultiDbTransactional)")
public Object multiDbTransaction(ProceedingJoinPoint proceedingJoinPoint) throws Throwable {
// Get class and method, needed for parent invocation. We need to cast to the actual
// implementation
MethodInvocationProceedingJoinPoint mipJoinPoint = (MethodInvocationProceedingJoinPoint) proceedingJoinPoint;
MethodSignature signature = (MethodSignature) mipJoinPoint.getSignature();
Class<?> clazz = mipJoinPoint.getTarget().getClass();
Method method = signature.getMethod();
return invokeWithinTransaction(method, clazz, new InvocationCallback() {
#Override
public Object proceedWithInvocation() throws Throwable {
// This class will create the other transactions, not of interest here.
MultiDbTxnContext ctx = new MultiDbTxnContext();
ctx.startTransactions();
/*
* We have started the transactions, so do the job. We mimic DEFAULT spring behavior
* regarding exceptions, so runtime exceptions roll back, the rest commits.
*/
try {
Object result = proceedingJoinPoint.proceed();
ctx.finishTransactions();
return result;
} catch (Error | RuntimeException re) {
ctx.rollbackTransactions();
throw re;
} catch (Throwable t) {
ctx.commitTransactions();
throw t;
}
}
});
}
In my service code, I am trying to create or update a Person domain object:
#Transactional
def someServiceMethod(some params....) {
try{
def person = Person.findByEmail(nperson.email.toLowerCase())
if (!person) {
person = new Person()
person.properties = nperson.properties
} else {
// update the person parameters (first/last name)
person.firstName = nperson.firstName
person.lastName = nperson.lastName
person.phone = nperson.phone
}
if (person.validate()) {
person.save(flush: true)
//... rest of code
}
// rest of other code....
} catch(e) {
log.error("Unknown error: ${e.getMessage()}", e)
e.printStackTrace()
return(null)
}
Now above code OCCASIONALLY when trying to save a Person object with already existing email throws following exception:
Hibernate operation: could not execute statement; SQL [n/a]; Duplicate entry 'someemail#gmail.com' for key 'email_UNIQUE'; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry 'someemail#gmail.com' for key 'email_UNIQUE'
This is very strange because I am already finding the person by email and hence the save() should try to update the record instead of creating the new one.
I was wondering why is this happening!
EDIT:
I am on grails 2.4.5 and Hibernate plugin in BuildConfig is:
runtime ':hibernate4:4.3.8.1'
EDIT2:
My application is on multiple servers hence synchronized block won't work
If this is concurrency issue, here is what we do in such case. We have a lot of concurrent background processes which work on the same tables. If there is such operation it indeed is in synchronized block, so code may look like:
class SomeService {
static transactional = false //service cannot be transactional
private Object someLock = new Object() //synchronized block on some object must be used
def someConcurrentSafeMethod(){
synchronized(someLock){
def person = Person.findByEmail(nperson.email.toLowerCase())
...
person.save(flush: true) // flush is very important, must be done in synchronized block
}
}
}
There are few important points to make this working (from our experience, not official):
Service cannot be transactional - if service is transactional, transaction is commited after the method returns value and synchronization inside method will not be enough. Programmatic transactions may be another way
synchronized method is not enough synchronized def someConcurrentSafeMethod() will not work - probably because service is wrapped in proxy
Session MUST be flushed inside synchronized block
every object which will be saved, should be read in synchronized block, if you pass it from external method, you may run into optimistic locking failed exception
UPDATED
Because application is deployed on distributed system, above will not solve the issue here (still may help others). After discussion we had on Slack, I just summarize potential ways to do that:
pessimistic locking of updated objects and lock of whole table for inserts (if possible)
moving 'dangerous' database related methods to single server with some API like REST and calling it from other deployments (and using synchronized approach from above)
using multiple save approach - if operation fails, catch exception and try again. This is supported by integration libraries like Spring Integration or Apache Camel and is one of enterprise patterns. See request-handler-advice-chain for Spring Integration as an example
use something to queue operations, for example JMS server
If anyone has more ideas please share them.
I have a problem with establishing a transaction manager/scope for my REST API (JAVA), I have below functions in my API back end and i want to excute all below functions as one transaction,
Call third party WS end point
Decrypt the response
Save the response in to DB1
Save the response in to DB2
I want to make sure all above steps are completed or rollback if any one fail, I have enough information to do the rollback, but i have no idea what would be the best practice to implement proper transaction management mechanism, because above mentioned steps happen in 3 separate classes per API call,
This is a pseudo code for my class structure
class CallWS {
public People callThWS() {
// functions related to call third party WS and decryption (step 1,2)
}
}
class People peopleServices {
public People getPeopleData() {
callThWS ppl= new callThWS();
People pplObj = ppl.callThWS();
// save to DB1, (step 3)
return pplObj;
}
}
class People peopleContr {
public People getAllPeople() {
peopleServices ppSer= new peopleServices();
People pplObj2 = ppSer.getPeopleData();
// save to DB2, (Step 4)
return pplObj2;
}
}
Please help me on this,
Thanks
What you need is Distributed Transactions(XA). Check for examples of various transaction managers which support XA. Check this article for using XA provider in standalone applications(Warning: Old article).
If you control sources of all classes listed and you can refactor your code the way you have a single entry point, you can do it quite easily, except the call to an external web service. The pseudo code is below.
Here we should agree that all resources your are calling in your methods are transactional. As I mentioned earlier call to an external WS would not fall to that category because calls to external web services are not transactional by their nature. Again if you do not change data withing a call to the external service you may consider just leave it outside transaction. You still have a bit of control. Like rolling back transaction in case a call to the external service was unsuccessful and as far as you have not changed anything on the other side, you may not care about rolling back a transaction there.
However you still have some options for a transaction call to an external WS call like Web Services Atomic Transactions, but I bet you would need control for sources and maybe even environment on the other side. In such a lucky circumstances you would rather want to achieve it by avoiding the WS call.
class RestAPIEntryPointResource {
#Inject
CallWS callWS;
#Inject
PeopleServices peopleServices ;
#Inject
PeopleContr peopleContr;
/*Put some transaction demarcation here.
If your class is an EJB, it is already done for you.
With Spring you have various options to mark the method transactional.
You also may want to take a manual control, but it look redundant here. */
public void entryPointMethod() {
callWS.callThWS();
peopleServices.getPeopleData();
peopleContr.getAllPeople();
}
}
class CallWS {
public People callThWS() {
// functions related to call third party WS and decryption (step 1,2)
}
}
class PeopleServices {
public People getPeopleData() {
..........
}
}
class PeopleContr {
public People getAllPeople() {
.........
}
}
I am working on a program that uses Spring and obtains Hibernate transactions transparently using a TransactionInterceptor. This makes it very convenient to say "when this method is invoked from some other class, wrap it in a transaction if it's not already in one."
However, I have a class that needs to attempt a write and must find out immediately whether or not it has succeeded. While I want two methods anyway, I was hoping that there was a way to keep them in the same class without needing to explicitly create an transaction procedurally. In effect, I'd like something like this:
public void methodOne() {
//..do some stuff
try {
transactionalMethod();//won't do what I want
} catch(OptimisticLockingFailure e) {
//..recover
}
}
#Transactional
public void transactionalMethod() {
//...do some stuff to database
}
Unfortunately, as I understand it, this wouldn't work because I'd just be directly calling transactionalMethod. Is there a way to ask Spring to call a local method for me and wrap it in a transaction if needed, or does it have to be in another class that I wire to this one?
Define an interface which the class implements which does the transactionalMethod(); use dependency injection to set the class' value of that to its own implementation; in your bean factory, allow Spring to insert an Around aspect around that interface implementation. That should work for your needs.
If you want the transactionalMethod to be part of it's own transaction and not simply join onto the transaction that is already active you have to set the propagation to REQUIRES_NEW. Like so
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void transactionalMethod() {
//...do some stuff to database
}
You should also check that your transaction manager supports this propagation. the means that transactionalMethos is completely seperate from the other transaction that it was called from and it will commit / rollback completely seperately as well.