I am trying to implement logging in DB table using Spring AOP. By "logging in table" I mean to write in special log table information about record that was CREATED/UPDATED/DELETED in usual table for domain object.
I wrote some part of the code and all is working good except one thing - when transaction is rolled back then changes in log table still commit successfully. It's strange for me because in my AOP advice the same transaction is using as in my business and DAO layer. (From my AOP advice I called methods of special manager class with Transaction propagation MANDATORY and also I checked transaction name TransactionSynchronizationManager.getCurrentTransactionName() in business layer, dao layer and AOP advice and it is the same).
Does anyone tried to implement similar things in practice? Is it possible to use in AOP advice the same transaction as in the business layer and rollback changes made in AOP advice if some error in business layer occurs?
Thank you in advance for unswers.
EDIT
I want to clarify that problem with rollback occurs only for changes that was made from AOP advice. All changes that is made in DAO layer are rollbacked successfully. I mean that, for example, if some exception is thrown then changes made in DAO layer will be successfully rollbacked, but in log table information will be saved (commited). But I can't understand why it is like that because as I wrote above in AOP advice the same transaction is using.
EDIT 2
I checked with debugger the piece of the code where I am writting to the log table in AOP advice and it seems to me that JdbcTemplate's update method executes outside transaction because changes had been commited to the DB directly after execution of the statement and before transactional method was finished.
EDIT 3
I solved this problem. Actually, that was my stupid fault. I'm using MySQL. After creation of the log table I did't change DB engine and HeidySQL set MyIsam by default. But MyIsam doesn't support transaction so I changed DB engine to InnoDB (as for all other tables) and now all is working perfectly.
Thank you all for help and sorry for disturbing.
If someone is interested, here is simplified example that illustrate my approach.
Consider DAO class that has save method:
#Repository(value="jdbcUserDAO")
#Transactional(propagation=Propagation.SUPPORTS, readOnly=true, rollbackFor=Exception.class)
public class JdbcUserDAO implements UserDAO {
#Autowired
private JdbcTemplate jdbcTemplate;
#LoggedOperation(affectedRows = AffectedRows.ONE, loggedEntityClass = User.class, operationName = OperationName.CREATE)
#Transactional(propagation=Propagation.REQUIRED, readOnly=false, rollbackFor=Exception.class)
#Override
public User save(final User user) {
if (user == null || user.getRole() == null) {
throw new IllegalArgumentException("Input User object or nested Role object should not be null");
}
KeyHolder keyHolder = new GeneratedKeyHolder();
jdbcTemplate.update(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection connection)
throws SQLException {
PreparedStatement ps = connection.prepareStatement(SQL_INSERT_USER, new String[]{"ID"});
ps.setString(1, user.getUsername());
ps.setString(2, user.getPassword());
ps.setString(3, user.getFullName());
ps.setLong(4, user.getRole().getId());
ps.setString(5, user.geteMail());
return ps;
}
}, keyHolder);
user.setId((Long) keyHolder.getKey());
VacationDays vacationDays = user.getVacationDays();
vacationDays.setId(user.getId());
// Create related vacation days record.
vacationDaysDAO.save(vacationDays);
user.setVacationDays(vacationDays);
return user;
}
}
Here is how aspect looks like:
#Component
#Aspect
#Order(2)
public class DBLoggingAspect {
#Autowired
private DBLogManager dbLogManager;
#Around(value = "execution(* com.crediteuropebank.vacationsmanager.server.dao..*.*(..)) " +
"&& #annotation(loggedOperation)", argNames="loggedOperation")
public Object doOperation(final ProceedingJoinPoint joinPoint,
final LoggedOperation loggedOperation) throws Throwable {
Object[] arguments = joinPoint.getArgs();
/*
* This should be called before logging operation.
*/
Object retVal = joinPoint.proceed();
// Execute logging action
dbLogManager.logOperation(arguments,
loggedOperation);
return retVal;
}
}
And here is how my db log manager class LooksLike:
#Component("dbLogManager")
public class DBLogManager {
#Autowired
private JdbcTemplate jdbcTemplate;
#InjectLogger
private Logger logger;
#Transactional(rollbackFor={Exception.class}, propagation=Propagation.MANDATORY, readOnly=false)
public void logOperation(final Object[] inputArguments, final LoggedOperation loggedOperation) {
try {
/*
* Prepare query and array of the arguments
*/
jdbcTemplate.update(insertQuery.toString(),
insertedValues);
} catch (Exception e) {
StringBuilder sb = new StringBuilder();
// Prepare log string
logger.error(sb.toString(), e);
}
}
It could be to do with the order of the advice - you would want your #Transaction related advice to take effect around(or before and after) your logging related advice. If you are using Spring AOP you can probably control it using the order attribute of the advice - give your transaction related advice the highest precedence so that it executes last on the way out.
Nothing to do with AOP, set datasource property autocommit to false like :
<bean id="datasource" ...>
<property name="autoCommit" value="false/>
</bean>
If you are using xml configuration
Related
While I have Java Batch jobs that read data, process it and store it in other places in the database, now I need a step to actually remove data from the database. All I need to run is a delete query via JPA.
The chunk based Reader/Processor/Writer pattern does not make sense here. But the Batchlet alternative is giving me a headache either. What did I do?
I created a Batchlet that gets invoked via CDI. At that moment it is easy to inject my JPA EntityManager. What is not easy is to run the update query. Code looks like this:
package ...;
import javax.batch.api.BatchProperty;
import javax.inject.Inject;
import javax.inject.Named;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
#Named("CleanerBatchlet")
public class CleanerBatchlet extends AbstractBatchlet {
public static final Logger log = LogManager.getLogger(CleanerBatchlet.class);
#PersistenceContext(unitName = "...")
private EntityManager entityManager;
#Inject
#BatchProperty(name = "technologyIds")
private String technologyIds;
private void clearQueue(long technologyId) {
//EntityManager entityManager = ...getEntityManager();
//entityManager.getTransaction().begin();
Query q = entityManager.createQuery("delete from Record r where r.technologyId=:technologyId");
q.setParameter("technologyId", technologyId);
int count = q.executeUpdate();
//entityManager.getTransaction().commit();
log.debug("Deleted {} entries from queue {}", count, technologyId);
//entityManager.close();
}
#Override
public String doProcess() throws Exception {
log.debug("doProcess()");
out.println("technologyIds=" + technologyIds);
log.info("technologyIds=" + technologyIds);
try {
String[] parts = technologyIds.split(",");
for (String part: parts) {
long technologyId = Long.parseLong(part);
clearQueue(technologyId);
}
} catch (NullPointerException | NumberFormatException e) {
throw new IllegalStateException("technologyIds must be set to a string of comma-separated numbers.", e);
}
return "COMPLETED";
}
}
As you can see some lines are commented out - these are the ones I am experimenting with.
So if I run the code as-is, I get an exception telling me that the update query requires a transaction. This is regardless of which of the two persistence units in my project I use (one is configured for JTA, the other is not).
javax.persistence.TransactionRequiredException: Executing an update/delete query
It also does not matter whether I uncomment the transaction handling code begin/commit. I still get the same error that a transaction is required to run the update query.
Even when I try to circumvent CDI and JTA completely by creating my own EntityManager via the Persistence API (and close it afterwards, respectively) I do get the very same exception.
So how can I run this delete query or other update queryies from within the batch job?
I'd suggest using plain jdbc to run this delete query, with either auto commit or manual transaction commit.
During the batchlet processing, the incoming transaction is suspended. So the entity manager does not have a transaction context.
Ultimately I made it work by following this tutorial: https://dzone.com/articles/resource-local-vs-jta-transaction-types-and-payara
and going for the Classic RESOURCE_LOCAL Application pattern.
It involves injecting the nonJTA EntityManagerFactory, using that to create the entitymanager and closing it after use. Of course the transaction has to be managed manually but after all now it works.
The essential excerpt of my code looke like this:
#PersistenceUnit(unitName = "...")
private EntityManagerFactory emf;
#Inject
#BatchProperty(name = "technologyIds")
private String technologyIds;
private void clearQueue(long technologyId) {
EntityManager entityManager = emf.createEntityManager();
entityManager.getTransaction().begin();
Query q = entityManager.createQuery("delete from Record r where r.technologyId=:technologyId");
q.setParameter("technologyId", technologyId);
q.executeUpdate();
entityManager.getTransaction().commit();
entityManager.close();
}
Is #Transactional support for NamedParameterTemplate.batchUpdate?
If something went wrong during the batch execution, will it rollback as expected? Personally, I am not experienced that. That's why I am asking.
Is there any document to check #Transactional supported methods.
public class JdbcActorDao implements ActorDao {
private NamedParameterTemplate namedParameterJdbcTemplate;
public void setDataSource(DataSource dataSource) {
this.namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(dataSource);
}
#Transactional
public int[] batchUpdate(List<Actor> actors) {
return this.namedParameterJdbcTemplate.batchUpdate(
"update t_actor set first_name = :firstName, last_name = :lastName where id = :id",
SqlParameterSourceUtils.createBatch(actors));
}
// ... additional methods
}
NamedParameterTemplate is just an abstraction around Jdbc. In spring it is the Transaction Manager that is responsible for managing transactions, not that you can not do it via plain JDBC but this is the spring way. Spring uses AOP internaly to inspect the annotated methods and delegates its transaction managment. But this role is separate from the NamedParameterTemplate.
So you can use it freely and annotate your methods as long as they are within a Spring managed component/bean with #Transactional
I want to know Best practices for initilizing JOOQ generated DAO. Now,I am using following approach for initilization of JOOQ generated DAO. In following case StudentDao is JOOQ generated.
public class ExtendedStudentDAO extends StudentDao {
public ExtendedStudentDAO () {
super();
}
public ExtendedStudentDAO (Connection connection) {
Configuration configuration = DSL.using(connection,
JDBCUtils.dialect(connection)).configuration();
this.setConfiguration(configuration);
}
//adding extra methods to DAO using DSL
public String getStudentName(Long ID)
throws SQLException {
try (Connection connection = ServiceConnectionManager.getConnection()) {
DSLContext dslContext = ServiceConnectionManager
.getDSLContext(connection);
Record1<String> record = dslContext
.select(Student.Name)
.from(Student.Student)
.where(Student.ID
.equal(ID)).fetchOne();
if (record != null) {
return record.getValue(Student.Name);
}
return null;
}
}
}
and I have doubt with using above DAO my example code is below.
try (Connection connection = ServiceConnectionManager.getConnection()) {
ExtendedStudentDAO extendedStudentDAO =new ExtendedStudentDAO(connection);
Student stud=new Student();
.....
....
//insert method is from Generated DAO
extendedStudentDAO.insert(stud);
//this method is added in extended class
extendedStudentDAO.getStudentName(12);
}
There are two ways to look at this kind of initialisation:
Create DAOs every time you need them
Your approach is correct, but might be considered a bit heavy. You're creating a new DAO every time you need it.
As of jOOQ 3.7, a DAO is a pretty lightweight object. The same is true for the Configuration that wraps your Connection.
Once your project evolves (or in future jOOQ versions), that might no longer be true, as your Configuration initialisation (or jOOQ's DAO initialisation) might become heavier.
But this is a small risk, and it would be easy to fix:
Use dependency injection to manage DAO or Configuration references
Most people will set up only a single jOOQ Configuration for their application, and also only a single DAO instance (per DAO type), somewhere in a service. In this case, your Configuration must not share the Connection reference, but provide a Connection to jOOQ via the ConnectionProvider SPI. In your case, that seems trivial enough:
class MyConnectionProvider implements ConnectionProvider {
#Override
public Connection acquire() {
return ServiceConnectionManager.getConnection();
}
#Override
public void release(Connection connection) {
try {
connection.close();
}
catch (SQLException e) {
throw new DataAccessException("Error while closing", e);
}
}
}
Below is a quick outline of what I'm trying to do. I want to push a record to two different tables in the database from one method call. If anything fails, I want everything to roll back. So if insertIntoB fails, I want anything that would be committed in insertIntoA to be rolled back.
public class Service {
MyDAO dao;
public void insertRecords(List<Record> records){
for (Record record : records){
insertIntoAAndB(record);
}
}
#Transactional (rollbackFor = Exception.class, propagation = Propagation.REQUIRES_NEW)
public void insertIntoAAndB(Record record){
insertIntoA(record);
insertIntoB(record);
}
#Transactional(propagation = Propagation.REQUIRED)
public void insertIntoA(Record record){
dao.insertIntoA(record);
}
#Transactional(propagation = Propagation.REQUIRED)
public void insertIntoB(Record record){
dao.insertIntoB(record);
}
public void setMyDAO(final MyDAO dao) {
this.dao = dao;
}
}
Where MyDAO dao is an interface that is mapped to the database using mybatis and is set using Spring injections.
Right now if insertIntoB fails, everything from insertIntoA still gets pushed to the database. How can I correct this behavior?
EDIT:
I modified the class to give a more accurate description of what I'm trying to achieve. If I run insertIntoAAndB directly, the roll back works if there are any issues, but if I call insertIntoAAndB from insertRecords, the roll back doesn't work if any issues arise.
I found the solution!
Apparently Spring can't intercept internal method calls to transactional methods. So I took out the method calling the transactional method, and put it into a separate class, and the rollback works just fine. Below is a rough example of the fix.
public class Foo {
public void insertRecords(List<Record> records){
Service myService = new Service();
for (Record record : records){
myService.insertIntoAAndB(record);
}
}
}
public class Service {
MyDAO dao;
#Transactional (rollbackFor = Exception.class, propagation = Propagation.REQUIRES_NEW)
public void insertIntoAAndB(Record record){
insertIntoA(record);
insertIntoB(record);
}
#Transactional(propagation = Propagation.REQUIRED)
public void insertIntoA(Record record){
dao.insertIntoA(record);
}
#Transactional(propagation = Propagation.REQUIRED)
public void insertIntoB(Record record){
dao.insertIntoB(record);
}
public void setMyDAO(final MyDAO dao) {
this.dao = dao;
}
}
I think the behavior you encounter is dependent on what ORM / persistence provider and database you're using. I tested your case using hibernate & mysql and all my transactions rolled back alright.
If you do use hibernate enable SQL and transaction logging to see what it's doing:
log4j.logger.org.hibernate.SQL=DEBUG
log4j.logger.org.hibernate.transaction=DEBUG
// for hibernate 4.2.2
// log4j.logger.org.hibernate.engine.transaction=DEBUG
If you're on plain jdbc (using spring JdbcTemplate), you can also debug SQL & transaction on Spring level
log4j.logger.org.springframework.jdbc.core=DEBUG
log4j.logger.org.springframework.transaction=DEBUG
Double check your autocommit settings and database specific peciular (eg: most DDL will be comitted right away, you won't be able to roll it back although spring/hibernate did so)
Just because jdk parses aop annotation not only with the method, also parse annotation with the target class.
For example, you have method A with #transactional, and method B which calls method A but without #transactional, When you invoke the method B with reflection, Spring AOP will check the B method with the target class has any annotations.
So if your calling method in this class is not with the #transactional, it will not parse any other method in this method.
At last, show you the source code:
org.springframework.aop.framework.jdkDynamicAopProxy.class
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
......
// Get the interception chain for this method.
List<Object> chain = this.advised.getInterceptorsAndDynamicInterceptionAdvice(method, targetClass);
// Check whether we have any advice. If we don't, we can fallback on direct
// reflective invocation of the target, and avoid creating a MethodInvocation.
if (chain.isEmpty()) {
// We can skip creating a MethodInvocation: just invoke the target directly
// Note that the final invoker must be an InvokerInterceptor so we know it does
// nothing but a reflective operation on the target, and no hot swapping orfancy proxying.
retVal = AopUtils.invokeJoinpointUsingReflection(target, method, args);
}
else {
// We need to create a method invocation...
invocation = new ReflectiveMethodInvocation(proxy, target, method, args, targetClass, chain);
// Proceed to the joinpoint through the interceptor chain.
retVal = invocation.proceed();
}
}
Firstly, I can't use the declarative #Transactional approach as the application has multiple JDBC data-sources, I don't want to bore with the details, but suffice it to say the DAO method is passed the correct data-source to perform the logic. All JDBC data sources have the same schema, they're separated as I'm exposing rest services for an ERP system.
Due to this legacy system there are a lot of long lived locked records which I do not have control over, so I want dirty reads.
Using JDBC I would perform the following:
private Customer getCustomer(DataSource ds, String id) {
Customer c = null;
PreparedStatement stmt = null;
Connection con = null;
try {
con = ds.getConnection();
con.setTransactionIsolation(Connection.TRANSACTION_READ_UNCOMMITTED);
stmt = con.prepareStatement(SELECT_CUSTOMER);
stmt.setString(1, id);
ResultSet res = stmt.executeQuery();
c = buildCustomer(res);
} catch (SQLException ex) {
// log errors
} finally {
// Close resources
}
return c;
}
Okay, lots' of boiler-plate, I know. So I've tried out JdbcTemplate since I'm using spring.
Use JdbcTemplate
private Customer getCustomer(JdbcTemplate t, String id) {
return t.queryForObject(SELECT_CUSTOMER, new CustomerRowMapper(), id);
}
Much nicer, but it's still using default transaction isolation. I need to somehow change this. So I thought about using a TransactionTemplate.
private Customer getCustomer(final TransactionTemplate tt,
final JdbcTemplate t,
final String id) {
return tt.execute(new TransactionCallback<Customer>() {
#Override
public Customer doInTransaction(TransactionStatus ts) {
return t.queryForObject(SELECT_CUSTOMER, new CustomerRowMapper(), id);
}
});
}
But how do I set the transaction isolation here? I can't find it anywhere on the callback or the TransactionTemplate to do this.
I'm reading Spring in Action, Third Edition which explains as far as I've done, though the chapter on transactions continues on to using declarative transactions with annotations, but as mentioned I can't use this as my DAO needs to determine at runtime which data-source to used based on provided arguments, in my case a country code.
Any help would be greatly appreciated.
I've currently solved this by using the DataSourceTransactionManager directly, though it seems like I'm not saving as much boiler-plate as I first hoped. Don't get me wrong, it's cleaner, though I still can't help but feel there must be a simpler way. I don't need a transaction for the read, I just want to set the isolation.
private Customer getCustomer(final DataSourceTransactionManager txMan,
final JdbcTemplate t,
final String id) {
DefaultTransactionDefinition def = new DefaultTransactionDefinition();
def.setIsolationLevel(TransactionDefinition.ISOLATION_READ_UNCOMMITTED);
TransactionStatus status = txMan.getTransaction(def);
Customer c = null;
try {
c = t.queryForObject(SELECT_CUSTOMER, new CustomerRowMapper(), id);
} catch (Exception ex) {
txMan.rollback(status);
throw ex;
}
txMan.commit(status);
return c;
}
I'm still going to keep this one unanswered for a while as I truly believe there must be a better way.
Refer to Spring 3.1.x Documentation - Chapter 11 - Transaction Management
Using the TransactionTemplate helps you here, you need to configure it appropriately. The transaction template also contains the transaction configuration. Actually the TransactionTemplate extends DefaultTransactionDefinition.
So somewhere in your configuration you should have something like this.
<bean id="txTemplate" class=" org.springframework.transaction.support.TransactionTemplate">
<property name="isolationLevelName" value="ISOLATION_READ_UNCOMMITTED"/>
<property name="readOnly" value="true" />
<property name="transactionManager" ref="transactionManager" />
</bean>
If you then inject this bean into your class you should be able to use the TransactionTemplate based code you posted/tried earlier.
However there might be a nicer solution which can clean up your code. For one of the projects I worked on, we had a similar setup as yours (single app multiple databases). For this we wrote some spring code which basically switches the datasource when needed. More information can be found here.
If that is to far fetched or overkill for your application you can also try and use Spring's AbstractRoutingDataSource, which based on a lookup key (country code in your case) selects the proper datasource to use.
By using either of those 2 solutions you can start using springs declarative transactionmanagement approach (which should clean up your code considerably).
Define a proxy data source, class being org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy and set the transaction isolation level. Inject actual data source either through setter or constructor.
<bean id="yourDataSource" class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
<constructor-arg index="0" ref="targetDataSource"/>
<property name="defaultTransactionIsolationName" value="TRANSACTION_READ_UNCOMMITTED"/>
</bean>
I'm not sure you can do it without working with at the 'Transactional' abstraction-level provided by Spring.
A more 'xml-free' to build your transactionTemplate could be something like this.
private TransactionTemplate getTransactionTemplate(String executionTenantCode, boolean readOnlyTransaction) {
TransactionTemplate tt = new TransactionTemplate(transactionManager);
tt.setReadOnly(readOnlyTransaction);
tt.setIsolationLevel(TransactionDefinition.ISOLATION_READ_UNCOMMITTED);
tt.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRES_NEW);
return tt;
}
In any case I would "leverage" the #Transactional annotation specifying the appropriate transaction manager, binded with a separate data-source. I've done this for a multi-tenant application.
The usage:
#Transactional(transactionManager = CATALOG_TRANSACTION_MANAGER,
isolation = Isolation.READ_UNCOMMITTED,
readOnly = true)
public void myMethod() {
//....
}
The bean(s) declaration:
public class CatalogDataSourceConfiguration {
#Bean(name = "catalogDataSource")
#ConfigurationProperties("catalog.datasource")
public DataSource catalogDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = ENTITY_MANAGER_FACTORY)
public EntityManagerFactory entityManagerFactory(
#Qualifier("catalogEntityManagerFactoryBean") LocalContainerEntityManagerFactoryBean emFactoryBean) {
return emFactoryBean.getObject();
}
#Bean(name= CATALOG_TRANSACTION_MANAGER)
public PlatformTransactionManager catalogTM(#Qualifier(ENTITY_MANAGER_FACTORY) EntityManagerFactory emf) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(emf);
return transactionManager;
}
#Bean
public NamedParameterJdbcTemplate catalogJdbcTemplate() {
return new NamedParameterJdbcTemplate(catalogDataSource());
}
}