I'm developing an import module to load in a bulk way from an excel file. I'm using spring to manage the transactional context and spring-data-jpa like DAO, to read a excel file I'm using POI.
#Service
public class ImportService{
#Autowired
UserRepository userRepository;
#Autowired
UserDetailRepository userDetailRepository;
#Autowired
AddressRepository addressRepository;
#Transactional(propagation = Propagation.REQUIRES_NEW)
public Map<String,StringBuffer> import(byte[] bytesUploaded) {
wb = new XSSFWorkbook(new ByteArrayInputStream(bytesUploaded));
XSSFSheet mySheet = (XSSFSheet) wb.getSheetAt((short)0);
for(Row row: mySheet){
if(row.getRowNum() >= 2 ){
readRowFromExcel(row);
}
}
}
#Transactional(propagation = Propagation.NESTED,rollbackFor=Exception.class)
private void readRowFromExcel(Row row) {
try{
User user = readUserFromFile(row);
UserDetail = readUserDetailFromFile(row,user);
Address address = readAddressFromFile(row,user);
//A few check to verify the corretly load of the entities
userDetailRepository.save(userDetail);
addressRepository.save(address);
userRepository.save(user);
}
catch (Exception e){
//Do something
}
}
}
I would want that if an exception occurred during read or the save of the row nothing become persistent, however an internal exception mustn't cause rollback the outer transaction.
How can I manage this kind of situation?
Am I facing the problem correctly?
Can you give me some suggestion about this (I'm new of spring)?
Thanks
A good start is the Transaction management section of the Spring framework reference documentation
There are a few things in your sample code that needs improvements:
Declarative transaction management
See section 11.5.1
The most important concepts to grasp with regard to the Spring Frameworkâs
declarative transaction support are that this support is enabled via AOP
proxies, and that the transactional advice is driven by metadata (currently
XML- or annotation-based). The combination of AOP with transactional metadata
yields an AOP proxy that uses a TransactionInterceptor in conjunction with
an appropriate PlatformTransactionManager implementation to drive transactions
around method invocations.
What that means typically is that calling your private method will not honour the transaction demarcation as you expect. Calling other methods of your service internally won't work either by default as it does not go through the proxy. In your case, Using #Transactional provides additional useful references.
The #Transactional on readRowFromExcel seems useless as you have the guarantee that a fresh transaction will be created before calling import.
Exception handling
In order for the AOP proxy to manage an exception thrown during a transactional method, you have to actually make sure the exception is thrown outside the method boundary so that the proxy can see it. In the case of your example, the do something part should make sure to throw the appropriate exception. If you want fine-grained management of the exception, you should probably define another exception type than Exception.
Wrapping up
If you want to read your excel file and save the content to your store using Spring data, you should probably have a single public annotated method whose purpose is to read an item and store it using Spring data. This would need some refactoring of your API. If the excel sheet is fairly large, you should also consider Spring Batch to process it.
What would be the purpose of the outer transaction here? You should also consider failure scenario such as when one particular row cannot be read (or written to the store). Are you leaving the file half-processed or are you flagging this error somewhere?
Related
I have an integration test on a endpoint of creating a user with its related entities. It turns out that the related entities were not persisted with the user entity.
However, it is working fine when running the normal spring boot application. Is it possible to achieve this during testing?
This is the log when running the integration test of the endpoint
And this is the log when calling from Postman to the normal application
as you can notice the roles are not inserted to the rel_mi_user__mi_user_role table during integration testing.
The setup source code for the integration test is shown below
#Autowired
private PasswordEncoder passwordEncoder;
#Autowired
private MockMvc mockMvc;
#Autowired
private MiMiUserRepository miMiUserRepository;
private static Logger log = LoggerFactory.getLogger(MiMiUserResourceIT.class);
#Test
#Transactional
void testRegisterBackOfficeUser() throws Exception {
MiMiUserRegistrationDTO userRegistrationDTO = new MiMiUserRegistrationDTO();
userRegistrationDTO.setPassword("test12345");
userRegistrationDTO.setEmail("john#gmail.com");
userRegistrationDTO.setContactNo("0188991122");
userRegistrationDTO.setUserRoles(List.of("BACKOFFICE"));
MiMiUserProfileRegistrationDTO profile = new MiMiUserProfileRegistrationDTO();
profile.setSalutation("Mr");
profile.setFirstName("John");
profile.setLastName("Lee");
userRegistrationDTO.setProfile(profile);
mockMvc
.perform(
post("/v1/p/user/register/back-office")
.contentType(MediaType.APPLICATION_JSON)
.content(TestUtil.convertObjectToJsonBytes(userRegistrationDTO))
)
.andExpect(status().isCreated());
MiUser u = miMiUserRepository.findOneWithMiUserRolesByEmailIgnoreCase(userRegistrationDTO.getEmail()).get();
assertThat(u.getUserStatus()).isEqualTo(UserStatus.NEW);
}
I guess it has to do with the fact that your whole test is annotated with #Transactional. When you annotate your test method with #Transactional, everything that happens during that test happens within one single transaction. As you might know, changes are only flushed to the database at the end of a transaction unless you call the flush method manually which is generally discouraged unless you know what you are doing because you interfere with your JPA provider's flushing optimization.
Only when the changes are flushed to the DB, auto-generated properties like an ID is set on your entity and DB constraints are evaluated. Yes, you're reading it right, you might even violate a DB constraint within your test without the test failing!
Here you can find more reasons why you should not annotate your tests with #Transactional.
If you need to save several entities in the arrange part of your test, you can autowire the Spring TransactionTemplate and create all your entities within a manual transaction. The same might apply to asserts where you load an entity from the DB and want to evaluate a lazy-loaded collection of said entity. As far as I can tell, neither of said scenarios apply to your test, just try omitting the #Transactional and your test should work.
Side note: if you write #DataJpaTests you always need to call the saveAndFlush method of the repository instead of the save method because #DataJpaTests are always transactional.
I am having trouble finding information about this issue I am running into. I am interested in implementing row level security on my Postgres db and I am looking for a way to be able to set postgres session variables automatically through some form of an interceptor. Now, I know that with hibernate you are able to do row-level-security using #Filter and #FilterDef, however I would like to additionally set policies on my DB.
A very simple way of doing this would be to execute the SQL statement SET variable=value prior to every query, though I have not been able to find any information on this.
This is being used on a spring-boot application and every request is expected to will have access to a request-specific value of the variable.
Since your application uses spring, you could try accomplishing this in one of a few ways:
Spring AOP
In this approach, you write an advice that you ask spring to apply to specific methods. If your methods use the #Transactional annotation, you could have the advice be applied to those immediately after the transaction has started.
Extended TransactionManager Implementation
Lets assume your transaction is using JpaTransactionManager.
public class SecurityPolicyInjectingJpaTransactionManager extends JpaTransactionManager {
#Autowired
private EntityManager entityManager;
// constructors
#Override
protected void prepareSynchronization(DefaultTransactionStatus status, TransactionDefinition definition) {
super.prepareSynchronization(status, definition);
if (status.isNewTransaction()) {
// Use entityManager to execute your database policy param/values
// I would suggest you also register an after-completion callback synchronization
// This after-completion would clear all the policy param/values
// regardless of whether the transaction succeeded or failed
// since this happens just before it gets returned to the connection pool
}
}
}
Now simply configure your JPA environment to use your custom JpaTransactionManager class.
There are likely others, but these are the two that come to mind that I've explored.
We have been developing a web site for about a year now, using Spring 4, Hibernate, repositories and JPA. We use the OpenEntityManagerInViewFilter to keep sessions open when retrieving data via endpoints, but recently have been working with scheduled actions and are now running into problems.
We have one method in one class with a #Scheduled annotation, and another method in another class with a #Transactional annotation. The calling method finds data on the file system and passes it to the second class/method.
Here's a simplified version of the code:
<code>
// metadataRepository
#Transactional
public Metadata findOneById(String uuid);
#Transactional
public void processQueueMessages(Map<String, MessageAttributeValue> attributes ) throws IOException{
Metadata metadata = null;
try{
metadata = metadataRepository.findOneById(uuid);
// REST call to get list of locations
GeotaggerList locationList = response.getResult();
// this call just maps fields, no database interaction
Map<String, GeoLocation> locationNameMap = metadataService.getKwebLocations(locationList, null);
for (Map.Entry<String, GeoLocation> entry : locationNameMap.entrySet()){
entry.getValue().setMetadata(metadata);
metadata.getGeoLocations().add(entry.getValue()); //fails
}
metadataService.save(metadata, username);
}
} catch (Exception e){
e.printStackTrace();
return;
}
</code>
The method/class that calls processQueueMessages has a #Scheduled annotation. I tried adding a #Transactional annotation on the caller, to no avail.
I've tried a few variants of placing the annotations, but always get this error:
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: .domain.Metadata.geoLocations, could not initialize proxy - no Session
I can't do an eager fetch in the definition of the Metadata domain object because it's used a lot of places where we don't want eager loading.
I'm open to suggestions.
After much searching, the answer was pretty simple, as it usually is. I use a configuration class, and it needed an annotation of
#EnableTransactionManagement
The answer was in the spring data documentation
http://docs.spring.io/spring/docs/current/spring-framework-reference/html/transaction.html
I must have read that page a dozen times, but missed the key statement.
You could try using hibernate fetch profiles.
https://docs.jboss.org/hibernate/orm/3.5/reference/en/html/performance.html#performance-fetching-profiles
Basically, you keep the same lazy loading by default on your model, but create a fetch profile which eagerly loads your geolocation property. Then when you load the metadata from your scheduled method, activate your fetch profile.
It seems that metadata object is already out of transactional session after its retrieval from metadataRepository, otherwise there is no point in throwing lazy exception. Check that metadataRepository.findOneById(uuid) is not running in a separate transaction, e.g. by marking it as REQUIRES_NEW
In order to prevent the lazy loading exception, you should preload the data inside call to metadataRepository.findOneById either by simply calling metadata.getGeoLocations().isEmpty() or by using a fetch join query to retrieve data eagerly
I've migrated from old style of transaction management with TransactionProxyFactoryBean to a Declarative transaction management recommended by Spring to avoid exceptions with transactions that appear from time to time.
For transactions save update delete I added the annotation:
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
It works good for me.
The question is:
Should I avoid using the annotation #Transactional when the transaction only reads data?
Example:
public TradeData getTrade(long tradeId) throws Exception {
return em.find(TradeData.class, tradeId);
}
After reading this article, which says:
"Better yet, just avoid using the #Transactional annotation altogether when doing read operations, as shown in Listing 10:"
I'm a little confused and I don't quite understand it.
For read-only operations with JDBC you don't need the #Transactional because it doesn't need a transaction, Hibernate does! So you can use #Transactional(readOnly = true). Then you are sure you don't update, delete, insert something by accident.
If a particular method of your service just read information from the database yes, you can put it as read-only
#Transactional(readOnly = true)
public Object yourReadOnlyMethod(){}
Environment: Spring 3, Custom Transaction Management, JDBC Transactions
I just read the Spring docs on using the transaction template to handle transaction management. It seemed overly complex so I want to ask:
Most of my transactions are JDBC related, meaning I just declare an #Transactional on my service. But now I am making a REST service call to another site which needs to rollback if any of the following JDBC operations fail, I'll provide the rollback code in this case.
As I progress in my method, in my transaction - I want to save a reference to the REST service call (needed to roll back that action), and upon exception I just want a method myCustomRollback() called which can access the previously stored object.
Why not just provide a map in the transactionTemplate for storing stuff and define a custom rollback method on the #Transactional annotation?
This is the way I think about it, I'm not following the way Spring thinks about this. Can someone help me bridge the gap between what I want and how I accomplish it most efficiently in Spring? I only need to do this for a few special case operations.
To anyone still reading this:
I solved a similar problem with spring events - as suggested by Den Roman in option 3.
Here's the basic idea (scenario is fictional):
Whenever I perform external operations that need to be rolled back together with the transaction, I publish an event inside my #Transactional method using support from spring (org.springframework.context.ApplicationEventPublisher):
#Transactional
public String placeOrder(Order order) {
String orderId = orderServiceGateway.createOrder(order);
applicationEventPublisher.publishEvent(new OrderCreatedEvent(orderId));
workflowService.startWorkflow(orderId);
return orderId;
}
The event itself can be any object - I created a POJO with details about the remote entity to be deleted.
Then I registered a special event listener that is bound to a transaction phase - in my case to the rollback:
#TransactionalEventListener(phase = TransactionPhase.AFTER_ROLLBACK)
public void rollBackOrder(OrderCreatedEvent orderCreatedEvent) {
String orderId = orderCreatedEvent.getOrderId();
orderServiceGateway.deleteOrder(orderId);
}
Of course, it's recommended to catch & log the exception from rollback operation, not to lose the original exception from the placeOrder() method.
By default these events are synchronous, but they can be made async by additional configuration.
Here's a very good article on this mechanism, including detailed configuration and pitfalls: Transaction Synchronization and Spring Application Events (DZone)
While I don't like the solution 100% because it clutters the business logic with event publishing stuff and binds to spring, it definitely does what I expect it to do and makes it possible to pass context from the transactional method to the rollback method - which is not available through a traditional try/catch block outside of the transactional method (unless you put your context in the exception itself, which is not very nice).
I've re-read your question a few times and am not sure I understand your question completely. I assume your executing someCode and if that fails you would like to execute myCustomRollback which has some information about someCode. So I'll try to provide a Generic answer.
If you want spring to rollback some code. It will only rollback that which is rollBackAble, like jdbc transactions. Assume you have a method which performs 2 calls.
#Transactional
public void doStuff(SomeEntity entity, File file) {
persist(entity);
customFileService.createOnFileSystem(file);
throw new RunTimeException();
}
So the code above will always rollback. It will undo the persisting of your entity, but not the creation of your file, since that is not managed by Spring transactions, unless you provide custom implementation for it to be.
Second, Spring provides 2 ways of working with transactions:
Spring AOP: a proxy is created at runtime which will decorate your code with transactional stuff. If your class would be named MyClass, then Spring will create a class names MyClassProxy, which will wrap your code in transactional code.
AspectJ: at compile time your .class file will be adjusted and transactional code will be embedded inside your method.
The aspectJ approach seems harder to configure, but isn't so much and is way easier to use. Since anything which is annotated with #Transactional will be embedded (weaved) with code. For Spring AOP this is not the case. Transactional inner method calls for instance in Spring will be ignored! So aspectJ provides a more intuitive approach.
Back to what I think your question is (the code is all in 1 class):
public void doSomeCode() {
Object restCall = initialize();
try {
execute(restCall);
} catch (CustomException e) {
myCustomRollback(restCall; e);
}
}
#Transactional(rollbackFor = CustomException.class)
private void execute(Object restCall) throws CustomException {
// jdbc calls..
restCall = callRest(restCall);
throw new CustomException();
}
void myCustomRollback(Object restCall, CustomException e) {
...
}
The code above will only work with AspectJ! Since your making inner method calls which also seems to be private! AOP at runtime cannot handle this.
So what happens is everything (which is rollbackAble) in execute will be rollbacked. And in doStuff you have information about the objects which were used in execute, you now can use in myCustomRollback to rollback your REST stuff manually.
Not sure if I answered this question properly, but I hope it helps someone with a similar problem.
1 solution is to implement your own transactional manager by extending a one
2 solution is to use TransactionSynchronizationManager class
3 solution is to use #TransactionalEventListener in case you have Spring 4
Spring transaction management the default behavior for automatic rollback is for unchecked exceptions
so for a custom exception,
#Transactional(rollbackFor = CustomException.class, noRollbackFor = RuntimeException.class)
public void doSomething(...
)
the transaction be rolled back if it there is an exception that matches the specified. If an exception not matches, it is propagated to caller of the service or TransactionRolledBackException wrapper
if you use use the org.springframework.transaction.PlatformTransactionManager it is more manageable handling exceptions than template
check the documentation http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/transaction.html
you can use the AfterThrowing advice (when an exception is thrown) & call your method (myCustmRollback()) there, you can use TransactionSynchronizationManager class to get thecurrent transaction & roll it back...
alternatively.. you can use the AroundAdvice to begin & commit/rollback your transaction (this way you can use the spring provided transaction manager by using the TransactionSynchronizationManager class)