How to test spring batch job within #Transactional SpringBootTest test case? - java

I just can't seem to win today...
Is there a way to read from a OneToMany relationship in a Spock SpringBootTest integration test, without annotating the test as #Transactional or adding the unrealistic spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true?
OR, is there a way to launch a Spring-Batch Job from within a #Transactional test case?
Let me elaborate...
I'm trying to get a simple Spring Boot Integration test working for my Spring Batch reporting process, which reads from tangled web of DB2 tables and generates a series of change messages for interested systems. I'm using the Groovy Spock testing framework and an H2 in-memory database filled with a representative slice of my DB2 tables' data.
At the beginning of the test, I'm attempting to use every entity from a given Table to generate entries in a change-tracking table that drives my messaging.
setup:
List allExistingTestPeople = peopleRepository.findAll()
Collections.shuffle(allExistingTestPeople)
allExistingTestPeople?.each { Person person ->
Nickname nicknames = person.nicknames
nicknames?.each { Nickname nickname ->
changeTrackingRepository.save(new Change(personId: person.id, nicknameId: nickname.id, status: NEW))
}
}
Given these as my DB2 domain classes:
#Entity
#Table(name = "T_PERSON")
public class Person {
#Id
#Column(name = "P_ID")
private Integer id;
#Column(name = "P_NME")
private String name;
#OneToMany(targetEntity = Nickname.class, mappedBy = "person")
private List<Nickname> nicknames;
}
#Entity
#Table(name = "T_NICKNAME")
public class Nickname{
#EmbeddedId
private PersonNicknamePK id;
#Column(name = "N_NME")
private String nickname;
#ManyToOne(optional = false, targetEntity = Person.class)
#JoinColumn(name = "P_ID", referencedColumnName="P_ID", insertable = false, updatable = false)
private Person person;
}
#Embeddable
public class PersonNicknamePK implements Serializable {
#Column(name="P_ID")
private int personId;
#Column(name="N_ID")
private short nicknameId;
}
But I'm getting this LazyInitializationException, even though I'm reading from that OneToMany relationship within the context of a test case...
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.my.package.db2.model.Person.nicknames, could not initialize proxy - no Session
at org.hibernate.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:602)
at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:217)
at org.hibernate.collection.internal.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:161)
at org.hibernate.collection.internal.PersistentBag.size(PersistentBag.java:350)
I came across the advice online to annotate my test case with the #Transactional annotation, which definitely got me a little further, allowing me to read from this OneToMany relationship. However, when I then attempt to launch the Spring Batch Job I'd like to test from my when clause:
#Transactional
def "Happy path test to validate I can generate a report of changes"() {
setup:
//... See above
when:
service.launchBatchJob()
then:
//... Messages are generated
}
I'm getting the exception that a Spring Batch Job can't be launched from the context of a transaction! Even though I'm using an in-memory Job manager via ResourcelessTransactionManager and MapJobRepositoryFactoryBean, since this is just a short lived scheduled script I'm writing...
java.lang.IllegalStateException: Existing transaction detected in JobRepository. Please fix this and try again (e.g. remove #Transactional annotations from client).
at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:177)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy125.createJobExecution(Unknown Source)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:134)
at com.my.package.service.MyService.launchBatchJob(MyService.java:30)
The only thing that seems to work so far is if I scrap the #Transactional annotation and instead add spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true to my application-test.properties file. BUT, this doesn't seem like a very good idea, because it's not realistic - if I add this, then even if there's a bug in my code due to a lazy-initialization-exception, I'd never see it in the tests.
Sorry for the novel, hoping someone can point me in the right direction :(
EDIT:
Also here's my In-memory Spring-Batch configuration, in which I've tried turning off the transaction validation. Unfortunately, while this gets me a little further, the Spring Batch partioner's autowired EntityManager is suddenly failing to run queries in the H2 database.
#Configuration
#EnableBatchProcessing
public class InMemoryBatchManagementConfig {
#Bean
public ResourcelessTransactionManager resourceslessTransactionManager() {
ResourcelessTransactionManager resourcelessTransactionManager = new ResourcelessTransactionManager();
resourcelessTransactionManager.setNestedTransactionAllowed(true);
resourcelessTransactionManager.setValidateExistingTransaction(false);
return resourcelessTransactionManager;
}
#Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactory(ResourcelessTransactionManager txManager)
throws Exception {
MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(txManager);
factory.setValidateTransactionState(false);
factory.afterPropertiesSet();
return factory;
}
#Bean
public JobRepository jobRepository(MapJobRepositoryFactoryBean factory) throws Exception {
return factory.getObject();
}
#Bean
public SimpleJobLauncher jobLauncher(JobRepository jobRepository) throws Exception {
SimpleJobLauncher launcher = new SimpleJobLauncher();
launcher.setJobRepository(jobRepository);
launcher.afterPropertiesSet();
return launcher;
}
#Bean
public JobExplorer jobExplorer(MapJobRepositoryFactoryBean factory) {
return new SimpleJobExplorer(factory.getJobInstanceDao(), factory.getJobExecutionDao(),
factory.getStepExecutionDao(), factory.getExecutionContextDao());
}
#Bean
public BatchConfigurer batchConfigurer(MapJobRepositoryFactoryBean mapJobRepositoryFactory,
ResourcelessTransactionManager resourceslessTransactionManager,
SimpleJobLauncher jobLauncher,
JobExplorer jobExplorer) {
return new BatchConfigurer() {
#Override
public JobRepository getJobRepository() throws Exception {
return mapJobRepositoryFactory.getObject();
}
#Override
public PlatformTransactionManager getTransactionManager() throws Exception {
return resourceslessTransactionManager;
}
#Override
public JobLauncher getJobLauncher() throws Exception {
return jobLauncher;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return jobExplorer;
}
};
}
}

This error happens because your code will be already executed in a transaction driven by Spring Batch. So running the job in the scope of a transaction is not correct. However, if you still want to disable the transaction validation done by the job repository, you can set the validateTransactionState to false, see AbstractJobRepositoryFactoryBean#setValidateTransactionState.
That said, running the job in a transaction is not the way to fix org.hibernate.LazyInitializationException. The property spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true is there for a reason, and if it works for you, I believe it is a better approach than running the entire job in a transaction (and btw, if I had to use a transaction for that, I would narrow its scope to the minimum (for example the step) and not the entire job).

You can do transactions programmatically using TransactionTemplate to run only the "setup" inside a transaction (instead of having everything in #Transactional). Unfortunately this way the transaction will be committed and you will need to do some manual cleanup.
It can be autowired as any other bean:
#Autowired
private TransactionTemplate transactionTemplate;
...and it's used this way:
transactionTemplate.execute((transactionStatus) -> {
// ...setup...
return null; // alternatively you can return some data out of the callback
});

Related

Hibernate isInitialized outside of session

Is there any way to check whether an entity's lazy collection attribute is initialized once we're outside the Transactional session?
For example, my Job entity has a lazy association to a list of Step, which is only sometimes initialized. In my controller, I would like to know if the steps collection has been initialized, although I'm met with a LazyInitializationException instead:
org.hibernate.LazyInitializationException: Unable to perform requested lazy initialization [com.overflow.stack.Job.steps] - no session and settings disallow loading outside the Session
at org.hibernate.bytecode.enhance.spi.interceptor.EnhancementHelper.throwLazyInitializationException(EnhancementHelper.java:199) ~[hibernate-core-5.4.23.Final.jar:5.4.23.Final]
at org.hibernate.bytecode.enhance.spi.interceptor.EnhancementHelper.performWork(EnhancementHelper.java:89) ~[hibernate-core-5.4.23.Final.jar:5.4.23.Final]
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.loadAttribute(LazyAttributeLoadingInterceptor.java:76) ~[hibernate-core-5.4.23.Final.jar:5.4.23.Final]
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.fetchAttribute(LazyAttributeLoadingInterceptor.java:72) ~[hibernate-core-5.4.23.Final.jar:5.4.23.Final]
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.handleRead(LazyAttributeLoadingInterceptor.java:53) ~[hibernate-core-5.4.23.Final.jar:5.4.23.Final]
at org.hibernate.bytecode.enhance.spi.interceptor.AbstractInterceptor.readObject(AbstractInterceptor.java:153) ~[hibernate-core-5.4.23.Final.jar:5.4.23.Final]
at com.overflow.stack.Job.$$_hibernate_read_steps(Job.java) ~[classes/:?]
at com.overflow.stack.Job.getSteps(Job.java:135) ~[classes/:?]
...
#Entity
public class Step {
...
}
#Entity
public class Job {
#OneToMany(mappedBy = "job", fetch = FetchType.LAZY)
private List<Step> steps = new ArrayList<>();
public List<Step> getSteps() {
return steps;
}
...
}
#Controller
public JobController {
public List<Step> getJobSteps(Long id) {
// Transactional service class method which fetches by id (does not initialize steps)
Job job = jobService.getJob(id);
if (Hibernate.isInitialized(job.getSteps())) { // Throws LazyInitializationException
return job.getSteps();
} else {
return List.of();
}
}
}
#Service
#Transactional
public JobService {
#Autowired JobRepository jobRepo;
public Job getJob(Long id) {
Job job = jobRepo.findById(id).get();
Hibernate.unproxy(job);
return job;
}
}
In this case, the exception is thrown when it attempts to execute job.getSteps() in the if statement.
I know that the session is closed by the time I leave the service call (the controller is not Transactional), although I'm wondering if there's any way to check if it's been initialized, or is the getSteps() method completely out of reach at that point?
Hibernate.isInitialized(job.getSteps()) shouldn't throw LazyInitializationException (LIE).
job.getSteps() returns a proxy, to have LIE you need to call a method of a proxy, for example job.getSteps().size().
job.getSteps() can throw LIE, if job is a proxy itself.
Anyway using entities as a response of REST API is a very bad idea. You will have a lot of issues with that.
Better to use JobEntity for an entity and Job for DTO. Just convert from JobEntity to Job on the service level. You can call Hibernate.isInitialized(jobEntity.getSteps()) on the service level and set an empty collection for the steps to the Job.
To unproxy entity
Hibernate.unproxy() returns a different object (not the same proxy). Also keep in mind that "If the proxy is uninitialized, it automatically triggers an initialization".
public Job getJob(Long id) {
Job job = jobRepo.findById(id).get();
return Hibernate.unproxy(job, Job.class);
}

Hibernate LazyInitializationException if entity is fetched in JWTAuthorizationFilter

I'm using Spring Rest. I have an Entity called Operator that goes like this:
#Entity
#Table(name = "operators")
public class Operator {
//various properties
private List<OperatorRole> operatorRoles;
//various getters and setters
#LazyCollection(LazyCollectionOption.TRUE)
#OneToMany(mappedBy = "operator", cascade = CascadeType.ALL)
public List<OperatorRole> getOperatorRoles() {
return operatorRoles;
}
public void setOperatorRoles(List<OperatorRole> operatorRoles) {
this.operatorRoles = operatorRoles;
}
}
I also have the corresponding OperatorRepository extends JpaRepository
I defined a controller that exposes this API:
#RestController
#RequestMapping("/api/operators")
public class OperatorController{
private final OperatorRepository operatorRepository;
#Autowired
public OperatorController(OperatorRepository operatorRepository) {
this.operatorRepository = operatorRepository;
}
#GetMapping(value = "/myApi")
#Transactional(readOnly = true)
public MyResponseBody myApi(#ApiIgnore #AuthorizedConsumer Operator operator){
if(operator.getOperatorRoles()!=null) {
for (OperatorRole current : operator.getOperatorRoles()) {
//do things
}
}
}
}
This used to work before I made the OperatorRoles list lazy; now if I try to iterate through the list it throws LazyInitializationException.
The Operator parameter is fetched from the DB by a filter that extends Spring's BasicAuthenticationFilter, and is then somehow autowired into the API call.
I can get other, non-lazy initialized, properties without problem. If i do something like operator = operatorRepository.getOne(operator.getId());, everything works, but I would need to change this in too many points in the code.
From what I understand, the problem is that the session used to fetch the Operator in the BasicAuthenticationFilter is no longer open by the time i reach the actual API in OperatorController.
I managed to wrap everything in a OpenSessionInViewFilter, but it still doesn't work.
Anyone has any ideas?
I was having this very same problem for a long time and was using FetchType.EAGER but today something has clicked in my head ...
#Transactional didn't work so I thought "if declarative transactions don't work? Maybe programmatically do" And they do!
Based on Spring Programmatic Transactions docs:
public class JwtAuthorizationFilter extends BasicAuthenticationFilter {
private final TransactionTemplate transactionTemplate;
public JwtAuthorizationFilter(AuthenticationManager authenticationManager,
PlatformTransactionManager transactionManager) {
super(authenticationManager);
this.transactionTemplate = new TransactionTemplate(transactionManager);
// Set your desired propagation behavior, isolation level, readOnly, etc.
this.transactionTemplate.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
}
private void doSomething() {
transactionTemplate.execute(transactionStatus -> {
// execute your queries
});
}
}
It could be late for you, but I hope it helps others.

#Transaction annotated method does not save data into a database

In my unit tests I want to persist some entities and test their retrieval from the database. They were not being saved and I figured out that when the test method was also annotated with #Transaction, anything that happened inside it did not get persisted, even though the method finished without an error.
I had previously encountered a LazyInitializationException when messing with a many-to-many lazy-loaded association and annotating the method with #Transaction seemed to fix the issue, that's why I have been using it.
What could be the cause why the entities don't get saved? There is no reason for the transaction to be rolled back, since it does not fail.
Code of related classes:
#Test
#Transactional
public void plainPersistence() throws NullParameterException {
User user = userHelper.createUser("User1", "password", null, null);
Assert.assertNotNull(userDAO.findByUsername("User1"));
}
userHelper:
#Service
public class UserHelper {
#Autowired
private UserDAO userDAO;
public User createUser(...) throws NullParameterException {
User newUser = new User(username, ...);
userDAO.save(newUser);
return newUser;
}
UserDAO's save() method subsequently calls save() on UserRepository:
#Repository
public interface UserRepository extends CrudRepository<User, Long> {
public User findByUsername(String username);
}
Since you're likely using Spring test, you should note that they are configured so that the default behaviour is to rollback the changes. To change this you should annotate your test classes with, if you are using Spring < 4.2
#TransactionConfiguration(defaultRollback = false)
otherwise, annotate the class with #Rollback(value = false)

Get org.hibernate.LazyInitializationException in spring boot integration test

I'm trying to write integration test for Spring Boot application. I have Product and GalleryImage domain model. They are in one-to-many relationship.
public class Product {
...
#OneToMany(mappedBy = "product")
private List<GalleryImage> galleryImages;
}
I have a integration test as below:
#Test
public void testProductAndGalleryImageRelationShip() throws Exception {
Product product = productRepository.findOne(1L);
List<GalleryImage> galleryImages = product.getGalleryImages();
assertEquals(1, galleryImages.size());
}
However, this test gives me a LazyInitializationException. I searched on Google and StackOverFlow, it says that the session is closed after productRepository.findOne(1L), since galleryImages are lazily loaded, so galleryImages.size() gives me this exception.
I have tried to add a #Transactional annotation on the test, but it's still not working.
Hibernate Session has been closed after following line productRepository.findOne(1L).
You can try to do Hibernate.initialize(product.getGalleryImages())
public static void initialize(Object proxy)
throws HibernateException
Force initialization of a proxy or persistent collection.
Note: This only ensures intialization of a proxy object or collection; it is not guaranteed that the elements INSIDE the collection will be initialized/materialized.
To avoid Hibernate.initialize you can create a service.
#Service
#Transactional
public class ProductService {
#Transactional(readOnly = true)
public List<GalleryImage> getImages(final long producId) throws Exception {
Product product = productRepository.findOne(producId);
return product.getGalleryImages();
}
}
If you do use Spring Data JPA in you application then dynamic finder is a good alternative.
I had similar issues in the past and the solution was indeed adding the #Transactional annotation, but contrarily to what is proposed by #Anton M answer I believe that in this case we should annotate the test instead as proposed here.
It works both ways but the annotation should be placed if and where necessary, i.e. add it on the service if you need it there and not just for testing purposes.
For example, keep this as it was:
public class Product {
...
#OneToMany(mappedBy = "product")
private List<GalleryImage> galleryImages;
}
And add the annotation here:
#Test
#Transactional
public void testProductAndGalleryImageRelationShip() throws Exception {
Product product = productRepository.findOne(1L);
List<GalleryImage> galleryImages = product.getGalleryImages();
assertEquals(1, galleryImages.size());
}

Combining JBehave with SpringJUnit4ClassRunner to enable transaction rollback

Essence:
How can I auto-rollback my hibernate transaction in a JUnit Test run with JBehave?
The problem seems to be that JBehave wants the SpringAnnotatedEmbedderRunner but annotating a test as #Transactional requires the SpringJUnit4ClassRunner.
I've tried to find some documentation on how to implement either rollback with SpringAnnotatedEmbedderRunner or to make JBehave work using the SpringJUnit4ClassRunner but I couldn't get either to work.
Does anyone have a (preferably simple) setup that runs JBehave storries with Spring and Hibernate and transaction auto-rollback?
Further infos about my setup so far:
Working JBehave with Spring - but not with auto-rollback:
#RunWith(SpringAnnotatedEmbedderRunner.class)
#Configure(parameterConverters = ParameterConverters.EnumConverter.class)
#UsingEmbedder(embedder = Embedder.class, generateViewAfterStories = true, ignoreFailureInStories = false, ignoreFailureInView = false)
#UsingSpring(resources = { "file:src/main/webapp/WEB-INF/test-context.xml" })
#UsingSteps
#Transactional // << won't work
#TransactionConfiguration(...) // << won't work
// both require the SpringJUnit4ClassRunner
public class DwStoryTests extends JUnitStories {
protected List<String> storyPaths() {
String searchInDirectory = CodeLocations.codeLocationFromPath("src/test/resources").getFile();
return new StoryFinder().findPaths(searchInDirectory, Arrays.asList("**/*.story"), null);
}
}
In my test steps I can #Inject everything nicely:
#Component
#Transactional // << won't work
public class PersonServiceSteps extends AbstractSmockServerTest {
#Inject
private DatabaseSetupHelper databaseSetupHelper;
#Inject
private PersonProvider personProvider;
#Given("a database in default state")
public void setupDatabase() throws SecurityException {
databaseSetupHelper.createTypes();
databaseSetupHelper.createPermission();
}
#When("the service $service is called with message $message")
public void callServiceWithMessage(String service, String message) {
sendRequestTo("/personService", withMessage("requestPersonSave.xml")).andExpect(noFault());
}
#Then("there should be a new person in the database")
public void assertNewPersonInDatabase() {
Assert.assertEquals("Service did not save person: ", personProvider.count(), 1);
}
(yes, the databaseSetupHelper methods are all transactional)
PersonProvider is basicly a wrapper around org.springframework.data.jpa.repository.support.SimpleJpaRepository. So there is access to the entityManager but taking control over the transactions (with begin/rollback) didn't work, I guess because of all the #Transactionals that are done under the hood inside that helper class.
Also I read that JBehave runs in a different context?session?something? which causes loss of controll over the transaction started by the test? Pretty confusing stuff..
edit:
Editet the above rephrasing the post to reflect my current knowledge and shortening the whole thing so that the question becomes more obvious and the setup less obstrusive.
I think you can skip the SpringAnnotatedEmbedderRunner and provide the necessary configuration to JBehave yourself. For example instead of
#UsingEmbedder(embedder = Embedder.class, generateViewAfterStories = true, ignoreFailureInStories = false, ignoreFailureInView = false)
you can do
configuredEmbedder()
.embedderControls()
.doGenerateViewAfterStories(true)
.doIgnoreFailureInStories(false)
.doIgnoreFailureInView(false);
Besides: why do you want to rollback the transaction? Typically you are using JBehave for acceptance tests, which run in a production-like environment. For example you first setup some data in the database, access it via Browser/Selenium and check for the results. For that to work the DB transaction has to be committed. Do you need to clean-up manually after your tests, which you can do in #AfterStories or #AfterScenario annotated methods.
I made it work by controlling transaction scope manually, rolling it back after each scenario. Just follow the official guide how to use Spring with JBehave and then do the trick as shown below.
#Component
public class MySteps
{
#Autowired
MyDao myDao;
#Autowired
PlatformTransactionManager transactionManager;
TransactionStatus transaction;
#BeforeScenario
public void beforeScenario() {
transaction = transactionManager.getTransaction(new DefaultTransactionDefinition());
}
#AfterScenario
public void afterScenario() {
if (transaction != null)
transactionManager.rollback(transaction);
}
#Given("...")
public void persistSomething() {
myDao.persist(new Foo());
}
}
I'm not familiar with JBehave, but it appears you're searching for this annotation.
#TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true).
You could also set defaultRollback to true in your testContext.

Categories