I'm using an H2 embedded database for testing, and after the tests complete, I'm seeing the system trying to close the database twice and then it hangs waiting on the last log line shown here:
...
2019-07-14 07:58:47.115 INFO 44844 --- [ Thread-2] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2019-07-14 07:58:47.115 INFO 44844 --- [ Thread-4] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2019-07-14 07:58:47.116 INFO 44844 --- [ Thread-2] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2019-07-14 07:58:47.116 INFO 44844 --- [ Thread-4] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2019-07-14 07:58:47.117 INFO 44844 --- [ Thread-4] o.s.j.d.e.EmbeddedDatabaseFactory : Shutting down embedded database: url='jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
2019-07-14 07:58:47.117 INFO 44844 --- [ Thread-2] o.s.j.d.e.EmbeddedDatabaseFactory : Shutting down embedded database: url='jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
This is happening with Spring Boot 2.1.5 and 2.1.6
In the test class I set up the database this way
#RunWith(SpringRunner.class)
#SpringBootTest
#TestPropertySource(locations = "classpath:application.yml")
#Slf4j
public class DBTest {
...
static EmbeddedDatabase informixDB;
static JdbcTemplate informixJDBCTemplate;
#BeforeClass
public static void initDb() {
informixDB = new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.H2).build();
informixJDBCTemplate = new JdbcTemplate(informixDB);
ClassPathResource initSchema = new ClassPathResource("data/informix/InformixUp.sql");
DatabasePopulator databasePopulator = new ResourceDatabasePopulator(initSchema);
DatabasePopulatorUtils.execute(databasePopulator, informixDB);
}
#AfterClass
public static void dropDb() {
ClassPathResource drop = new ClassPathResource("data/informix/InformixDown.sql");
DatabasePopulator databasePopulator = new ResourceDatabasePopulator(drop);
DatabasePopulatorUtils.execute(databasePopulator, informixDB);
}
I have this in my test/application.yml though it seems to be being ignored
spring:
jpa:
database-platform: org.hibernate.dialect.H2Dialect
h2:
console:
path: /h2-console
enabled: true
settings:
web-allow-others: true
# trace: true
datasource:
url: jdbc:h2:mem:informixDB;AUTO_SERVER=TRUE
username: sa
password:
Related
I'm using Testcontainers 1.15.3 with Spring Boot 2.4 and Junit5.
When I run my test, testcontainers starts the first container and execute flyway scripts and then stop the first container. Immediatly a second container is started (without launching flyway scripts).
My test fail because the second container does not contain data.
Abstract class:
#ExtendWith({RestDocumentationExtension.class, SpringExtension.class})
#TestPropertySource(locations = "classpath:application-test.properties")
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
public abstract class AbstractIntegrationTest {
//...
}
Test class:
class ClassTest extends AbstractIntegrationTest{
#Test
void getById () throws Exception {
//...
}
}
Property file for test (jdbc url contains jdbc:tc to launch testcontainer):
spring.flyway.locations = classpath:database/structure,classpath:database/data
spring.datasource.url=jdbc:tc:postgresql:13.3:///databasename?TC_INITSCRIPT=file:src/test/resources/database/dataset/add_user.sql
Logs after launching test :
...
...
2021-06-21 12:56:52 [main] INFO 🐳 [postgres:13.3] - Creating container for image: postgres:13.3
2021-06-21 12:56:52 [main] INFO 🐳 [postgres:13.3] - Starting container with ID: 6a41054e8ec0f9045f8db9e945134234458a0e60b6157618f6f139cdf77d0cc4
2021-06-21 12:56:52 [main] INFO 🐳 [postgres:13.3] - Container postgres:13.3 is starting: 6a41054e8ec0f9045f8db9e945134234458a0e60b6157618f6f139cdf77d0cc4
...
...
2021-06-21 12:56:53 [main] INFO o.f.core.internal.command.DbMigrate - Migrating schema "public" to version "1.1.001 - init structure"
...
...
2021-06-21 12:56:55 [main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
2021-06-21 12:56:55 [main] INFO 🐳 [postgres:13.3] - Creating container for image: postgres:13.3
2021-06-21 12:56:55 [main] INFO 🐳 [postgres:13.3] - Starting container with ID: f02fccb0706f047918d849f897ce52bf41870a53821663b21212760c779db05f
2021-06-21 12:56:55 [main] INFO 🐳 [postgres:13.3] - Container postgres:13.3 is starting: f02fccb0706f047918d849f897ce52bf41870a53821663b21212760c779db05f
As we see in the logs above, two containers are created.
Could you help me to solve this problem ?
Thank you.
The way I fixed it is by adding ?TC_DAEMON=true to the datasource url.
(in my case I used postgis, so just replace it with jdbc:tc:postgresql:13.3
spring:
datasource:
driver-class-name: org.testcontainers.jdbc.ContainerDatabaseDriver
url: jdbc:tc:postgis:9.6-2.5:///dbname?TC_DAEMON=true
username: xxx
password: xxx
flyway:
enabled: true
locations: 'classpath:db/migration'
url: ${spring.datasource.url}
user: ${spring.datasource.username}
password: ${spring.datasource.password}
validate-on-migrate: true
I found a solution for my case: remove flyway user and password properties to use only spring ones. The duplication of these properties caused the double launch of the datasourse.
Before
spring:
flyway:
locations: [ classpath:flyway-scripts ]
user: xxx
password: xxx
datasource:
url: jdbc:postgresql://localhost:5432/postgres
username: xxx
password: xxx
After
spring:
flyway:
locations: [ classpath:flyway-scripts ]
datasource:
url: jdbc:postgresql://localhost:5432/postgres
username: xxx
password: xxx
I'm currently learning Hibernate and I'm stuck at fetching data from many-to-many relationship in hibernate (spring-jpa). I'm trying to get data by id and it just doesn't work.
I know it's not realistic but one book can be taken from many persons.
The problem is, it does not matter in which class I put fetch type EAGER and in the second I put LAZY fetch type, it's throwing:
LazyInitializationException: failed to lazily initialize a collection of role
But if I put EAGER fetch type in both classes is throwing a StackOverFlowError.
The many-to-many logic goes like this we have a books in library and we have persons and many persons can take many books and book can be taken from many persons.
I have tried to put EAGER fetching in both classes but it gives me StackOverFlowError.
I have to mention that I'm using the JpaRepository interface.
The Person class:
#Entity
#Table(name = "persons")
public class Person { // (in library)
#Id
#Column(name = "id")
#GeneratedValue(strategy= GenerationType.IDENTITY)
private int id;
#Column(name = "name")
private String name;
#Column(name = "number_card")
private int numberCard;
#Column(name = "time_of_account_creating")
#Temporal(TemporalType.DATE)
private Date date;
#ManyToMany(cascade=CascadeType.ALL, fetch = FetchType.EAGER)
List<Book> books = new ArrayList<>();
public Person() {
this.date = new Date();
}
public Person(String name, int numberCard) {
this.name = name;
this.numberCard = numberCard;
this.date = new Date();
}
public void addBook(Book book) {
this.books.add(book);
}
// gettters and setters
The Book class:
#Entity
#Table(name = "books")
public class Book { // (in library)
#Id
#Column(name = "id")
#GeneratedValue(strategy= GenerationType.IDENTITY)
private int id;
#Column(name = "release_year")
private int releaseYear;
#Column(name = "name")
private String name;
#ManyToMany(mappedBy = "books", cascade=CascadeType.ALL, fetch = FetchType.LAZY)
private List<Person> persons = new ArrayList<>();
public Book() {
}
public Book(int releaseYear, String name) {
this.releaseYear = releaseYear;
this.name = name;
}
public List<Person> getPersons() {
return persons;
}
public void addPerson(Person person) {
this.persons.add(person);
}
// getters and setters
Main method in the #SpringBootApplication:
public static void main(String[] args) {
ApplicationContext context = SpringApplication.run(HibernateDemoApplication.class, args);
PersonRepository personRepository = context.getBean(PersonRepository.class);
System.out.println(personRepository.findById(4));
}
application.property:
spring.datasource.url = jdbc:mysql://localhost:3306/${DB}
spring.datasource.username = ${username}
spring.datasource.password = ${password}
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto = update
spring.jpa.show-sql=true
The output console:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.2.5.RELEASE)
2020-03-26 15:59:27.228 INFO 18912 --- [ main] c.e.h.HibernateDemoApplication : Starting HibernateDemoApplication on abu with PID 18912 (/home/yoav/hibernateDemo/target/classes started by yoav in /home/yoav/hibernateDemo)
2020-03-26 15:59:27.232 INFO 18912 --- [ main] c.e.h.HibernateDemoApplication : No active profile set, falling back to default profiles: default
2020-03-26 15:59:28.450 INFO 18912 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2020-03-26 15:59:28.625 INFO 18912 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 131ms. Found 6 JPA repository interfaces.
2020-03-26 15:59:31.886 INFO 18912 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2020-03-26 15:59:31.928 INFO 18912 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2020-03-26 15:59:31.929 INFO 18912 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.31]
2020-03-26 15:59:32.101 INFO 18912 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2020-03-26 15:59:32.101 INFO 18912 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 4687 ms
2020-03-26 15:59:32.636 INFO 18912 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2020-03-26 15:59:33.001 INFO 18912 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.4.12.Final
2020-03-26 15:59:33.631 INFO 18912 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.0.Final}
2020-03-26 15:59:35.055 INFO 18912 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2020-03-26 15:59:37.374 INFO 18912 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2020-03-26 15:59:37.689 INFO 18912 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MySQL5InnoDBDialect
Hibernate: alter table students add constraint FKrpifpqwvgu2pg2lib5c787vs foreign key (laptop_id) references laptops (id)
2020-03-26 15:59:41.092 INFO 18912 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2020-03-26 15:59:41.105 INFO 18912 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2020-03-26 15:59:42.982 WARN 18912 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2020-03-26 15:59:43.406 INFO 18912 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2020-03-26 15:59:44.775 INFO 18912 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2020-03-26 15:59:45.009 INFO 18912 --- [ main] c.e.h.HibernateDemoApplication : Started HibernateDemoApplication in 18.339 seconds (JVM running for 18.756)
Hibernate: select person0_.id as id1_5_0_, person0_.time_of_account_creating as time_of_2_5_0_, person0_.name as name3_5_0_, person0_.number_card as number_c4_5_0_, books1_.persons_id as persons_1_6_1_, book2_.id as books_id2_6_1_, book2_.id as id1_0_2_, book2_.name as name2_0_2_, book2_.release_year as release_3_0_2_ from persons person0_ left outer join persons_books books1_ on person0_.id=books1_.persons_id left outer join books book2_ on books1_.books_id=book2_.id where person0_.id=?
Exception in thread "main" org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.example.hibernateDemo.models.Book.persons, could not initialize proxy - no Session
at org.hibernate.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:606)
at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:218)
at org.hibernate.collection.internal.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:585)
at org.hibernate.collection.internal.AbstractPersistentCollection.read(AbstractPersistentCollection.java:149)
at org.hibernate.collection.internal.PersistentBag.toString(PersistentBag.java:621)
at java.lang.String.valueOf(String.java:2994)
And I have to say it again, I am getting a StackOverFlowError if I put EAGER fetch type in both classes.
I know this thread is a few months old, but for asker in the future:
Issue:
You hide the Person#toString method. In the method you access the books field. It is fine since the fetchtype is EAGER. The issue is that you access the Book#toString where you access the persons field. This field is LAZY. Since you will not access this field within transaction you will always get the LazyInitializationException.
Solution:
Open a transaction. (google jpa transaction)
Load the entity.
Access the lazy fields. It will work!
Close the transaction: Note, all changes applied to loaded entities within transaction will be save to database automaticallay, there is no need to save them through repository DAO manually!
I have a few milions of records and I migrate it from one Oracle DB to another. We have performance problem and after discussing with my colleagues I decided to process data multithread.
We have the following artifacts:
Spring boot 2.1.6.RELEASE
HikariCP 3.2.0
Hibernate 5.3.10.Final
JDK 11.0.2
OJDBC8 12.2.0.1
I have Service class annotated #Service and inside the class is method
#Async("threadPoolTaskExecutor")
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void processMigration(int from, int to) {
int progressInterval = to - from;
ProgressBar progressBar = new ProgressBar("Progress " + Thread.currentThread().getName(), progressInterval, ProgressBarStyle.UNICODE_BLOCK);
try (Stream<SomeEntity> entityStream = someEntityRepository.streamAllInInterval(from, to)) {
progressBar.start();
entityStream.forEach(entity -> progressBar.step());
progressBar.stop();
} catch (Exception e) {
progressBar.stop();
throw e;
}
}
inside the foreach there will be some logic for processing data and this service is injected into another class call it Migrator (containing injected ThreadPoolTaskExecutor) with the following method:
public void migrate() {
crimeService.processMigration(0, 500000);
crimeService.processMigration(500000, 1000000);
}
and my SpringBootApplication class (main configuration):
#SpringBootApplication
#EnableAsync
#EnableTransactionManagement
public class MigrationApplication {
public static void main(String[] args) {
SpringApplication.run(MigrationApplication.class, args);
}
#Bean("threadPoolTaskExecutor")
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(25);
executor.setQueueCapacity(30);
executor.afterPropertiesSet();
return executor;
}
}
application.yaml looks like this:
spring:
jpa:
properties:
hibernate:
jdbc:
batch_size: 100
fetch_size: 400
dialect: org.hibernate.dialect.Oracle12cDialect
order_inserts: true
order_updates: true
datasource:
url: jdbc:oracle:thin::1521:something
username: username
password: password
hikari:
maximum-pool-size: 10
leak-detection-threshold: 30000
driver-class-name: oracle.jdbc.OracleDriver
I supposed that when I ran the mentioned code then Hikari will create 2 connections, because I have called twice the method processMigration() annotated with #Transactional. I saw in log that it created only at the begining, but when the one of the threads waiting to another then there was only one connection as active. My computer has available 4 cores, so I would expect that with the HW problem does not exist. I understand why there is only one active connection because there is only one thread running, but why the second thread waiting I cannot figure out. Please help and If someone have better approach how to migrate data than I chose, I appriciate your suggestion.
UPDATE
I found out that the JpaTransactionManager creates the transaction for both threads, but one transaction is committed immediately at the begining. When the second thread had finished the task, it has not finished the previous task.
2019-08-15 15:02:39.707 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [org.springframework.data.jpa.repository.support.SimpleJpaRepository.count]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT,readOnly
2019-08-15 15:02:39.707 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(1773290233<open>)] for JPA transaction
2019-08-15 15:02:39.717 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#698ef9d1]
2019-08-15 15:02:40.156 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:40.157 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(1773290233<open>)]
2019-08-15 15:02:40.163 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(1773290233<open>)] after transaction
Migrating 1799449 CRIMES
2019-08-15 15:02:40.186 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [com.aliter.mvsmigration.dvs.service.CrimeServiceImpl.processMigration]: PROPAGATION_REQUIRES_NEW,ISOLATION_DEFAULT
2019-08-15 15:02:40.186 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [com.aliter.mvsmigration.dvs.service.CrimeServiceImpl.processMigration]: PROPAGATION_REQUIRES_NEW,ISOLATION_DEFAULT
TASK duration: 830ms
2019-08-15 15:02:40.187 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(346995150<open>)] for JPA transaction
2019-08-15 15:02:40.187 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(1848267920<open>)] for JPA transaction
2019-08-15 15:02:40.187 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#1a69552b]
2019-08-15 15:02:40.188 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [org.springframework.data.jpa.repository.support.SimpleJpaRepository.save]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2019-08-15 15:02:40.189 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(282270616<open>)] for JPA transaction
2019-08-15 15:02:40.191 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#22c0dffa]
2019-08-15 15:02:40.192 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#817e0aa]
2019-08-15 15:02:40.212 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:40.214 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(282270616<open>)]
2019-08-15 15:02:40.229 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(282270616<open>)] after transaction
PROGRAM duration: 1507ms
mvs-migration-shell:>2019-08-15 15:02:40.389 WARN 3939 --- [lTaskExecutor-2] org.jline : Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
2019-08-15 15:02:40.389 WARN 3939 --- [lTaskExecutor-1] org.jline : Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
Progress MVSThreadPoolTaskExecutor-2 0% │ │ 0/1000 (0:00:00 / ?)
2019-08-15 15:02:41.820 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:41.821 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(346995150<open>)]
2019-08-15 15:02:41.825 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(346995150<open>)] after transaction
Progress MVSThreadPoolTaskExecutor-1 100% │██████████│ 1000/1000 (0:00:08 / 0:
2019-08-15 15:02:49.084 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:49.085 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(1848267920<open>)]
2019-08-15 15:02:49.585 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(1848267920<open>)] after transaction
2019-08-15 15:03:02.981 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-08-15 15:03:32.984 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-08-15 15:04:02.989 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-08-15 15:04:32.995 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
I found out where the problem is, in the mentioned method processMigration() there is the method streamAllInInterval(from, to), which returns the data in specified interval. It contains native SQL query:
Select f.* from someView f left join someTable em on em.id = f.utvar where em.id is null AND ROWNUM BETWEEN :from AND :to
This query returns data when I set from 0 and to 500000 but not for 500000 and 1000000. So I rewrote the query with the first mentioned answer in here SQL ROWNUM how to return rows between a specific range and now everything is working.
I wanted to make use of #Transactional, expecting that annotated methods get a separate transaction which will be committed at the end of the method.
However, if I check the DB, there was nothing committed:
#Transactional
public boolean borrowLibraryItem(Long libraryUserId, Long uniqueLibraryItemNumber) {
boolean success = false;
LibraryUser borrower = libraryUserRepository.findByLibraryUserId(libraryUserId);
LibraryItem borrowItem = libraryItemRepository.findByUniqueLibraryItemNumber(uniqueLibraryItemNumber);
success = borrower != null && borrowItem != null;
if (success) {
BorrowedByRel borrowedByRel = new BorrowedByRel(borrower, borrowItem);
borrowedByRel.setBorrowDate(LocalDateTime.now());
borrowItem.setBorrowedByRel(borrowedByRel);
// libraryItemRepository.save(borrowItem);
}
return success;
}
The code commits the changes perfectly fine when using the repository.save-method, but not without.
Configuration is done via spring boot - as far as I understood, things should work out of the box this way (this might be the part where I got something wrong):
#SpringBootApplication
#EnableNeo4jRepositories(basePackages = "yalms.libraryapi.repositories")
#EntityScan("yalms.libraryapi.entities")
#EnableTransactionManagement
public class YalmsLibraryApplication {
public static void main(String[] args) {
SpringApplication.run(YalmsLibraryApplication.class, args);
}
}
Something regarding transactions seems to be happening, as the following logging.level.org.springframework.transaction.interceptor=TRACE shows:
Getting transaction for borrowLibraryItem()..
Don't need to create transaction for findByLibraryUserId, not transactional..
Request: MATCH (n:`LibraryUser`)..
Don't need to create transaction for findByUniqueLibraryItemNumber, not transactional..
Request: MATCH (n:`LibraryItem`)..
Completing transaction for borrowLibraryItem().
I would expect though that the changes (the addition of a relationship) would be committed. Am I misunderstanding something here or do I have something not configured right? Any help would be very much appreciated, thanks in advance!
UPDATE:
I've added a Neo4jTransactionManager-Bean as suggested in the comments, unfortunately it didn't help to solve my issue:
#Bean
public SessionFactory sessionFactory() {
org.neo4j.ogm.config.Configuration configuration = new org.neo4j.ogm.config.Configuration.Builder()
.uri(databaseUrl)
.credentials(userName, password)
.build();
return new SessionFactory(configuration,"yalms.libraryapi");
}
#Bean
public Neo4jTransactionManager transactionManager() {
return new Neo4jTransactionManager(sessionFactory());
}
I've enabled trace-output regarding everything coming from spring.data.*, and it's quite surprising to me as it seems that everything seems to be working fine:
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Initializing transaction synchronization
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Getting transaction for [yalms.libraryapi.services.BorrowService.borrowLibraryItem]
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Don't need to create transaction for [org.springframework.data.neo4j.repository.support.SimpleNeo4jRepository.findByLibraryUserId]: This method isn't transactional.
DEBUG 19634 --- [nio-8080-exec-2] .s.d.n.r.q.d.DerivedGraphRepositoryQuery : Executing query for method findByLibraryUserId
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Retrieved value [org.springframework.data.neo4j.transaction.SessionHolder#4adaed6] for key [org.neo4j.ogm.session.SessionFactory#5f4fecd0] bound to thread [http-nio-8080-exec-2]
INFO 19634 --- [nio-8080-exec-2] o.n.o.drivers.bolt.request.BoltRequest : Request: MATCH (n:`LibraryUser`) WHERE n.`libraryUserId` = { `libraryUserId_0` } WITH n RETURN n,[ [ (n)<-[r_b1:`BORROWED_BY`]-(l1:`LibraryItem`) | [ r_b1, l1 ] ] ], ID(n) with params {libraryUserId_0=0}
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Don't need to create transaction for [org.springframework.data.neo4j.repository.support.SimpleNeo4jRepository.findByUniqueLibraryItemNumber]: This method isn't transactional.
DEBUG 19634 --- [nio-8080-exec-2] .s.d.n.r.q.d.DerivedGraphRepositoryQuery : Executing query for method findByUniqueLibraryItemNumber
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Retrieved value [org.springframework.data.neo4j.transaction.SessionHolder#4adaed6] for key [org.neo4j.ogm.session.SessionFactory#5f4fecd0] bound to thread [http-nio-8080-exec-2]
INFO 19634 --- [nio-8080-exec-2] o.n.o.drivers.bolt.request.BoltRequest : Request: MATCH (n:`LibraryItem`) WHERE n.`uniqueLibraryItemNumber` = { `uniqueLibraryItemNumber_0` } WITH n RETURN n,[ [ (n)-[r_b1:`BORROWED_BY`]->(l1:`LibraryUser`) | [ r_b1, l1 ] ] ], ID(n) with params {uniqueLibraryItemNumber_0=2}
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Completing transaction for [yalms.libraryapi.services.BorrowService.borrowLibraryItem]
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering beforeCommit synchronization
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering beforeCompletion synchronization
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Initiating transaction commit
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Committing Neo4j OGM transaction [org.neo4j.ogm.drivers.bolt.transaction.BoltTransaction#5a5172dc] on Session [org.neo4j.ogm.session.Neo4jSession#7dc575ae]
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering afterCommit synchronization
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Clearing transaction synchronization
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering afterCompletion synchronization
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Not closing pre-bound Neo4j Session after transaction
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Removed value [org.springframework.data.neo4j.transaction.SessionHolder#4adaed6] for key [org.neo4j.ogm.session.SessionFactory#5f4fecd0] from thread [http-nio-8080-exec-2]
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.w.s.OpenSessionInViewInterceptor : Closed Neo4j OGM Session in OpenSessionInViewInterceptor
But the update still does not end up in DB - makes kinda sense as there is no query that creates the new relation, but I do not understand why not: The retrieved entities seem to be attached to the transaction, the entities get modified within the transaction, so the changes should be committed once the transaction completes. Or did I misunderstand something here fundamentally?
Neo4j-OGM (the object graph mapper behind Spring Data Neo4j) needs an explicit save call. This is currently done by the explicit Spring Data Neo4j save call. There is no auto-commit when using Spring's transactional boundary.
Basically you have no error in your application and the explicit save call needs to be un-commented.
Some notes on the comments: You do not need to define a TransactionManager on your own within a Spring Boot applications. The spring-boot-starter-neo4j takes care of initialising Configuration, SessionFactory and an appropriate TransactionManager.
I have a spring boot app, it pulls in a jar which has two database configs, both mysql db's. Both databases look to be starting up correctly in the logs, but are not uniquely identified. It looks like default is registered and then reregistered so it appears to overwrite.
2017-07-19 10:24:16,817 INFO DriverManagerDataSource - Loaded JDBC driver: com.mysql.jdbc.Driver [tx-id=]
2017-07-19 10:24:16,937 INFO LocalContainerEntityManagerFactoryBean - Building JPA container EntityManagerFactory for persistence unit 'default' [tx-id=]
2017-07-19 10:24:16,947 INFO LogHelper - HHH000204: Processing PersistenceUnitInfo [
name: default
...] [tx-id=]
2017-07-19 10:24:16,991 INFO Version - HHH000412: Hibernate Core {5.0.12.Final} [tx-id=]
2017-07-19 10:24:16,992 INFO Environment - HHH000206: hibernate.properties not found [tx-id=]
2017-07-19 10:24:16,993 INFO Environment - HHH000021: Bytecode provider name : javassist [tx-id=]
2017-07-19 10:24:17,021 INFO Version - HCANN000001: Hibernate Commons Annotations {5.0.1.Final} [tx-id=]
2017-07-19 10:24:17,379 INFO Dialect - HHH000400: Using dialect: org.hibernate.dialect.MySQL5InnoDBDialect [tx-id=]
2017-07-19 10:24:17,593 WARN RootClass - HHH000038: Composite-id class does not override equals(): com.example.EmployeeGroupEntity$EmployeeGroupId [tx-id=]
2017-07-19 10:24:17,593 WARN RootClass - HHH000039: Composite-id class does not override hashCode(): com.example.EmployeeGroupEntity$EmployeeGroupId [tx-id=]
2017-07-19 10:24:17,943 INFO LocalContainerEntityManagerFactoryBean - Initialized JPA EntityManagerFactory for persistence unit 'default' [tx-id=]
2017-07-19 10:24:17,970 INFO DriverManagerDataSource - Loaded JDBC driver: net.sourceforge.jtds.jdbc.Driver [tx-id=]
2017-07-19 10:24:17,972 INFO LocalContainerEntityManagerFactoryBean - Building JPA container EntityManagerFactory for persistence unit 'default' [tx-id=]
2017-07-19 10:24:17,972 INFO LogHelper - HHH000204: Processing PersistenceUnitInfo [
name: default
...] [tx-id=]
2017-07-19 10:24:18,370 INFO Dialect - HHH000400: Using dialect: org.hibernate.dialect.SQLServerDialect [tx-id=]
2017-07-19 10:24:18,389 INFO LobCreatorBuilderImpl - HHH000423: Disabling contextual LOB creation as JDBC driver reported JDBC version [3] less than 4 [tx-id=]
2017-07-19 10:24:18,394 WARN RootClass - HHH000038: Composite-id class does not override equals(): com.example.EmployeeGroupEntity$EmployeeGroupId [tx-id=]
2017-07-19 10:24:18,394 WARN RootClass - HHH000039: Composite-id class does not override hashCode(): com.example.entity.EmployeeGroupEntity$EmployeeGroupId [tx-id=]
2017-07-19 10:24:18,405 WARN EntityManagerFactoryRegistry - HHH000436: Entity manager factory name (default) is already registered. If entity manager will be clustered or passivated, specify a unique value for property 'hibernate.ejb.entitymanager_factory_name' [tx-id=]
I have two classes for configuring the database connections, both of these are located in the imported jar, basically a common library.
#Configuration
#EnableTransactionManagement
#PropertySource(value = { "classpath:/${app.execution.environment}/application.properties" })
#EnableJpaRepositories(basePackages = "com.example", entityManagerFactoryRef = "mysqlEntityManager", transactionManagerRef = "mysqlTransactionManager")
#EntityScan(basePackages = { "com.example" })
public class MysqlHibernateConfig {
// beans configured here marked with #Primary
...
}
and
#Configuration
#EnableTransactionManagement
#PropertySource(value = { "classpath:/${app.execution.environment}/application.properties" })
#EnableJpaRepositories(basePackages = "com.example.another_package", entityManagerFactoryRef = "mysqlEntityManager2", transactionManagerRef = "mysqlTransactionManager2")
#EntityScan(basePackages = { "com.example.another_package" })
public class MysqlHibernateConfig2 {
...
}
I try to get data from each database through classes like this:
#Transactional("mysqlTransactionManager") // workspace complains about not being able to find this bean
#Service
public class PriceService {
}
#Transactional("mysqlTransactionManager2") // workspace complains about not being able to find this bean
#Service
public class PriceService2 {
}
PriceService gets data just fine, but PriceService2 tries to get data from mysqlTransactionManager instead of from mysqlTransactionManager2.
I also have the following classes to run my boot app:
#Configuration
#ComponentScan(basePackages = "com.example")
public class WebMvcConfig extends WebMvcConfigurerAdapter {
//config stuff
}
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
How can I force one class to use a specific transaction manager/data source? I have been trying various configs and as far as I can tell this is set up correctly. If I put both of these datasource config classes into my spring boot app, I can connect to the correct datasources no problem. But when I move them to an outside library it does not work. I have to keep these in a common library because of the structure of my projects, two separate apps use this database config.
I do have the freedom to move the non primary db into my spring boot app if necessary, but that is not working either. Ends up with the same result.