I wanted to make use of #Transactional, expecting that annotated methods get a separate transaction which will be committed at the end of the method.
However, if I check the DB, there was nothing committed:
#Transactional
public boolean borrowLibraryItem(Long libraryUserId, Long uniqueLibraryItemNumber) {
boolean success = false;
LibraryUser borrower = libraryUserRepository.findByLibraryUserId(libraryUserId);
LibraryItem borrowItem = libraryItemRepository.findByUniqueLibraryItemNumber(uniqueLibraryItemNumber);
success = borrower != null && borrowItem != null;
if (success) {
BorrowedByRel borrowedByRel = new BorrowedByRel(borrower, borrowItem);
borrowedByRel.setBorrowDate(LocalDateTime.now());
borrowItem.setBorrowedByRel(borrowedByRel);
// libraryItemRepository.save(borrowItem);
}
return success;
}
The code commits the changes perfectly fine when using the repository.save-method, but not without.
Configuration is done via spring boot - as far as I understood, things should work out of the box this way (this might be the part where I got something wrong):
#SpringBootApplication
#EnableNeo4jRepositories(basePackages = "yalms.libraryapi.repositories")
#EntityScan("yalms.libraryapi.entities")
#EnableTransactionManagement
public class YalmsLibraryApplication {
public static void main(String[] args) {
SpringApplication.run(YalmsLibraryApplication.class, args);
}
}
Something regarding transactions seems to be happening, as the following logging.level.org.springframework.transaction.interceptor=TRACE shows:
Getting transaction for borrowLibraryItem()..
Don't need to create transaction for findByLibraryUserId, not transactional..
Request: MATCH (n:`LibraryUser`)..
Don't need to create transaction for findByUniqueLibraryItemNumber, not transactional..
Request: MATCH (n:`LibraryItem`)..
Completing transaction for borrowLibraryItem().
I would expect though that the changes (the addition of a relationship) would be committed. Am I misunderstanding something here or do I have something not configured right? Any help would be very much appreciated, thanks in advance!
UPDATE:
I've added a Neo4jTransactionManager-Bean as suggested in the comments, unfortunately it didn't help to solve my issue:
#Bean
public SessionFactory sessionFactory() {
org.neo4j.ogm.config.Configuration configuration = new org.neo4j.ogm.config.Configuration.Builder()
.uri(databaseUrl)
.credentials(userName, password)
.build();
return new SessionFactory(configuration,"yalms.libraryapi");
}
#Bean
public Neo4jTransactionManager transactionManager() {
return new Neo4jTransactionManager(sessionFactory());
}
I've enabled trace-output regarding everything coming from spring.data.*, and it's quite surprising to me as it seems that everything seems to be working fine:
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Initializing transaction synchronization
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Getting transaction for [yalms.libraryapi.services.BorrowService.borrowLibraryItem]
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Don't need to create transaction for [org.springframework.data.neo4j.repository.support.SimpleNeo4jRepository.findByLibraryUserId]: This method isn't transactional.
DEBUG 19634 --- [nio-8080-exec-2] .s.d.n.r.q.d.DerivedGraphRepositoryQuery : Executing query for method findByLibraryUserId
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Retrieved value [org.springframework.data.neo4j.transaction.SessionHolder#4adaed6] for key [org.neo4j.ogm.session.SessionFactory#5f4fecd0] bound to thread [http-nio-8080-exec-2]
INFO 19634 --- [nio-8080-exec-2] o.n.o.drivers.bolt.request.BoltRequest : Request: MATCH (n:`LibraryUser`) WHERE n.`libraryUserId` = { `libraryUserId_0` } WITH n RETURN n,[ [ (n)<-[r_b1:`BORROWED_BY`]-(l1:`LibraryItem`) | [ r_b1, l1 ] ] ], ID(n) with params {libraryUserId_0=0}
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Don't need to create transaction for [org.springframework.data.neo4j.repository.support.SimpleNeo4jRepository.findByUniqueLibraryItemNumber]: This method isn't transactional.
DEBUG 19634 --- [nio-8080-exec-2] .s.d.n.r.q.d.DerivedGraphRepositoryQuery : Executing query for method findByUniqueLibraryItemNumber
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Retrieved value [org.springframework.data.neo4j.transaction.SessionHolder#4adaed6] for key [org.neo4j.ogm.session.SessionFactory#5f4fecd0] bound to thread [http-nio-8080-exec-2]
INFO 19634 --- [nio-8080-exec-2] o.n.o.drivers.bolt.request.BoltRequest : Request: MATCH (n:`LibraryItem`) WHERE n.`uniqueLibraryItemNumber` = { `uniqueLibraryItemNumber_0` } WITH n RETURN n,[ [ (n)-[r_b1:`BORROWED_BY`]->(l1:`LibraryUser`) | [ r_b1, l1 ] ] ], ID(n) with params {uniqueLibraryItemNumber_0=2}
TRACE 19634 --- [nio-8080-exec-2] o.s.t.i.TransactionInterceptor : Completing transaction for [yalms.libraryapi.services.BorrowService.borrowLibraryItem]
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering beforeCommit synchronization
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering beforeCompletion synchronization
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Initiating transaction commit
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Committing Neo4j OGM transaction [org.neo4j.ogm.drivers.bolt.transaction.BoltTransaction#5a5172dc] on Session [org.neo4j.ogm.session.Neo4jSession#7dc575ae]
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering afterCommit synchronization
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Clearing transaction synchronization
TRACE 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Triggering afterCompletion synchronization
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.t.Neo4jTransactionManager : Not closing pre-bound Neo4j Session after transaction
TRACE 19634 --- [nio-8080-exec-2] .s.t.s.TransactionSynchronizationManager : Removed value [org.springframework.data.neo4j.transaction.SessionHolder#4adaed6] for key [org.neo4j.ogm.session.SessionFactory#5f4fecd0] from thread [http-nio-8080-exec-2]
DEBUG 19634 --- [nio-8080-exec-2] o.s.d.n.w.s.OpenSessionInViewInterceptor : Closed Neo4j OGM Session in OpenSessionInViewInterceptor
But the update still does not end up in DB - makes kinda sense as there is no query that creates the new relation, but I do not understand why not: The retrieved entities seem to be attached to the transaction, the entities get modified within the transaction, so the changes should be committed once the transaction completes. Or did I misunderstand something here fundamentally?
Neo4j-OGM (the object graph mapper behind Spring Data Neo4j) needs an explicit save call. This is currently done by the explicit Spring Data Neo4j save call. There is no auto-commit when using Spring's transactional boundary.
Basically you have no error in your application and the explicit save call needs to be un-commented.
Some notes on the comments: You do not need to define a TransactionManager on your own within a Spring Boot applications. The spring-boot-starter-neo4j takes care of initialising Configuration, SessionFactory and an appropriate TransactionManager.
Related
I have a Spring Boot application where I have a thin layer between the Controller and the Service which's only purpose it to try-catch and if an exception is thrown, to persist the failed entity with a JpaRepository, for subsequent inspection.
I designed my "interceptor" like:
#Transactional
public void upload(byte[] bytes) {
try {
service.upload(bytes);
} catch (Exception e) {
failRepo.save(new Failure(bytes, e)); // code trimmed for brevity
throw e;
}
}
And my service method looks like:
#Transactional(propagation = Propagation.REQUIRES_NEW, rollbackFor = Exception.class)
public void upload(byte[] bytes);
What I expect is that, in case of an exception thrown in the service, the inner transaction will be rollbacked, the exception will pop out and the outer transaction will persist my data, but the hibernate logs show that for some reason the outer transaction also rollbacks and the failure data is not persisted.
Do I need another propagation level on the outer layer also?
Edit: the relevant logs
o.s.o.h.HibernateTransactionManager : Creating new transaction with name [com.company.interceptor.upload]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
o.s.o.h.HibernateTransactionManager : Opened new Session [SessionImpl(480607911<open>)] for Hibernate transaction
o.h.e.t.internal.TransactionImpl : On TransactionImpl creation, JpaCompliance#isJpaTransactionComplianceEnabled == false
o.h.e.t.internal.TransactionImpl : begin
o.s.o.h.HibernateTransactionManager : Exposing Hibernate transaction as JDBC [org.springframework.orm.hibernate5.HibernateTransactionManager$$Lambda$1317/0x0000000800d0c440#52318830]
o.s.o.h.HibernateTransactionManager : Found thread-bound Session [SessionImpl(480607911<open>)] for Hibernate transaction
o.s.o.h.HibernateTransactionManager : Suspending current transaction, creating new transaction with name [com.company.service.upload]
o.s.o.h.HibernateTransactionManager : Opened new Session [SessionImpl(1041560268<open>)] for Hibernate transaction
o.h.e.t.internal.TransactionImpl : On TransactionImpl creation, JpaCompliance#isJpaTransactionComplianceEnabled == false
o.h.e.t.internal.TransactionImpl : begin
o.s.o.h.HibernateTransactionManager : Exposing Hibernate transaction as JDBC [org.springframework.orm.hibernate5.HibernateTransactionManager$$Lambda$1317/0x0000000800d0c440#778c9da3]
------ other irrelevant for us logs
o.s.o.h.HibernateTransactionManager : Initiating transaction rollback
o.s.o.h.HibernateTransactionManager : Rolling back Hibernate transaction on Session [SessionImpl(1041560268<open>)]
o.h.e.t.internal.TransactionImpl : rolling back
o.s.o.h.HibernateTransactionManager : Closing Hibernate Session [SessionImpl(1041560268<open>)] after transaction
o.s.o.h.HibernateTransactionManager : Resuming suspended transaction after completion of inner transaction
stomAnnotationTransactionAttributeSource : Adding transactional method 'save' with attribute: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
o.s.o.h.HibernateTransactionManager : Found thread-bound Session [SessionImpl(480607911<open>)] for Hibernate transaction
o.s.o.h.HibernateTransactionManager : Participating in existing transaction
org.hibernate.engine.spi.ActionQueue : Executing identity-insert immediately
org.hibernate.SQL : insert into failure_table (content_type, created_time, exception_message, bytes, user_agent) values (?, ?, ?, ?, ?)
o.h.id.IdentifierGeneratorHelper : Natively generated identity: 15
o.h.r.j.i.ResourceRegistryStandardImpl : HHH000387: ResultSet's statement was not registered
o.s.o.h.HibernateTransactionManager : Initiating transaction rollback
o.s.o.h.HibernateTransactionManager : Rolling back Hibernate transaction on Session [SessionImpl(480607911<open>)]
o.h.e.t.internal.TransactionImpl : rolling back
o.s.o.h.HibernateTransactionManager : Closing Hibernate Session [SessionImpl(480607911<open>)] after transaction
Problem solution helped found by: https://stackoverflow.com/a/7125918/3214777.
The problem was that Spring rollbacks by default on runtime exception so I needed an
#Transactional(noRollbackFor = Exception.class)
on my interceptor to get it going as I would've expected.
We have written a basic streams processor using Spring Cloud Stream (3.0.9.RELEASE) and the Kafka streams binder. The actual project contains multiple classes annotated with #SpringBootApplication, each with their own functions, but only one is used at a time.
The issue we're having is that, when bundling our app into a JAR and running it as a k8s deployment, if we run any more than 1 replica at a time, those additional replicas terminate after a few minutes with no meaningful error message or obvious reason why. For example, if we run 5 replicas, 1 will stay up indefinitely and process events, but the other 4 (although they too initially start to process events) will exit shortly after. Obviously if we set our deployment to run only 1 replica there is no issue, but the processor needs to be scalable.
This section of the documentation regarding setting an application ID seems relevant:
For production deployments, it is highly recommended to explicitly specify the application ID
through configuration. This is especially going to be very critical if you are auto scaling your
application in which case you need to make sure that you are deploying each instance with the
same application ID.
but neither setting this value at the binding level (for our normalize function):
spring:
cloud:
stream:
kafka:
streams:
binder:
functions:
normalize:
applicationId: normalizer-full
nor setting it at the binder level:
spring:
cloud:
stream:
kafka:
streams:
binder:
applicationId: normalizer-full
seems to be able to keep the subsequent pods running.
Any help appreciated.
-- EDIT --
Here is a capture of the trace logging When the processor shuts down after several minutes:
2020-11-20 06:32:09.721 DEBUG 1 --- [extShutdownHook] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext#34f7cfd9, started on Fri Nov 20 06:26:59 GMT 2020
2020-11-20 06:32:09.722 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'org.springframework.integration.config.IdGeneratorConfigurer#0'
2020-11-20 06:32:09.722 DEBUG 1 --- [extShutdownHook] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext#5e1dde44, started on Fri Nov 20 06:27:23 GMT 2020, parent: org.springframework.context.annotation.AnnotationConfigApplicationContext#34f7cfd9
2020-11-20 06:32:09.722 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'lifecycleProcessor'
2020-11-20 06:32:09.723 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory#38830ea: defining beans [org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.event.internalEventListenerProcessor,org.springframework.context.event.internalEventListenerFactory,KStreamBinderConfiguration,org.springframework.boot.autoconfigure.internalCachingMetadataReaderFactory,org.springframework.boot.autoconfigure.kafka.KafkaAnnotationDrivenConfiguration,org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration,org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor,org.springframework.boot.context.internalConfigurationPropertiesBinderFactory,org.springframework.boot.context.internalConfigurationPropertiesBinder,org.springframework.boot.context.properties.BoundConfigurationProperties,org.springframework.boot.context.properties.ConfigurationBeanFactoryMetadata,org.springframework.cloud.stream.binder.kafka.streams.MultiBinderPropertiesConfiguration,provisioningProvider,kStreamBinder]; parent: org.springframework.beans.factory.support.DefaultListableBeanFactory#7cb502c
2020-11-20 06:32:09.723 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Retrieved dependent beans for bean 'org.springframework.boot.autoconfigure.internalCachingMetadataReaderFactory': [org.springframework.context.annotation.internalConfigurationAnnotationProcessor]
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'lifecycleProcessor'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'outputBindingLifecycle'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'inputBindingLifecycle'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'streamsBuilderFactoryManager'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean '_org.springframework.integration.errorLogger'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'integrationHeaderChannelRegistry'
2020-11-20 06:32:09.724 TRACE 1 --- [extShutdownHook] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'stream-builder-normalize'
2020-11-20 06:32:09.724 DEBUG 1 --- [extShutdownHook] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483547
...
I have a few milions of records and I migrate it from one Oracle DB to another. We have performance problem and after discussing with my colleagues I decided to process data multithread.
We have the following artifacts:
Spring boot 2.1.6.RELEASE
HikariCP 3.2.0
Hibernate 5.3.10.Final
JDK 11.0.2
OJDBC8 12.2.0.1
I have Service class annotated #Service and inside the class is method
#Async("threadPoolTaskExecutor")
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void processMigration(int from, int to) {
int progressInterval = to - from;
ProgressBar progressBar = new ProgressBar("Progress " + Thread.currentThread().getName(), progressInterval, ProgressBarStyle.UNICODE_BLOCK);
try (Stream<SomeEntity> entityStream = someEntityRepository.streamAllInInterval(from, to)) {
progressBar.start();
entityStream.forEach(entity -> progressBar.step());
progressBar.stop();
} catch (Exception e) {
progressBar.stop();
throw e;
}
}
inside the foreach there will be some logic for processing data and this service is injected into another class call it Migrator (containing injected ThreadPoolTaskExecutor) with the following method:
public void migrate() {
crimeService.processMigration(0, 500000);
crimeService.processMigration(500000, 1000000);
}
and my SpringBootApplication class (main configuration):
#SpringBootApplication
#EnableAsync
#EnableTransactionManagement
public class MigrationApplication {
public static void main(String[] args) {
SpringApplication.run(MigrationApplication.class, args);
}
#Bean("threadPoolTaskExecutor")
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(10);
executor.setMaxPoolSize(25);
executor.setQueueCapacity(30);
executor.afterPropertiesSet();
return executor;
}
}
application.yaml looks like this:
spring:
jpa:
properties:
hibernate:
jdbc:
batch_size: 100
fetch_size: 400
dialect: org.hibernate.dialect.Oracle12cDialect
order_inserts: true
order_updates: true
datasource:
url: jdbc:oracle:thin::1521:something
username: username
password: password
hikari:
maximum-pool-size: 10
leak-detection-threshold: 30000
driver-class-name: oracle.jdbc.OracleDriver
I supposed that when I ran the mentioned code then Hikari will create 2 connections, because I have called twice the method processMigration() annotated with #Transactional. I saw in log that it created only at the begining, but when the one of the threads waiting to another then there was only one connection as active. My computer has available 4 cores, so I would expect that with the HW problem does not exist. I understand why there is only one active connection because there is only one thread running, but why the second thread waiting I cannot figure out. Please help and If someone have better approach how to migrate data than I chose, I appriciate your suggestion.
UPDATE
I found out that the JpaTransactionManager creates the transaction for both threads, but one transaction is committed immediately at the begining. When the second thread had finished the task, it has not finished the previous task.
2019-08-15 15:02:39.707 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [org.springframework.data.jpa.repository.support.SimpleJpaRepository.count]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT,readOnly
2019-08-15 15:02:39.707 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(1773290233<open>)] for JPA transaction
2019-08-15 15:02:39.717 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#698ef9d1]
2019-08-15 15:02:40.156 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:40.157 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(1773290233<open>)]
2019-08-15 15:02:40.163 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(1773290233<open>)] after transaction
Migrating 1799449 CRIMES
2019-08-15 15:02:40.186 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [com.aliter.mvsmigration.dvs.service.CrimeServiceImpl.processMigration]: PROPAGATION_REQUIRES_NEW,ISOLATION_DEFAULT
2019-08-15 15:02:40.186 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [com.aliter.mvsmigration.dvs.service.CrimeServiceImpl.processMigration]: PROPAGATION_REQUIRES_NEW,ISOLATION_DEFAULT
TASK duration: 830ms
2019-08-15 15:02:40.187 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(346995150<open>)] for JPA transaction
2019-08-15 15:02:40.187 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(1848267920<open>)] for JPA transaction
2019-08-15 15:02:40.187 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#1a69552b]
2019-08-15 15:02:40.188 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Creating new transaction with name [org.springframework.data.jpa.repository.support.SimpleJpaRepository.save]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
2019-08-15 15:02:40.189 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Opened new EntityManager [SessionImpl(282270616<open>)] for JPA transaction
2019-08-15 15:02:40.191 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#22c0dffa]
2019-08-15 15:02:40.192 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#817e0aa]
2019-08-15 15:02:40.212 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:40.214 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(282270616<open>)]
2019-08-15 15:02:40.229 DEBUG 3939 --- [ main] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(282270616<open>)] after transaction
PROGRAM duration: 1507ms
mvs-migration-shell:>2019-08-15 15:02:40.389 WARN 3939 --- [lTaskExecutor-2] org.jline : Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
2019-08-15 15:02:40.389 WARN 3939 --- [lTaskExecutor-1] org.jline : Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
Progress MVSThreadPoolTaskExecutor-2 0% │ │ 0/1000 (0:00:00 / ?)
2019-08-15 15:02:41.820 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:41.821 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(346995150<open>)]
2019-08-15 15:02:41.825 DEBUG 3939 --- [lTaskExecutor-2] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(346995150<open>)] after transaction
Progress MVSThreadPoolTaskExecutor-1 100% │██████████│ 1000/1000 (0:00:08 / 0:
2019-08-15 15:02:49.084 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Initiating transaction commit
2019-08-15 15:02:49.085 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Committing JPA transaction on EntityManager [SessionImpl(1848267920<open>)]
2019-08-15 15:02:49.585 DEBUG 3939 --- [lTaskExecutor-1] o.s.orm.jpa.JpaTransactionManager : Closing JPA EntityManager [SessionImpl(1848267920<open>)] after transaction
2019-08-15 15:03:02.981 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-08-15 15:03:32.984 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-08-15 15:04:02.989 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
2019-08-15 15:04:32.995 DEBUG 3939 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)
I found out where the problem is, in the mentioned method processMigration() there is the method streamAllInInterval(from, to), which returns the data in specified interval. It contains native SQL query:
Select f.* from someView f left join someTable em on em.id = f.utvar where em.id is null AND ROWNUM BETWEEN :from AND :to
This query returns data when I set from 0 and to 500000 but not for 500000 and 1000000. So I rewrote the query with the first mentioned answer in here SQL ROWNUM how to return rows between a specific range and now everything is working.
I'm using an H2 embedded database for testing, and after the tests complete, I'm seeing the system trying to close the database twice and then it hangs waiting on the last log line shown here:
...
2019-07-14 07:58:47.115 INFO 44844 --- [ Thread-2] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2019-07-14 07:58:47.115 INFO 44844 --- [ Thread-4] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2019-07-14 07:58:47.116 INFO 44844 --- [ Thread-2] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2019-07-14 07:58:47.116 INFO 44844 --- [ Thread-4] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2019-07-14 07:58:47.117 INFO 44844 --- [ Thread-4] o.s.j.d.e.EmbeddedDatabaseFactory : Shutting down embedded database: url='jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
2019-07-14 07:58:47.117 INFO 44844 --- [ Thread-2] o.s.j.d.e.EmbeddedDatabaseFactory : Shutting down embedded database: url='jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
This is happening with Spring Boot 2.1.5 and 2.1.6
In the test class I set up the database this way
#RunWith(SpringRunner.class)
#SpringBootTest
#TestPropertySource(locations = "classpath:application.yml")
#Slf4j
public class DBTest {
...
static EmbeddedDatabase informixDB;
static JdbcTemplate informixJDBCTemplate;
#BeforeClass
public static void initDb() {
informixDB = new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.H2).build();
informixJDBCTemplate = new JdbcTemplate(informixDB);
ClassPathResource initSchema = new ClassPathResource("data/informix/InformixUp.sql");
DatabasePopulator databasePopulator = new ResourceDatabasePopulator(initSchema);
DatabasePopulatorUtils.execute(databasePopulator, informixDB);
}
#AfterClass
public static void dropDb() {
ClassPathResource drop = new ClassPathResource("data/informix/InformixDown.sql");
DatabasePopulator databasePopulator = new ResourceDatabasePopulator(drop);
DatabasePopulatorUtils.execute(databasePopulator, informixDB);
}
I have this in my test/application.yml though it seems to be being ignored
spring:
jpa:
database-platform: org.hibernate.dialect.H2Dialect
h2:
console:
path: /h2-console
enabled: true
settings:
web-allow-others: true
# trace: true
datasource:
url: jdbc:h2:mem:informixDB;AUTO_SERVER=TRUE
username: sa
password:
Hi i am trying to insert a record in Mysql and get the ID of the inserted user in Spring boot JPA. i am getting following error. i have seen couple of questions like this but there they provided answer as USE JDBC Template for these kind of output. is it not possible to do it??. or this is related to some other issue.
Controller
#RequestMapping(value = "/signup", method = RequestMethod.POST)
public String createsignup(#RequestParam String name,#RequestParam String email,#RequestParam String password, ModelMap model) {
int userid = 0;
User user = new User(name,email);
userrepository.save(user);
userid = user.getId();
Authentication auth = new Authentication(email,password,userid);
authrepository.save(auth);
model.put("remember_token", auth.getRemember_token());
return "redirect:/profile";
}
Model
#Entity
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int id;
private String name;
private String email;
private String location;
public User() {
}
}
Error
018-01-31 23:45:32.890 INFO 2512 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization completed in 7 ms
Hibernate: insert into user (email, location, name) values (?, ?, ?)
Hibernate: select next_val as id_val from hibernate_sequence for update
2018-01-31 23:45:41.783 ERROR 2512 --- [nio-8080-exec-2] o.hibernate.id.enhanced.TableStructure : could not read a hi value
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'blog.hibernate_sequence' doesn't exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_45]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_45]
UPDATE
application properties
spring.mvc.view.prefix=/WEB-INF/jsp/
spring.mvc.view.suffix=.jsp
logging.level.org.springframework.web=INFO
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
spring.jpa.show-sql = true
spring.jpa.hibernate.ddl-auto=none
spring.datasource.url=jdbc:mysql://localhost:3306/blog
spring.datasource.username=pranava
spring.datasource.password=**********
This the message from server log
2018-02-01 22:21:29.691 INFO 5648 --- [ restartedMain] org.hibernate.Version : HHH000412: Hibernate Core {5.2.10.Final}
2018-02-01 22:21:29.693 INFO 5648 --- [ restartedMain] org.hibernate.cfg.Environment : HHH000206: hibernate.properties not found
o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.0.1.Final}
2018-02-01 22:21:29.928 INFO 5648 --- [ restartedMain] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MySQL5Dialect
2018-02
After lot of search and try i found that User Table is default table in Postgresql. that's why it did not allowed. i have changed the table name and it worked. is there any other way is there to reuse it ?.
Make sure you have specified the correct dialect for hibernate and MySQL. It seems that value for identity column can't be set correctly.
For instance, if you use Spring Boot, check your application properties for something like:
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
Or if you use Spring MVC without Spring Boot, that you have:
hibernateJpa.setDatabasePlatform("org.hibernate.dialect.MySQL5Dialect");
in LocalContainerEntityManagerFactoryBean configuration.