We Keep Coding

REQUIRES_NEW: Spring outer transaction rollbacks also instead of persisting when inner rollbacks

I have a Spring Boot application where I have a thin layer between the Controller and the Service which's only purpose it to try-catch and if an exception is thrown, to persist the failed entity with a JpaRepository, for subsequent inspection.
I designed my "interceptor" like:
#Transactional
public void upload(byte[] bytes) {
try {
service.upload(bytes);
} catch (Exception e) {
failRepo.save(new Failure(bytes, e)); // code trimmed for brevity
throw e;
}
}
And my service method looks like:
#Transactional(propagation = Propagation.REQUIRES_NEW, rollbackFor = Exception.class)
public void upload(byte[] bytes);
What I expect is that, in case of an exception thrown in the service, the inner transaction will be rollbacked, the exception will pop out and the outer transaction will persist my data, but the hibernate logs show that for some reason the outer transaction also rollbacks and the failure data is not persisted.
Do I need another propagation level on the outer layer also?
Edit: the relevant logs
o.s.o.h.HibernateTransactionManager : Creating new transaction with name [com.company.interceptor.upload]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
o.s.o.h.HibernateTransactionManager : Opened new Session [SessionImpl(480607911<open>)] for Hibernate transaction
o.h.e.t.internal.TransactionImpl : On TransactionImpl creation, JpaCompliance#isJpaTransactionComplianceEnabled == false
o.h.e.t.internal.TransactionImpl : begin
o.s.o.h.HibernateTransactionManager : Exposing Hibernate transaction as JDBC [org.springframework.orm.hibernate5.HibernateTransactionManager$$Lambda$1317/0x0000000800d0c440#52318830]
o.s.o.h.HibernateTransactionManager : Found thread-bound Session [SessionImpl(480607911<open>)] for Hibernate transaction
o.s.o.h.HibernateTransactionManager : Suspending current transaction, creating new transaction with name [com.company.service.upload]
o.s.o.h.HibernateTransactionManager : Opened new Session [SessionImpl(1041560268<open>)] for Hibernate transaction
o.h.e.t.internal.TransactionImpl : On TransactionImpl creation, JpaCompliance#isJpaTransactionComplianceEnabled == false
o.h.e.t.internal.TransactionImpl : begin
o.s.o.h.HibernateTransactionManager : Exposing Hibernate transaction as JDBC [org.springframework.orm.hibernate5.HibernateTransactionManager$$Lambda$1317/0x0000000800d0c440#778c9da3]
------ other irrelevant for us logs
o.s.o.h.HibernateTransactionManager : Initiating transaction rollback
o.s.o.h.HibernateTransactionManager : Rolling back Hibernate transaction on Session [SessionImpl(1041560268<open>)]
o.h.e.t.internal.TransactionImpl : rolling back
o.s.o.h.HibernateTransactionManager : Closing Hibernate Session [SessionImpl(1041560268<open>)] after transaction
o.s.o.h.HibernateTransactionManager : Resuming suspended transaction after completion of inner transaction
stomAnnotationTransactionAttributeSource : Adding transactional method 'save' with attribute: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
o.s.o.h.HibernateTransactionManager : Found thread-bound Session [SessionImpl(480607911<open>)] for Hibernate transaction
o.s.o.h.HibernateTransactionManager : Participating in existing transaction
org.hibernate.engine.spi.ActionQueue : Executing identity-insert immediately
org.hibernate.SQL : insert into failure_table (content_type, created_time, exception_message, bytes, user_agent) values (?, ?, ?, ?, ?)
o.h.id.IdentifierGeneratorHelper : Natively generated identity: 15
o.h.r.j.i.ResourceRegistryStandardImpl : HHH000387: ResultSet's statement was not registered
o.s.o.h.HibernateTransactionManager : Initiating transaction rollback
o.s.o.h.HibernateTransactionManager : Rolling back Hibernate transaction on Session [SessionImpl(480607911<open>)]
o.h.e.t.internal.TransactionImpl : rolling back
o.s.o.h.HibernateTransactionManager : Closing Hibernate Session [SessionImpl(480607911<open>)] after transaction

Problem solution helped found by: https://stackoverflow.com/a/7125918/3214777.
The problem was that Spring rollbacks by default on runtime exception so I needed an
#Transactional(noRollbackFor = Exception.class)
on my interceptor to get it going as I would've expected.

How to fix org.hibernate.PropertyAccessException: IllegalArgumentException exception in spring boot application

I am getting the below exception when i run the spring boot application.
But the same code works fine when we execute as a plain java.
Exception:
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: org.hibernate.PropertyAccessException: IllegalArgumentException occurred calling getter of com.sample.model.TRSN_LOOKUP.emClntId
at org.hibernate.property.access.spi.GetterMethodImpl.get(GetterMethodImpl.java:64)
at org.hibernate.tuple.entity.AbstractEntityTuplizer.getIdentifier(AbstractEntityTuplizer.java:230)
at org.hibernate.persister.entity.AbstractEntityPersister.getIdentifier(AbstractEntityPersister.java:4725)
This is for spring-boot standalone application running in windows
Session session = getSession();
if (!session.getTransaction().isActive() )
session.beginTransaction();
TRSN_LOOKUP trsnlookup = new TRSN_LOOKUP();
trsnlookup.setPrsnid(PrsnId);
trsnlookup.setEmClntId(clientId);
session.save(trsnlookup);
session.getTransaction().commit();

Continuation from my above comments:
Just try with naming-strategy property which will help you to use a similar name.
For example,: Spring will change entity name from TRSN_LOOKUP to tRSN_LOOKUP. So, please try adding one more property in hibernate.
There are two types of strategy, you can use 2nd one:
1. org.hibernate.cfg.ImprovedNamingStrategy
2. org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
which help you keep the same name in all DB fields.

How to create schema in Postgres DB, before liquibase start to work?

I have standalone application. It’s on java, spring-boot, postgres and it has liquibase.
I need to deploy my app and liquibase should create all tables, etc. But it should do it into custom schema not in public. All service tables of liquibase (databasechangelog and databasechangeloglock) should be in custom schema too. How can I create my schema in DB before liquibase start to work? I must do it inside my app when it’s deploying, in config or some like. Without any manual intervention into the DB.
application.properties:
spring.datasource.jndi-name=java:/PostgresDS
spring.jpa.properties.hibernate.default_schema=my_schema
spring.jpa.show-sql = false
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
spring.datasource.continue-on-error=true
spring.datasource.sql-script-encoding=UTF-8
liquibase.change-log = classpath:liquibase/changelog-master.yaml
liquibase.default-schema = my_schema
UPD:
When liquibase start, it's create two tables databasechangelogs and one more table. After that, liquibase start working. But I want liquibase in liquibase.default-schema = my_schema, but it's not exist when liquibase start to work and it an error: exception is liquibase.exception.LockException: liquibase.exception.DatabaseException: ERROR: schema "my_schema" does not exist
I want liquibase work in custom schema, not in public:
liquibase.default-schema = my_schema
but before liquibase can do it, the schema must be created. Liquibase can't do this because it not started yet and for start it needs schema.
Vicious circle.

I found a solution with my application.properties.
org.springframework.jdbc.datasource.init.ScriptUtils worked before liquibase.
Logs:
14:11:14,760 INFO [org.springframework.jdbc.datasource.init.ScriptUtils]
(ServerService Thread Pool -- 300) Executing SQL script from URL
[vfs:/content/app.war/WEB-INF/classes/schema.sql]
14:11:14,761 INFO [org.springframework.jdbc.datasource.init.ScriptUtils]
(ServerService Thread Pool -- 300) Executed SQL script from URL
[vfs:/content/app.war/WEB-INF/classes/schema.sql] in 1 ms.
14:11:14,912 ERROR [stderr] (ServerService Thread Pool -- 300) INFO 9/27/18
2:11 PM: liquibase: Successfully acquired change log lock
14:11:15,292 ERROR [stderr] (ServerService Thread Pool -- 300) INFO 9/27/18
2:11 PM: liquibase: Reading from my_schema.databasechangelog
14:11:15,320 ERROR [stderr] (ServerService Thread Pool -- 300) INFO 9/27/18
2:11 PM: liquibase: Successfully released change log lock
I just put schema.sql with CREATE SCHEMA IF NOT EXISTS my_schema; into resources dir and all working properly.
Thanks all for help.
Update: It's work for Spring boot 1.X. If you are use Spring Boot 2, you should enable schema.sql in properties file, with spring.datasource.initialization-mode=always.
More info in Spring Boot - Loading Initial Data
Update 2: In Spring Boot 2.5.2 (maybe in earlier versions too) this solution is not working now, as #peterh has wrote in comment. Sad but true. The last version I was try this solution and it's work was Spring Boot 2.0.9 In docs Spring Boot says that it was redesigned from Spring Boot 2.5.x
Update 3: Some information why they kill this feature -> https://github.com/spring-projects/spring-boot/issues/22741

You can use Spring Boot Pre-Liquibase module for this. It is exactly what it is meant for. It executes some SQL prior to executing Liquibase itself. Pre-Liquibase sets itself up in the Spring Boot AutoConfigure chain so that it is guaranteed to always execute before Liquibase.
Step 1
Add the following Starter to your project:
<dependency>
<groupId>net.lbruun.springboot</groupId>
<artifactId>preliquibase-spring-boot-starter</artifactId>
<version> ---latest-version--- </version>
</dependency>
Step 2
Add a SQL file to src/main/resources/preliquibase with a name of postgresql.sql and content like this:
CREATE SCHEMA IF NOT EXISTS ${spring.liquibase.default-schema};
The ${} syntax denotes a placeholder variable. Pre-Liquibase will resolve it from the properties in your Spring Environment.
Step 3
Set application properties like this:
spring.liquibase.default-schema=${my.db.schemaname}
spring.jpa.properties.hibernate.default_schema=${my.db.schemaname}
Now - in this example - the only thing left to decide is where the my.db.schemaname value comes from. That is your choice. The example project advocates that it should come from an OS environment variable, in particular if your are deploying to a cloud.
Final words
WARNING: Pre-Liquibase is possibly way too flexible in that it allows to execute any SQL code. Don't be tempted to put stuff in Pre-Liquibase files which rightfully belong in an Liquibase ChangeSet. Honestly, the only usage I can think of for Pre-Liquibase is to set up a database "home" (meaning a schema or a catalog) where Liquibase db objects can live so that instances of the same application can be separated by schema or catalog while residing on the same database server.
(Disclosure: I'm the author of Pre-Liquibase module)

To solve this, we need to run a SQL statement that creates the schema during Spring Boot initialization at the point when DataSource bean had been already initialized so DB connections can be easily obtained but before Liquibase runs.
By default, Spring Boot runs Liquibase by creating an InitializingBean named SpringLiquibase. This happens in LiquibaseAutoConfiguration.
Knowing this, we can use AbstractDependsOnBeanFactoryPostProcessor to configure SpringLiquibase to depend on our custom schema creating bean (SchemaInitBean in the example below) which depends on DataSource. This arranges the correct execution order.
My application.properties:
db.schema=my_schema
spring.jpa.hibernate.ddl-auto=validate
spring.jpa.open-in-view=false
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
spring.jpa.properties.hibernate.default_schema=${db.schema}
spring.datasource.url=jdbc:postgresql://localhost:5432/postgres
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.liquibase.enabled=true
spring.liquibase.change-log=classpath:db/changelog/db.changelog-master.xml
spring.liquibase.defaultSchema=${db.schema}
Add the #Configuration class below to the project, for example put it in a package processed by component scan.
#Slf4j
#Configuration
#ConditionalOnClass({ SpringLiquibase.class, DatabaseChange.class })
#ConditionalOnProperty(prefix = "spring.liquibase", name = "enabled", matchIfMissing = true)
#AutoConfigureAfter({ DataSourceAutoConfiguration.class, HibernateJpaAutoConfiguration.class })
#Import({SchemaInit.SpringLiquibaseDependsOnPostProcessor.class})
public class SchemaInit {
#Component
#ConditionalOnProperty(prefix = "spring.liquibase", name = "enabled", matchIfMissing = true)
public static class SchemaInitBean implements InitializingBean {
private final DataSource dataSource;
private final String schemaName;
#Autowired
public SchemaInitBean(DataSource dataSource, #Value("${db.schema}") String schemaName) {
this.dataSource = dataSource;
this.schemaName = schemaName;
}
#Override
public void afterPropertiesSet() {
try (Connection conn = dataSource.getConnection();
Statement statement = conn.createStatement()) {
log.info("Going to create DB schema '{}' if not exists.", schemaName);
statement.execute("create schema if not exists " + schemaName);
} catch (SQLException e) {
throw new RuntimeException("Failed to create schema '" + schemaName + "'", e);
}
}
}
#ConditionalOnBean(SchemaInitBean.class)
static class SpringLiquibaseDependsOnPostProcessor extends AbstractDependsOnBeanFactoryPostProcessor {
SpringLiquibaseDependsOnPostProcessor() {
// Configure the 3rd party SpringLiquibase bean to depend on our SchemaInitBean
super(SpringLiquibase.class, SchemaInitBean.class);
}
}
}
This solution does not require external libraries like Spring Boot Pre-Liquibase and not affected by limitations on data.sql / schema.sql support. My main motivation for finding this solution was a requirement I had that schema name must be a configurable property.
Putting everything in one class and using plain JDBC is for brevity.

Simple solution based on Pavel D. answer.
It also can be used without liquibase
#Slf4j
#Component
public class SchemaConfig implements BeanPostProcessor {
#Value("${db.schema}")
private String schemaName;
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (!StringUtils.isEmpty(schemaName) && bean instanceof DataSource) {
DataSource dataSource = (DataSource) bean;
try (Connection conn = dataSource.getConnection();
Statement statement = conn.createStatement()) {
log.info("Going to create DB schema '{}' if not exists.", schemaName);
statement.execute("create schema if not exists " + schemaName);
} catch (SQLException e) {
throw new RuntimeException("Failed to create schema '" + schemaName + "'", e);
}
}
return bean;
}
}

Spring #Transactional(Propagation.NEVER) should create Hibernate session?

Let's assume that we have correctly configured JPA backed by Hibernate (version 4.3.11) in Spring (version 4.2.7). Hibernate first level cache is enabled. We use declarative transactions.
We have OuterBean:
#Service
public class OuterBean {
#Resource
private UserDao userDao;
#Resource
private InnerBean innerBean;
#Transactional(propagation = Propagation.NEVER)
public void withoutTransaction() {
User user = userDao.load(1l);
System.out.println(user.getName()); //return userName
innerBean.withTransaction();
user = userDao.load(1l);
System.out.println(user.getName()); //return userName instead of newUserName
}
}
And InnerBean that is called from OuterBean:
#Service
public class InnerBean {
#Resource
private UserDao userDao;
#Transactional
public void withTransaction() {
User user = userDao.load(1l);
user.setName("newUserName");
}
}
Is it correct behaviour that method user.getName() in OuterBean returns the same value twice (second time is after update name in database)?
In other words is it correct behaviour that #Transactional(propagation = Propagation.NEVER) creates Hibernate session for method withoutTransaction() that causes that second call user.getName() reads from Hibernate first level cache instead of database?
EDIT
To explain problem more I attache trace from creation of hibernate sessions
TRACE org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl - Opening Hibernate Session. tenant=null, owner=org.hibernate.jpa.internal.EntityManagerImpl#c17285e
TRACE org.hibernate.internal.SessionImpl - Opened session at timestamp: 14689173439
TRACE org.hibernate.internal.SessionImpl - Setting flush mode to: AUTO
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
userName
TRACE org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl - Opening Hibernate Session. tenant=null, owner=org.hibernate.jpa.internal.EntityManagerImpl#715c48ca
TRACE org.hibernate.internal.SessionImpl - Opened session at timestamp: 14689173439
TRACE org.hibernate.internal.SessionImpl - Setting flush mode to: AUTO
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Automatically flushing session
TRACE org.hibernate.internal.SessionImpl - before transaction completion
TRACE org.hibernate.internal.SessionImpl - after transaction completion
TRACE org.hibernate.internal.SessionImpl - Closing session
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
userName
TRACE org.hibernate.internal.SessionImpl - Closing session
Now let's compare trace when I remove #Transactional(propagation = Propagation.NEVER)
TRACE org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl - Opening Hibernate Session. tenant=null, owner=org.hibernate.jpa.internal.EntityManagerImpl#4ebd2c5f
TRACE org.hibernate.internal.SessionImpl - Opened session at timestamp: 14689203905
TRACE org.hibernate.internal.SessionImpl - Setting flush mode to: AUTO
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Closing session
userName
TRACE org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl - Opening Hibernate Session. tenant=null, owner=org.hibernate.jpa.internal.EntityManagerImpl#5af84083
TRACE org.hibernate.internal.SessionImpl - Opened session at timestamp: 14689203905
TRACE org.hibernate.internal.SessionImpl - Setting flush mode to: AUTO
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Automatically flushing session
TRACE org.hibernate.internal.SessionImpl - before transaction completion
TRACE org.hibernate.internal.SessionImpl - after transaction completion
TRACE org.hibernate.internal.SessionImpl - Closing session
TRACE org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl - Opening Hibernate Session. tenant=null, owner=org.hibernate.jpa.internal.EntityManagerImpl#35f4f41f
TRACE org.hibernate.internal.SessionImpl - Opened session at timestamp: 14689203906
TRACE org.hibernate.internal.SessionImpl - Setting flush mode to: AUTO
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Setting cache mode to: NORMAL
TRACE org.hibernate.internal.SessionImpl - Closing session
newUserName
Please notice when I omit #Transactional(propagation = Propagation.NEVER) separate session is create for every invocation of method from userDao.
So my question can be formulated also as
Shouldn’t be #Transactional(propagation = Propagation.NEVER)
implemented in Spring as a guardian that prevents us from accidental use of
transaction, without any side effect (session creation)?

The behavior is correct - Hibernate will always create a session (how else would you expect it to perform any operation?), and by loading the entity you have associated it with that session. Since withoutTransaction is not participating in a transaction, the changes made within withTransaction will happen within a new transaction and shouldn't be visible unless you call refresh, which will force a re-load from the database.
I'm quoting Hibernate's official documentation:
The main function of the Session is to offer create, read and delete operations for instances of mapped entity classes. Instances may exist in one of three states:
transient: never persistent, not associated with any Session
persistent: associated with a unique Session detached: previously
persistent, not associated with any Session
Transient instances may be made persistent by calling save(), persist() or saveOrUpdate(). Persistent instances may be made transient by calling delete(). Any instance returned by a get() or load() method is persistent.
Taken from Java Persistence With Hibernate, Second Edition by Christian Bauer, Gavin King, and Gary Gregory:
The persistence context acts as a first-level cache; it remembers all entity instances you’ve handled in a particular unit of work. For example, if you ask Hibernate to load an entity instance using a primary key value (a lookup by identifier), Hibernate can first check the current unit of work in the persistence context. If Hibernate finds the entity instance in the persistence context, no database hit occurs—this is a repeatable read for an application. Consecutive em.find(Item.class, ITEM_ID) calls with the same persistence context will yield the same result.
Also from Java Persistence With Hibernate, Second Edition:
The persistence context cache is always on—it can’t be turned off. It ensures the following:
The persistence layer isn’t vulnerable to stack overflows in the case of circular references in an object graph.
There can never be conflicting representations of the same database row at the end of a unit of work. The provider can safely write all changes made to an entity instance to the database.
Likewise, changes made in a particular persistence context are always immediately visible to all other code executed inside that unit of work and its persistence context. JPA guarantees repeatable entity-instance reads.
Concerning transactions, here's an excerpt taken from official Hibernate's documentation:
Defines the contract for abstracting applications from the configured underlying means of transaction management. Allows the application to define units of work, while maintaining abstraction from the underlying transaction implementation (eg. JTA, JDBC).
So, to sum it up, withTransaction and withoutTransaction will not share UnitOfWork and therefore will not share the first-level cache, which is why the second load returns the original value.
As to the reasons why these two methods do not share the unit of work, you can refer to Shailendra's answer.
EDIT:
You seem to misunderstand something. A session must always be created - that's how Hibernate works, period. Your expectation of no sessions being created is equal to expecting to execute a JDBC query without having a JDBC connection :)
The difference between your two examples is that with #Transactional(propagation = Propagation.NEVER) your method is intercepted and proxied by Spring and only a single session is created for the queries in withoutTransaction. When you remove the annotation you exclude your method from Spring's transactional interceptor so a new session will be created for each DB-related operation. I repeat again, and I cannot stress this enough - you must have an open session to perform any queries.
As far as guarding goes - try swapping the annotations on the two methods by making withTransaction use Propagation.NEVER and withoutTransaction use the default #Transactional annotation and see what happens (spoiler: you'll get an IllegalTransactionStateException).
EDIT2:
As for why the session is shared between two loads in the outer bean - that's just what JpaTransactionManager is supposed to do, and by annotating your method with #Transactional you've notified Spring that it should use the configured transaction manager to wrap your method. Here's what the official documentation says about JpaTransactionManager's expected behavior:
PlatformTransactionManager implementation for a single JPA EntityManagerFactory. Binds a JPA EntityManager from the specified factory to the thread, potentially allowing for one thread-bound EntityManager per factory. SharedEntityManagerCreator and #PersistenceContext are aware of thread-bound entity managers and participate in such transactions automatically. Using either is required for JPA access code supporting this transaction management mechanism.
Also, to know how Spring is handling declarative transaction management (i.e. #Transactional annotations on methods), refer to the official documentation. For ease of navigation, I'll include a quote:
The most important concepts to grasp with regard to the Spring Framework’s declarative transaction support are that this support is enabled via AOP proxies, and that the transactional advice is driven by metadata (currently XML- or annotation-based). The combination of AOP with transactional metadata yields an AOP proxy that uses a TransactionInterceptor in conjunction with an appropriate PlatformTransactionManager implementation to drive transactions around method invocations.

First of all, as you use hibernate behind JPA API I will use the term EntityManager instead of session (strictly the same thing, just a matter of terminology).
Every access to the database using JPA will involve an EntityManager, you are fetching entities, you need an EntityManager (EM). What's called 1st level cache is nothing more than the EM managed entities state.
Theoretically the lifecycle of the EM is short and bound to a unit of work (and so generally to a transaction, see Struggling to understand EntityManager proper use).
Now JPA can be used in different way : Container-Managed or User-Managed persistence. When the EM is managed by the container (your case, here spring is the container) this last is in charge of managing the EM scope / lifecycle (create, flush and destroy it for you). As the EM is bounded to a transaction / Unit of Work, this task is delegated to the TransactionManager (the object handling the #Transactional annotations).
When you annotate a method using #Transactional(propagation = Propagation.NEVER), you are creating a spring logical transaction scope which will ensure that there is no existing underlying JDBC transaction bound to an eventual existing EM, which will not create one and will use JDBC autocommit mode but which will create an EM for this logical transaction scope if none already exists.
Regarding the fact that a new EM instance is created for each DAO call when no transaction logical scope is defined, you have to remember that you cannot access database using JPA outside of the EM. AFAIK hibernate used to throw a no session bound to thread error in this case but this may have evolved with later releases, otherwise your DAO may be annotated with #Transactional(propagation = Propagation.SUPPORT) which would also automatically create an EM if no enclosing logical scope exists. This is a bad practice as transaction should be defined at the unit of work, eg. the service level and not the DAO one.

#Transactional(propagation = Propagation.NEVER) would still create a session. If you are using Spring/Hibernate/JPA combination for non distributed transactions then you are most certainly using JpaTransactionManager as the Spring
transaction manager. The answer to your question lies in this class. A good idea would be do use a debugger in your IDE to follow what's happening. The doBegin method of this class ( which is called by Spring transaction infrastructure is :-
protected void doBegin(Object transaction, TransactionDefinition definition) {
JpaTransactionObject txObject = (JpaTransactionObject) transaction;
if (txObject.hasConnectionHolder() && !txObject.getConnectionHolder().isSynchronizedWithTransaction()) {
throw new IllegalTransactionStateException(
"Pre-bound JDBC Connection found! JpaTransactionManager does not support " +
"running within DataSourceTransactionManager if told to manage the DataSource itself. " +
"It is recommended to use a single JpaTransactionManager for all transactions " +
"on a single DataSource, no matter whether JPA or JDBC access.");
}
try {
if (txObject.getEntityManagerHolder() == null ||
txObject.getEntityManagerHolder().isSynchronizedWithTransaction()) {
EntityManager newEm = createEntityManagerForTransaction();
if (logger.isDebugEnabled()) {
logger.debug("Opened new EntityManager [" + newEm + "] for JPA transaction");
}
txObject.setEntityManagerHolder(new EntityManagerHolder(newEm), true);
}
EntityManager em = txObject.getEntityManagerHolder().getEntityManager();
// Delegate to JpaDialect for actual transaction begin.
final int timeoutToUse = determineTimeout(definition);
Object transactionData = getJpaDialect().beginTransaction(em,
new DelegatingTransactionDefinition(definition) {
#Override
public int getTimeout() {
return timeoutToUse;
}
});
txObject.setTransactionData(transactionData);
// Register transaction timeout.
if (timeoutToUse != TransactionDefinition.TIMEOUT_DEFAULT) {
txObject.getEntityManagerHolder().setTimeoutInSeconds(timeoutToUse);
}
// Register the JPA EntityManager's JDBC Connection for the DataSource, if set.
if (getDataSource() != null) {
ConnectionHandle conHandle = getJpaDialect().getJdbcConnection(em, definition.isReadOnly());
if (conHandle != null) {
ConnectionHolder conHolder = new ConnectionHolder(conHandle);
if (timeoutToUse != TransactionDefinition.TIMEOUT_DEFAULT) {
conHolder.setTimeoutInSeconds(timeoutToUse);
}
if (logger.isDebugEnabled()) {
logger.debug("Exposing JPA transaction as JDBC transaction [" +
conHolder.getConnectionHandle() + "]");
}
TransactionSynchronizationManager.bindResource(getDataSource(), conHolder);
txObject.setConnectionHolder(conHolder);
}
else {
if (logger.isDebugEnabled()) {
logger.debug("Not exposing JPA transaction [" + em + "] as JDBC transaction because " +
"JpaDialect [" + getJpaDialect() + "] does not support JDBC Connection retrieval");
}
}
}
// Bind the entity manager holder to the thread.
if (txObject.isNewEntityManagerHolder()) {
TransactionSynchronizationManager.bindResource(
getEntityManagerFactory(), txObject.getEntityManagerHolder());
}
txObject.getEntityManagerHolder().setSynchronizedWithTransaction(true);
}
catch (TransactionException ex) {
closeEntityManagerAfterFailedBegin(txObject);
throw ex;
}
catch (Throwable ex) {
closeEntityManagerAfterFailedBegin(txObject);
throw new CannotCreateTransactionException("Could not open JPA EntityManager for transaction", ex);
}
}
The transactional resource when using JPA is actually the entity manager (underlying implementation is session in hibernate) as you can see and this is the first thing this method does
EntityManager em = txObject.getEntityManagerHolder().getEntityManager();
So definitely an entity manager / session is created. The transaction attributes are then passed to the underlying JpaDialect (HibernateJpaDialect) via TransactionDefinition. This class in turn actually gets the underlying Hibernate Session and the transaction API of session.
HibernateJpaDialect {
........
public Object beginTransaction(EntityManager entityManager, TransactionDefinition definition)
Session session = getSession(entityManager);
entityManager.getTransaction().begin();
......
......
}
......

I don't think this is correct behavior. It is true what the colleagues are saying that even without transaction the hibernate is creating a session. But this mean that we are facing two sessions S1 and S2 for the two separate reads from the DAO. At the same time L1 cache is always per session, so it does not make sense for two separate sessions to have a hit for the L1 cache. It seems like your Spring is not respecting the #Transactional(propagation = Propagation.NEVER)
The #Transactional(propagation = Propagation.NEVER) should be equivalent to if you just initialize your service from a main method and do the subsequent calls to the DAO yourself.
Try it in a main class and see how it will react. I doubt it will hit the L1 cache again.
Also I will copy paste the doc from Sprint on propagation NEVER:
NEVER Execute non-transactionally, throw an exception if a transaction
exists.
One more question - Is the hibernate configured to AutoCommit. Is it possible that the "runInTransaction" - method is not committing ?

Thread lock after opening new EntityManager

I have a very weird error working with Spring JPA transactions. The thread is locked around 16 minutes and then continues without any problem.
Here is the situation:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class A {
public String encrypt(String str){
LOG.debug("encrypting...");
// just data base read operations
}
public String encrypt(String str, String str2){
// read and write database operations.
}
public String foo(...){
// read and write database operations.
}
public String bar(...){
// read and write database operations.
}
}
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class B {
public String doSomething(...){
LOG.debug("calling encrypt method...");
String chain1 = this.a.encrypt("whatever");
LOG.debug("calling encrypt method...");
String chain2 = this.a.encrypt("again");
LOG.debug("calling encrypt method...");
String chain3 = this.a.encrypt("and again");
...
}
}
Taking a look to the log file I see that it takes 16 minutes from log "calling encrypt method" to "encripting". So, have activated JTA logs and this is what I see:
15:09:04.317 DEBUG e.i.n.p.d.TipoMensajeDaoDelegate [45] - obteniendo mensaje para tipo operacion 0104 y protocolo 03
15:09:04.318 DEBUG o.s.orm.jpa.JpaTransactionManager [332] - Found thread-bound EntityManager [org.hibernate.ejb.EntityManagerImpl#4e6b01e9] for JPA transaction
15:09:04.319 DEBUG o.s.orm.jpa.JpaTransactionManager [471] - Participating in existing transaction
15:09:04.320 DEBUG o.s.orm.jpa.JpaTransactionManager [332] - Found thread-bound EntityManager [org.hibernate.ejb.EntityManagerImpl#4e6b01e9] for JPA transaction
15:09:04.321 DEBUG o.s.orm.jpa.JpaTransactionManager [471] - Participating in existing transaction
15:09:04.324 DEBUG e.i.n.c.p.p.b.B [485] - calling encrypt method...
15:09:04.325 DEBUG o.s.orm.jpa.JpaTransactionManager [332] - Found thread-bound EntityManager [org.hibernate.ejb.EntityManagerImpl#4e6b01e9] for JPA transaction
15:09:04.326 DEBUG o.s.orm.jpa.JpaTransactionManager [416] - Suspending current transaction, creating new transaction with name [es.indra.nnp.gestorclaves.GestorClavesServiceImpl.cifrar]
15:09:04.326 DEBUG o.s.orm.jpa.JpaTransactionManager [369] - Opened new EntityManager [org.hibernate.ejb.EntityManagerImpl#27f2b012] for JPA transaction
...
15:24:29.954 DEBUG o.s.orm.jpa.JpaTransactionManager [408] - Not exposing JPA transaction [org.hibernate.ejb.EntityManagerImpl#27f2b012] as JDBC transaction because JpaDialect [org.springframework.orm.jpa.DefaultJpaDialect#4d832b01] does not support JDBC Connection retrieval
15:24:29.955 DEBUG e.i.n.g.A [146] - encrypting
15:24:29.956 DEBUG o.s.orm.jpa.JpaTransactionManager [332] - Found thread-bound EntityManager [org.hibernate.ejb.EntityManagerImpl#27f2b012] for JPA transaction
15:24:29.957 DEBUG o.s.orm.jpa.JpaTransactionManager [471] - Participating in existing transaction
15:24:29.958 DEBUG o.s.orm.jpa.JpaTransactionManager [332] - Found thread-bound EntityManager [org.hibernate.ejb.EntityManagerImpl#27f2b012] for JPA transaction
15:24:29.958 DEBUG o.s.orm.jpa.JpaTransactionManager [471] - Participating in existing transaction
15:24:29.962 DEBUG o.s.orm.jpa.JpaTransactionManager [332] - Found thread-bound EntityManager [org.hibernate.ejb.EntityManagerImpl#27f2b012] for JPA transaction
15:24:29.962 DEBUG o.s.orm.jpa.JpaTransactionManager [471] - Participating in existing transaction
...
Here, the facts:
Error does not happen always, but when it does it is always in the same point.
After 16 minutes more or less, the thread continues and calls the same method few times with no problem and finish correctly.
When it happens, it is always around 15 minutes and 30 seconds.
It happens with no concurrency. Anyway, when a thread is locked, if I start another thread there is no problem. The second thread is processed while the first is still locked.
DDBB has being checked looking for database locks while the lock was happening. No database locks were found.
Others methods form class A are called from others points of the code with no problem.
Just happens on production environment. You can imagine how difficult is to do changes.
Database connection is done via JNDI to MySql and application run in Tomcat.
I know that with this information is difficult to find out where the problem is. Just I hope someone can contribute with some thoughts that help me to find what is happening.

For me this sounds pretty much like this SO question.
Using REQUIRES_NEW will always ensure a new Transaction, so if there is an already existing one that should be suspended.
But since nested transactions are not supported by the JPATransactionManager:
On JDBC 3.0, this transaction manager supports nested transactions via
JDBC 3.0 Savepoints. The
AbstractPlatformTransactionManager.setNestedTransactionAllowed(boolean)
"nestedTransactionAllowed"} flag defaults to "false", though, as
nested transactions will just apply to the JDBC Connection, not to the
JPA EntityManager and its cached objects. You can manually set the
flag to "true" if you want to use nested transactions for JDBC access
code which participates in JPA transactions (provided that your JDBC
driver supports Savepoints). Note that JPA itself does not support
nested transactions! Hence, do not expect JPA access code to
semantically participate in a nested transaction.
So the two transactions will share the same JDBC connection and there might be some locking involved. Is it that the transaction timeout is set to 15 minutes and that's why you see it hanging around for this amount of time?