We Keep Coding

Disable transactions in my Spring Batch job

I need to insert a record in my database so another system can read that record. After that, I will write (with my ItemWriter) the response I recieve from that system in a CSV file.
My problem is that the system can't read the record since Spring Batch is transactional. How can I disable that property?

Disable spring batch transaction:
User .transactionAttribute(attributes()) method from AbstractTaskletStepBuilder with a parameter DefaultTransactionAttribute.
Build DefaultTransactionAttribute object with transaction propagation.
Example:
#Bean
public Step springBatchStep() {
return this.stepBuilderFactory.get("springBatchStep")
...
.reader()
.processor()
.writer()
.transactionAttribute(attributesForTransaction())
...
.build();
}
private DefaultTransactionAttribute attributesForTransaction() {
DefaultTransactionAttribute attribute = new DefaultTransactionAttribute();
attribute.setPropagationBehavior(Propagation.NOT_SUPPORTED.value());
attribute.setIsolationLevel(Isolation.DEFAULT.value());
return attribute;
}

You do not want to disable transaction enforcement, that can open you up to a serious data integrity problem. Even a 1% error issue can easily cause cause ten's of thousands, or even tens of millions of incomplete or bad records. Especially if say the one of the databases you are interacting with, or file systems you are writing to become unavailable. Which over time WILL happen. It would also break the job retry features.
A better option would be to break the process up into multiple steps, or jobs so the transaction boundaries fit your process. So one of them write out to this other database, and then the next step or job does the reading and writing to the file.

Spring Batch is highly opinionated Framework. You can split Step or do your job inside REQUIRES_NEW transaction. Or choose alternative less restrictive Framework.

how to make spring boot rest api as stateful

I want to make web application with REST and spring boot. Rest web service is stateless. I want to make it stateful so that the information server send to client after first request should be used in upcoming request. Or the execution done in first or second request will be used further.
Can we generate some session id for this and this session id client can send to sever in followed requests ? If yes, then
if state is changing (values get modified due to some manipulation) of some objects/beans. So how can we save the state inorder of objects/beans to make it stateful and what is the scope of these beans (whose value will be modified) and those classes/beans which will give call to these beans as multiple clients or users will be using this web application ?

Restful API's are not stateful by design, if you make them stateful using a server side then its not REST!
What you need a correlation Id which is a recognised pattern in a distributed system design. Correlation Id will let you tie requests together.

Sessions are typically an optimization to improve performance when running multiple servers. They improve performance by ensuring that a clients requests always get sent to the same server which has cached the clients data.
If you only want to run a single server, you won't have to worry about sessions. There are two common approaches to solve this problem.
1. In Memory State
If the state you want to maintain is small enough to fit into memory, and you don't mind losing it in the event of a server crash or reboot, you can save it in memory. You can create a spring service which holds a data structure. Then you can inject that service into your controllers and change the state in your http handlers.
Services are singletons by default. So state stored in a service is accessible to all controllers, components, and user requests. A small pseudo example is bellow.
Service Class
#Service
public class MyState
{
private Map<String, Integer> sums = new HashMap<>();
public synchronized int get(String key) {
return sums.get(key);
}
public synchronized void add(String key, int val) {
int sum = 0;
if (sums.contains(key)) {
sum = sum.get(key);
}
sum += val;
sums.put(key, (Integer)sum);
}
}
Controller Class
#RestController
#RequestMapping("/sum")
public class FactoryController
{
#Autowired
private MyState myState;
#PostMapping(consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
#ResponseStatus(HttpStatus.OK)
#ResponseBody
public SuccessResponse saveFactory(#RequestBody KeyVal keyVal)
{
myState.add(keyVal.getKey(), keyVal.getValue());
}
}
Note: This approach will not work if you are running multiple servers behind a load balancer, unless you use a more complex solution like a distributed cache. Sessions can be used to optimize performance in this case.
2. Database
The other option is to just use a database to store your state. This way you won't lose data if you crash or reboot. Spring supports the hibernate persistence framework and you can run a database like Postgres.
Note: If you are running multiple servers you will need a more complex solution since hibernate caches data in memory. You will have to plug hibernate into a distributed cache to synchronize in memory state across multiple servers. Sessions could be used as a performance optimization here.
Important
Whenever you are modifying state you need to make sure you are doing it in a thread safe manner, otherwise your state may be incorrect.

Spring + Hibernate+ HikariCP: how to handle DB connection while doing long running REST call?

I have a project running on Spring Boot 1.3.8, Hikari CP 2.6.1 and Hibernate (Spring ORM 4.2.8). The code on service layer looks like this:
public void doStuff() {
A a = dao.findByWhatever();
if (a.hasProperty()) {
B b = restService.doRemoteRequestWithRetries(); // May take long time
}
a.setProp(b.getSomethig());
dao.save(b);
}
Hikari configuration has this: spring.datasource.leakDetectionThreshold=2000.
The problem is that external REST service is quite slow and often takes 2+ seconds to respond, as a result we see a lot of java.lang.Exception: Apparent connection leak detected which are nothing else but false negatives, though the problem can be clearly seen: we hold DB connection for the time we executing rest request.
The question would be: how to properly decouple DB and REST stuff? Or how to tell hibernate to release connection in between? So that we return DB connection to pool while waiting for REST response.
I have tried setting hibernate.connection.release_mode=AFTER_TRANSACTION and it kind of helps, at least we do not have connection leak exceptions. The only problem is that our tests started showing this:
2018-04-17 15:48:03.438 WARN 94029 --- [ main] o.s.orm.jpa.vendor.HibernateJpaDialect : JDBC Connection to reset not identical to originally prepared Connection - please make sure to use connection release mode ON_CLOSE (the default) and to run against Hibernate 4.2+ (or switch HibernateJpaDialect's prepareConnection flag to false`
The tests are using injected DAO to insert records in DB and later check them via application API. They are not annotated with #Transactional and the list of listeners looks like this:
#TestExecutionListeners({
DependencyInjectionTestExecutionListener.class,
TransactionalTestExecutionListener.class,
TransactionDbUnitTestExecutionListener.class
})
Any ideas what could be the problem with tests?

In the code
public void doStuff() {
A a = dao.findByWhatever();
if (a.hasProperty()) {
B b = restService.doRemoteRequestWithRetries(); // May take long time
}
a.setProp(b.getSomethig());
dao.save(b);
}
I see three tasks here - fetching entity A, connecting to remote service and updating entity A. And all these are in same transaction, so the underlying connection will be held till the method is complete.
So the idea is to split the tasks one and three into separate transactions, there by allowing the connection to be releases before making the call to remote service.
Basically, with spring boot you need to add spring.jpa.open-in-view=false. This will not register OpenEntityManagerInViewInterceptor and thus entityManager (in-turn connection) is not bound to the current thread/request.
Subsequently, split the three tasks into separate methods with #Transactional. This helps us bind the entityManager to the transaction scope and releasing connection at end of transaction method.
NOTE: And do ensure that there isn't any transaction started/in progress before (i.e., caller - like Controller etc) calling these methods. Else the purpose is defeated and these new #Transactional methods will run in the same transaction as before.
So the high-level approach could look like below:
In spring boot application.properties add property spring.jpa.open-in-view=false.
Next you need to split doStuff method into three methods in new service class. Intent is to ensure they use different transactions.
First method with #Transactionalwill call A a = dao.findByWhatever();`.
Second method makes remote call.
Third method with #Transactionalwill call rest of the code with JPA merge or hibernate saveOrUpdate on objecta`.
Now Autowired this new service in your current code and call the 3 methods.

Hooking into start of JDBC transaction

I have a Spring Boot webapp connected to a Postgres 9.6 database.
I use Spring's JdbcTemplate for executing SQL statements.
Each table in my database has triggers for INSERT, CREATE and DELETE statments. These triggers copy the affected rows into a history table.
I want the triggers to also save the application user ID of the user who made the change.
According to https://stackoverflow.com/a/13172964/2591231 I can achieve my goal by having the application insert the current user id into a temporary table at the start of each transaction and having the triggers read from the temporary table.
A similar method, mentioned in several other places, is executing:
SET LOCAL application_name = "my_application_user", then reading application_name inside the triggers. Again, this has to be done at the start of each transaction.
I'm looking for way, which is orthogonal to business code (I don't want each DAO to explicitly set user ID), to hook into the start of each transaction in order to run a specific SQL statement before any other statement in the same transaction.
I need this to work for both implicit transactions (single invocations of JdbcTemplate) and transactions demarcated declaratively with Spring's #Transactional annotation.

First of all, JdbcTemplate does not provide transaction support out-of-the-box (see here). So, in order to intercept all #Transaction annotated code AND every call to JdbcTemplate, this could be done at DataSource level, as commented earlier by Serge Bogatyrev.
I have a Spring Web project where I tested this approach. I defined a replacement DataSource #Bean called MyDataSource that extends BasicDataSource, and overwrites its getConnection() method so that it creates the temp table and insert the user_id before returning the connection.
It worked for #Transaction calls and pure JdbcTemplate calls.
If you want to strictly tie this temp table update at the start of each transaction, do this same strategy for defining the PlatformTransactionManager #Bean. You only need to overwrite the doBegin() method. And don't forget to annotate with #Transaction all methods calling JdbcTemplate.
PS1: Make sure to call DROP TABLE IF EXISTS temp_table_name prior creating the temp table, in order to replace the DISCARD ALL on connection returning to pool, as mentioned here.
PS2: This whole solution of creating a temp table doesn't smell well. I wouldn't implement it myself. I would prefer to take a deep breath and add created_by and updated_by columns to all my tables.

You can take advantage of Spring AOP for setting up the user. The aspect will make a call to the database to set up the application user.
In my example, a stored procedure is used to set up the application user responsible for creating, modifying, or deleting a record . You can customize it according to your requirements. Here is the example aspect which retrieves the user from the HTTP request and then makes a call to the stored procedure,
#Component
#Aspect
#Slf4j
public class SetUserAspect {
private final JdbcTemplate jdbcTemplate;
#Autowired
public SetUserAspect(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
#Before("execution(* com.basaki.example.service.BookService.*(..) )")
public void setUser(JoinPoint jp) {
log.info("In class: " + jp.getSignature().getDeclaringTypeName() +
" - before method: " + jp.getSignature().getName());
HttpServletRequest request =
((ServletRequestAttributes) RequestContextHolder.currentRequestAttributes()).getRequest();
if (request != null) {
String user = request.getHeader("user");
if (user != null) {
log.info("Setting user " + user);
SimpleJdbcCall
jdbcCall = new SimpleJdbcCall(jdbcTemplate)
.withSchemaName("example_book_schema")
.withFunctionName("set_user");
SqlParameterSource
in =
new MapSqlParameterSource().addValue("audit_user",
user);
jdbcCall.executeFunction(String.class, in);
}
}
}
}
All the CRUD operations are performed in BookService, essentially a DAO.
Here is the stored procedure used for setting up the user,
CREATE OR REPLACE FUNCTION example_book_schema.set_user(
audit_user TEXT
) RETURNS BOOLEAN STABLE LANGUAGE SQL AS $$
SELECT set_config('book.audit_user', audit_user, true);
SELECT TRUE;
Restricting Pointcuts to Only Transactional Methods
You can restrict the points cuts to only transactional methods in BookService by adding an additional clause in the Before advice.
#Before("execution(* com.basaki.example.service.BookService.*(..) ) " +
"&& #annotation(annotation)")
public void setUser(final JoinPoint jp, final Transactional annotation) {
...
}

You can use #EntityListeners to listen change of entity in application context, then collect whatever information (entity value, authentication user, etc...) and then insert to your history table. Example you can follow here: http://www.baeldung.com/database-auditing-jpa

You can create view, add user id column, and use your triggers to deal with updates. So that is yet another way to code it at DB side. That way you are supposed to pass it every time, so no other changes are needed.
Going to Java/Spring side.
A bit oldish style: TransactionTemplate - that way you have full control, but your dao needs more code, since transaction management needs to be done there.
Other option is to create proxy on
org.springframework.jdbc.datasource.DataSourceTransactionManager and do your job at doBegin, then your proxy needs to be passed to transaction manager. And that is the way to go for me.

Apache Wicket, OpenJPA and Spring/ROO

I've set up a Spring ROO application, set persistence to OpenJPA and created some entities.
Then I replaced Spring MVC with Apache Wicket. Stuff seems to be working fine and I've successfully displayed a list of Customer entities.
Next up was the editing view for the customer. For now I've made a Wicket form that uses the OpenJPA entity directly as the form model, and thus I had to make the entity class implement Serializable.
Now, I'm not sure how to correctly implement the OpenJPA persistence, what I've got now is this:
#Override
protected void onSubmit() {
try {
if (customer.getId()!=null) {
customer.merge();
}
else {
customer.persist();
}
}
catch (Exception e) {
throw new Error(e);
}
super.onSubmit();
}
That works, but only once per Customer object. Somehow.
That is, I submit my form once and it works both with a new customer (.persist()) and a existing customer (.merge()). However, I submit the form again for the same customer I get this error (I added some linebreaks here):
<openjpa-2.0.0-r422266:935683 nonfatal store error>
org.apache.openjpa.persistence.OptimisticLockException:
An optimistic lock violation was detected when flushing object instance "no.magge.iumb.domain.crm.PrivateCustomer-379" to the data store.
This indicates that the object was concurrently modified in another transaction.
My question is, what is the correct way to persist with OpenJPA and why am I getting that error ?
Wicket-wise: Should I have created a separate Wicket IModel with a detachable Customer model and could that be the reason that I have these problems?
Thanks a bunch for any advice!

Do yourself a favor and separate your application layers. Code in a view should not ever access a Database.
Create a Service layer and / or a Dao layer, unit test the code of those layers to see that they are working and then inject a dao or service object into the wicket component. (I'd recommend you to use spring for that, but you can also do it manually)
With your scenario, there are so many different things that can fail in one spot, and it's nearly impossible to separate them.
Here are some pointers:
Dao (J2ee patterns)
Business logic (wikipedia)
Wicket / Spring / Hibernate
Configuration (it's very similar for JPA)
JPA using Spring