In our application we are using both MySQL server and Redis databases. we use Redis as a database and not just a cache. we use both of them in a service method and I want to make the method #Transactional to let the spring manages my transactions. Hence if in the middle of an transactional method a RuntimeException is thrown all works on both Redis and MySQL are rolled back. I have followed the spring docs and configured my #SpringBootApplication class as following:
#SpringBootApplication
#EnableTransactionManagement
public class TransactionsApplication {
#Autowired
DataSource dataSource;
public static void main(String[] args) {
SpringApplication.run(TransactionsApplication.class, args);
}
#Bean
public StringRedisTemplate redisTemplate() {
StringRedisTemplate template = new StringRedisTemplate(redisConnectionFactory());
// explicitly enable transaction support
template.setEnableTransactionSupport(true);
return template;
}
#Bean
public LettuceConnectionFactory redisConnectionFactory() {
return new LettuceConnectionFactory(new RedisStandaloneConfiguration("localhost", 6379));
}
#Bean
public PlatformTransactionManager transactionManager() throws SQLException, IOException {
return new DataSourceTransactionManager(dataSource);
}
}
and this is my service method:
#Service
#RequiredArgsConstructor
#Slf4j
public class FooService {
private final StringRedisTemplate redisTemplate;
private final FooRepository fooRepository;
#Transactional
public void bar() {
Foo foo = Foo.builder()
.id(1001)
.name("fooName")
.email("foo#mail.com")
.build();
fooRepository.save(foo);
ValueOperations<String, String> values = redisTemplate.opsForValue();
values.set("foo-mail", foo.getEmail());
}
However after the test method of TestService is called there is no user in MySQL db and I think it's because there is no active transaction for it.Is there any solution for this problem? Should I use spring ChainedTransactionManager class and then how? or I can only manage Redis transactions manually through MULTI?
After playing around with FooService class I found that using #Transactional in a service method that is intended to work with Redis and specially read a value from Redis (which I think most service methods are supposed to read some value from DB) is somehow useless since any read operation would result a null value and that is because Redis queues all operations of a transaction and executes them at the end. Summing up, I think using MULTI and EXEC operations is more preferable since it gives more control to use data in Redis.
After all, any suggestions to use Redis transactions is appreciated.
Related
I have a data source configuration class that looks as follows, with separate DataSource beans for testing and non-testing environments using JOOQ. In my code, I do not use DSLContext.transaction(ctx -> {...} but rather mark the method as transactional, so that JOOQ defers to Spring's declarative transactions for transactionality. I am using Spring 4.3.7.RELEASE.
I have the following issue:
During testing (JUnit), #Transactional works as expected. A single method is transactional no matter how many times I use the DSLContext's store() method, and a RuntimeException triggers a rollback of the entire transaction.
During actual production runtime, #Transactional is completely ignored. A method is no longer transactional, and TransactionSynchronizationManager.getResourceMap() holds two separate values: one showing to my connection pool (which is not transactional), and one showing the TransactionAwareDataSourceProxy).
In this case, I would have expected only a single resource of type TransactionAwareDataSourceProxy which wraps my DB CP.
After much trial and error using the second set of configuration changes I made (noted below with "AFTER"), #Transactional works correctly as expected even during runtime, though TransactionSynchronizationManager.getResourceMap() holds the following value:
In this case, my DataSourceTransactionManager seems to not even know the TransactionAwareDataSourceProxy (most likely due to my passing it the simple DataSource, and not the proxy object), which seems to completely 'skip' the proxy anyway.
My question is: the initial configuration that I had seemed correct, but did not work. The proposed 'fix' works, but IMO should not work at all (since the transaction manager does not seem to be aware of the TransactionAwareDataSourceProxy).
What is going on here? Is there a cleaner way to fix this issue?
BEFORE (not transactional during runtime)
#Configuration
#EnableTransactionManagement
#RefreshScope
#Slf4j
public class DataSourceConfig {
#Bean
#Primary
public DSLContext dslContext(org.jooq.Configuration configuration) throws SQLException {
return new DefaultDSLContext(configuration);
}
#Bean
#Primary
public org.jooq.Configuration defaultConfiguration(DataSourceConnectionProvider dataSourceConnectionProvider) {
org.jooq.Configuration configuration = new DefaultConfiguration()
.derive(dataSourceConnectionProvider)
.derive(SQLDialect.POSTGRES_9_5);
configuration.set(new DeleteOrUpdateWithoutWhereListener());
return configuration;
}
#Bean
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
#Bean
public DataSourceConnectionProvider dataSourceConnectionProvider(DataSource dataSource) {
return new DataSourceConnectionProvider(dataSource);
}
#Configuration
#ConditionalOnClass(EmbeddedPostgres.class)
static class EmbeddedDataSourceConfig {
#Value("${spring.jdbc.port}")
private int dbPort;
#Bean(destroyMethod = "close")
public EmbeddedPostgres embeddedPostgres() throws Exception {
EmbeddedPostgres embeddedPostgres = EmbeddedPostgresHelper.startDatabase(dbPort);
return embeddedPostgres;
}
#Bean
#Primary
public DataSource dataSource(EmbeddedPostgres embeddedPostgres) throws Exception {
DataSource dataSource = embeddedPostgres.getPostgresDatabase();
return new TransactionAwareDataSourceProxy(dataSource);
}
}
#Configuration
#ConditionalOnMissingClass("com.opentable.db.postgres.embedded.EmbeddedPostgres")
#RefreshScope
static class DefaultDataSourceConfig {
#Value("${spring.jdbc.url}")
private String url;
#Value("${spring.jdbc.username}")
private String username;
#Value("${spring.jdbc.password}")
private String password;
#Value("${spring.jdbc.driverClass}")
private String driverClass;
#Value("${spring.jdbc.MaximumPoolSize}")
private Integer maxPoolSize;
#Bean
#Primary
#RefreshScope
public DataSource dataSource() {
log.debug("Connecting to datasource: {}", url);
HikariConfig hikariConfig = buildPool();
DataSource dataSource = new HikariDataSource(hikariConfig);
return new TransactionAwareDataSourceProxy(dataSource);
}
private HikariConfig buildPool() {
HikariConfig config = new HikariConfig();
config.setJdbcUrl(url);
config.setUsername(username);
config.setPassword(password);
config.setDriverClassName(driverClass);
config.setConnectionTestQuery("SELECT 1");
config.setMaximumPoolSize(maxPoolSize);
return config;
}
}
AFTER (transactional during runtime, as expected, all non-listed beans identical to above)
#Configuration
#EnableTransactionManagement
#RefreshScope
#Slf4j
public class DataSourceConfig {
#Bean
public DataSourceConnectionProvider dataSourceConnectionProvider(TransactionAwareDataSourceProxy dataSourceProxy) {
return new DataSourceConnectionProvider(dataSourceProxy);
}
#Bean
public TransactionAwareDataSourceProxy transactionAwareDataSourceProxy(DataSource dataSource) {
return new TransactionAwareDataSourceProxy(dataSource);
}
#Configuration
#ConditionalOnMissingClass("com.opentable.db.postgres.embedded.EmbeddedPostgres")
#RefreshScope
static class DefaultDataSourceConfig {
#Value("${spring.jdbc.url}")
private String url;
#Value("${spring.jdbc.username}")
private String username;
#Value("${spring.jdbc.password}")
private String password;
#Value("${spring.jdbc.driverClass}")
private String driverClass;
#Value("${spring.jdbc.MaximumPoolSize}")
private Integer maxPoolSize;
#Bean
#Primary
#RefreshScope
public DataSource dataSource() {
log.debug("Connecting to datasource: {}", url);
HikariConfig hikariConfig = buildPoolConfig();
DataSource dataSource = new HikariDataSource(hikariConfig);
return dataSource; // not returning the proxy here
}
}
}
I'll turn my comments into an answer.
The transaction manager should NOT be aware of the proxy. From the documentation:
Note that the transaction manager, for example
DataSourceTransactionManager, still needs to work with the underlying
DataSource, not with this proxy.
The class TransactionAwareDataSourceProxy is a special purpose class that is not needed in most cases. Anything that is interfacing with your data source through the Spring framework infrastructure should NOT have the proxy in their chain of access. The proxy is intended for code that cannot interface with the Spring infrastructure. For example, a third party library that was already setup to work with JDBC and did not accept any of Spring's JDBC templates. This is stated in the same docs as above:
This proxy allows data access code to work with the plain JDBC API and
still participate in Spring-managed transactions, similar to JDBC code
in a J2EE/JTA environment. However, if possible, use Spring's
DataSourceUtils, JdbcTemplate or JDBC operation objects to get
transaction participation even without a proxy for the target
DataSource, avoiding the need to define such a proxy in the first
place.
If you do not have any code that needs to bypass the Spring framework then do not use the TransactionAwareDataSourceProxy at all. If you do have legacy code like this then you will need to do what you already configured in your second setup. You will need to create two beans, one which is the data source, and one which is the proxy. You should then give the data source to all of the Spring managed types and the proxy to the legacy types.
I am trying to update a node on Neo4J, but what ends up happening is that it creates a duplicate Node. I read that the update has to be in a single transaction and I added #Transactional, but still same result. Here is what I have. I tried the approach of reading and deleting the old node, and saving the new one and it appears to be working. But, I think that is not the right approach. Why the #Transactional annotation not working. Thank you.
#EnableNeo4JRepositories(com.example.graph.repo)
#EnableTransactionManagement
#org.springframework.contect.annotation.Configuration
public class Neo4JConfig {
#Bean
public Configuration configuration() {
Configuration cfg = new Configuration();
cfg.driverConfiguration()
.setDriverClassName("org.neo4j.ogm.drivers.http.driver.HttpDriver")
.setURI("http://neo4j:neo4j#localhost:7474");
return cfg;
}
#Bean
public SessionFactory sessionFactory() {
return new SessionFactory(configuration(), "com.example");
}
#Bean
public Neo4jTransactionManager transactionManager() {
return new Neo4JTransactionManager(sessionFactory());
}
}
#Service
public class UserService{
#Autowired
UserRepository userRepository;
#Transactional
public void updateUser(User user) {
User existingUser = userRepository.getExistingUser(user.getUserName());
if(existingUser != null ) {
user.setSomeValue(existingUser.getSomeValue());
userRepository.save(user);
}
}
}
Spring AOP uses JDK Proxy mechanism by default. It means that you must invoke #Transactional method via interface method.
So you should split your service into interface UserService and implementation (say UserServiceImpl), autowire the interface into the code where you currently autowire the impementation, and then invoke transactional method via interface.
P.S. Another approach is to force Spring to use CGLIB as long as this mechanism is not limited to interfaces. More details for both mechanisms https://docs.spring.io/spring/docs/3.0.0.M3/reference/html/ch08s06.html
As far as I know spring provides some ways to inject beans into non-managed classes.
It can be done explicitly with AutowireCapableBeanFactory. (How to inject dependencies into a self-instantiated object in Spring?)
But I've faced strange (IMHO) behavior, when spring performs such injection automatically.
Here is an example with spring batch,
Configuration:
#SpringBootConfiguration
public class ProcessorJobConfig {
//.....
#Bean(name = "pullRestTemplate")
public RestTemplate createPullRestTemplate() {
RestTemplate restTemplate = restTemplateBuilder.build();
return restTemplate;
}
#Bean(name = "step")
public Step step(#Autowired ItemReader<Measurement> itemReader,
#Autowired ItemProcessor<Measurement, Event> itemProcessor,
#Autowired ItemWriter<Event> itemWriter) {
return stepBuilderFactory.get("step")
.<Measurement, Event>chunk(Integer.MAX_VALUE)
.reader(itemReader)
.processor(itemProcessor)
.writer(itemWriter)
.build();
}
#Bean(name = "restProcessorJob")
public Job job(#Qualifier("step") Step step) throws Exception {
return jobBuilderFactory.get("restProcessorJob")
.start(step)
.build();
}
#Bean
public ItemReader<Measurement> itemReader() {
RestMeasureReader restMeasureReader = new RestMeasureReader(); // Use new() explicitly
return restMeasureReader;
}
//.....
}
Reader:
public class RestMeasureReader implements ItemReader<Measurement> {
private static final Logger LOGGER = LoggerFactory.getLogger(RestMeasureReader.class);
/**
* NOTE: This field will be injected automatically by spring, even we are using new() to create instance of this class.
*/
#Autowired
#Qualifier("pullRestTemplate")
private RestTemplate restTemplate;
#Override
public Measurement read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
// do some stuff
}
}
And application itself
#EnableBatchProcessing
#EnableTask
#SpringBootApplication
public class TestAutowiredTaskApplication {
public static void main(String[] args) {
SpringApplication.run(TestAutowiredTaskApplication.class, args);
}
}
Even I use explicit new() to instantiate RestMeasureReader, its RestTemplate field will be injected afterwards.
Is it normal behavior? I do not expect spring to automatically inject fields when creating object with new().
If you are talking about using new inside of your #Configuration class, then yes it is normal behavior. This is you Spring java configs. So it's is Spring managed context. You are not going to call itemReader() in your code explicitly.
So, when you are going to do this:
#Autowired
private ItemReader<Measurement> iterReader;
you will get instance of your RestMeasureReader from Spring's IoC.
But if you will try to do explicitly call new RestMesureReader() inside of your code, you will get a new instance of RestMesureReader not a Spring Proxy with injected #Autowired fields.
Try to remove #Bean from your itemReader() method declaration and won't event be able to autowire RestMesureReader.
So basically #Configuration classes are just a Spring configuration, not a real java code. Even though you call new Spring will still return you a proxy class.
For more information check this guide.
Spring processes beans returned by methods that are annotated with #Bean
This allows you to use autowiring or livecycle callbacks when using Java configuration.
A more minimalistic example:
#Configuration
public class MyConfiguration {
#Bean
public A a() {
return new A();
}
static class A {
#Autowired
private B b;
#PostConstruct
public void onPostConstruct() {
System.out.println("postConstruct: " + b);
}
}
#Component
static class B {
}
}
Here, even if the bean named a is created manually, Spring will inject dependencies (b) and call #PostConstruct callbacks.
I have seen a lot of examples of Spring Batch projects where either (a) a dataSource is defined, or (b) no dataSource is defined.
However, in my project, I would like my business logic to have access to a dataSource, but I want Spring Batch to NOT use the dataSource. Is this possible?
This guy has a similar problem: Spring boot + spring batch without DataSource
Generally, using spring-batch without a database is not a good idea, since there could be concurrency issues depending on the kind of job you define. So at least an using an inmemory db is strongly advised, especially if you plan to use the job in production.
Using SpringBatch with SpringBoot will initialize an inmemory datasource, if you do not configure your own datasource(s).
Taking this into account, let me redefine your question as follows: Can my businesslogic use another datasource than springbatch is using to update its BATCH-tables?
Yes, it can. As a matter of fact, you can use as many datasources as you want inside your SpringBatch Jobs. Just use by-name autowiring.
Here is how I do it:
I always use Configuration class, which defines all the datasources I have to use in my Jobs
Configuration
public class DatasourceConfiguration {
#Bean
#ConditionalOnMissingBean(name = "dataSource")
public DataSource dataSource() {
// create datasource, that is used by springbatch
// for instance, create an inmemory datasource using the
// EmbeddedDatabaseFactory
return ...;
}
#Bean
#ConditionalOnMissingBean(name = "bl1datasource")
public DataSource bl1datasource() {
return ...; // your first datasource that is used in your businesslogic
}
#Bean
#ConditionalOnMissingBean(name = "bl2datasource")
public DataSource bl2datasource() {
return ...; // your second datasource that is used in your businesslogic
}
}
Three points to note:
SpringBatch is looking for a datasource with the name "dataSource", if you do not provide this EXACT (uppercase 'S') name as the name, spring batch will try to autowire by type and if it finds more than one instance of DataSource, it will throw an exception.
Put your datasource configuration in its own class. Do not put them in the same class as your jobdefinitions are. Spring needs to be able to instantiate the datasource-SpringBean with the name "dataSource" very early when it loads the context. Before it starts to instantiate your Job- and Step-Beans. Spring will not be able to do it correctly, if you put your datasource definitions in the same class as you have your job/step definitions.
Using #ConditionalOnMissingBean is not mandatory, but I found it a good practics. It makes it easy to change the datasources for unit/integration tests. Just provide an additional test configuration in the ContextConfiguration of your unit/IT test which, for instance, overwrites the "bl1Datasource" with an inMemoryDataSource:
Configuration
public class TestBL1DatasourceConfiguration {
// overwritting bl1datasource with an inMemoryDatasource.
#Bean
public DataSource bl1datasource() {
return new EmbeddedDatabaseFactory.getDatabase();
}
}
In order to use the businesslogic datasources, use injection by name:
#Component
public class PrepareRe1Re2BezStepCreatorComponent {
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private DataSource bl1datasource;
#Autowired
private DataSource bl2datasource;
public Step createStep() throws Exception {
SimpleStepBuilder<..., ...> builder =
stepBuilderFactory.get("astep") //
.<..., ...> chunk(100) //
.reader(createReader(bl1datasource)) //
.writer(createWriter(bl2datasource)); //
return builder.build();
}
}
Furthermore, you probably want to consider using XA-Datasources if you'd like to work with several datasources.
Edited:
Since it seems that you really don't want to use a datasource, you have to implement your own BatchConfigurer (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/configuration/annotation/BatchConfigurer.html) (as Michael Minella - the SpringBatch project lead - pointed out above).
You can use the code of org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer as a starting point for your own implementation. Simply remove all the datasource/transactionmanager code and keep the content of the if (datasource === null) part in the initialize method. This will initialize a MapBasedJobRepository and MapBasedJobExplorer. But again, this is NOT a useable solution in a productive environment, since it is not threadsafe.
Edited:
How to implement it:
Configuration class that defines the "businessDataSource":
#Configuration
public class DataSourceConfigurationSimple {
DataSource embeddedDataSource;
#Bean
public DataSource myBusinessDataSource() {
if (embeddedDataSource == null) {
EmbeddedDatabaseFactory factory = new EmbeddedDatabaseFactory();
embeddedDataSource = factory.getDatabase();
}
return embeddedDataSource;
}
}
The implementation of a specific BatchConfigurer:
(of course, the methods have to be implemented...)
public class MyBatchConfigurer implements BatchConfigurer {
#Override
public JobRepository getJobRepository() throws Exception {
return null;
}
#Override
public PlatformTransactionManager getTransactionManager() throws Exception {
return null;
}
#Override
public JobLauncher getJobLauncher() throws Exception {
return null;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return null;
}
}
And finally the main configuration and launch class:
#SpringBootApplication
#Configuration
#EnableBatchProcessing
// Importing MyBatchConfigurer will install your BatchConfigurer instead of
// SpringBatch default configurer.
#Import({DataSourceConfigurationSimple.class, MyBatchConfigurer.class})
public class SimpleTestJob {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
public Job job() throws Exception {
SimpleJobBuilder standardJob = this.jobs.get(JOB_NAME)
.start(step1());
return standardJob.build();
}
protected Step step1() throws Exception {
TaskletStepBuilder standardStep1 = this.steps.get("SimpleTest_step1_Step")
.tasklet(tasklet());
return standardStep1.build();
}
protected Tasklet tasklet() {
return (contribution, context) -> {
System.out.println("tasklet called");
return RepeatStatus.FINISHED;
};
}
public static void main(String[] args) throws Exception {
SpringApplication.run(SimpleTestJob.class, args);
}
}
I am trying to learn how to use transaction in java spring. I am a java novice so please bear with me :-) The unit test im trying to achieve below is testing:
rollback if a runtime exception is thrown.
The problem im having is a
java.lang.NullPointerException?
Ok here goes.. I have stripped out some code to help improve readability
TutorialTest.java
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = AppConfig.class, loader=AnnotationConfigContextLoader.class)
public class TutorialTest {
#Autowired
private Dao dao;
#Test
public void that_if_a_runtime_exception_is_thrown_transaction_rolledback(){
User u = new User();
u.setUsername("FAIL_TEST");
u.setPhone("0161");
u.setEmail("FAIL_TEST#gmail.com");
//dao.addUser(u); // -- this will insert if uncommented out so I know it works
OuterService os = new OuterService();
os.addUserThrowError(u);
}
}
OuterService.java
#ContextConfiguration(classes = AppConfig.class, loader=AnnotationConfigContextLoader.class)
public class OuterService {
#Autowired
private Dao dao;
#Transactional
public void addUserThrowError(User user) throws RuntimeException{
dao.addUser(user); // gives me a java.lang.NullPointerException?
throw new RuntimeException("This should roll back DB entry");
}
}
Beans are declared in
AppConfig.java
#Configuration
#EnableTransactionManagement
#ComponentScan(value = {"com.training.spring.tx.tutorial.dao",
"com.training.spring.tx.tutorial.service"})
public class AppConfig {
public DataSource dataSource() {
// Create a BasicDataSource object and configure database
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost/spring_training_tx");
dataSource.setUsername("training");
dataSource.setPassword("training");
return dataSource;
}
#Bean
public DataSourceTransactionManager transactionManager() {
DataSourceTransactionManager transactionManager = new DataSourceTransactionManager();
transactionManager.setDataSource(dataSource());
return transactionManager;
}
#Bean
public JdbcTemplate jdbcTemplate(){
return new JdbcTemplate(transactionManager().getDataSource());
}
}
First of all, to quote the javadoc of #ContextConfiguration
#ContextConfiguration defines class-level metadata that is used to
determine how to load and configure an ApplicationContext for
integration tests.
Consider how you are using it on OuterService. Does it seem right? Is OuterService meant to be used to load and configure an ApplicationCOntext for integration tests? Unless I'm missing something essential, the answer is: No.
So what is OuterService? It's some kind of service. You seem to want to use it as a bean. What is a bean? A bean is an object whose lifecycle is managed by Spring. This includes instantiation of the bean class, initialization of the object, post processing, and, finally, destruction of the object.
If you created the object like so
OuterService os = new OuterService();
then Spring is not involved. You created the object and there is no way for Spring to hook into that. You cannot therefore expect Spring to autowire its field
#Autowired
private Dao dao;
And since you haven't initialized the field, it remains null, which causes the NullPointerException.
So how do we get an OuterService bean, which is managed by Spring? You either declare a #Bean method for OuterService or you annotate OuterService with #Component or any of its specializations and component-scan the package it is in. You then inject the bean into any other bean that uses it. For example,
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = AppConfig.class, loader=AnnotationConfigContextLoader.class)
public class TutorialTest {
#Autowired
private Dao dao;
#Autowired
private OuterService os;
You can then use that variable directly.