I have a Spring Boot application that happens to use Camunda for BPMN. Everything works fine. I have the Hikairi DBPool and the datasource properties in my application.properties file. Every thing runs fine, and workflows work etc...
I now want to access my DB via JdbcTemplate, using the same DataSource, as all the tables are on the same DB. I add this class:
#Component
public class MyDao extends JdbcDaoSupport {
public MyRow getMyRowById(int id) {
String sql = "select * from MyTable where id = ?";
try {
MyRow myRow = (MyRow)getJdbcTemplate().queryForObject(sql, new Object[] { id }, new MyRowMapper());
return myRow;
} catch (Exception e) {
System.out.println(e.getMessage());
}
return null;
}
}
And I get the error:
Caused by: java.lang.IllegalArgumentException: 'dataSource' or 'jdbcTemplate' is required
How is that possible when I know it's there. I see in the logs that Hikari is using it and adding itself as the DataSource for pooling. If I simply remove the #Component and it at least deploys, but as you would think, it throws a null pointer at the getJdbcTemplate() call.
Is there an annotation I am missing to get this to autowire correctly and expose the DataSource to my JdbcTemplate?
First, you should annotate your MyDao with the #Repository annotation and not with just the #Component one. For this reason, please take a moment to read What's the difference between #Component, #Repository & #Service annotations in Spring?.
Second, looking at your exception, seems you are missing the injection of the jdbcTemplate/datasource in the MyDao. For this point, if you are working with the datasource itself and not with the JdbcTemplate, you can inject the datasource as following:
#Autowired
public void setDs(DataSource dataSource) {
setDataSource(dataSource);
}
However, if you are using the JdbcTemplate, you can add a setter injection inside you MyDao as following:
#Autowired
public void setJt(JdbcTemplate jdbcTemplate) {
setJdbcTemplate(jdbcTemplate);
}
Related
I am using Spring Boot for my application. I am defining JNDI name in the application.properties file.
When I am trying to get JdbcTemplate in below class, its null:
#Configuration
public class DemoClass
{
#Autowired
private JdbcTemplate template;
#Bean
private DataSource getDS(){
return template.getDataSource(); //NPE
}
}
Another Class
#Component
public class SecondClass {
#Autowired
private JdbcTemplate template;
public void show(){
template.getDataSource(): // Working Fine
}
}
I am not sure this configured by default.. In case it is not, then maybe you can try configuring it yourself:
#Autowired
DataSoure dataSource;
#Bean
public JdbcTemplate getJdbcTemplate() {
return new JdbcTemplate(dataSource);
}
in any case if you need only the DataSource, I think it is auto-configured by Spring Boot so you can autowire it directly when you need it.
#Repository
public class DataRepository {
private JdbcTemplate jdbcTemplate;
#Autowired
public void setJdbcTemplate(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
public int updateCandidate() {
return this.jdbcTemplate.update("update .... from table where ....");
}
}
application.properties
database connection details
spring.datasource.url=jdbc:oracle:thin:***
spring.datasource.username=Scott
spring.datasource.password=Tiger
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
spring.datasource.tomcat.initial-size=1
spring.datasource.tomcat.max-active=1
spring.datasource.tomcat.min-idle=1
spring.datasource.tomcat.max-idle=1
If you're getting a NPE at getDS. This means JdbcTemplate hasn't been injected yet, maybe it couldn't be injected.
Give spring a hint at bean dependencies by
#Bean
public JdbcTemplate getJdbcTemplate(DataSource dataSource){
return new JdbcTemplate(dataSource)
}
Or
#Bean
#DependsOn("template")
public DataSouce getDS(){
return template.getDataSource();
}
By default #Autowired sets required=true so the DemoClass should not be constructed by Spring.
Most likely you are creating new DemoClass() or disabled the annotation config altogether and the DemoClass class is registered manually e.g. using XML.
Instead ensure that the DemoClass class is discovered using Spring's component scan e.g. using #SpringBootApplication or #ComponentScan e.g. as per this example.
I have a data source configuration class that looks as follows, with separate DataSource beans for testing and non-testing environments using JOOQ. In my code, I do not use DSLContext.transaction(ctx -> {...} but rather mark the method as transactional, so that JOOQ defers to Spring's declarative transactions for transactionality. I am using Spring 4.3.7.RELEASE.
I have the following issue:
During testing (JUnit), #Transactional works as expected. A single method is transactional no matter how many times I use the DSLContext's store() method, and a RuntimeException triggers a rollback of the entire transaction.
During actual production runtime, #Transactional is completely ignored. A method is no longer transactional, and TransactionSynchronizationManager.getResourceMap() holds two separate values: one showing to my connection pool (which is not transactional), and one showing the TransactionAwareDataSourceProxy).
In this case, I would have expected only a single resource of type TransactionAwareDataSourceProxy which wraps my DB CP.
After much trial and error using the second set of configuration changes I made (noted below with "AFTER"), #Transactional works correctly as expected even during runtime, though TransactionSynchronizationManager.getResourceMap() holds the following value:
In this case, my DataSourceTransactionManager seems to not even know the TransactionAwareDataSourceProxy (most likely due to my passing it the simple DataSource, and not the proxy object), which seems to completely 'skip' the proxy anyway.
My question is: the initial configuration that I had seemed correct, but did not work. The proposed 'fix' works, but IMO should not work at all (since the transaction manager does not seem to be aware of the TransactionAwareDataSourceProxy).
What is going on here? Is there a cleaner way to fix this issue?
BEFORE (not transactional during runtime)
#Configuration
#EnableTransactionManagement
#RefreshScope
#Slf4j
public class DataSourceConfig {
#Bean
#Primary
public DSLContext dslContext(org.jooq.Configuration configuration) throws SQLException {
return new DefaultDSLContext(configuration);
}
#Bean
#Primary
public org.jooq.Configuration defaultConfiguration(DataSourceConnectionProvider dataSourceConnectionProvider) {
org.jooq.Configuration configuration = new DefaultConfiguration()
.derive(dataSourceConnectionProvider)
.derive(SQLDialect.POSTGRES_9_5);
configuration.set(new DeleteOrUpdateWithoutWhereListener());
return configuration;
}
#Bean
public DataSourceTransactionManager transactionManager(DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
#Bean
public DataSourceConnectionProvider dataSourceConnectionProvider(DataSource dataSource) {
return new DataSourceConnectionProvider(dataSource);
}
#Configuration
#ConditionalOnClass(EmbeddedPostgres.class)
static class EmbeddedDataSourceConfig {
#Value("${spring.jdbc.port}")
private int dbPort;
#Bean(destroyMethod = "close")
public EmbeddedPostgres embeddedPostgres() throws Exception {
EmbeddedPostgres embeddedPostgres = EmbeddedPostgresHelper.startDatabase(dbPort);
return embeddedPostgres;
}
#Bean
#Primary
public DataSource dataSource(EmbeddedPostgres embeddedPostgres) throws Exception {
DataSource dataSource = embeddedPostgres.getPostgresDatabase();
return new TransactionAwareDataSourceProxy(dataSource);
}
}
#Configuration
#ConditionalOnMissingClass("com.opentable.db.postgres.embedded.EmbeddedPostgres")
#RefreshScope
static class DefaultDataSourceConfig {
#Value("${spring.jdbc.url}")
private String url;
#Value("${spring.jdbc.username}")
private String username;
#Value("${spring.jdbc.password}")
private String password;
#Value("${spring.jdbc.driverClass}")
private String driverClass;
#Value("${spring.jdbc.MaximumPoolSize}")
private Integer maxPoolSize;
#Bean
#Primary
#RefreshScope
public DataSource dataSource() {
log.debug("Connecting to datasource: {}", url);
HikariConfig hikariConfig = buildPool();
DataSource dataSource = new HikariDataSource(hikariConfig);
return new TransactionAwareDataSourceProxy(dataSource);
}
private HikariConfig buildPool() {
HikariConfig config = new HikariConfig();
config.setJdbcUrl(url);
config.setUsername(username);
config.setPassword(password);
config.setDriverClassName(driverClass);
config.setConnectionTestQuery("SELECT 1");
config.setMaximumPoolSize(maxPoolSize);
return config;
}
}
AFTER (transactional during runtime, as expected, all non-listed beans identical to above)
#Configuration
#EnableTransactionManagement
#RefreshScope
#Slf4j
public class DataSourceConfig {
#Bean
public DataSourceConnectionProvider dataSourceConnectionProvider(TransactionAwareDataSourceProxy dataSourceProxy) {
return new DataSourceConnectionProvider(dataSourceProxy);
}
#Bean
public TransactionAwareDataSourceProxy transactionAwareDataSourceProxy(DataSource dataSource) {
return new TransactionAwareDataSourceProxy(dataSource);
}
#Configuration
#ConditionalOnMissingClass("com.opentable.db.postgres.embedded.EmbeddedPostgres")
#RefreshScope
static class DefaultDataSourceConfig {
#Value("${spring.jdbc.url}")
private String url;
#Value("${spring.jdbc.username}")
private String username;
#Value("${spring.jdbc.password}")
private String password;
#Value("${spring.jdbc.driverClass}")
private String driverClass;
#Value("${spring.jdbc.MaximumPoolSize}")
private Integer maxPoolSize;
#Bean
#Primary
#RefreshScope
public DataSource dataSource() {
log.debug("Connecting to datasource: {}", url);
HikariConfig hikariConfig = buildPoolConfig();
DataSource dataSource = new HikariDataSource(hikariConfig);
return dataSource; // not returning the proxy here
}
}
}
I'll turn my comments into an answer.
The transaction manager should NOT be aware of the proxy. From the documentation:
Note that the transaction manager, for example
DataSourceTransactionManager, still needs to work with the underlying
DataSource, not with this proxy.
The class TransactionAwareDataSourceProxy is a special purpose class that is not needed in most cases. Anything that is interfacing with your data source through the Spring framework infrastructure should NOT have the proxy in their chain of access. The proxy is intended for code that cannot interface with the Spring infrastructure. For example, a third party library that was already setup to work with JDBC and did not accept any of Spring's JDBC templates. This is stated in the same docs as above:
This proxy allows data access code to work with the plain JDBC API and
still participate in Spring-managed transactions, similar to JDBC code
in a J2EE/JTA environment. However, if possible, use Spring's
DataSourceUtils, JdbcTemplate or JDBC operation objects to get
transaction participation even without a proxy for the target
DataSource, avoiding the need to define such a proxy in the first
place.
If you do not have any code that needs to bypass the Spring framework then do not use the TransactionAwareDataSourceProxy at all. If you do have legacy code like this then you will need to do what you already configured in your second setup. You will need to create two beans, one which is the data source, and one which is the proxy. You should then give the data source to all of the Spring managed types and the proxy to the legacy types.
I'm trying to convert a Spring 4 project to use HibernateTransactionManager to manage the transactions, but transactional services throw this exception:
org.hibernate.HibernateException: createSQLQuery is not valid without
active transaction
I've searched on google, and 80 different people have 80 different ways of doing it. I've tried them all, and either they don't work or they require using EntityManager instead of Hibernate Sessions. Since I'm converting existing code that uses Hibernate sessions, things would be far easier if I could get them to work. How do I get rid of this exception?
Spring version: 4.2.5.RELEASE
Hibernate version: 5.1.0.Final
Hikari version: 2.7.6
JDK version: 1.8.0_161
DataSource config:
#Configuration
public class DataSourceConfig {
#Bean
public DataSource dataSource() {
// return a Hikari connection pool as the data source
try {
HikariDataSource bean = new HikariDataSource();
// set connection pool and JDBC properties
return bean;
} catch (Exception e) {
throw new RuntimeException("could create data source", e);
}
}
}
Session factory config:
#Configuration
public class SessionFactoryConfig {
#Autowired
private DataSource dataSource;
#Bean
public LocalSessionFactoryBean sessionFactory() {
// create a session factory
try {
LocalSessionFactoryBean bean = new LocalSessionFactoryBean();
// set Hibernate properties
bean.setDataSource(dataSource);
bean.setPackagesToScan(packagesToScan);
bean.setHibernateProperties(properties);
bean.afterPropertiesSet(); // tried without this line as well
return bean;
} catch (Exception e) {
throw new RuntimeException("could not create session factory", e);
}
}
}
Transaction config:
#Configuration
#EnableTransactionManagement
#ComponentScan("com.mydomain.*")
public class TransactionConfig {
#Autowired
private SessionFactory sessionFactory;
#Bean
public PlatformTransactionManager transactionManager() {
HibernateTransactionManager bean = new HibernateTransactionManager(sessionFactory);
// bean.setHibernateManagedSession(true); // similar exception when set
bean.afterPropertiesSet(); // tried without this line as well
return bean;
}
}
Service:
#Service
#Transactional // tried using only the #Transactional on the method
public class TestTransactionService {
#Autowired
private SessionFactory sessionFactory;
#Transactional // tried using only the #Transactional on the class
public String get(String key) {
Session session = sessionFactory.getCurrentSession();
// the below line throws the exception!
SQLQuery q = session.createSQLQuery("SELECT value FROM test_transaction WHERE key = :key");
q.setParameter("key", key);
q.addScalar("value", StringType.INSTANCE);
String out = (String)q.uniqueResult();
return out;
}
}
Any ideas? I hope so because I'm out.
Buried in our settings was this innocent-looking one, in there since the initial git checkin of our project, so I cannot find out who put it in (they're lucky):
hibernate.current_session_context_class = thread
Since no one knew what this setting was for, no one dared touch it since then. This breaks Spring managed transactions, and had no business being in there.
Let's this be a lesson, boys and girls: if you're trying to get something to work, DON'T just fiddle with settings when you don't know what they are. It will come back to bite you in the ass--sometimes years later. And then I will come find you and get you.
I run a Spring Boot Application wiht #EnableTransactionManagement and want to use #Transactional(readOnly = true) for some database queries.
But I receive a confusing error message.
I'm using Spring, Spring Boot and and Spring Data JPA.
MySpringBootApplication.java
#SpringBootApplication
#EnableTransactionManagement
#ComponentScan("com.deutscheboerse.regrephub")
#EntityScan(basePackages = "com.deutscheboerse.regrephub")
#EnableJpaRepositories(basePackages = "com.deutscheboerse.regrephub")
#Slf4j
public class MySpringBootApplication
{
... Some #Autowired variables ...
public static void main(String[] args)
{
SpringApplication.run(MySpringBootApplication.class, args);
}
...
}
MySpringBootApplicationConfiguration.java
#Configuration
#EnableEncryptableProperties
#EnableTransactionManagement
#EnableAsync
#Slf4j
public class MySpringBootApplicationConfiguration
{
... Some #Autowired variables ...
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource()
{
return DataSourceBuilder
.create(this.dataSourceProperties.getClassLoader())
.url(this.dataSourceProperties.getUrl())
.username(this.dataSourceProperties.getUsername())
.password(this.dataSourceProperties.getPassword())
.build();
}
...
}
MyBeanDao.java
#Repository
public interface MyBeanDao extends JpaRepository<MyBeanData, Long>
{
#QueryHints(value = #QueryHint(name = HINT_FETCH_SIZE, value = "" + Integer.MIN_VALUE))
#Query(value = "SELECT * FROM MY_TABLE", nativeQuery = true)
#Transactional(readOnly = true)
Stream<MyBeanData> streamAll();
}
MyBeanService.java
#Service
#Slf4j
public class MyBeanService extends AbstractService
{
#Autowired
public MyBeanService(...)
{
...
}
#Override
#Transactional(readOnly = true, propagation = Propagation.SUPPORTS)
public void handleRequest(Object request, Message msg)
{
try (Stream<MyBeanData> data = myBeanDao.streamAll())
{
...
}
catch (Exception e)
{
...
}
}
}
When I run my SpringBootApplication I will receive the following log messages / errors:
[TransactionInterceptor:474] Getting transaction for [org.springframework.data.jpa.repository.support.SimpleJpaRepository.streamAll]
[TransactionInterceptor:517] Completing transaction for [org.springframework.data.jpa.repository.support.SimpleJpaRepository.streamAll] after exception: org.springframework.dao.InvalidDataAccessApiUsageException: You're trying to execute a streaming query method without a surrounding transaction that keeps the connection open so that the Stream can actually be consumed. Make sure the code consuming the stream uses #Transactional or any other way of declaring a (read-only) transaction.
[RuleBasedTransactionAttribute:131] Applying rules to determine whether transaction should rollback on org.springframework.dao.InvalidDataAccessApiUsageException: You're trying to execute a streaming query method without a surrounding transaction that keeps the connection open so that the Stream can actually be consumed. Make sure the code consuming the stream uses #Transactional or any other way of declaring a (read-only) transaction.
[RuleBasedTransactionAttribute:148] Winning rollback rule is: null
[RuleBasedTransactionAttribute:153] No relevant rollback rule found: applying default rules
First JPA opens a transaction and close it immediately with an exception, that I want to execute a streaming query method without a surrounding transaction.
Does someone had this before?!
I fixed it!
It was a problem with the spring context.
When I initialize MyBeanService, I stored the bean into an HashMap with an corresponding request-object.
dispatcher.put(MyBeanRequest.class, this);
...
((MyAbstractService) dispatcher.get(MyBeanRequest.class).handleRequest(...);
it works, when I search for the bean in the spring context:
dispatcher.put(MyBeanRequest.class, this.getClass());
...
((MyAbstractService) appContext.getBean(dispatcher.get(requestObject.getClass()))).handleRequest(...);
I have seen a lot of examples of Spring Batch projects where either (a) a dataSource is defined, or (b) no dataSource is defined.
However, in my project, I would like my business logic to have access to a dataSource, but I want Spring Batch to NOT use the dataSource. Is this possible?
This guy has a similar problem: Spring boot + spring batch without DataSource
Generally, using spring-batch without a database is not a good idea, since there could be concurrency issues depending on the kind of job you define. So at least an using an inmemory db is strongly advised, especially if you plan to use the job in production.
Using SpringBatch with SpringBoot will initialize an inmemory datasource, if you do not configure your own datasource(s).
Taking this into account, let me redefine your question as follows: Can my businesslogic use another datasource than springbatch is using to update its BATCH-tables?
Yes, it can. As a matter of fact, you can use as many datasources as you want inside your SpringBatch Jobs. Just use by-name autowiring.
Here is how I do it:
I always use Configuration class, which defines all the datasources I have to use in my Jobs
Configuration
public class DatasourceConfiguration {
#Bean
#ConditionalOnMissingBean(name = "dataSource")
public DataSource dataSource() {
// create datasource, that is used by springbatch
// for instance, create an inmemory datasource using the
// EmbeddedDatabaseFactory
return ...;
}
#Bean
#ConditionalOnMissingBean(name = "bl1datasource")
public DataSource bl1datasource() {
return ...; // your first datasource that is used in your businesslogic
}
#Bean
#ConditionalOnMissingBean(name = "bl2datasource")
public DataSource bl2datasource() {
return ...; // your second datasource that is used in your businesslogic
}
}
Three points to note:
SpringBatch is looking for a datasource with the name "dataSource", if you do not provide this EXACT (uppercase 'S') name as the name, spring batch will try to autowire by type and if it finds more than one instance of DataSource, it will throw an exception.
Put your datasource configuration in its own class. Do not put them in the same class as your jobdefinitions are. Spring needs to be able to instantiate the datasource-SpringBean with the name "dataSource" very early when it loads the context. Before it starts to instantiate your Job- and Step-Beans. Spring will not be able to do it correctly, if you put your datasource definitions in the same class as you have your job/step definitions.
Using #ConditionalOnMissingBean is not mandatory, but I found it a good practics. It makes it easy to change the datasources for unit/integration tests. Just provide an additional test configuration in the ContextConfiguration of your unit/IT test which, for instance, overwrites the "bl1Datasource" with an inMemoryDataSource:
Configuration
public class TestBL1DatasourceConfiguration {
// overwritting bl1datasource with an inMemoryDatasource.
#Bean
public DataSource bl1datasource() {
return new EmbeddedDatabaseFactory.getDatabase();
}
}
In order to use the businesslogic datasources, use injection by name:
#Component
public class PrepareRe1Re2BezStepCreatorComponent {
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private DataSource bl1datasource;
#Autowired
private DataSource bl2datasource;
public Step createStep() throws Exception {
SimpleStepBuilder<..., ...> builder =
stepBuilderFactory.get("astep") //
.<..., ...> chunk(100) //
.reader(createReader(bl1datasource)) //
.writer(createWriter(bl2datasource)); //
return builder.build();
}
}
Furthermore, you probably want to consider using XA-Datasources if you'd like to work with several datasources.
Edited:
Since it seems that you really don't want to use a datasource, you have to implement your own BatchConfigurer (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/configuration/annotation/BatchConfigurer.html) (as Michael Minella - the SpringBatch project lead - pointed out above).
You can use the code of org.springframework.batch.core.configuration.annotation.DefaultBatchConfigurer as a starting point for your own implementation. Simply remove all the datasource/transactionmanager code and keep the content of the if (datasource === null) part in the initialize method. This will initialize a MapBasedJobRepository and MapBasedJobExplorer. But again, this is NOT a useable solution in a productive environment, since it is not threadsafe.
Edited:
How to implement it:
Configuration class that defines the "businessDataSource":
#Configuration
public class DataSourceConfigurationSimple {
DataSource embeddedDataSource;
#Bean
public DataSource myBusinessDataSource() {
if (embeddedDataSource == null) {
EmbeddedDatabaseFactory factory = new EmbeddedDatabaseFactory();
embeddedDataSource = factory.getDatabase();
}
return embeddedDataSource;
}
}
The implementation of a specific BatchConfigurer:
(of course, the methods have to be implemented...)
public class MyBatchConfigurer implements BatchConfigurer {
#Override
public JobRepository getJobRepository() throws Exception {
return null;
}
#Override
public PlatformTransactionManager getTransactionManager() throws Exception {
return null;
}
#Override
public JobLauncher getJobLauncher() throws Exception {
return null;
}
#Override
public JobExplorer getJobExplorer() throws Exception {
return null;
}
}
And finally the main configuration and launch class:
#SpringBootApplication
#Configuration
#EnableBatchProcessing
// Importing MyBatchConfigurer will install your BatchConfigurer instead of
// SpringBatch default configurer.
#Import({DataSourceConfigurationSimple.class, MyBatchConfigurer.class})
public class SimpleTestJob {
#Autowired
private JobBuilderFactory jobs;
#Autowired
private StepBuilderFactory steps;
#Bean
public Job job() throws Exception {
SimpleJobBuilder standardJob = this.jobs.get(JOB_NAME)
.start(step1());
return standardJob.build();
}
protected Step step1() throws Exception {
TaskletStepBuilder standardStep1 = this.steps.get("SimpleTest_step1_Step")
.tasklet(tasklet());
return standardStep1.build();
}
protected Tasklet tasklet() {
return (contribution, context) -> {
System.out.println("tasklet called");
return RepeatStatus.FINISHED;
};
}
public static void main(String[] args) throws Exception {
SpringApplication.run(SimpleTestJob.class, args);
}
}