I've been tearing my hair out with what should be a pretty common use case for a web application. I have a Spring-Boot application which uses REST Repositories, JPA, etc. The problem is that I have two data sources:
Embedded H2 data source containing user authentication information
MySQL data source for actual data which is specific to the authenticated user
Because the second data source is specific to the authenticated user, I'm attempting to use AbstractRoutingDataSource to route to the correct data source according to Principal user after authentication.
What's absolutely driving me crazy is that Spring-Boot is fighting me tooth and nail to instantiate this data source at startup. I've tried everything I can think of, including Lazy and Scope annotations. If I use Session scope, the application throws an error about no session existing at startup. #Lazy doesn't appear to help at all. No matter what annotations I use, the database is instantiated at startup by Spring Boot and doesn't find any lookup key which essentially crashes the entire application.
The other problem is that the Rest Repository API has IMO a terrible means of specifying the actual data source to be used. If you have multiple data sources with Spring Boot, you have to juggle Qualifier annotations which is a runtime debugging nightmare.
Any advice would be very much appreciated.
Your problem is with the authentication manager configuration. All the samples and guides set this up in a GlobalAuthenticationConfigurerAdapter, e.g. it would look like this as an inner class of your SimpleEmbeddedSecurityConfiguration:
#Configuration
public static class AuthenticationConfiguration extends GlobalAuthenticationConfigurerAdapter
{
#Bean(name = Global.AUTHENTICATION_DATA_QUALIFIER + "DataSource")
public DataSource dataSource()
{
return new EmbeddedDatabaseBuilder().setName("authdb").setType(EmbeddedDatabaseType.H2).addScripts("security/schema.sql", "security/data.sql").build();
}
#Override
public void init(AuthenticationManagerBuilder auth) throws Exception
{
auth.jdbcAuthentication().dataSource(dataSource()).passwordEncoder(passwordEncoder());
}
}
If you don't use GlobalAuthenticationConfigurerAdapter then the DataSource gets picked up by Spring Data REST during the creation of the Security filters (before the #Primary DataSource bean has even been registered) and the whole JPA initialization starts super early (bad idea).
UPDATE: the authentication manager is not the only problem. If you need to have a session-scoped #Primary DataSource (pretty unusual I'd say), you need to switch off everything that wants access to the database on startup (Hibernate and Spring Boot in various places). Example:
spring.datasource.initialize: false
spring.jpa.hibernate.ddlAuto: none
spring.jpa.properties.hibernate.temp.use_jdbc_metadata_defaults: false
spring.jpa.properties.hibernate.dialect: H2
FURTHER UPDATE: if you're using the Actuator it also wants to use the primary data source on startup for a health indicator. You can override that by prividing a bean of the same type, e.g.
#Bean
#Scope(value="session", proxyMode=ScopedProxyMode.TARGET_CLASS)
#Lazy
public DataSourcePublicMetrics dataSourcePublicMetrics() {
return new DataSourcePublicMetrics();
}
P.S. I believe the GlobalAuthenticationConfigurerAdapter might not be necessary in Spring Boot 1.2.2, but it is in 1.2.1 or 1.1.10.
Related
I need to configure multiple LDAP data sources / LdapTemplates in my Spring Boot 2 application. The first LdapTemplate will be used for most of the work, while the second will be used for a once-in-a-while subset of data (housed elsewhere).
I have read these StackOverflow questions regarding doing that, but they seem to be for Spring Boot 1.
Can a spring ldap repository project access two different ldap directories?
Multiple LDAP repositories with Spring LDAP Repository
From what I can gather, much of that configuration/setup had to be done anyway, even for just one LDAP data source, back in Spring Boot 1. With Spring Boot 2, I just put the properties in my config file like so
ldap.url=ldap://server.domain.com:389
ldap.base:DC=domain,DC=com
ldap.username:domain\ldap.svc.acct
ldap.password:secret
and autowire the template in my repository like so
#Autowired
private final LdapTemplate ldapTemplate;
and I'm good to go. (See: https://stackoverflow.com/a/53474188/3669288)
For a second LDAP data source, can I just add the properties and configuration elements for "ldap2" and be done (see linked questions)? Or does adding this configuration cause Spring Boot 2's auto configuration to think I'm overriding it and so now I lose my first LdapTemplate, meaning I now need to go explicitly configure that as well?
If so, do I need to configure everything, or will only a partial configuration work? For example, if I add the context source configuration and mark it as #Primary (does that work for LDAP data sources?), can I skip explicitly assigning it to the first LdapTemplate? On a related note, do I still need to add the #EnableLdapRepositories annotation, which is otherwise autoconfigured by Spring Boot 2?
TLDR: What's the minimum configuration I need to add in Spring Boot 2 to wire in a second LdapTemplate?
This takes what I've learned over the weekend and applies it as an answer to my own question. I'm still not an expert in this so I welcome more experienced answers or comments.
The Explanation
First, I still don't know for certain if I need the #EnableLdapRepositories annotation. I don't yet make use of those features, so I can't say if not having it matters, or if Spring Boot 2 is still taking care of that automatically. I suspect Spring Boot 2 is, but I'm not certain.
Second, Spring Boot's autoconfigurations all happen after any user configurations, such as my code configuring a second LDAP data source. The autoconfiguration is using a couple of conditional annotations for whether or not it runs, based on the existence of a context source or an LdapTemplate.
This means that it sees my "second" LDAP context source (the condition is just that a context source bean exists, regardless of what its name is or what properties it is using) and skips creating one itself, meaning that I no longer have that piece of my primary data source configured.
It will also see my "second" LdapTemplate (again, the condition is just that an LdapTemplate bean exists, regardless of what its name is or what context source or properties it is using) and skip creating one itself, so I again no longer have that piece of my primary data source configured.
Unfortunately, those conditions mean that in this case there is no in-between either (where I can manually configure the context source, for example, and then allow the autoconfiguration of the LdapTemplate to still happen). So the solution is to either make my configuration run after the autoconfiguration, or to not leverage the autoconfiguration at all and set them both up myself.
As for making my configuration run after the autoconfiguration: the only way to do that is to make my configuration an autoconfiguration itself and specify its order to be after Spring's built-in autoconfiguration (see: https://stackoverflow.com/a/53474188/3669288). That's not appropriate for my use case, so for my situation (because Spring Boot's setup does make sense for a standard single-source situation) I'm stuck forgoing the autoconfiguration and setting them both up myself.
The Code
Setting up two data sources is pretty well covered in the following two answers (though partly for other reasons), as linked in my question, but I'll also detail my setup here.
Can a spring ldap repository project access two different ldap directories?
Multiple LDAP repositories with Spring LDAP Repository
First up, the configuration class needs to be created, as one was not previously needed at all with Spring Boot 2. Again, I left out the #EnableLdapRepositories annotation partly because I don't use it yet, and partly because I think Spring Boot 2 will still cover that for me. (Note: All of this code was typed up in the Stack Overflow answer box as I don't have a development environment where I'm writing this, so imports are skipped and the code may not be perfectly compilable and function correctly, though I hope it's good.)
#Configuration
public class LdapConfiguration {
}
Second is manually configuring the primary data source; the one that used to be autoconfigured but no longer will be. There is one piece of Spring Boot's autoconfiguration that can be leveraged here, and that is its reading in of the standard spring.ldap.* properties (into a properties object), but since it wasn't given a name, you have to reference it by its fully qualified class name. This means you can skip straight to setting up the context source for the primary data source. This code is not quite as full featured as the actual autoconfiguration code (See: Spring Code)
I marked this LdapTemplate as #Primary because for my use, this is the primary data source and so it's what all other autowired calls should default to. This also means you don't need a #Qualifier where you autowire this source up (as seen later).
#Configuration
public class LdapConfiguration {
#Bean(name="contextSource")
public LdapContextSource ldapContextSource(#Qualifier("spring.ldap-org.springframework.boot.autoconfigure.ldap.LdapProperties") LdapProperties properties) {
LdapContextSource source = new LdapContextSource();
source.setUrls(properties.getUrls());
source.setUserDn(properties.getUsername());
source.setPassword(properties.getPassword());
source.setBaseEnvironmentProperties(Collections.unmodifiableMap(properties.getBaseEnvironment()));
return source;
}
#Bean(name="ldapTemplate")
#Primary
public LdapTemplate ldapTemplate(#Qualifier("contextSource") LdapContextSource source) {
return new LdapTemplate(source);
}
}
Third is to manually configure the secondary data source, the one that caused all of this to begin with. For this one, you do need to configure the reading of your properties into an LdapProperties object. This code builds on the previous code, so you can see the complete class for context.
#Configuration
public class LdapConfiguration {
#Bean(name="contextSource")
public LdapContextSource ldapContextSource(#Qualifier("spring.ldap-org.springframework.boot.autoconfigure.ldap.LdapProperties") LdapProperties properties) {
LdapContextSource source = new LdapContextSource();
source.setUrls(properties.getUrls());
source.setUserDn(properties.getUsername());
source.setPassword(properties.getPassword());
source.setBaseEnvironmentProperties(Collections.unmodifiableMap(properties.getBaseEnvironment()));
return source;
}
#Bean(name="ldapTemplate")
#Primary
public LdapTemplate ldapTemplate(#Qualifier("contextSource") LdapContextSource source) {
return new LdapTemplate(source);
}
#Bean(name="ldapProperties2")
#ConfigurationProperties("app.ldap2")
public LdapProperties ldapProperties2() {
return new LdapProperties();
}
#Bean(name="contextSource2")
public LdapContextSource ldapContextSource2(#Qualifier("ldapProperties2") LdapProperties properties) {
LdapContextSource source = new LdapContextSource();
source.setUrls(properties.getUrls());
source.setUserDn(properties.getUsername());
source.setPassword(properties.getPassword());
source.setBaseEnvironmentProperties(Collections.unmodifiableMap(properties.getBaseEnvironment()));
return source;
}
#Bean(name="ldapTemplate2")
public LdapTemplate ldapTemplate2(#Qualifier("contextSource2") LdapContextSource source) {
return new LdapTemplate(source);
}
}
Finally, in your class that uses these LdapTemplates, you can autowire them as normal. This uses constructor autowiring instead of the field autowiring the other two answers used. Either is technically valid though constructor autowiring is recommended.
#Component
public class LdapProcessing {
protected LdapTemplate ldapTemplate;
protected LdapTemplate ldapTemplate2;
#Autowired
public LdapProcessing(LdapTemplate ldapTemplate, #Qualifier("ldapTemplate2") LdapTemplate ldapTemplate2) {
this.ldapTemplate = ldapTemplate;
this.ldapTemplate2 = ldapTemplate2;
}
}
TLDR: Defining a "second" LDAP data source stops the autoconfiguration of the first LDAP data source, so both must be (nearly fully) manually configured if using more than one; Spring's autoconfiguration can not be leveraged even for the first LDAP data source.
I recently migrated my spring boot/batch Java application from spring-boot/spring-framework (respectively) 1.x.x/4.x.x to => 2.x.x/5.x.x (2.2.4/5.2.3 to be specific). The problem is something is definitely wrong (in my opinion) with the transaction/entity manager, as when the .saveAll() method is called from the JpaRepository class of my database persistance layer, it jumps into the SpringAOP framework/libarary code and into a infinite loop. I see it returning a "DefaulTransaction" object from a method (invoke()). My application on 1.x.x/4.x.x when it worked, would return the actual ArrayList here of my entities. I am using spring-boot-starter, spring-boot-starter-web, spring-boot-starter-data-jpa, spring-boot-starter-batch, and hibernate/hibernate-envers/hibernate-entitymanager (also of course many other dependencies, let me know if you would like me to list them).
After some research, I'm finding people are saying that Spring Batch #EnableBatchProcessing annotation sets up a default transaction manager, which if I'm using JPA could be causing issues. Reference:
https://github.com/spring-projects/spring-boot/issues/2363
wilkinsona suggested defining this Bean in my #Configuration class:
#Bean
public BatchConfigurer batchConfigurer(DataSource dataSource, EntityManagerFactory entityManagerFactory) {
return new BasicBatchConfigurer(dataSource, entityManagerFactory);
}
I'm getting an error when I do this because its saying the BasicBatchConfigurer() has protected access. What is the best way to instantiate this?
I also saw some people saying removing the #EnableBatchProcessing annotation fixes the persistance to database issue, but when I remove this, I lose the ability to Autowire my JobBuilderFactory and StepBuilderFactory. Is there a way to remove the annotation and get these objects in my code so I can at-least test if this works? Sorry, I'm not completely a master with Spring Batch/Spring.
In my #Configuration class, I am using the PlatformTransactionManager. I am setting up my JobRepository something like this.:
#Bean
public JobRepository jobRepository(PlatformTransactionManager transactionManager,
#Qualifier("dataSource") DataSource dataSource) throws Exception {
JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
jobRepositoryFactoryBean.setDataSource(dataSource);
jobRepositoryFactoryBean.setTransactionManager(transactionManager);
jobRepositoryFactoryBean.setDatabaseType("POSTGRES");
return jobRepositoryFactoryBean.getObject();
}
I can provide any other information if needed. Another question is - if I was using the same code basically, transaction manager, entity manager etc.. how was old my code working on 1.x.x? Could I have a wrong dependency somewhere in my pom.xml such that my new migrated code is using a wrong method or something from the wrong dependency?
By default, #EnableBatchProcessing configures Spring Batch to use a DataSourceTransactionManager if you provide a DataSource. This transaction manager knows nothing about your JPA context. So if you want to use a JPA repository to save data, you need to configure Spring Batch to use a JpaTransactionManager.
Now in order to provide a custom transaction manager, you need to register a BatchConfigurer and override the getTransactionManager() method, something like:
#Bean
public BatchConfigurer batchConfigurer(DataSource dataSource) {
return new DefaultBatchConfigurer(dataSource) {
#Override
public PlatformTransactionManager getTransactionManager() {
return new JpaTransactionManager();
}
};
}
This is explained in the Configuring A Job section and in the Javadoc of #EnableBatchProcessing.
I was working on a project using Spring boot, Spring MVC, and Hibernate. I encountered this problem which had already taken me 2 days.
My project was an imitation of twitter. When I started to work on the project, I used the JPA to get the Hibernate Session. Here is the code in my BaseDaoImpl class:
#Autowired
private EntityManagerFactory entityManagerFactory;
public Session getSession(){
return entityManagerFactory.createEntityManager().unwrap(Session.class);
}
In my Service class, I used the #Transactional annotation:
#Service("userServ")
#Transactional(propagation=Propagation.REQUIRED, readOnly=false,rollbackFor={Exception.class, RuntimeException.class})
public class UserServImpl implements IUserServ {}
And finally, an overview of my main class:
#SpringBootApplication
#EnableTransactionManagement
#EntityScan(basePackages = {"edu.miis.Entities"})
#ComponentScan({"edu.miis.Controllers","edu.miis.Service","edu.miis.Dao"})
#EnableAutoConfiguration
#Configuration
public class FinalProjectSpringbootHibernateDruidApplication {
public static void main(String[] args) {
SpringApplication.run(FinalProjectSpringbootHibernateDruidApplication.class, args);
}
}
When I was using this setting, everything seemed fine - until I was able to move up to a the extent where I started to add "post" function. I could add posts and comments into the database. However, I could not do this a lot of times. Every time I added up to 4 posts, the program ceased to run - no exceptions, no errors - the page just got stuck there.
I looked up online, realizing that the problem was probably due to the entityManagerFactory. I was told that entityManagerFactory.createEntityManager().unwrap(Session.class)opens new Hibernate sessions, instead of the traditional sessionFactory.getCurrentSession() that returns an existing session.
So I started to work on it. I changed my Dao configuration into this:
#Autowired
private EntityManagerFactory entityManagerFactory;
public Session getSession(){
Session session = entityManagerFactory.unwrap(SessionFactory.class).getCurrentSession();
return session;
}
My idea was to use the autowired EntityManagerFactory to return a Hibernate SessionFactory so that the getCurrentSession method can be then used.
But then I got problem:
Since I configured to this setting, any operation that involves input from controller to the service-dao-database invokes an exception: No Transaction Is In Progress
But the weird thing is: although the system broke due to no visible transaction in progress, Hibernate still generates new SQL statements, and data still get synchronized into the database.
Can anybody help me over how to get this issue resolved?
Sincerely thanks!
Following #M. Deinum's suggestion, I finally had this issue resolved.
The reason why the #Transactional annotation didn't work in my code in the first place, was because in my original code, I used plain Hibernate features - Session, SessionFactory, getCurrentSession() etc.
In order for these features to work, I need to specifically configure the transaction manager into a Hibernate Transaction Manager (under default setting, Spring boot autowires a JPA transaction manager).
But the problem is: most of methods that were used to support Hibernate features are now deprecated. Now, the JPA method is the mainstream.
Use EntityManager instead of Session/
Use EntityManager.persist instead of Session.save
Use EntityManager.merge instead of Session.update
Use EntityManager.remove instead of Session.remove.
That's all.
Thanks!
We're using Spring Boot 1.5.10 with Spring Data for Apache Cassandra and that's all working well.
We've had a new requirement coming along where we need to connect to a different keyspace while the service is up and running.
Through the use of Spring Cloud Config Server, we can easily set the value of spring.data.cassandra.keyspace-name, however, we're not certain if there's a way that we can dynamically switch (force) the service to use this new keyspace without having to restart if first?
Any ideas or suggestions?
Using #RefreshScope with properties/repositories doesn't work as the keyspace is bound to the Cassandra Session bean.
Using Spring Data Cassandra 1.5 with Spring Boot 1.5 you have at least two options:
Declare a #RefreshScope CassandraSessionFactoryBean, see also CassandraDataAutoConfiguration. This will interrupt all Cassandra operations upon refresh and re-create all dependant beans.
Listen to RefreshScopeRefreshedEvent and change the keyspace via USE my-new-keyspace;. This approach is less invasive and doesn't interrupt running queries. You'd basically use an event listener.
#Component
class MySessionRefresh {
private final Session session;
private final Environment environment;
// omitted constructors for brevity
#EventListener
#Order(Ordered.LOWEST_PRECEDENCE)
public void handle(RefreshScopeRefreshedEvent event) {
String keyspace = environment.getProperty("spring.data.cassandra.keyspace-name");
session.execute("USE " + keyspace + ";");
}
}
With Spring Data Cassandra 2, we introduced the SessionFactory abstraction providing AbstractRoutingSessionFactory for code-controlled routing of CQL/session calls.
Yes, you can use the #RefreshScope annotation on a the bean(s) holding the spring.data.cassandra.keyspace-name value.
After changing the config value through Spring Cloud Config Server, you have to issue a POST on the /refresh endpoint of your application.
From the Spring cloud documentation:
A Spring #Bean that is marked as #RefreshScope will get special treatment when there is a configuration change. This addresses the problem of stateful beans that only get their configuration injected when they are initialized. For instance if a DataSource has open connections when the database URL is changed via the Environment, we probably want the holders of those connections to be able to complete what they are doing. Then the next time someone borrows a connection from the pool he gets one with the new URL.
From the RefreshScope class javadoc:
A Scope implementation that allows for beans to be refreshed dynamically at runtime (see refresh(String) and refreshAll()). If a bean is refreshed then the next time the bean is accessed (i.e. a method is executed) a new instance is created. All lifecycle methods are applied to the bean instances, so any destruction callbacks that were registered in the bean factory are called when it is refreshed, and then the initialization callbacks are invoked as normal when the new instance is created. A new bean instance is created from the original bean definition, so any externalized content (property placeholders or expressions in string literals) is re-evaluated when it is created.
How can I set the name of an embedded database started in a Spring Boot app running in a test?
I'm starting two instances of the same Spring Boot app as part of a test, as they collaborate. They are both correctly starting an HSQL database, but defaulting to a database name of testdb despite being provided different values for spring.datasource.name.
How can I provide different database names, or some other means of isolating the two databases? What is going to be the 'lightest touch'? If I can avoid it, I'd rather control this with properties than adding beans to my test config - the test config classes shouldn't be cluttered up because of this one coarse-grained collaboration test.
Gah - setting spring.datasource.name changes the name of the datasource, but not the name of the database.
Setting spring.datasource.url=jdbc:hsql:mem:mydbname does exactly what I need it to. It's a bit crap that I have to hardcode the embedded database implementation, but Spring Boot is using an enum for default values, which would mean a bigger rewrite if it were to try getting the name from a property.
You can try it so:
spring.datasource1.name=testdb
spring.datasource2.name=otherdb
And then declare datasource in your ApplicationConfig like this
#Bean
#ConfigurationProperties(prefix="spring.datasource1")
public DataSource dataSource1() {
...
}
#Bean
#ConfigurationProperties(prefix="spring.datasource2")
public DataSource dataSource2() {
...
}
See official docs for more details: https://docs.spring.io/spring-boot/docs/current/reference/html/howto-data-access.html#howto-configure-a-datasource