Single API, multiple Elasticsearch instances - java

We have a Spring Boot Restful API that needs to get data from 2 different Elasticsearch instances (on different servers), 1 for "shared" data (with about 5 different indexes on it) and 1 for "private" data (with about 3 different indexes). Currently running against just the "private" data instance, everything is good. But we now need to get at the "shared" data now.
In our Spring Boot application, we have enabled Elasticsearch repositories like this
#SpringBootApplication
#EnableElasticsearchRepositories(basePackages = {
"com.company.core.repositories", //<- private repos here...
"com.company.api.repositories" //<-- shared repos here...
})
public class Application { //... }
Then we access the "private" data with an ElasticsearchRepository like:
package com.company.core.repositories
public interface DocRepository extends ElasticsearchRepository<Doc, Integer> { ... }
In our endpoint, we have...
#RestController
#CrossOrigin
#RequestMapping("/v2/statuses/")
public class StatusEndpoint {
#Resource
private ElasticsearchTemplate template;
#Autowired
private DocRepository docRepository;
#Autowired
private Validator validator;
//...
}
Now we want to add another repository like:
package com.company.api.repositories
public interface LookupRepository extends ElasticsearchRepository<Lookup, Integer> { ... }
Then in our API layer we would add an auto-wired instance...
#Autowired
private LookupRepository lookupRepo;
We were thinking that we could define multiple Beans with different names, but how do we associate each of the "elasticsearchTemplate" beans with the different ElasticsearchRepository instances that need them? Also, how do we associate the "private" bean/configuration with injected instances of
#Resource
private ElasticsearchTemplate template;
Where we need to use that natively?

You can solve this with 2 unique Elasticsearch configuration beans and an #Resource(name="XXX") annotation for the template injection in your StatusEndpoint controller.
If you segregate your repositories into different packages depending on which Elasticsearch cluster they should use, you can associate them with different configurations using the #EnableElasticsearchRepositories annotation.
For example:
If you have these packages and classes:
com.company.data.repositories.private.YourPrivateRepository
com.company.data.repositories.shared.YourSharedRepository
And then these Configurations:
#Configuration
#EnableElasticsearchRepositories(
basePackages = {"com.company.data.repositories.private"},
elasticsearchTemplateRef = "privateElasticsearchTemplate")
public class PrivateElasticsearchConfiguration {
#Bean(name="privateElasticsearchTemplate")
public ElasticsearchTemplate privateTemplate() {
//code to create connection to private ES cluster
}
}
#Configuration
#EnableElasticsearchRepositories(
basePackages = {"com.company.data.repositories.shared"},
elasticsearchTemplateRef = "sharedElasticsearchTemplate")
public class SharedElasticsearchConfiguration {
#Bean(name="sharedElasticsearchTemplate")
public ElasticsearchTemplate sharedTemplate() {
//code to create connection to shared ES cluster
}
}
Because of the elasticsearchTemplateRef parameter in the #EnableElasticsearchRepositories annotation, the JPA code that implements the repositories will use the specified template for repositories in the basePackages list.
For the StatusEndpoint portion, you would just provide your #Resource annotation with the correct template bean name. Your StatusEndpoint would look like this:
#RestController
#CrossOrigin
#RequestMapping("/v2/statuses/")
public class StatusEndpoint {
#Resource(name="privateElasticsearchTemplate")
private ElasticsearchTemplate template;
#Autowired
private DocRepository docRepository;
#Autowired
private Validator validator;
//...
}

There might be multiple ways to do this. Here is one that takes advantage of the #Bean name and the #Resource name.
#Configuration
public class MyElasticConfig{
#Bean //this is your private template
public ElasticsearchTemplate template(){
//construct your template
return template;
}
#Bean //this is your public template
public ElasticsearchTemplate publicTemplate(){
//construct your template
return template;
}
}
then you can get them like this...
#Resource
private ElasticsearchTemplate template;
#Resource
private ElasticsearchTemplate publicTemplate;
or
#Resource(name="template")
private ElasticsearchTemplate anyName;
#Resource(name="publicTemplate")
private ElasticsearchTemplate anyOtherName;
also you can name your #Bean's directly instead of relying on the #Bean's method name.
#Bean (name="template")
public ElasticsearchTemplate myPrivateTemplate(){
//construct your template
return template;
}
#Bean (name="publicTemplate")
public ElasticsearchTemplate myPubTemplate(){
//construct your template
return template;
}
Check out these wonderful resources on the topic.
SPRING INJECTION WITH #RESOURCE, #AUTOWIRED AND #INJECT
Bean Annotation Type
Autowired vs Resource

Related

Spring: Implement two instances of a #Service with two instances of #Configuration linked to each respective service

An existing utility exists that I am importing to my project. It is written similarly as these two classes
#Service
public class ServiceAccessorImpl implements ServiceAccessor {
#Autowired private ServiceConfiguration serviceConfiguration;
public Response executeCall(){
return callEndPoint(serviceConfiguration.getServiceEndPoint());
}
}
#Configuration
#Data //automatically adds getters and setters
#ConfigurationProperties(prefix="config")//pulls serviceEndPoint value from external config
//Assume external config has config.serviceEndPoint = "www.endpoint1.com"
public class ServiceConfiguration {
private String serviceEndPoint;
}
In a separate project below I am importing the above into my project. I would like to have two instances of the same service with two unique and respective configuration classes. so that service1 is linked to config1 and service2 is linked to config2. My reasoning is I want one instance that only pulls the endpoint from the external configuration and another instance that I can use to set the endpoint. I have tried using things like #Qualifier but I cant figure out how to link the correct config with the correct service. I have a feeling that this may not be possible because ServiceConfiguration is privately scoped within ServiceAccessorImpl and I have no access through setters or constructors.
Controller Endpoint. The below is psuedo code of how I would like to implement my design. Autowiring in a single instance and using either endpoint works for that endpoint but not for both as shown below.
#ComponentScan(basePackageClass = ServiceAccessorImpl.class)
public class ServiceAccessorController {
#Autowired private ServiceAccessor serviceAccessor1;
#Autowired private ServiceConfiguration serviceConfiguration1;
#Autowired private ServiceAccessor serviceAccessor2;
#Autowired private ServiceConfiguration serviceConfiguration2;
Response CallEndpoint1(){
//www.endpoint1.com is already set here from external config
return serviceAccessor1.executeCall();
}
Response CallEndpoint1(){
serviceConfiguration2.setServiceEndPoint("www.endpoint2.com")
return serviceAccessor2.executeCall();
}
}
Thank you in advance
If you need multiple instances of the same implementation, it's easier to not annotate it as a bean, and have a #Configuration provide the beans instead.
public class ServiceAccessorImpl implements ServiceAccessor {
private ServiceConfiguration serviceConfiguration;
public ServiceAccessorImpl(ServiceConfiguration configuration) {
this.serviceConfiguration = configuration;
}
public Response executeCall(){
return callEndPoint(serviceConfiguration.getServiceEndPoint());
}
}
// this one should just have #ConfigurationProperties, not #Configuration
#Data
#ConfigurationProperties(prefix="config")
public class ServiceConfiguration {
private String serviceEndPoint;
}
then in your service you can have a Configuration providing both instances:
#Configuration
public class BeansConfiguration {
#Qualifier("service1")
#Primary // optional, Spring will autowire this instance by default if no qualifier is given
#Bean
public service1(#Autowired ServiceConfiguration config) {
// use the default config bean
return new ServiceAccessorImpl(config);
}
#Qualifier("service2")
#Bean
public service2() {
return new ServiceAccessorImpl(new ServiceConfiguration("www.endpoint2.com"));
}
}
then you may consume both by using the qualifiers (note that you don't have to inject the configs here):
public class ServiceAccessorController {
#Autowired
private ServiceAccessor serviceAccessor1;
#Autowired
#Qualifier("service2")
private ServiceAccessor serviceAccessor2;
Response CallEndpoint1(){
return serviceAccessor1.executeCall();
}
Response CallEndpoint2(){
return serviceAccessor2.executeCall();
}
}

How to get a list of beans?

I have something like :
#Configuration
#EnableTransactionManagement
public class JdbcTemplateConfig{
#Bean("JdbcTemplateOne")
public NamedParameterJdbcTemplate (#Qualifier(firstDataSource final DataSource ds)){
return new NamedParameterJdbcTemplate(ds);
}
#Bean("JdbcTemplateTwo")
public NamedParameterJdbcTemplate (#Qualifier(secondDataSource final DataSource ds)){
return new NamedParameterJdbcTemplate(ds);
}
#Bean("JdbcTemplateThree")
public NamedParameterJdbcTemplate (#Qualifier(thirdDataSource final DataSource ds)){
return new NamedParameterJdbcTemplate(ds);
}
}
and now I need a list of above templates. To get a one by one (for example in an IT-Test) I can make something like:
#SpringBootTest
public class SomeITCase{
#Autowired
#Qualifier("JdbcTemplateOne")
private NamedParameterJdbcTemplate jdbcTemplate1;
#Autowired
#Qualifier("JdbcTemplateTwo")
private NamedParameterJdbcTemplate jdbcTemplate2;
#Autowired
#Qualifier("JdbcTemplateThree")
private NamedParameterJdbcTemplate jdbcTemplate3;
???
List<NamedParameterJdbcTemplate> myList = ???
}
Question: How to get all templates in a list without to have declaring them one by one?
What you a re looking for is a straight forward #Autowired annotation.
Let's suppose we have several beans defined like in your above example.
Now, we create a test like below:
#SpringBootTest
#Slf4j
public class SomeITCase {
#Autowired
#Qualifier("JdbcTemplateTwo")
private NamedParameterJdbcTemplate jdbcTemplate2;
#Autowired
List<NamedParameterJdbcTemplate> allTemplates;
#Test
public void testMyList() {
assertThat(allTemplates)
.hasSize(3)
.contains(jdbcTemplate2);
allTemplates.forEach(template -> {
log.info(template.toString());
});
}
}
Executing this test should make it really obvious: Spring injects all beans of matching type in your list. The test ensures that the size is three and at least the one known bean reference is contained.
For further information on getting started with #Autowired, maybe take a look at Injecting collections - Injecting Bean references.
To get list of all the beans from the Application context, you can do:
public void beanNames(ApplicationContext ctx) {
String[] beanNames = ctx.getBeanDefinitionNames();
for (String beanName : beanNames) {
System.out.println(beanName);
}
}
You can also look into Spring boot acutator. It provides endpoints like /bean to get all the beans registered with Spring.
If you just want a list of all NamedParameterJdbcTemplate beans in your application, you could just ask spring to autowire it for you. It allows you to autowire a list of all beans of a specific type:
{
...
#Autowired
private List<NamedParameterJdbcTemplate> allJdbcTemplates;
...
}

Spring specify implementation in configuration

I'm writing application using spring mvc/boot, and I have two storage implementations: database storage and in memory storage. My global idea is choose in configuration file what storage application should use.
My idea is
put #Qualifier annotation on each storage implementation
create two configurations, like databaseStorageConfiguration and InMemoryStorageConfiguration
depends on profile, apply first or second configuration
The thing is I don't know how to bind implementation and configuration.
I tried something like this:
#Configuration
public class InMemoryStorageConfig {
#Autowired
#Qualifier("inMemoryStorage")
private Storage storage;
#Bean
public Storage getStorage() {
return storage;
}
}
But I get an error, that 3 beans were found: 2 beans with dfferent implementation and the 3rd one - in config
UPDATE 1
I've added #Profile("InMemory") to Configuration and activated that profile in properties. That gave no changes but looks more logical now
UPDATE 2
Full configuration:
#SpringBootApplication
#ImportResource("classpath:spring-config.xml")
public class Application {
public static void main(String... args) {
SpringApplication.run(Application.class, args);
}
}
#Service
public class WidgetService {
private WidgetCache widgetCache;
#Autowired
public WidgetService(WidgetCache widgetCache) {
this.widgetCache = widgetCache;
}
....
#Qualifier("databaseWidgetCache")
#Transactional
#Repository
public class DatabaseWidgetCache implements WidgetCache {
private WidgetRepository widgetRepository;
#Autowired
public DatabaseWidgetCache(WidgetRepository widgetRepository) {
this.widgetRepository = widgetRepository;
}
#Qualifier("inMemoryWidgetCache")
#Repository
public class InMemoryWidgetCache implements WidgetCache {
private WidgetLayersStorage widgetLayersStorage;
#Autowired
public InMemoryWidgetCache(WidgetLayersStorage widgetLayersStorage) {
this.widgetLayersStorage = widgetLayersStorage;
}
#Profile("InMemory")
#Configuration
public class InMemoryStorageConfig {
#Autowired
#Qualifier("inMemoryWidgetCache")
private WidgetCache widgetCache;
#Bean
public WidgetCache getWidgetCache() {
return widgetCache;
}
}
Stacktrace:
Parameter 0 of constructor in
com.widgets.service.widget.WidgetService required a single
bean, but 3 were found:
- inMemoryWidgetCache: defined in file [..../MemoryWidgetCache.class]
- databaseWidgetCache: defined in file [..../DatabaseWidgetCache.class]
- getWidgetCache: defined by method 'getWidgetCache' in class path resource
[......../InMemoryStorageConfig.class]
Action:
Consider marking one of the beans as #Primary, updating the consumer
to accept multiple beans, or using #Qualifier to identify the bean
that should be consumed
Your WidgetService should be changed to
#Service
public class WidgetService {
private WidgetCache widgetCache;
/** or
private List<WidgetCache> widgetCaches;
public WidgetService(List<WidgetCache> widgetCaches) {
this.widgetCaches = widgetCaches;
}
*/
public WidgetService(#Qualifier(<desired impl>) WidgetCache widgetCache) {
this.widgetCache = widgetCache;
}
}
and need to annotate your InMemoryWidgetCache and DatabaseWidgetCache with #Qualifier annotation. since you are using default convention.
and please remove
#Bean
public WidgetCache getWidgetCache() {
return widgetCache;
}
i don't see a real use there
In order to specify implementation in Configuration class, you don't need "Qualifier" annotation, and configuration should be changed to:
#Profile("inMemoryStorage")
#Import(InMemoryWidgetCache.class)
#Configuration
public class InMemoryStorageConfig {
}
thus, by activating profile, you choose the desire implementation

Creating an object of an #Service bean, is it possible?

i have an restful application with all the #Service,#RestController,#Repository beans. and im autowiring required beans.
Now i want to use the #service class in another class that is not managed by spring, is that possible?
These 2 classes are also in 2 diffrent maven projects if that makes any diffrence
i have tried creating a new object , as expected to no awail.
i have also tried creating diffrent constructors also to no awail.
i have googled for some anwsers but havent found any so now i turn to you experts ;)
The class i want to use!
#Service
public class ProductService {
ProductRepository repository;
#Autowired
public ProductService(ProductRepository repository){
this.repository = repository;
}
}
the Restcontroller
#RestController
#RequestMapping(path = "/product")
public class ProductResource {
#Autowired
ProductService service;
}
Repo
public interface ProductRepository extends JpaRepository<Product,Long> {
}
Here is where i want to create the service.
public static void main(String[] args) {
String chromeDriver = args[0];
String method = args[1];
String domainName = args[2];
ProductService service = new ProductService();
System.setProperty("webdriver.chrome.driver", chromeDriver);
Runner runner = new Runner(method,domainName);
runner.run();
}
As far as I know, you can instantiate a Spring-managed class in a non-Spring-managed class as long as the Spring-managed class (in this case the service) doesn't contain any Spring dependencies in it, as those will remain null (non initialized) because they are never gonna be injected by Spring.
Your service looks good to me because the ProductRepository field is not #autowired, but in your code you're creating the ProductService like this:
ProductService service = new ProductService();
While this parameterless constructor doesn't exist in the ProductService class:
#Service
public class ProductService
{
ProductRepository repository;
#Autowired
public ProductService(ProductRepository repository)
{
this.repository = repository;
}
}
I think this is not possible (I mean, it technically is, but it is not going to work).
To have access to your service class your second application needs to have access to the same Spring Context.
This is because Spring Context in your base application defines necessary dependencies that your second application doesn't have access to. One of the examples for this is ProductRepository repository (which is a way Spring Boot app will talk with the database), have no Spring Context and thus information on DB location and connection URL.
In your case you need to replicate the class and mechanism to connect to DB with DB configuration in your second project.
There's a workaround to "inject" dependencies manually. Create a managed component that stores the reference to the ApplicationContext in a static variable, so that you can access from non-managed classes:
#Component
public class ApplicationContextProvider implements ApplicationContextAware {
private static ApplicationContext applicationContext;
public static ApplicationContext applicationContext() {
return applicationContext;
}
#Override
public void setApplicationContext(ApplicationContext applicationContext) {
ApplicationContextProvider.applicationContext = applicationContext;
}
}
public class NoManagedClass {
private final ManagedClass bean;
public NoManagedClass() {
this.bean = ApplicationContextProvider.applicationContext().getBean(ManagedClass.class);
}
}
However, be aware that your non-managed objects should be created after the application context is set. You still have to start your Spring application in the main method so that the beans are initialized.

Injecting library class as dependencies in spring project

I have multiple library classes in my project which need to be injected into a service class. This is the error statement for IntegrationFactory class:
Consider defining a bean of type 'com.ignitionone.service.programmanager.integration.IntegrationFactory' in your configuration.
This error is coming on almost every injection where this library class is injected.
I have already added the Library package in #ComponentScan, but, as it is read-only file, I can not annotate the library class. I came to know from some answer here that Spring can not inject classes which it does not manage. This library is not built on spring.
I have tried to create a #Bean method which returns the IntegrationFactory(class in question) in the class where #Inject is used, but this too does not seem to work.
How can this be done, preferably without creating a stub/copy class?
This is EngagementServiceImpl class snippet:
#Inject
public EngagementServiceImpl(EngagementRepository engagementRepository,
#Lazy IntegrationFactory integrationFactory, TokenRepository tokenRepository,
EngagementPartnerRepository engagementPartnerRepository, MetricsService metricsService) {
this.engagementRepository = engagementRepository;
this.integrationFactory = integrationFactory;
this.tokenRepository = tokenRepository;
this.engagementPartnerRepository = engagementPartnerRepository;
this.metricsService = metricsService;
}
This is injection part:
#Autowired
private EngagementService engagementService;
This is ConfigClass:
#Configuration
public class ConfigClass {
#Bean
public IntegrationFactory getIntegrationFactory(){
Map<String, Object> globalConfig = new HashMap<>();
return new IntegrationFactory(globalConfig);
}
#Bean
#Primary
public EntityDataStore getEntityDataStore(){
EntityModel entityModel = Models.ENTITY;
return new EntityDataStore(this.dataSource(), entityModel );
}
#ConfigurationProperties(prefix = "datasource.postgres")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder
.create()
.build();
}
}
You need to add your bean definitions in a configuration class.
#Configuration
public class ServiceConfig {
#Bean
public IntegrationFactory getIntegrationFactory(){
// return an IntegrationFactory instance
}
}
Then you have to make sure your #Configuration class gets detected by Spring, either by having it within your scanned path or by manually importing it via #Import from somewhere withing you scanned path. An example of #Import, considering you are using Spring Boot.
#Import(ServiceConfig.class)
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Hope this helps!
Your Bean IntegrationFactory can't be found, as it is not annotated with any Spring stereotype and therefore not recognized by the component scan.
As you have multiple options to provide an instance of your class to the application context, read the Spring documentation (which also includes samples) to find out which one fits you the most:
https://docs.spring.io/spring/docs/5.1.0.RELEASE/spring-framework-reference/core.html#beans-java-basic-concepts
One Option would be to create a factory which provides an instance of your class to the application context, like it is stated in the documentation:
#Configuration
public class AppConfig {
#Bean
public IntegrationFactory myIntegrationFactory() {
return new IntegrationFactory();
}
}
Do not forget to add the Configuration to the application context.

Categories