Single API, multiple Elasticsearch instances

Question

We have a Spring Boot Restful API that needs to get data from 2 different Elasticsearch instances (on different servers), 1 for "shared" data (with about 5 different indexes on it) and 1 for "private" data (with about 3 different indexes). Currently running against just the "private" data instance, everything is good. But we now need to get at the "shared" data now.

In our Spring Boot application, we have enabled Elasticsearch repositories like this

@SpringBootApplication
@EnableElasticsearchRepositories(basePackages = {
    "com.company.core.repositories",  //<- private repos here...
    "com.company.api.repositories"  //<-- shared repos here...
})
public class Application { //... }

Then we access the "private" data with an ElasticsearchRepository like:

package com.company.core.repositories

public interface DocRepository extends ElasticsearchRepository<Doc, Integer> { ... }

In our endpoint, we have...

@RestController
@CrossOrigin
@RequestMapping("/v2/statuses/")
public class StatusEndpoint {
    @Resource
    private ElasticsearchTemplate template;

    @Autowired
    private DocRepository docRepository;

    @Autowired
    private Validator validator;
    //...
}

Now we want to add another repository like:

package com.company.api.repositories

public interface LookupRepository extends ElasticsearchRepository<Lookup, Integer> { ... }

Then in our API layer we would add an auto-wired instance...

@Autowired
private LookupRepository lookupRepo;

We were thinking that we could define multiple Beans with different names, but how do we associate each of the "elasticsearchTemplate" beans with the different ElasticsearchRepository instances that need them? Also, how do we associate the "private" bean/configuration with injected instances of

@Resource
private ElasticsearchTemplate template;

Where we need to use that natively?

score 1 · Accepted Answer · answered Dec 21 '16 at 23:49

You can solve this with 2 unique Elasticsearch configuration beans and an @Resource(name="XXX") annotation for the template injection in your StatusEndpoint controller.

If you segregate your repositories into different packages depending on which Elasticsearch cluster they should use, you can associate them with different configurations using the @EnableElasticsearchRepositories annotation.

For example:

If you have these packages and classes:

com.company.data.repositories.private.YourPrivateRepository
com.company.data.repositories.shared.YourSharedRepository

And then these Configurations:

@Configuration
@EnableElasticsearchRepositories( 
    basePackages = {"com.company.data.repositories.private"},
    elasticsearchTemplateRef = "privateElasticsearchTemplate")
public class PrivateElasticsearchConfiguration {

    @Bean(name="privateElasticsearchTemplate")
    public ElasticsearchTemplate privateTemplate() {
       //code to create connection to private ES cluster
    }
}

@Configuration
@EnableElasticsearchRepositories( 
    basePackages = {"com.company.data.repositories.shared"},
    elasticsearchTemplateRef = "sharedElasticsearchTemplate")
public class SharedElasticsearchConfiguration {

    @Bean(name="sharedElasticsearchTemplate")
    public ElasticsearchTemplate sharedTemplate() {
       //code to create connection to shared ES cluster
    }
}

Because of the elasticsearchTemplateRef parameter in the @EnableElasticsearchRepositories annotation, the JPA code that implements the repositories will use the specified template for repositories in the basePackages list.

For the StatusEndpoint portion, you would just provide your @Resource annotation with the correct template bean name. Your StatusEndpoint would look like this:

@RestController
@CrossOrigin
@RequestMapping("/v2/statuses/")
public class StatusEndpoint {

    @Resource(name="privateElasticsearchTemplate")
    private ElasticsearchTemplate template;

    @Autowired
    private DocRepository docRepository;

    @Autowired
    private Validator validator;
    //...
}

This worked out just right for the scenario. Allows me to use repositories and ElasticSearchTemplate. — HenryC, Dec 28 '16 at 16:44

score 0 · Answer 2 · edited May 23 '17 at 11:53

There might be multiple ways to do this. Here is one that takes advantage of the @Bean name and the @Resource name.

@Configuration
public class MyElasticConfig{

    @Bean //this is your private template
    public ElasticsearchTemplate template(){
        //construct your template
        return template;
    }

    @Bean //this is your public template
    public ElasticsearchTemplate publicTemplate(){
        //construct your template
        return template;
    }
}

then you can get them like this...

@Resource
private ElasticsearchTemplate template;

@Resource
private ElasticsearchTemplate publicTemplate;

or

@Resource(name="template")
private ElasticsearchTemplate anyName;

@Resource(name="publicTemplate")
private ElasticsearchTemplate anyOtherName;

also you can name your @Bean's directly instead of relying on the @Bean's method name.

    @Bean (name="template")
    public ElasticsearchTemplate myPrivateTemplate(){
        //construct your template
        return template;
    }

    @Bean (name="publicTemplate")
    public ElasticsearchTemplate myPubTemplate(){
        //construct your template
        return template;
    }

Check out these wonderful resources on the topic.

SPRING INJECTION WITH @RESOURCE, @AUTOWIRED AND @INJECT
Bean Annotation Type
Autowired vs Resource

Single API, multiple Elasticsearch instances

2 Answers2