How do I create beans programmatically in Spring Boot?

Question

I have an app that has a number of datasource settings listed in application.properties. I have a @ConfigurationProperties class that loads up these settings. Now I want to take the values from this ConfigurationProperties class and use them to create DataSource beans on-the-fly. I've tried using @PostConstruct and implementing BeanFactoryPostProcessor. However, with BeanFactoryPostProcessor, the processing seems to happen to early - before my ConfigurationProperties class has been populated. How can I read properties and create DataSource beans on the fly with Spring Boot?

Here's what my application.properties looks like:

ds.clients[0]=client1|jdbc:db2://server/client1
ds.clients[1]=client2,client3|jdbc:db2://server/client2
ds.clients[2]=client4|jdbc:db2://server/client4
ds.clients[3]=client5|jdbc:db2://server/client5

And my ConfigurationProperties class:

@Component
@ConfigurationProperties(prefix = "ds")
public class DataSourceSettings {
    public static Map<String, String> CLIENT_DATASOURCES = new LinkedHashMap<>();

    private List<String> clients = new ArrayList<>();

    public List<String> getClients() {
        return clients;
    }

    public void setClients(List<String> clients) {
        this.clients = clients;
    }

    @PostConstruct
    public void configure() {
        for (String client : clients) {
            // extract client name
            String[] parts = client.split("\\|");
            String clientName = parts[0];
            String url = parts[1];
            // client to datasource mapping
            String dsName = url.substring(url.lastIndexOf("/") + 1);
            if (clientName.contains(",")) {
                // multiple clients with same datasource
                String[] clientList = clientName.split(",");
                for (String c : clientList) {
                    CLIENT_DATASOURCES.put(c, dsName);
                }
            } else {
                CLIENT_DATASOURCES.put(clientName, dsName);
            }
        }
    }

At the end of this @PostConstruct method, I'd like to create a BasicDataSource with these settings and add it to the ApplicationContext. However, if I try to do this by implement BeanFactoryPostProcessor and implementing postProcessBeanFactory, the clients property is null, as is the CLIENT_DATASOURCES that I've populated with @PostConstruct.

@Override
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
    System.out.println("clients: " + CLIENT_DATASOURCES);
}

What's the best way to create datasources on-the-fly with Spring Boot?

This is an old question, so it deserves an [old answer](https://stackoverflow.com/questions/28374000/spring-programmatically-generate-a-set-of-beans/28550486#28550486). It's not specific to Spring Boot, but general to Spring. I think it still holds. — fps, Jan 05 '18 at 16:07

Stephane Nicoll · Accepted Answer · 2020-02-09T08:37:39.683

How about creating your beans and ask Spring Boot to inject values into it?

Something like

@Bean
@ConfigurationProperties("ds.client1")
public DataSource dataSourceClient1() {
    DataSourceBuilder.create().build();
}

@Bean
@ConfigurationProperties("ds.client2")
public DataSource dataSourceClient2() {
    DataSourceBuilder.create().build();
}

Then, any setting in the ds.client1 namespace belongs to the first data source (i.e. ds.client1.password is the data source password for that DataSource).

But maybe you don't know how much data sources you'll have? This is getting more complicated, especially if you need to inject those dynamic data sources in other objects. If you only need to lookup them by name, you could register them yourself as singletons. Here is an example that works

@ConfigurationProperties(prefix = "ds")
public class DataSourceSettings implements BeanFactoryAware {

    private List<String> clients = new ArrayList<>();

    private BeanFactory beanFactory;

    public List<String> getClients() {
        return clients;
    }

    public void setClients(List<String> clients) {
        this.clients = clients;
    }

    @Override
    public void setBeanFactory(BeanFactory beanFactory) {
        this.beanFactory = beanFactory;
    }

    @PostConstruct
    public void configure() {
        Map<String, String> clientDataSources = new HashMap<String, String>();
        for (String client : clients) {
            // extract client name
            String[] parts = client.split("\\|");
            String clientName = parts[0];
            String url = parts[1];
            // client to datasource mapping
            String dsName = url.substring(url.lastIndexOf("/") + 1);
            if (clientName.contains(",")) {
                // multiple clients with same datasource
                String[] clientList = clientName.split(",");
                for (String c : clientList) {
                    clientDataSources.put(c, url);
                }
            }
            else {
                 clientDataSources.put(clientName, url);
            }
        }
        Assert.state(beanFactory instanceof ConfigurableBeanFactory, "wrong bean factory type");
        ConfigurableBeanFactory configurableBeanFactory = (ConfigurableBeanFactory) beanFactory;
        for (Map.Entry<String, String> entry : clientDataSources.entrySet()) {
            DataSource dataSource = createDataSource(entry.getValue());
            configurableBeanFactory.registerSingleton(entry.getKey(), dataSource);
        }
    }

    private DataSource createDataSource(String url) {
        return DataSourceBuilder.create().url(url).build();
    }
}

Note that those beans are only available by bean name lookup. Let me know if that works out for you.

I don't think the first option will work because don't know how many datasources I'll have. — Matt Raible, Aug 06 '14 at 14:52
Alright, let me give some more thoughts on the second case then. — Stephane Nicoll, Aug 06 '14 at 15:20
I am guessing that those data sources are bound to other objects (i.e. you need to wire them in other beans)? — Stephane Nicoll, Aug 06 '14 at 16:45
The datasources I want to create dynamically are not bound to other objects. They're referenced by SQL statements in Apache Camel. For example: sql:select * from table where id=# order by name?dataSource=myDS — Matt Raible, Aug 07 '14 at 02:28
Thanks Stéphane - that worked! I did need to add a check to see if the bean was already registered (for unit tests). Also, I changed from List clients to a List to get rid of some string parsing. Cheers! — Matt Raible, Aug 07 '14 at 11:09
@Matt - I have same requirement where I have to hit multiple databases with same query. Can you please share small piece of code where I can pick the data source from this list of ds beans to execute query. — kapil gupta, Aug 17 '20 at 10:25

score 17 · Answer 2 · answered Aug 07 '14 at 06:42

I created an example project on github to demonstrate your usecase.

https://github.com/lhotari/dynamic-datasources

I implemented a ImportBeanDefinitionRegistrar to add the beans. You can get a hold of the configuration by implementing EnvironmentAware. There might be other ways to achieve your goal, but this was the way I used in GspAutoConfiguration to register beans dynamicly. GspAutoConfiguration makes Grails GSP available in Spring Boot applications.

Here's the relevant configuration class in the dynamic-datasource sample: https://github.com/lhotari/dynamic-datasources/blob/master/src/main/groovy/sample/DynamicDataSourcesConfiguration.java

package sample;

import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;

import org.springframework.beans.FatalBeanException;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.beans.factory.support.GenericBeanDefinition;
import org.springframework.boot.bind.PropertiesConfigurationFactory;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportBeanDefinitionRegistrar;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.Environment;
import org.springframework.core.type.AnnotationMetadata;
import org.springframework.jdbc.datasource.SingleConnectionDataSource;
import org.springframework.validation.BindException;

@Configuration
public class DynamicDataSourcesConfiguration implements ImportBeanDefinitionRegistrar, EnvironmentAware {
    private ConfigurableEnvironment environment;
    private static Map<String, Object> defaultDsProperties = new HashMap<String, Object>() {
        {
            put("suppressClose", true);
            put("username", "sa");
            put("password", "");
            put("driverClassName", "org.h2.Driver");
        }
    };

    @Override
    public void setEnvironment(Environment environment) {
        this.environment = (ConfigurableEnvironment)environment;
    }

    @Override
    public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry registry) {
        DataSourceSettings settings = resolveSettings();
        for (Entry<String, String> entry : settings.clientDataSources().entrySet()) {
            createDsBean(registry, entry.getKey(), entry.getValue());
        }
    }

    private void createDsBean(BeanDefinitionRegistry registry, String beanName, String jdbcUrl) {
        GenericBeanDefinition beanDefinition = createBeanDefinition(SingleConnectionDataSource.class);
        beanDefinition.getPropertyValues().addPropertyValues(defaultDsProperties).addPropertyValue("url", jdbcUrl);
        registry.registerBeanDefinition(beanName, beanDefinition);
    }

    private GenericBeanDefinition createBeanDefinition(Class<?> beanClass) {
        GenericBeanDefinition beanDefinition = new GenericBeanDefinition();
        beanDefinition.setBeanClass(beanClass);
        beanDefinition.setAutowireMode(GenericBeanDefinition.AUTOWIRE_NO);
        return beanDefinition;
    }

    private DataSourceSettings resolveSettings() {
        DataSourceSettings settings = new DataSourceSettings();
        PropertiesConfigurationFactory<Object> factory = new PropertiesConfigurationFactory<Object>(settings);
        factory.setTargetName("ds");
        factory.setPropertySources(environment.getPropertySources());
        factory.setConversionService(environment.getConversionService());
        try {
            factory.bindPropertiesToTarget();
        }
        catch (BindException ex) {
            throw new FatalBeanException("Could not bind DataSourceSettings properties", ex);
        }
        return settings;
    }

}

That's cool, but how to select the appropriate DS for CrudRepository and JdbcTemplate? — Eugene Hoza, Mar 31 '20 at 07:46

How do I create beans programmatically in Spring Boot?

2 Answers2

Linked