35

I've read many questions and answers about dynamic datasource routing and have implemented a solution using AbstractRoutingDataSource and another(see below). That's fine, but requires hardcoded properties for all datasources. As the number of users using the application increases, this isn't a suitable way of routing any more. Also it would require to add an entry to the properties every time a new user registers. The situation is as follows

  • 1 database server
  • many schemas on that server, every user has their own schema.
  • I only need to change the schema name during runtime
  • schema name is retainable by logged in user

I'm using spring boot 1.4.0 together with hibernate 5.1 and spring data jpa

I can't find a way to change the schema completely dynamically. Does someone know how to do it in spring?

EDIT:

Thanks to @Johannes Leimer's answer, I got a working implemantation.

Here's the code:

User Provider:

@Component
public class UserDetailsProvider {
    @Bean
    @Scope("prototype")
    public CustomUserDetails customUserDetails() {
        return (CustomUserDetails) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
    }
}

UserSchemaAwareRoutingDatasource:

public class UserSchemaAwareRoutingDataSource extends AbstractDataSource {
@Inject
Provider<CustomUserDetails> customUserDetails;

@Inject
Environment env;
private LoadingCache<String, DataSource> dataSources = createCache();

@Override
public Connection getConnection() throws SQLException {
    try {
        return determineTargetDataSource().getConnection();
    } catch (ExecutionException e){
        e.printStackTrace();

        return null;
    }
}

@Override
public Connection getConnection(String username, String password) throws SQLException {
    System.out.println("getConnection" + username);
    System.out.println("getConnection2" + password);
    try {
        return determineTargetDataSource().getConnection(username, password);
    } catch (ExecutionException e) {
        e.printStackTrace();
        return null;
    }
}

private DataSource determineTargetDataSource() throws SQLException, ExecutionException {
    try {
        String schema = customUserDetails.get().getUserDatabase();
        return dataSources.get(schema);
    } catch (NullPointerException e) {
        e.printStackTrace();

        return dataSources.get("fooooo");
    }

}
baao
  • 71,625
  • 17
  • 143
  • 203
  • don't want to dive deeply into the stuff, can you just put all the logic into the method that returns prototype DataSource instances? – AdamSkywalker Sep 06 '16 at 20:40
  • Right. A multi-schema database makes far more sense than a multi-database server, which is what your question advertised before your edit. – Dawood ibn Kareem Sep 06 '16 at 20:47
  • Yes, changed the text, it was missleading. Thanks and sorry for the wrong wording! @DavidWallace – baao Sep 06 '16 at 20:48
  • I definitely wanna know the answer of this. I have a really big application that needs to switch schema for a particular job. But whenever that particular even occurs, it *hangs*. – Jay Sep 10 '16 at 18:47
  • 1
    http://forum.spring.io/forum/spring-projects/data/93914-dynamic-datasources-at-runtime This covers up most of the thing which you need. The user name you can get it from the thread local variable which you can set earlier. – Meherzad Sep 11 '16 at 05:41
  • @Meherzad the post is eight years old and unfortunately doesn't fit my requirements – baao Sep 13 '16 at 08:49
  • @baao can you elaborate on why exactly solution from that thread doesn`t fit your requirments? ThreadLocal context approach is pretty common and should cover your case. – chimmi Sep 13 '16 at 09:20
  • @baao You can try overriding AbstractRoutingDataSource.determineTargetDataSource() [Link](http://grepcode.com/file/repo1.maven.org/maven2/org.springframework/spring-jdbc/4.2.0.RELEASE/org/springframework/jdbc/datasource/lookup/AbstractRoutingDataSource.java#AbstractRoutingDataSource.determineTargetDataSource%28%29) method to have a different behavior. If the datasource is not found for your key then create one as per the user name and insert it into the map which holds all the datasources. – Meherzad Sep 13 '16 at 12:15
  • Continuation for previous comment **Note** You need add proper synchronization here to cater to the usecase where 2 threads request for the same datasource which is not created yet and we dont end up creating same datasource multiple times. – Meherzad Sep 13 '16 at 12:20
  • Could you please share example how you implement this one. – dnvsp Jul 02 '18 at 10:18
  • Could you please share example how you implement this one.It would be helpful for me if you provide example.here is my problem https://stackoverflow.com/questions/51130898/dynamic-data-source-creating-in-spring-boot – dnvsp Jul 02 '18 at 11:07
  • @bambam could you please share git repo of POC ? – Irfan Nasim Jun 11 '19 at 06:45

2 Answers2

32

Assumptions

Because I don't have the reputation yet to post a comment below your question, my answer is based on the following assumptions:

  • The current schema name to be used for the current user is accessible through a Spring JSR-330 Provider like private javax.inject.Provider<User> user; String schema = user.get().getSchema();. This is ideally a ThreadLocal-based proxy.

  • To build a DataSource which is fully configured in a way you need it requires the same properties. Every time. The only thing which is different is the schema name. (It would easily possible to obtain other different parameters as well, but this would be too much for this answer)

  • Each schema is already set up with the needed DDL, so there is no need for hibernate to create tables or something else

  • Each database schema looks completely the same except for its name

  • You need to reuse a DataSource every time the corresponding user makes a request to your application. But you don't want to have every DataSource of every user permanently in the memory.

My solution idea

Use a combination of ThreadLocal proxys to get the schema name and a Singleton-DataSource which behaves different on every user request. This solution is inspired by your hint to AbstractRoutingDataSource, Meherzad's comments and own experience.

A dynamic DataSource

I suggest to facilitate the AbstractDataSource of Spring and implement it like the AbstractRoutingDataSource. Instead of a static Map-like approach we use a Guava Cache to get an easy to use cache.

public class UserSchemaAwareRoutingDataSource extends AbstractDataSource {
    private @Inject javax.inject.Provider<User> user;
    private @Inject Environment env;
    private LoadingCache<String, DataSource> dataSources = createCache();

    @Override
    public Connection getConnection() throws SQLException {
        return determineTargetDataSource().getConnection();
    }

    @Override
    public Connection getConnection(String username, String password) throws SQLException {
        return determineTargetDataSource().getConnection(username, password);
    }

    private DataSource determineTargetDataSource() {
        String schema = user.get().getSchema();
        return dataSources.get(schema);
    }

    private LoadingCache<String, DataSource> createCache() {
        return CacheBuilder.newBuilder()
           .maximumSize(100)
           .expireAfterWrite(10, TimeUnit.MINUTES)
           .build(
               new CacheLoader<String, DataSource>() {
                 public DataSource load(String key) throws AnyException {
                   return buildDataSourceForSchema(key);
                 }
               });
    }

    private DataSource buildDataSourceForSchema(String schema) {
        // e.g. of property: "jdbc:postgresql://localhost:5432/mydatabase?currentSchema="
        String url = env.getRequiredProperty("spring.datasource.url") + schema;
        return DataSourceBuilder.create()
            .driverClassName(env.getRequiredProperty("spring.datasource.driverClassName"))
            [...]
            .url(url)
            .build();
    }
}

Now you have a `DataSource´ which acts different for every user. Once a DataSource is created it's gonna be cached for 10 minutes. That's it.

Make the application aware of our dynamic DataSource

The place to integrate our newly created DataSource is the DataSource singleton known to the spring context and used in all beans e.g. the EntityManagerFactory

So we need an equivalent to this:

@Primary
@Bean(name = "dataSource")
@ConfigurationProperties(prefix="spring.datasource")
public DataSource dataSource() {
    return DataSourceBuilder.create().build();
}

but it has to be more dynamic, than a plain property based DataSourceBuilder:

@Primary
@Bean(name = "dataSource")
public UserSchemaAwareRoutingDataSource dataSource() {
    return new UserSchemaAwareRoutingDataSource();
}

Conclusion

We have a transparent dynamic DataSource which uses the correct DataSource everytime.

Open questions

  • What to do, when no user is logged in? Is there no database access allowed?
  • Who sets up the schemes?

Disclaimer

I haven't tested this code!

EDIT: To implement a Provider<CustomUserDetails> with Spring you need to define this as prototype. You can utilize Springs support of JSR-330 and Spring Securitys SecurityContextHolder:

@Bean @Scope("prototype")
public CustomUserDetails customUserDetails() {
    return return (CustomUserDetails) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
}

You don't need a RequestInterceptor, the UserProvider or the controller code to update the user anymore.

Does this help?

EDIT2 Just for the record: do NOT reference the CustomUserDetails bean directly. Since this is a prototype, Spring will try to create a proxy for the class CustomUserDetails, which is not a good idea in our case. So just use Providers to access this bean. Or make it an interface.

Johannes Leimer
  • 1,339
  • 12
  • 23
  • If you need a schema aware jdbc url of other databases e.g. Oracle, please let me know – Johannes Leimer Sep 15 '16 at 06:25
  • Hi, thank you for your answer, I think it's the right direction. I'll try a little longer and will post my current code in a few minutes. Would be nice if you could have a look! – baao Sep 15 '16 at 10:32
  • Edited the question! – baao Sep 15 '16 at 11:23
  • Edited my answer :-) – Johannes Leimer Sep 15 '16 at 11:42
  • Improved my answer to utilize Spring prototypes more efficiently – Johannes Leimer Sep 15 '16 at 11:50
  • I've edited the question with the current code, is that ok in terms of what you mean with you second edit? – baao Sep 15 '16 at 13:07
  • Yes, but you should not define a bean inside a `@Component` annotated class. The `@Bean` annotation is meant to be used inside a `@Configuration` annotated class. – Johannes Leimer Sep 16 '16 at 18:38
  • Thank you again! I'm still learning java and spring - it's very interesting to see how much I can still learn with spring from your answer and comments! I will read up on bean definition! – baao Sep 16 '16 at 18:40
  • @JohannesLeimer could you please see this my question which is my requirement in which i am fixed. https://stackoverflow.com/questions/53297213/how-to-switch-database-on-runtime-in-springboot-and-springdatajpa-hibernate – Irfan Nasim Nov 15 '18 at 05:31
  • Would this also work for an Oracle Database? How would the jdbc url look like? – Sebastian Apr 24 '19 at 07:37
  • Hi @Sebastian, unfortunately you can't put a pre-selected schema into the jdbc url (see https://stackoverflow.com/questions/2353594/default-schema-in-oracle-connection-url). You have to find out a different way to achieve this flexibility. But be aware: Oracle normally changes the schema on a connection basis. In this answer I used a java.sql.Datasource based technique. – Johannes Leimer Apr 29 '19 at 12:03
  • @JohannesLeimer could you post git repo link ? – Irfan Nasim Jun 11 '19 at 06:44
  • @bambam could you please provide link of git repo which you have created as POC ? – Irfan Nasim Jun 12 '19 at 04:39
  • @IrfanNasim, I don't think there is a git repo available where you can just pull a POC. At least I haven't created one. All we have is written here in the question and answers. – Johannes Leimer Jun 12 '19 at 08:52
  • @JohannesLeimer thanks for this answer, it helped me a lot. – alexandrum Feb 28 '20 at 13:14
  • 1
    @alexandrum, you are very welcome :-) I'm glad my answer is actually helping the people out there – Johannes Leimer Feb 29 '20 at 17:28
  • Thanks for this answer. Im just confused on how to implement the datasource. And another question, my schema only changes once after login. Does this works? – KenobiBastila Oct 28 '20 at 13:42
  • https://stackoverflow.com/questions/64575220/stuck-on-implementing-change-of-schema-after-user-login-following-a-stackoverfl – KenobiBastila Oct 28 '20 at 14:43
  • Hi @SHASHA, the question would be, does the schema change for every user to a separate schema after login, or are they (after login) all working on the same schema? For separate schemas you need something like I described in my answer here. For the same schema, you can just use two different datasources. But this case I'd discuss in your linked question. – Johannes Leimer Oct 29 '20 at 14:09
  • @JohannesLeimer Hi! I was even googling for you lol. So basically, I have 1 schema for each company and each company has 1 to 'n' users. After log-in the spring boot application should 're-connect' to the schema that pertains to that company. – KenobiBastila Oct 29 '20 at 14:11
  • @JohannesLeimer In my Java Desktop app I just re-initialize hibernate connection after the user login. So its easy, but im not sure how to do it on spring. Please help. I greatly appreciate it. Thanks alot!!!!! – KenobiBastila Oct 29 '20 at 14:12
  • Hi @SHASHA, I'd probably do something with the routing database where `user.get().getSchema();` is just returning the company specific schema name. Then you only create datasources for every company and reuse them over all users of the company. The caching is taking care of that. – Johannes Leimer Oct 29 '20 at 14:17
  • I tried but it doesnt work. Im not sure how to "use" those data sources. would you check my topic, please? You are the only one who know anything about this and im really stuck. – KenobiBastila Oct 29 '20 at 14:18
  • This may seem as a very good way to implement it, but the problem is with creating a new `DataSource` for each schema, then you have hundreds of DB connections. And that causes a lot of stress later ( I am talking from experience) so follow another way and check this to implement another way. https://callistaenterprise.se/blogg/teknik/2020/09/19/multi-tenancy-with-spring-boot-part1/ – Renis1235 Nov 22 '21 at 07:19
  • Jep, the combination of many users, open connections and connection-pooling can render this solution useless. – Johannes Leimer Nov 22 '21 at 15:42
0

Given that you do not specify the DBMS, here is a high-level idea that may help.

(Although I am using Spring Data JDBC-ext as reference, same approach can be easily adopted by using general AOP)

Please refer to http://docs.spring.io/spring-data/jdbc/docs/current/reference/html/orcl.connection.html , Section 8.2

In Spring Data JDBC-ext, there is ConnectionPreparer that can allow you to run arbitrary SQLs when you acquire a Connection from DataSource. You can simply execute the commands to switch schema (e.g. ALTER SESSION SET CURRENT SCHEMA = 'schemaName' in Oracle, using schemaName for Sybase etc).

e.g.

package foo;

import org.springframework.data.jdbc.support.ConnectionPreparer;

import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.SQLException;

public class SwitchSchemaConnectionPreparer implements ConnectionPreparer {

    public Connection prepare(Connection conn) throws SQLException {
        String schemaName = whateverWayToGetTheScehmaToSwitch();
        CallableStatement cs = conn.prepareCall("ALTER SESSION SET CURRENT SCHEMA " + scehmaName);
        cs.execute();
        cs.close();
        return conn;
    }
}

In App Context config

<aop:config>
    <aop:advisor 
        pointcut="execution(java.sql.Connection javax.sql.DataSource.getConnection(..))" 
        advice-ref="switchSchemaInterceptor"/>
</aop:config>

<bean id="switchSchemaInterceptor" 
      class="org.springframework.data.jdbc.aop.ConnectionInterceptor">
    <property name="connectionPreparer">
        <bean class="foo.SwitchSchemaConnectionPreparer"/>
    </property>
</bean>
Adrian Shum
  • 38,812
  • 10
  • 83
  • 131
  • I think there is a problem with my and Adrians solution. We both forgot the handling of the EntityManagers JDBC connection. For instance [OpenJPA](http://openjpa.apache.org/builds/1.2.3/apache-openjpa/docs/ref_guide_dbsetup_retain.html) (I know this is an old version, but the new one doesn't load at my laptop :-() uses by default always the same connection. So `getConnection(...)` on the DataSource is only called once. – Johannes Leimer Sep 15 '16 at 09:08
  • You mean OpenJPA do another layer of connection pooling by itself? Sounds weird. In most case you will find an EM always using same connection because Datasource is doing connection pooling, so that it will always return you a pooled connection. If it is the case my way should work. Anyway, at least my way works well with Hibernate (I have used this to set up Oracle VPD) – Adrian Shum Sep 15 '16 at 15:26