Spring boot fails to start with Hibernate when there are lots of objects in DB

Question

My Spring boot version is 2.0.3

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.0.3.RELEASE</version>
    <relativePath /> <!-- lookup parent from repository -->
</parent>

When there are thousands of tables, indexes, sequences in the DB, the application bootstrapping takes hours.

After some investigating, I find it's trying to retrieve database schema information when creating the EntityManagerFactory. Here are the 3 different callstacks during the initialization.

            SequenceInformationExtractorLegacyImpl.extractMetadata(ExtractionContext) line: 46  
        DatabaseInformationImpl.initializeSequences() line: 65  
        DatabaseInformationImpl.<init>(ServiceRegistry, JdbcEnvironment, DdlTransactionIsolator, Namespace$Name) line: 59   
        Helper.buildDatabaseInformation(ServiceRegistry, DdlTransactionIsolator, Namespace$Name) line: 132  
        GroupedSchemaMigratorImpl(AbstractSchemaMigrator).doMigration(Metadata, ExecutionOptions, TargetDescriptor) line: 96    
        SchemaManagementToolCoordinator.performDatabaseAction(Action, Metadata, SchemaManagementTool, ServiceRegistry, ExecutionOptions) line: 183  
        SchemaManagementToolCoordinator.process(Metadata, ServiceRegistry, Map, DelayedDropRegistry) line: 72   
        SessionFactoryImpl.<init>(MetadataImplementor, SessionFactoryOptions) line: 312 
        SessionFactoryBuilderImpl.build() line: 462 
        EntityManagerFactoryBuilderImpl.build() line: 892   
        SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(PersistenceUnitInfo, Map) line: 57    
        LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory() line: 365 
        LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).buildNativeEntityManagerFactory() line: 390    
        LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).afterPropertiesSet() line: 377 
        LocalContainerEntityManagerFactoryBean.afterPropertiesSet() line: 341   
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).invokeInitMethods(String, Object, RootBeanDefinition) line: 1767 
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).initializeBean(String, Object, RootBeanDefinition) line: 1704    
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).doCreateBean(String, RootBeanDefinition, Object[]) line: 581 
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).createBean(String, RootBeanDefinition, Object[]) line: 503   
        DefaultListableBeanFactory(AbstractBeanFactory).lambda$doGetBean$0(String, RootBeanDefinition, Object[]) line: 317  
        214649627.getObject() line: not available   
        DefaultListableBeanFactory(DefaultSingletonBeanRegistry).getSingleton(String, ObjectFactory<?>) line: 222   
        DefaultListableBeanFactory(AbstractBeanFactory).doGetBean(String, Class<T>, Object[], boolean) line: 315    
        DefaultListableBeanFactory(AbstractBeanFactory).getBean(String) line: 199   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).getBean(String) line: 1089   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).finishBeanFactoryInitialization(ConfigurableListableBeanFactory) line: 859   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).refresh() line: 550  
        AnnotationConfigServletWebServerApplicationContext(ServletWebServerApplicationContext).refresh() line: 140  
        SpringApplication.refresh(ApplicationContext) line: 759 
        SpringApplication.refreshContext(ConfigurableApplicationContext) line: 395  
        SpringApplication.run(String...) line: 327  


        InformationExtractorJdbcDatabaseMetaDataImpl.processTableResults(ResultSet) line: 393   
        InformationExtractorJdbcDatabaseMetaDataImpl.getTables(Identifier, Identifier) line: 336    
        DatabaseInformationImpl.getTablesInformation(Namespace) line: 120   
        GroupedSchemaMigratorImpl.performTablesMigration(Metadata, DatabaseInformation, ExecutionOptions, Dialect, Formatter, Set<String>, boolean, boolean, Set<Identifier>, Namespace, GenerationTarget[]) line: 65   
        GroupedSchemaMigratorImpl(AbstractSchemaMigrator).performMigration(Metadata, DatabaseInformation, ExecutionOptions, Dialect, GenerationTarget...) line: 207 
        GroupedSchemaMigratorImpl(AbstractSchemaMigrator).doMigration(Metadata, ExecutionOptions, TargetDescriptor) line: 114   
        SchemaManagementToolCoordinator.performDatabaseAction(Action, Metadata, SchemaManagementTool, ServiceRegistry, ExecutionOptions) line: 183  
        SchemaManagementToolCoordinator.process(Metadata, ServiceRegistry, Map, DelayedDropRegistry) line: 72   
        SessionFactoryImpl.<init>(MetadataImplementor, SessionFactoryOptions) line: 312 
        SessionFactoryBuilderImpl.build() line: 462 
        EntityManagerFactoryBuilderImpl.build() line: 892   
        SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(PersistenceUnitInfo, Map) line: 57    
        LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory() line: 365 
        LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).buildNativeEntityManagerFactory() line: 390    
        LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).afterPropertiesSet() line: 377 
        LocalContainerEntityManagerFactoryBean.afterPropertiesSet() line: 341   
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).invokeInitMethods(String, Object, RootBeanDefinition) line: 1767 
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).initializeBean(String, Object, RootBeanDefinition) line: 1704    
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).doCreateBean(String, RootBeanDefinition, Object[]) line: 581 
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).createBean(String, RootBeanDefinition, Object[]) line: 503   
        DefaultListableBeanFactory(AbstractBeanFactory).lambda$doGetBean$0(String, RootBeanDefinition, Object[]) line: 317  
        1800649922.getObject() line: not available  
        DefaultListableBeanFactory(DefaultSingletonBeanRegistry).getSingleton(String, ObjectFactory<?>) line: 222   
        DefaultListableBeanFactory(AbstractBeanFactory).doGetBean(String, Class<T>, Object[], boolean) line: 315    
        DefaultListableBeanFactory(AbstractBeanFactory).getBean(String) line: 199   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).getBean(String) line: 1089   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).finishBeanFactoryInitialization(ConfigurableListableBeanFactory) line: 859   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).refresh() line: 550  
        AnnotationConfigServletWebServerApplicationContext(ServletWebServerApplicationContext).refresh() line: 140  
        SpringApplication.refresh(ApplicationContext) line: 759 
        SpringApplication.refreshContext(ConfigurableApplicationContext) line: 395

        TableInformationImpl.indexes() line: 122    
        TableInformationImpl.getIndex(Identifier) line: 138 
        GroupedSchemaMigratorImpl(AbstractSchemaMigrator).applyUniqueKeys(Table, TableInformation, Dialect, Metadata, Formatter, ExecutionOptions, GenerationTarget...) line: 370   
        GroupedSchemaMigratorImpl.performTablesMigration(Metadata, DatabaseInformation, ExecutionOptions, Dialect, Formatter, Set<String>, boolean, boolean, Set<Identifier>, Namespace, GenerationTarget[]) line: 85   
        GroupedSchemaMigratorImpl(AbstractSchemaMigrator).performMigration(Metadata, DatabaseInformation, ExecutionOptions, Dialect, GenerationTarget...) line: 207 
        GroupedSchemaMigratorImpl(AbstractSchemaMigrator).doMigration(Metadata, ExecutionOptions, TargetDescriptor) line: 114   
        SchemaManagementToolCoordinator.performDatabaseAction(Action, Metadata, SchemaManagementTool, ServiceRegistry, ExecutionOptions) line: 183  
        SchemaManagementToolCoordinator.process(Metadata, ServiceRegistry, Map, DelayedDropRegistry) line: 72   
        SessionFactoryImpl.<init>(MetadataImplementor, SessionFactoryOptions) line: 312 
        SessionFactoryBuilderImpl.build() line: 462 
        EntityManagerFactoryBuilderImpl.build() line: 892   
        SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(PersistenceUnitInfo, Map) line: 57    
        LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory() line: 365 
        LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).buildNativeEntityManagerFactory() line: 390    
        LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).afterPropertiesSet() line: 377 
        LocalContainerEntityManagerFactoryBean.afterPropertiesSet() line: 341   
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).invokeInitMethods(String, Object, RootBeanDefinition) line: 1767 
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).initializeBean(String, Object, RootBeanDefinition) line: 1704    
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).doCreateBean(String, RootBeanDefinition, Object[]) line: 581 
        DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).createBean(String, RootBeanDefinition, Object[]) line: 503   
        DefaultListableBeanFactory(AbstractBeanFactory).lambda$doGetBean$0(String, RootBeanDefinition, Object[]) line: 317  
        473666452.getObject() line: not available   
        DefaultListableBeanFactory(DefaultSingletonBeanRegistry).getSingleton(String, ObjectFactory<?>) line: 222   
        DefaultListableBeanFactory(AbstractBeanFactory).doGetBean(String, Class<T>, Object[], boolean) line: 315    
        DefaultListableBeanFactory(AbstractBeanFactory).getBean(String) line: 199   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).getBean(String) line: 1089   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).finishBeanFactoryInitialization(ConfigurableListableBeanFactory) line: 859   
        AnnotationConfigServletWebServerApplicationContext(AbstractApplicationContext).refresh() line: 550  
        AnnotationConfigServletWebServerApplicationContext(ServletWebServerApplicationContext).refresh() line: 140  
        SpringApplication.refresh(ApplicationContext) line: 759 
        SpringApplication.refreshContext(ConfigurableApplicationContext) line: 395  
        SpringApplication.run(String...) line: 327

It's fetching the data from DB, unfortunately no fetchSize is specified on ResultSet or Statement. So, when there are thousands of DB objects, such kind of operations will be very time consuming. By default each time only fetching 10 records. The "jdbc.fetch_size" setting doesn't apply to these code logics.

        try {
        final ResultSet resultSet = statement.executeQuery( lookupSql );
        try {
            final List<SequenceInformation> sequenceInformationList = new ArrayList<SequenceInformation>();
            while ( resultSet.next() ) {
                sequenceInformationList.add(
                        new SequenceInformationImpl(
                                new QualifiedSequenceName(
                                        null,
                                        null,
                                        identifierHelper.toIdentifier(
                                                resultSet.getString( 1 )
                                        )
                                ),
                                -1
                        )
                );
            }
            return sequenceInformationList;
        }

Is there any suggestion to mitigate this performance issue? Or could it be improved in the future Hibernate versions?

Have you looked at the hibernate.temp.use_jdbc_metadata_defaults property? I'm wondering if that could be used to just switch off this loading behaviour rather than speed it up. See https://stackoverflow.com/questions/14445838/hibernate-startup-very-slow — Ryan Dawson, Aug 03 '18 at 08:59
Thank you, Ryan. However it seems this setting doesn't work, the above code logic will still be triggered. — Alex Lv, Aug 07 '18 at 01:16
Might be worth also trying hibernate official forums or raising an issue as I don't know whether the hibernate team monitors SO — Ryan Dawson, Aug 07 '18 at 06:42

score 0 · Accepted Answer · answered Aug 07 '18 at 07:33

0

Currently it seems the only workaround is to disable ddl-auto. Comment the below setting can mitigate the issue

#spring.jpa.hibernate.ddl-auto=update

answered Aug 07 '18 at 07:33

Alex Lv

11
2

This is correct. It is also mentioned in the docs of spring boot: https://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html if someone needs further information. But also be aware that this will switch on the default behaviour of spring boot which varies between embedded databases and not embedded ones. – the hand of NOD Aug 07 '18 at 07:51

Spring boot fails to start with Hibernate when there are lots of objects in DB

1 Answers1