As several other answers suggest you should create unit tests for testing small pieces of code with mocking all external dependencies.
However sometimes ( a lot of times) it should worth to test whole features. Especially when you use some kind of framework like Spring. Or you use a lot of annotations. When your classes or methods have annotations on them the effects of those annotations usually cannot be tested via unit-tests. You need the whole framework running during the test to make sure it works as expected.
In our current project we have almost as much integration tests as unit tests. We use the H2 in-memory DB for these tests, this way we can avoid failures because of connectivity problems, and Spring's test package could collect and run multiple integration tests into the same test-context, so it has to build the context only once for multiple tests and this way running these tests are not too expensive.
Also you can create separate test context for different part of the project (with different settings and DB content), so this way the tests running under different context won't interfere with each-other.
Do not afraid of using a lot of integration tests. You need some anyway, and if you already have a test-context it's not a big deal adding some more tests into the same context.
Also there are a lot of cases which would take a LOT of effort to cover with unit-tests (or cannot be covered fully at all) but can be covered simply by an integration tests.
A personal experience:
Our numerous integration tests were extremely useful when we switched from Spring Boot to Spring Boot 2.
Back to the original question:
Unit tests should not connect to real DB, but feel free to use more integration tests. (with in-memory DB)