I've explained in this answer how difficult it is to be truly vendor agnostic using JDBC alone. There are two layers to this:
- JDBC is a very good abstraction of the network protocol used to connect to the database. Though you'll need a few vendor-specific quirks here and there. These quirks are fine if you're binding to a single RDBMS vendor, but if you need to support multiple products, it gets very hairy
- SQL is a very good standard for a query language. Though, again, you'll need quite a few vendor-specific quirks here and there. Same as above, it's OK to support 1 RDBMS and its quirks, but very hard to support multiple.
As others have mentioned, JPA/Hibernate help you abstract over some of the differences, mostly by removing access to more "advanced" SQL features (including derived tables, unions, etc. at least as of version 5). For a more SQL centric abstraction over dialects, jOOQ is a popular option.
Disclaimer: I work for the company behind jOOQ.