The SNI extension will add the server_name(hostname) in the client hello message at the SSL handshake stage.
It may be needed by a server that serves requests for different hostnames or virtual hosts. It is used to resolve the Certificate for the hostname the client intend to communicate with. The server may be configured with a default certificate if the host name is not added or not found. The default certificate may be a multisigned certificate for all the hosts it serves.
The first warning you get may be a server responding with a default certificate but that also warns that the hostname is not recognized. The server probably does not recognize the hostname simply because it is not included in the configuration. Instead of ignoring the warning and use the default certificate, Java seems to throw an exception.
In the other case where you disabled SNI, you do not include the hostname anymore and the request is sent to a server that does not have a default certificate. It will respond with a fatal error because it cannot resolve any certificate for you.