LWP::UserAgent Insists on verifying hostname

Question

The following script works returns a comprehensive headers on a host running libwww-perl-5.836 but not on the host using libwww-perl-6.30.0. In that case, the script displays the following:

    500 Can't connect to backend.mutegroup.org:443 (certificate verify failed)
    Content-Type: text/plain
    Client-Date: Mon, 28 Jul 2014 21:09:28 GMT
    Client-Warning: Internal response

    Can't connect to backend.mutegroup.org:443 (certificate verify failed)

    LWP::Protocol::https::Socket: SSL connect attempt failed with unknown error error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/lib64/perl5/vendor_perl/5.16.3/LWP/Protocol/http.pm line 51.

This is the script:

#!/usr/bin/perl

use strict;
use warnings;

use LWP::UserAgent;
use HTTP::Request::Common;

my $ua      = LWP::UserAgent->new(ssl_opts => { verify_hostname => 0, } );
my $url     = 'https://backend.mutegroup.org/api/getLastId';

my $request = POST $url;
print $ua->request($request)->as_string

Host names are not verified by default for libwww-perl-5.837 and prior. That explains why it works on the old host. However, I am explicitly disabling checks and it is still insisting on doing them.

This is on a Gentoo system.

This might help http://blogs.perl.org/users/brian_d_foy/2011/07/now-you-need-lwpprotocolhttps.html — oalders, Jul 28 '14 at 19:53
@oalders, that explains why the old hosts is working. Thanks. I've updated the question based on this information. However, I am explicitly turning off hostname checking and I still have the problem. — Stephen Rasku, Jul 28 '14 at 20:39

score 17 · Accepted Answer · edited Jul 29 '14 at 03:27

17

You have turned off verifying the hostname; you have not turned off verifying the certificate.

verify_hostname controls whether "LWP will for secure protocol schemes ensure it connects to servers that have a valid certificate matching the expected hostname" (my emphasis). Setting that to 0 allows you to connect to a server that has a valid certificate but not issued for the host / hostname that you are trying to reach.

To turn off checking that the certificate is valid (issued by a trusted CA), you want:

use IO::Socket::SSL;

my $ua = LWP::UserAgent->new(
   ssl_opts => {
      verify_hostname => 0,
      SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE,
   },
);

Note that turning off either of these options is a bad idea if you are transmitting any sensitive information or expect to trust the data returned. With either of these turned off, you are losing the benefits of SSL and are vulnerable to various man-in-the-middle attacks.

edited Jul 29 '14 at 03:27

ikegami

367,544
15
269
518

answered Jul 28 '14 at 22:16

Colin Phipps

908
5
8

I was in the process of submitting my own answer while you posted this. Yours is more comprehensive so I will delete mine and accept yours. Can you add `use IO::Socket::SSL;` somewhere in your answer (for completeness)? – Stephen Rasku Jul 28 '14 at 22:27
I've noticed that the equivalent configuration need to also be set as environment variables, e.g.: $ENV{'PERL_LWP_SSL_VERIFY_HOSTNAME'} = 0; – mike622867 Jun 29 '16 at 21:29
I have an old system with perl 5.14.2, where a WebInject monitoring task suddenly failed. Setting verify_hostname = 0 helped as work around until we can update the system. Thanks. – mvw May 18 '18 at 10:47
We installed cloudflare cert's on our backend machines, and couldn't use our perl scripts to activate URL end points, until we added the SSL_ca_file option to ssl_opts: – Dave Smith Sep 07 '22 at 20:12

score 0 · Answer 2 · answered Sep 07 '22 at 20:15

We installed cloudflare cert's on our backend machines, and couldn't use our perl scripts to activate URL end points, until we added the SSL_ca_file option to ssl_opts:

my $ua = LWP::UserAgent->new;
$ua->ssl_opts(
    SSL_ca_file => '/etc/ssl/certs/cloudflare-x.pem'
);

LWP::UserAgent Insists on verifying hostname

2 Answers2

Linked