2

I'm writing a simple script using AnyEvent::HTTP which fetches several HTML-pages in parallel. The module doesn't do any content decoding, it just provides the response body and headers to the callback.

What would be the proper way to decode the response into Perl's Unicode strings? Should I rely on the "Content-Type" header field or meta tags?

Eugene Yarmash
  • 142,882
  • 41
  • 325
  • 378

1 Answers1

1

Same modules as by LWP can be used:

use AnyEvent::HTTP qw(http_get);
use HTTP::Message;
use HTTP::Headers;
use HTTP::Response;
my $can_accept = HTTP::Message::decodable;
http_get(
    "https://apa.at/",
    headers => { 'Accept-Encoding' => $can_accept },
    sub {
        my ( $body, $hdr ) = @_;
        delete( $hdr->{URL} );
        my $status  = delete( $hdr->{Status} );
        my $reason  = delete( $hdr->{Reason} );
        my $headers = HTTP::Headers->new(%$hdr);
        my $resp = HTTP::Response->new( $status, $reason, $headers, $body );

        print $resp->decoded_content . "\n";
    }
);

(inspired by How can I accept gzip-compressed content using LWP::UserAgent?)

Jozef
  • 111
  • 7