Stack Overflow
Stack Overflow

Questions tagged [lwp]

The World-Wide Web library for Perl - LWP is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web.

The main focus of the library is to provide classes and functions that allow you to write WWW clients. The library also contain modules that are of more general use and even classes that help you implement simple HTTP servers.

Most modules in this library provide an object oriented API. The user agent, requests sent and responses received from the WWW server are all represented by objects. This makes a simple and powerful interface to these services. The interface is easy to extend and customize for your own needs.

The main features of the library are:

  • Contains various reusable components (modules) that can be used separately or together.
  • Provides an object oriented model of HTTP-style communication. Within this framework we currently support access to http, https, gopher, ftp, news, file, and mailto resources.
  • Provides a full object oriented interface or a very simple procedural interface.
  • Supports the basic and digest authorization schemes.
  • Supports transparent redirect handling.
  • Supports access through proxy servers.
  • Provides parser for robots.txt files and a framework for constructing robots.
  • Supports parsing of HTML forms. Implements HTTP content negotiation algorithm that can be used both in protocol modules and in server scripts (like CGI scripts).
  • Supports HTTP cookies.
  • Some simple command line clients, for instance lwp-request and lwp-download.

More information on the CPAN website.

365 questions
49
votes
8 answers

How can I get LWP to validate SSL server certificates?

How can I get LWP to verify that the certificate of the server I'm connecting to is signed by a trusted authority and issued to the correct host? As far as I can tell, it doesn't even check that the certificate claims to be for the hostname I'm…
cjm
  • 61,471
  • 9
  • 126
  • 175
33
votes
4 answers

How can I make a JSON POST request with LWP?

If you try to login at https://orbit.theplanet.com/Login.aspx?url=/Default.aspx (use any username/password combination), you can see that the login credentials are sent as a non-traditional set of POST data: just a lonesome JSON string and no normal…
Richard Simões
  • 12,401
  • 6
  • 41
  • 50
23
votes
1 answer

How can I accept gzip-compressed content using LWP::UserAgent?

I am fetching some pages over the Web using Perl's LWP::UserAgent and would like to be as polite as possible. By default, LWP::UserAgent does not seamlessly handle compressed content via gzip. Is there an easy way to make it do so, to save everyone…
Ryan Tate
  • 1,553
  • 2
  • 14
  • 21
21
votes
5 answers

Scripts broke after upgrading LWP "certificate verify failed"

I have a lot of scripts, most of them based around WWW::Mechanize that scrape data off of misc hardware that is accessible via HTTPs. After upgrading most of my perl installation and its modules, all scripts using HTTPS:// broke because of…
Jarmund
  • 3,003
  • 4
  • 22
  • 45
19
votes
2 answers

How do I send POST data with LWP?

I want to make a program that communicates with http://www.md5crack.com/crackmd5.php. My goal is to send the site a hash (md5) and hopefully the site will be able to crack it. After, I would like to display the plaintext of the hash. My problem is…
DisgruntledPerlLearner
  • 191
  • 1
  • 1
  • 3
16
votes
4 answers

How can I install LWP::Protocol::https?

I created a Perl script to run an https task. When I run it I get the error LWP::Protocol::https not installed. I cannot figure out, or find a tutorial or command on how exactly to install LWP::Protocol::http. Anyone have any idea how to install…
CRAIG
  • 977
  • 2
  • 11
  • 25
16
votes
4 answers

Installing Perl module LWP::Protocol::https

I was trying to connect to a remote host via LWP::UserAgent, but when I was trying to make it work with HTTPS I received a message that LWP::Protocol::https need to be installed. (Perl 5.10.1, CentOS 6) I've tried to install this module with…
tester3
  • 413
  • 1
  • 4
  • 12
13
votes
2 answers

Can I force LWP::UserAgent to accept an expired SSL certificate?

I would like to know whether it is possible to force LWP::UserAgent to accept an expired SSL certificate for a single, well-known server. The issue is slightly complicated by the Squid proxy in between. I went as far as to set up a debugging…
fB.
  • 358
  • 1
  • 2
  • 10
12
votes
4 answers

True timeout on LWP::UserAgent request method

I am trying to implement a request to an unreliable server. The request is a nice to have, but not 100% required for my perl script to successfully complete. The problem is that the server will occasionally deadlock (we're trying to figure out…
Philip R
  • 123
  • 1
  • 1
  • 6
12
votes
3 answers

How to POST content with an HTTP Request (Perl)

use LWP::UserAgent; use Data::Dumper; my $ua = new LWP::UserAgent; $ua->agent("AgentName/0.1 " . $ua->agent); my $req = new HTTP::Request POST => 'http://example.com'; $req->content('port=8', 'target=64'); #problem my $res =…
Takkun
  • 6,131
  • 16
  • 52
  • 69
11
votes
5 answers

Why can't I fetch wikipedia pages with LWP::Simple?

I'm trying to fetch Wikipedia pages using LWP::Simple, but they're not coming back. This code: #!/usr/bin/perl use strict; use LWP::Simple; print get("http://en.wikipedia.org/wiki/Stack_overflow"); doesn't print anything. But if I use some other…
Jesse Beder
  • 33,081
  • 21
  • 109
  • 146
11
votes
2 answers

Selecting SSL_VERIFY_NONE for SSL_verify_mode

I am trying to create a client connection to an internal ssl site that does not have a certificate and needs to bypass the proxy. I am able to bypass the proxy, and I am able to connect to the site and create a client connection, however, i am…
Rod Baldwin
  • 111
  • 1
  • 1
  • 3
10
votes
3 answers

How do I enable IPv6 support in LWP?

The following code ... my $user_agent = LWP::UserAgent->new; my $request = HTTP::Request->new(GET => $url); my $response = $user_agent->request($request); if ($response->is_success) { print "OK\n"; } else { die($response->status_line); } ..…
knorv
  • 49,059
  • 74
  • 210
  • 294
10
votes
1 answer

Installing a new ca certificate for Perl 5.14.2 LWP on Ubuntu 12.04

I am getting the following error trying to connect to a specific https website using LWP: LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at…
Ben Holness
  • 2,457
  • 3
  • 28
  • 49
10
votes
2 answers

perl save a file downloaded by lwp

Im using LWP to download an executable file type and with the response in memory, i am able to hash the file. However how can i save this file on my system? I think i'm on the wrong track with what i'm trying below. The download is successful as i…
Marcus Lim
  • 567
  • 2
  • 5
  • 14
1
2 3
…
24 25