IdHTTP.get returning HTTP1.1/ 403 Forbidden

Question

I'm trying to access an update.txt file on my website, using a DelphiXE compiled program and the IdHTTP component.

The code I'm using is as follows:

procedure TFormAbout.SpeedButtonUpdateClick(Sender: TObject);

function CheckUpdates: String;
var lIdHttp: TIdHTTP;
begin
  lIdHttp := TIdHTTP.Create(nil);
  result := lIdHttp.Get('http://www.test.com/test_down_load/update.txt');
end;

var
sWebVersion: String;
sVersionList: TStringList;

begin
try
  sWebVersion := Checkupdates;
except
  on E: Exception do
  begin 
    ShowMEssage(E.ErrorMessage);
    MessageDlg('An Error occured checking for an update.',mtError,[mbOK],0);
  end;
end;
if sWebVersion <> '' then
  begin
    sVersionList.CommaText := sWebVersion;
    ShowMessage('Version: ' + sVersionList[0] + ' - ' + 'Date: ' + sVersionList[1]);
  end;
end;

This however results in error: HTTP1.1/ 403 Forbidden

The IdHTTP component has been set up with the following properties.

HandleRedirects := true;
HTTPOptions [hoForceEncodeParams];
ProtocolVersion := pv1_1;
Request.UserAgent := Mozilla/5.0 (compatible; Test)

If I enter the URL in an IE browser, it returns the file without errors, but when accessing from my program, I get the error. Any pointers will be appreciated. .htaccess is correct for the site. Permissions for the file are correct on the website: 0644.

Do I have to set any of the other properties for the IdHTTP component. I only have this component on the about form. Do I need anything else.

The updateinfo.txt file simply contains the text in quotes: "18.3.5,2011/12/17"

I have simply used "test" here in place of my actual program name and URL.

Regards Adrian

score 10 · Answer 1 · edited May 23 '17 at 12:09

I run into the exact same problem when using Indy's Get() function.

It more than likely that you are getting this error because you haven't set the UserAgent property and the website knowing that you are not accessing the file as a Browser is kicking up a fuss.

function CheckUpdates: String;
var lIdHttp: TIdHTTP;
begin
  lIdHttp := TIdHTTP.Create(nil);
  //avoid getting '403 Forbidden' by setting UserAgent
  lIdHttp.Request.UserAgent :=
      'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0';
  result := lIdHttp.Get('http://www.test.com/test_down_load/update.txt');
end;

Similar question, but correct answer was logged here: https://stackoverflow.com/questions/10870730/why-do-i-get-403-forbidden-when-i-connect-to-whatismyip-com

score 6 · Accepted Answer · answered Dec 17 '11 at 17:18

6

403 means you do not have permission to access the requested URL. The server likely requires you to provide a username/password, especially since you are using an .htaccess file. Use the Request.UserName and Request.Password properties for that. As for why the browser does not ask for username/password, my guess is the browser has them cached from an earlier access.

BTW, your SpeedButtonUpdateClick() has a memory leak. You are creating a new TIdHTTP object but you are not freeing it.

answered Dec 17 '11 at 17:18

Remy Lebeau

555,201
31
458
770

Hello Remy, I have no username or password set up for the site. I can access the url from a webbrowser on another PC, that doesn't belong to me. Thx for the memory leak .. just sample code, but you are correct, it needs to be freed. – Adrian Wreyford Dec 17 '11 at 17:25
5

Then use a packet sniffer, such as Wireshark, to see what the HTTP requests look like from your browser and TIdHTTP, and then compare them for differences. Offhand, maybe your server does not like the value of the `Request.UserAgent` that you are using. Some servers are UserAgent-sensitive. – Remy Lebeau Dec 17 '11 at 17:39
1

I was worried about the Request.UserAgent too, and tried many variations, without any success. – Adrian Wreyford Dec 17 '11 at 17:42
6

Hello Remy, Actually the problem lies with me. As you can see in the code, I'm creating TIdHTTP, but was not changing its Request.UserAgent, but the IdHttp1 component I had dropped on the form, thus the default Mozilla/3.0 (compatible; Indy Library) was being sent, and not what I thought I was sending all along. So all the changes I was making to the component.properties and testing were never going through. Adding this line of code did it after the create of the lIdHttp: lIdHttp.request.useragent := 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; MAAU)'; Thx Adrian – Adrian Wreyford Dec 17 '11 at 18:18
@Remy Lebeau - TeamB, 403 is forbidden, 401 is for credential challenge. – OnTheFly Dec 18 '11 at 01:25
401 is used to ask for credentials in a particular format specified by the "WWW-Authentication" header(s). 403 is used when the request is rejected, such as if suitable credentials were specified but not valid for the requested resource. – Remy Lebeau Dec 18 '11 at 01:50
When using Indy's Get() function - if you set the UserAgent property, as shown in follow up answer, the request will have the correct credentials and not get a 403 error. – TheSteven Apr 07 '14 at 22:50
Changing the `Request.UserAgent` does not affect how Indy sends credentials in any way, it only affects how *some* HTTP servers process requests. – Remy Lebeau Apr 08 '14 at 02:21
1

FYI, I just a few minutes ago checked in an update where the default `TIdHTTP.Request.UserAgent` is now customizable. There is a new `GIdDefaultUserAgent` global variable in the `IdHTTPHeaderInfo.pas` unit. Instead of setting `TIdHTTP.Request.UserAgent` separately for every `TIdHTTP` object used, you can now set `GIdDefaultUserAgent` once at program startup and every `TIdHTTP` request will use it (you can still use `TIdHTTP.Request.UserAgent` to override the `UserAgent` on a per-request basis, if needed). – Remy Lebeau Apr 08 '14 at 02:23

score -1 · Answer 3 · edited Jun 30 '15 at 13:54

The answers given did not work for me until i put all of them together.

//add Request.Username and supply the correct mysql username
tidHttpObject.Request.Username := 'username';

//do the same for the password
tidHttpObject.Request.Password := 'password';

//then add a UserAgent property with the string below
tidHttpObject.Request.UserAgent :=  'Mozilla/5.0 (Windows NT 6.1; WOW64;
rv:12.0) Gecko/20100101 Firefox/12.0';

//finally call the get() url method of the tidHttp object
Result :=  tidHttpObject.Get(url);

IdHTTP.get returning HTTP1.1/ 403 Forbidden

3 Answers3

Linked