0

I send so much request to http://www.finansgundem.com/ for get news.I am thinking That's why They blocks me. I am using phantomjs as Browser. How can i fix this problem?

This is Broser Codes

 var driverService = PhantomJSDriverService.CreateDefaultService();
 driverService.HideCommandPromptWindow = true;                 
 driverService.IgnoreSslErrors = true;
 driverService.LocalToRemoteUrlAccess = true;

 Driver = new PhantomJSDriver(driverService);

and Request...

Driver.Navigate().GoToUrl(url);
var source=Driver.PageSource;

This is response of the requests...

<!DOCTYPE html PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
  <head>
     <title>403 Forbidden</title>
  </head>
  <body>
     <h1>Forbidden</h1>
     <p>You don't have permission to access /gundem/esad_i-soke-edecek-saldiri-705823.htm
        on this server.
     </p>
     <hr>
     <address>Apache/2.2.22 (Win32) mod_evasive2/1.10.1-win mod_jk/1.2.37 Server at www.finansgundem.com Port 80
     </address>
  </body>
</html>
Amit Kumar Ghosh
  • 3,618
  • 1
  • 20
  • 24
Must.Tek
  • 1,220
  • 10
  • 17
  • there is nothing you can do if you send too much requests, but you could set up a server which requests the news once every few minutes and your app is getting the news from your server. – Sebastian L Sep 03 '15 at 07:40
  • I would check about their terms of service. If you are violating them then there is no way to get help here to override their TOS – Steve Sep 03 '15 at 07:41
  • Take a look [here](http://stackoverflow.com/questions/19715970/remote-server-403-forbidden-error-while-using-webclient/19716283). You can either set a timeout between requests, or set a different user agent so that the server thinks you as a regular browser. – Tasos K. Sep 03 '15 at 07:54
  • The code for too many requests is 429, not 403. If you get 403 consistently and assuming they don't return the wrong code intentionally, you should call their support. Either they refuse anonymous access or they've explicitly denied access to your credentials. They may have simply blacklisted your IP by hand, in which case no rate limiting code will work. – Panagiotis Kanavos Sep 03 '15 at 08:00
  • @TasosK. yes this is a good idea. Actually i thougth this but i cant add header in phantomjs. There is a class that named PhantomJSOptions maybe i have to use this. – Must.Tek Sep 03 '15 at 08:07
  • @PanagiotisKanavos I have seen an IIS web server returning 403 when I was sending simultaneous requests and for a few seconds all requests had a 403 response. After waiting for about 20 seconds the 403 error stopped. – Tasos K. Sep 03 '15 at 10:49
  • @TasosK. as I said, not everyone implements the proper status codes, or wants to. 403 may be returned by the application, not IIS. Then again there may be a filter that bans IPs that exceed certain levels. High traffic IIS sites definitely *don't* return 403 when under load. – Panagiotis Kanavos Sep 03 '15 at 10:56

0 Answers0