1

I am not exactly sure if this is the right question, but I'm running a website on Windows Azure. On that site I am displaying data I collect via data scraping, which at the moment I'm using a single source. My goal is to have a worker thread operate off of my website that periodically goes out re-scrapes and updates the database. What's holding me back at the moment is that my server app (the worker thread) is making 500+ requests to another server with all the data. Is it possible for me to setup something to make the requests go through a proxy or something so that I don't get denied access for making to many requests from a single ip?

czifro
  • 784
  • 7
  • 24

1 Answers1

1

You mean like this? How to use CURL via a proxy?

(Plenty of free proxy lists online to use(paid or free, quality not guaranteed on free)

Community
  • 1
  • 1
g33k84
  • 138
  • 9
  • 1
    So i would do something like this in c#: [Download Files using HTTP and Proxy Server in C#](http://www.c-sharpcorner.com/UploadFile/mahesh/DownloadUsingHTTP12132005012752AM/DownloadUsingHTTP.aspx) – czifro Jul 23 '14 at 15:13