0

I a trying to web scraping from http://hydro.imd.gov.in/hydrometweb/(S(e4xftrnoogdzzz55q2l23i55))/Argaws.aspx. I am novice web-scraper in web pages with ViewState-generator and event validation. So far I have tried with the following code and it is returning the same page instead of redirecting to new page. Any help regarding it will be highly appreciated.

            string link = "http://hydro.imd.gov.in/hydrometweb/(S(e4xftrnoogdzzz55q2l23i55))/Argaws.aspx";
            WebClient client = new WebClient();
            string intiHtml = client.DownloadString(link);

            string viewStateNameDelimiter = "__VIEWSTATE";
            string valueDelimiter = "value=\"";
            int viewStateNamePosition = intiHtml.IndexOf(viewStateNameDelimiter);
            int viewStateValuePosition = intiHtml.IndexOf(valueDelimiter, viewStateNamePosition);
            int viewStateStartPosition = viewStateValuePosition + valueDelimiter.Length;
            int viewStateEndPosition = intiHtml.IndexOf("\"", viewStateStartPosition);
            string viewstatePos = intiHtml.Substring(viewStateStartPosition, viewStateEndPosition - viewStateStartPosition);

            Console.WriteLine(viewstatePos);
            Console.ReadKey();

            string eventValidationNameDelimiter = "__EVENTVALIDATION";
            int eventValidationNamePosition = intiHtml.IndexOf(eventValidationNameDelimiter);
            int eventValidationValuePosition = intiHtml.IndexOf(valueDelimiter, eventValidationNamePosition);
            int eventValidationStartPosition = eventValidationValuePosition + valueDelimiter.Length;
            int eventValidationEndPosition = intiHtml.IndexOf("\"", eventValidationStartPosition);
            string eventvalidate = intiHtml.Substring(eventValidationStartPosition, eventValidationEndPosition - eventValidationStartPosition);

            Console.WriteLine(eventvalidate);
            Console.ReadKey();

            string viewgeneratorNameDelimiter = "__VIEWSTATEGENERATOR";
            int viewgeneratorNamePosition = intiHtml.IndexOf(viewgeneratorNameDelimiter);
            int viewgeneratorValuePosition = intiHtml.IndexOf(valueDelimiter, viewgeneratorNamePosition);
            int viewgeneratorStartPosition = viewgeneratorValuePosition + valueDelimiter.Length;
            int viewgeneratorEndPosition = intiHtml.IndexOf("\"", viewgeneratorStartPosition);
            string viewgenerator = intiHtml.Substring(viewgeneratorStartPosition, viewgeneratorEndPosition - viewgeneratorStartPosition);
            Console.WriteLine(viewgenerator);
            Console.ReadKey();

            string postData = viewStateNameDelimiter + "=" + viewstatePos + "&" + viewgeneratorNameDelimiter + "=" + viewgenerator + "&" + eventValidationNameDelimiter + "=" + eventvalidate + "&DropDownList2=17-01-2018&DropDownList1=07&GoBtn=Go";
            Console.WriteLine(postData);

            WebRequest req = WebRequest.Create(link);

            //string postData = "";
            //postData = "";

            byte[] send = Encoding.Default.GetBytes(postData);
            req.Method = "POST";
            req.ContentType = "application/x-www-form-urlencoded";
            req.ContentLength = send.Length;

            Stream sout = req.GetRequestStream();
            sout.Write(send, 0, send.Length);
            sout.Flush();
            sout.Close();

            WebResponse res = req.GetResponse();
            StreamReader sr = new StreamReader(res.GetResponseStream());
            string returnvalue = sr.ReadToEnd();
            Console.Write(returnvalue);
            Console.ReadKey();
DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
nishanuw
  • 49
  • 1
  • 11
  • Your request is missing very important information such as `cookies`, `referer` ... which might be checked with the mentioned website to allow you navigate deeper in their website. – AaA Jan 18 '18 at 04:15
  • Thanks for your reply. Will you be a little more specific how I can incorporate those parameters in my code to navigate that website? Thanks in advance. – nishanuw Jan 18 '18 at 19:54
  • you can refer to [this](https://stackoverflow.com/questions/31129381/) and [this](https://stackoverflow.com/questions/17022408/). however to successfully scraping the website, you need to study headers sent by browser to website and clone them exactly in your application, so the website feels like a user is browsing it. – AaA Jan 19 '18 at 02:40
  • Still, I couldn't get it. I saw both of the examples and not fully clear what you wanted to say. However thanks for your effort. – nishanuw Jan 20 '18 at 00:17

0 Answers0