5

I am in the progress of making a mobile App for a website to view your schedule. They don't provide any API and has no intention to make one.

The website can only function with Ajax, however to fake these requests and scrape the website I need to fake the __EVENTVALIDATION post field.

I have no control whatsoever over the website and I have never built anything using ASP.NET or Microsoft Ajax.

Has anyone done this?

I have found that the __EVENTVALIDATION field has this pattern (... symbolises bytes changed depending on the request, hexdump of the base64 decoded version):

d8 01 16 13 02 4f 0a
...
f6 e0 84 d4 05 02 a0 3f
e2 3f 03 02 3f d8 d1 d5 0c 02 bb 82 cf ec 08 02
b4 b5 99 f8 0b 02 3f 89 3f eb 04 02 d5 83 90 88
0a 02 8a db 94 90 03 02 8b cf 3f 85 08 02 93 3f
b1 3f 06 02 9b 3f 8f a5 02 02 b5 b4 af 85 01 02
d1 fc ae 9c 0e 02 b4 e2 94 9e 0a 02 3f e2 94 9e
0a 02 3f e2 94 9e 0a 02 bb 92 80 a5 06
...
Tyilo
  • 28,998
  • 40
  • 113
  • 198

2 Answers2

10

I've dealt with this problem before in building scrapers for ASP.NET sites. You need to request the initial page that the browser user would ordinarily land on, extract the __VIEWSTATE and __EVENTVALIDATION hashes then use these in making the second request for the data which you actually need.

For example, if you're scraping the response from a form submission:

  1. make an AJAX request for the page that the form is on
  2. extract the viewstate and event validation hashes from the response
  3. make a new AJAX request that simulates form submission, passing the hashes as parameters

If you're looking for JavaScript functions to extract the hashes from markup, I've published the ones I use as ms-viewstate on GitHub.

Matthew
  • 15,464
  • 2
  • 37
  • 31
  • I remember it didn't work with just using the `__VIEWSTATE` and `__EVENTVALIDATION`, that the site initially provided. The site has (fortunately) changed its design, so I don't need this anymore, however I can't test it. – Tyilo Mar 13 '13 at 18:28
  • Ok. Just posting this here in case anyone has the same problem: the site might be setting an `ASP.NET_SessionId` cookie. You'll need to be on the same domain for that to be passed along with the AJAX request. The browser's security policy won't allow you to read the `Set-Cookie` header and manually pass the cookie along with any subsequent request. – Matthew Mar 13 '13 at 20:07
  • I was actually not using AJAX, as it was a crawler running on client. Thanks anyway :) – Tyilo Mar 13 '13 at 20:20
  • THANK YOU! I'd been using PhantomJS to crawl one of these sites and was about to do it again on another similar site until I found this advice. Saved me hours. – Jeff Allen May 06 '15 at 03:59
1

__EVENTVALIDATION is a security measure.

The feature prevents unauthorized requests sent by potentially malicious users from the client. To ensure that each and every postback and callback event originates from the expected user interface elements, the page adds an extra layer of validation on events. The page basically matches the contents of the request with the information in the __EVENTVALIDATION field to verify that no extra input field has been added on the client and that value is selected on a list that was already known on the server. The page generates the event validation field during rendering-that is at the last possible moment when the information is available. Like the view state, the event validation field contains a hash value to prevent client-side tampering.

The hash value is based on a key at the server level. So you cannot replicate that hash - or rather, if you did, without access to the server, I guess you found a security hole.

REF: MSDN

EdSF
  • 11,753
  • 6
  • 42
  • 83
  • How can the browser generate it then? – Tyilo Oct 30 '12 at 22:45
  • @Tyilo The browser isn't generating it. The server is, and renders it to the ASP.Net Page. – EdSF Oct 30 '12 at 23:57
  • @Tyilo Given that I have the source of the page and it's scripts before submitting the page, can I get the it then? – Tyilo Oct 31 '12 at 06:34
  • @Tyilo **"To ensure that each and every postback and callback event originates from the _expected user interface elements_.."** - this statement pretty much says you can't "spoof" Requests - events (that trigger postback/callback) are expected to come from interface elements.... – EdSF Oct 31 '12 at 17:23
  • @Tyilo already covered in past comments. ASP.Net Page generates all the elements (server-side) so it "knows" all the (user interface) elements in the page, **"To ensure that each and every postback and callback event originates from the expected user interface elements....Event validation may pose issues if used in the context of AJAX-enabled applications. In such applications, some client work can create new input elements on the fly, thus making the next postback _fail_ because of _unknown elements_"**. – EdSF Oct 31 '12 at 23:56