0

I am requesting an Ajax Web site with a Python script and fetching cities and branch offices of http://www.yurticikargo.com/bilgi-servisleri/Sayfalar/en-yakin-sube.aspx

I completed the first step with posting {cityID: 34} to this url and fetc the JSON output.

http://www.yurticikargo.com/_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-sswservices.aspx/GetTownByCity

But I can not retrive the JSON output with Python although i get succesfully with Chrome Advanced Rest Client Extension, posting {cityID:54,townID:5416,unitOnDutyFlag:null,closestFlag:2}

http://www.yurticikargo.com/_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-unitservices.aspx/GetUnit

All of the source code is here

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import requests
import json


class Yurtici(object):

    baseUrl = 'http://www.yurticikargo.com/'
    ajaxRoot = '_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-sswservices.aspx/'
    getTown = 'GetTownByCity'
    getUnit = 'GetUnit'
    urlGetTown = baseUrl + ajaxRoot + getTown
    urlGetUnit = baseUrl + ajaxRoot + getUnit 
    headers = {'content-type': 'application/json','encoding':'utf-8'}

    def __init__(self):
        pass

    def ilceler(self, plaka=34): # Default testing value
        payload = {'cityId':plaka}
        url = self.urlGetTown
        r = requests.post(url, data=json.dumps(payload), headers=self.headers)
        return r.json() # OK

    def subeler(self, ilceNo=5902): # Default testing value
        # 5902 Çerkezköy 
        payload=  {'cityID':59,'townID':5902,'unitOnDutyFlag':'null','closestFlag':0}
        url = self.urlGetUnit
        headers = {'content-type': 'application/json','encoding':'utf-8'}
        r = requests.post(url, data=json.dumps(payload), headers=headers)
        print  r.status_code, r.raw.read()

if __name__ == '__main__':        
    a = Yurtici()
    print a.ilceler(37) # OK
    print a.subeler()   # NOT OK !!!
guneysus
  • 6,203
  • 2
  • 45
  • 47
  • This is a trivial point, but [the documentation clearly states](http://docs.python-requests.org/en/latest/user/quickstart/#raw-response-content) that you should not use `Response.raw` unless you've set `stream=True` on your original request. – Lukasa Nov 24 '13 at 07:52

1 Answers1

2

Your code isn't posting to the same url you're using in your text example.

Let's walk through this backwards. First, let's look at the failing POST.

url = self.urlGetUnit
headers = {'content-type': 'application/json','encoding':'utf-8'}
r = requests.post(url, data=json.dumps(payload), headers=headers)

So we're posting to a URL that is equal to self.urlGetUnit. Ok, let's look at how that's defined:

baseUrl = 'http://www.yurticikargo.com/'
ajaxRoot = '_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-sswservices.aspx/'
getUnit = 'GetUnit'
urlGetUnit = baseUrl + ajaxRoot + getUnit

If you do the work in urlGetUnit, you get that the URL will be http://www.yurticikargo.com/_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-sswservices.aspx/GetUnit. Let's put this alongside the URL you used in Chrome to compare the differences:

http://www.yurticikargo.com/_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-sswservices.aspx/GetUnit
http://www.yurticikargo.com/_layouts/ArikanliHolding.YurticiKargo.WebSite/ajaxproxy-unitservices.aspx/GetUnit

See the difference? ajaxRoot is not the same for both URLs. Sort that out and you'll get back a JSON response.

Lukasa
  • 14,599
  • 4
  • 32
  • 34
  • Thanks a lot, i was sleepy :) Now i solved it. By fixing the ajax url. And also,the response was encoded gzip. I decompressed it with zlib as in here : http://stackoverflow.com/questions/2695152/in-python-how-do-i-decode-gzip-encoding – guneysus Nov 24 '13 at 08:41
  • 1
    Requests can do that for you. Use `Response.content`, `Response.text` or `Response.json()` and Requests will automatically decompress it. – Lukasa Nov 24 '13 at 08:59