POST Requests with headers and data results in 200 ok response but no user added

Question

Before creating this query checked few existing posts in this topic and followed the steps there, Still could not get it through hence raising this again here for solutions.

The following are the handshake codes could get from inspect element Network activity in chrome.

Request URL: https://example.com/ne_rm/add/  
Request Method: POST
Status Code: 200 
Remote Address: 11.130.11.19:413
Referrer Policy: no-referrer-when-downgrade

accept-ranges: bytes
age: 0
cache-control: max-age=0
content-encoding: gzip
content-length: 22
content-type: text/html; charset=utf-8
date: Wed, 05 Dec 2018 09:27:21 GMT
expires: Wed, 05 Dec 2018 09:27:21 GMT
last-modified: Wed, 05 Dec 2018 09:27:21 GMT
server: tv
status: 200
vary: Accept-Encoding, Cookie
via: 1.1 varnish, 1.1 c34ac5faa133414ef7dde72a4f32c43d.cloudfront.net (CloudFront)
x-amz-cf-id: K3-mr0hE2iObHSoWssicdKTZzCGsWEUnSUSws1v-fln9jP1gT668sQ==
x-cache: Miss from cloudfront
x-frame-options: SAMEORIGIN
x-varnish: 3925923656

:authority: example.com
:method: POST
:path: /ne_rm/add/
:scheme: https
accept: application/json, text/javascript, */*; q=0.01
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
content-length: 72
content-type: application/x-www-form-urlencoded; charset=UTF-8
cookie: km_lv=x; _ga=GA1.2.927532329.1495375443; km_ai=xxx@mail.com;  km_ni=xx@gmail.com; sessionid=0j95mvy7sssxss41i6woawmj3nyqw11a; csrftoken=SDmN222meuuKexz3333nPXue2yw22TGV7dfff; _sp_id.df1c=15d71e85-964f-42ee-965a->4a9d8c0902ec.1538974408.1.1538974410.1538974408.37c368eb-6de7-4716-8e84->06b17f6e6914; __utmz=226258911.1539246301.382.19.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); tv_ecuid=189b7a17-8451-4ae3-9443-3cec2dda407d; __utmc=226258911; _sp_ses.cf1a=*; __utma=226258911.927532329.1495375443.1543915752.1544001282.532; km_vs=1; _sp_id.cf1a=f53754cc-c8ba-42e2-a310-08566d71540d.1539169168.144.1544001661.1543915787.f1ddde18-9adf-4e88-a852-1485403f2587; kvcd=1544001661767; __utmb=226258911.10.9.1544001932454
origin: https://example.com
referer: https://example.com/script/Zo-Tester/
user-agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36
x-csrftoken: SDmN222meuuKexz3333nPXue2yw22TGV7dfff
x-language: in
x-requested-with: XMLHttpRequest

id: PUB;FMx5WvjpGrmETV username_recip: xxxxyyzzz

Here using the id and username as data I would like to call https://example.com/ne_rm/add/ and add this username in the list. While doing this using POST Chrome extension I am getting 200 Ok as response and the name is added in the list. At the same time while doing this with python code, I do get a 200 Ok as the response code, but the operation is not successful.

Note what is POST browser minimum worked that alone used in python but it fails.

Can you please help me make this work?

Here is the code

import requests
import json

# *optional*, the site may not care about these. If they *do* care, then
# they care about keeping out automated scripts and could in future 
# raise the stakes and require more 'browser-like' markers. Ask yourself
# if you want to anger the site owners and get into an arms race.
headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36',
    'origin':'https://www.example.com',
    'referer':'https://example.com/script/Zo-Tester/',
    'x-csrftoken': 'SDmN222meuuKexz333nPXue2yw22TGV7dfff',
}

payload = {
    'x-csrftoken': 'SDmN222meuuKexz333nPXue2yw22TGV7dfff',
    'username_recip':'xxxxyyzzz',
    'pine_id':'PUB;FMx5WvjpGrmETV',
}

url = 'https://www.example.com/ne_rm/add/'
# the URL from the Referer header, but others at the site would probably
# also work
#Referrer URL
initial_url = 'https://example.com/script/Zo-Tester/'

with requests.Session() as session:
    # obtain CSRF cookie
    #initial_response  = session.get(initial_url)
    #payload['csrf_test_name'] = session.cookies['csrf_cookie_name']

    # Now actually post with the correct CSRF cookie
    response = session.post(url, headers=headers, data=payload)
    print(response)
    input("wait")

You can't just reuse a CSRF token. You **must** first get the token sorted out. — Martijn Pieters, Dec 11 '18 at 12:07
The request/response pair you posted you can see that the CSRF token is stored in a cookie (the `Cookie` header that the browser sends has `csrftoken=SDmN222meuuKexz3333nPXue2yw22TGV7dfff;`, that cookie was first given to the browser in an earlier request. The code from my answer that you commented out *gets that token from the initial request*, and by disabling it you ensured that the server will not trust your requests now, as it will not contain a valid CSRF token. — Martijn Pieters, Dec 11 '18 at 12:11
how to achieve this any code sample, please . I am new to python and to this kind of headers handling. — Marx Babu, Dec 11 '18 at 13:54

ewwink · Answer 1 · 2018-12-11T11:29:22.297

-1

you need to uncomment part of # obtain CSRF cookie to obtain new x-csrftoken and if I see in your Chrome headers, its id not pine_id ?

import requests
import json


headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36',
    'origin':'https://www.example.com',
    'referer':'https://example.com/script/Zo-Tester/',
}

payload = {
    'username_recip':'xxxxyyzzz',
    'pine_id':'PUB;FMx5WvjpGrmETV' # it should be 'id' ?
}

url = 'https://www.example.com/ne_rm/add/'
initial_url = 'https://example.com/script/Zo-Tester/'

with requests.Session() as session:
    # obtain CSRF cookie
    initial_response  = session.get(initial_url)
    headers['x-csrftoken'] = session.cookies['csrftoken']

    # Now actually post with the correct CSRF cookie
    response = session.post(url, headers=headers, data=payload)
    print(response)
    input("wait")

edited Dec 11 '18 at 11:29

answered Dec 11 '18 at 09:58

ewwink

18,382
2
44
54

it is pine_id not header ID this is correct .I changed the code for uncommenting part and this is the errors. not successfull payload['x-csrftoken'] = session.cookies['x-csrftoken'] return self._find_no_duplicates(name) _find_no_duplicates raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path)) KeyError: "name='x-csrftoken', domain=None, path=None" – Marx Babu Dec 11 '18 at 10:28
is there any change in code than my issue reported code.Commenting this line was not working was communicated already. – Marx Babu Dec 11 '18 at 11:08
have you delete `x-csrftoken` in headers and payload? your code seem to be right. if you don't mind post the URL. – ewwink Dec 11 '18 at 11:11
in postman query as well x-csrftoken has to be included otherwise it was not working in chrom POSTMAN simulation ,now tried removing and testing still it is failing . Unfortunately sharing url not allowed. – Marx Babu Dec 11 '18 at 11:21
I just found it, `x-csrftoken` is stored in cookies as `csrftoken` and it need to be sent as headers, see edited answer. – ewwink Dec 11 '18 at 11:30
same error soon we will connect in chat to resolve this issue – Marx Babu Dec 11 '18 at 11:40
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/185064/discussion-between-marx-babu-and-ewwink). – Marx Babu Dec 11 '18 at 11:46

POST Requests with headers and data results in 200 ok response but no user added

1 Answers1

Linked