Python dryscrape scrape page with cookies

Question

I wanna get some data from site, which requires loggin in.
I log in by requests

url = "http://example.com"
response = requests.get(url, {"email":"a@gmail.com", "password":"12345"})
cookies = response.cookies

Then I wanna get data from some JS page. Through requests it isn't possible, so I have to use dryscrape for this.

import dryscrape
url = "http://example.com/js-page"
sess = dryscrape.Session()
sess.visit(url)

Is it possible to pass cookies to visit() or I have to look for another solution?

if you are trying to load a webpage (works same for .js or .html) then you can use the solution from here: http://stackoverflow.com/questions/1825438/download-html-page-and-its-content — Tadhg McDonald-Jensen, Feb 07 '16 at 18:58
Wait, do you mean you want to get the content after it has been modified by javascript? — Tadhg McDonald-Jensen, Feb 07 '16 at 19:08
@TadhgMcDonald-Jensen I need to access a page with JS, which requires login/password. So I log in with ```requests``` and I want to save cookies and pass them to ```dryscrape```. — GriMel, Feb 08 '16 at 11:34

score 6 · Accepted Answer · answered Jun 28 '16 at 14:14

Why not login by dryscrape?

session = dryscrape.Session()
session.visit('<url_where_is_login_form>')
name = session.at_xpath('//*[@name="username"]') # Where <input name="username">
name.set("<login>")
password = session.at_xpath('//*[@name="password"]') # Where <input name="password">
password.set("<password>")
# Push the button
name.form().submit()
session.visit("<url to visit with proper cookies>")

Python dryscrape scrape page with cookies

1 Answers1

Linked