unable to send data using urllib and urllib2 (python)

Question

Hello everybody (first post here).

I am trying to send data to a webpage. This webpage request two fields (a file and an e-mail address) if everything is ok the webpage returns a page saying "everything is ok" and sends a file to the provided e-mail address. I execute the code below and I get nothing in my e-mail account.

import urllib, urllib2

params = urllib.urlencode({'uploaded': open('file'),'email': 'user@domain.com'})
req = urllib2.urlopen('http://webpage.com', params)
print req.read()

the print command gives me the code of the home page (I assume instead it should give the code of the "everything is ok" page).

I think (based o google search) the poster module should do the trick but I need to keep dependencies to a minimum, hence I would like a solution using standard libraries (if that is possible).

Thanks in advance.

are you missing any headers? 'Content-type: application/x-www-form-urlencoded' — Corey Goldberg, Jun 25 '11 at 18:07
Please don't post your solution in the question. You *can* and *should* answer your own question, if none of the answers below helped. — Mike Pennington, Jun 27 '11 at 02:53

score 3 · Accepted Answer · answered Jun 27 '11 at 19:52

Thanks everybody for your answers. I solve my problem using the mechanize library.

import mechanize 

br = mechanize.Browser()
br.open('webpage.com') 

email='user@domain.com'

br.select_form(nr=0) 
br['email'] = email 
br.form.add_file(open('filename'), 'mime-type', 'filename')    
br.form.set_all_readonly(False) 
br.submit()

score 1 · Answer 2 · answered Jun 25 '11 at 18:17

1

This site could checks Referer, User-Agent and Cookies.

Way to handle all of this is using urllib2.OpenerDirector which you can get by urllib2.build_opener.

# Cookies handle
cj = cookielib.CookieJar()
CookieProcessor = urllib2.HTTPCookieProcessor(cj)
# Build OpenerDirector
opener = urllib2.build_opener(CookieProcessor)
# Valid User-Agent from Firefox 3.6.8 on Ubuntu 10.04
user_agent = 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.8) Gecko/20100723 Ubuntu/10.04 (lucid) Firefox/3.6.8'
# Referer says that you send request from web-site title page
referer = 'http://webpage.com'
opener.addheaders = [
    ('User-Agent', user_agent),
    ('Referer', referer),
    ('Accept-Charset', 'utf-8')
]

Then prepare parameters with urlencode and send request by opener.open(params)

Documentation for Python 2.7: cookielib, OpenerDirector

answered Jun 25 '11 at 18:17

Kirill

3,364
2
21
36

thanks for your answer, but your solution does not solve my problem. I think (i have just realized) that my problem have something to do with the fact that the page needs the data to be encoded as "multipart/form-data" – aloctavodia Jun 25 '11 at 18:32
OK. Then you can use what was advised: [MultiPartPostHandler](http://peerit.blogspot.com/2007/07/multipartposthandler-doesnt-work-for.html). It is few lines of code so it isn't really dependency. Consider it just your own class which you wrote yesterday for convenient file uploading :) – Kirill Jun 26 '11 at 03:46
I am trying to use this solution (with some minor changes, but still not working properly) [link]http://stackoverflow.com/questions/680305/using-multipartposthandler-to-post-form-data-with-python/681182#681182 – aloctavodia Jun 26 '11 at 15:50

unable to send data using urllib and urllib2 (python)

2 Answers2

Linked