1

I am trying to use urllib to fill out a form and then retrieve the results from the submitted data.The app I'm posting to is on a server that I have to login through appgate to access. The POST form (that I see in the response in the terminal) is like this: 

    <form action="accuracy-upload3" method="post" 
    enctype="multipart/form-data">
    Human: <input type="file" id="human" name="human"><br>
    Machine: <input type="file" id="machine" name="machine"><br>
    <input type="submit" class="submit" value="Submit" />
    </form>

But even if the method is "POST" I seem to be doing a GET instead since the html from the url is all that returns, not the response from the url on the 'action' parameter. My code is as follows:

    url = "https://accuracy3.html"
    filename1 = 'human.txt'
    filename2 = 'machine.txt'
    filedata1 = open(filename1).read()
    filedata2 = open(filename2).read()

    values = {
        "Human": filedata1,
        "id": "human",
        "name": "human",
        "Machine": filedata2,
        "id": "machine",
        "name": "machine",
        "value": "Submit"
     }
    req = urllib2.Request(url, data = urllib.urlencode(values), headers 
     = 
    {"Content-type": "application/x-www-form-urlencoded"})
    response = urllib2.urlopen(req)
    print response.read()

From Making a POST call instead of GET using urllib2 I see that it should be sending a POST since I've sent data, but it isn't doing that.

Can anyone see what the problem is with my script?

topplethepat
  • 531
  • 6
  • 23
  • 1
    I'm not very familiar with urllib2, but it looks like you're not doing anything with the response from the request with the form data. You store it in `response` but then don't do anything with it; everything after that point uses the response from a separate request. Have you checked to see if `response` contains the HTML you need? – Harry Cutts Jan 18 '19 at 22:38
  • Yes, if I do the_page = response.read(the_page) print the_page, it prints the form from the original html. – topplethepat Jan 18 '19 at 23:19
  • I edited the post now to show what happens when I try using the response variable. – topplethepat Jan 19 '19 at 00:55
  • @topplethepat You show that the form posts to `accuracy-upload3` however you use url `https://accuracy3.html` (which might be a GET). Also the POST can return valid HTML. In general I'd recommend using [requests](https://pypi.org/project/requests/), it'll be much easier to work with. – a_guest Jan 29 '19 at 01:10
  • You should use "type" instead of "method, this will solve you problem – An Nguyen Jan 29 '19 at 01:19
  • @a_guest Thanks -- to clarify, that url variable is where the form lives, but the response is shown at a different url, i.e., the one in the form action parameter. I'll try again with requests as well. – topplethepat Jan 29 '19 at 16:14
  • @AnNguyen I'm using an existing app, not one I'm creating myself, so the form fields are already set. – topplethepat Jan 29 '19 at 16:15
  • @topplethepat Exactly so you should post to the URL where the form would post to, not to where the form's HTML lives. – a_guest Jan 29 '19 at 16:15
  • @a_guest The part I don't understand is the sequence. I tried it right now with the 2nd url in, and it gave me the form again, this time with the value: "Upload" for the submit button. But no result appears, because I think it has to submit first on the first url, and then retrieve from the 2nd. This appeared to go directly to the 2nd url but no data had been submitted so no results. – topplethepat Jan 29 '19 at 16:49
  • 1
    @topplethepat So the question is really where does your form live? `https://accuracy3.html`is not a valid URL since `.html` is not a valid top-level domain. Say for example that the form lives at `https://accuracy3.com/index.html`. Since the form defines `action="accuracy-upload3" method="post"` this means it wants to POST to the location `accuracy-upload3` *relative* to the current location. For this example this would be `https://accuracy3.com/accuracy-upload3` (since `https://accuracy3.com/` is where the `index.html` sits which contains the form). So you need to find out the correct URL. – a_guest Jan 29 '19 at 18:05

0 Answers0