Converting cURL command with XML string to python request

Question

I was used to send XML strings to the API of my firewall with this cURL command:

curl -k https://192.1.1.1:90/APIController -F "reqxml=<myfile.xml"

Now, I would like to do this in python. Since the XML string will be created programmatically, I don't provide a XML file but a XML string. This is my code right now:

#!/usr/bin/env python
import requests

def xml_string():
    return '''
    reqxml=<?xml version="1.0" encoding="UTF-8"?>
    <Request>
        <Login>
            <Username>admin</Username>
            <Password>admin</Password>
        </Login>
        <Set>
        … further markup …
        </Set>
    </Request>
    '''

def send_xml():
  response = requests.post(
    "https://192.1.1.1:90/APIController",
    headers={"Content-Type": "application/xml"},
    data=xml_string(),
    verify=False
    )
  print(response.status_code)


send_xml()

The output of the verbose command curl -k -v https://192.1.1.1:90/APIController -F "reqxml=<testcategory.xml" is:

* Trying 192.1.1.1...
* TCP_NODELAY set
* Connected to 192.1.1.1 (192.1.1.1) port 90 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=NA …
*  start date: Aug  1 00:00:00 2010 GMT
*  expire date: Dec 31 23:59:59 2030 GMT
*  issuer: C=NA …
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> POST /APIController HTTP/1.1
> Host: 192.1.1.1:90
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 796
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------20be8a5c28d38998
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Date: Sun, 26 Apr 2020 10:07:59 GMT
< Server: xxxx
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=31536000
< X-Content-Type-Options: nosniff
< Content-Type: text/xml;  charset=UTF-8
< Cache-Control: max-age=2592000
< Expires: Tue, 26 May 2020 10:07:59 GMT
< Transfer-Encoding: chunked
< 
<?xml version="1.0" encoding="UTF-8"?>
<Response APIVersion="1702.1" IPS_CAT_VER="1">
  <Login>
    <status>Authentication Successful</status>
  </Login>
  <Status code="200">Configuration applied successfully.</Status>
</Response>
* Connection #0 to host 192.1.1.1 left intact

But despite the returned status code is 200, the command had no effect on the firewall, no settings were changed. Why doesn't work the python code as expected?

EDIT: I found out, that the XML string is completely ignored when firing the above python code. Maybe this working code in PHP can shed some light?

<?php

$xml = '<?xml version="1.0" encoding="UTF-8"?>
<Request>
    <Login>
        <Username>admin</Username>
        <Password>admin</Password>
    </Login>
    <Set>
    …
    </Set>
</Request>';
$url = "https://192.1.1.1:90/APIController";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_POSTFIELDS, "reqxml=" . $xml);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec($ch);
echo '<pre>';
echo htmlentities($data);
echo '</pre>';
if (curl_errno($ch)) {
print curl_error($ch);
} else {
curl_close($ch);
}
?>

have you also looked at the data returned by the remote endpoint? perhaps there's an error string and the status code is 200. Have a look at https://requests.readthedocs.io/en/master/user/quickstart/#response-content — Lmwangi, Apr 26 '20 at 10:05
See https://stackoverflow.com/a/22567429/113962 for an example of how to upload a file with requests. Try getting it to work with an actual file first, then once that’s working you can use stringio instead of a file. — Alasdair, Apr 26 '20 at 13:14

Lmwangi · Answer 1 · 2020-04-26T17:59:24.800

0

Your best bet is to compare what is sent over the wire using a tool/application that changes the actual setting and compare with your python requests request. If your remote host accepts plain HTTP, use that & fire up wireshark/tcpdump

---edit

Try post the content as a file using requests library.


xml_content = '''<?xml version="1.0" encoding="UTF-8"?>
<Request>
    <Login>
        <Username>admin</Username>
        <Password>admin</Password>
    </Login>
    <Set>
    …
    </Set>
</Request>
'''
url = "https://192.1.1.1:90/APIController"

files = {'file': ('reqxml', xml_content)}
r = requests.post(url, files=files)

edited Apr 26 '20 at 17:59

answered Apr 26 '20 at 10:03

Lmwangi

2,486
1
19
26

This seems to be beyond my technical skills. Could you give some hints where I can get more information about this? Unfortunately, my host doesn't accept plain HTTP. – Madamadam Apr 26 '20 at 10:45
You need to use a POST with content-type multipart/form-data or 'Content-Type: application/x-www-form-urlencoded'. That means you either upload a file or submit a form. Try change your content type to 'Content-Type: application/x-www-form-urlencoded' instead of "application/xml" – Lmwangi Apr 26 '20 at 14:21
Thank you, but changing Content-type and base64-encoding of the XML-part didn't work neither. – Madamadam Apr 26 '20 at 15:01
2

instead of data=... try files=... as seen in the last example of: https://requests.readthedocs.io/en/master/user/quickstart/#post-a-multipart-encoded-file – Lmwangi Apr 26 '20 at 17:52
your last edit seemed logic and promising, but for some reason it doesn't work. Command `print(r.url)` outputs just `https://192.1.1.1:90/APIController` without any values … – Madamadam Apr 26 '20 at 21:23

sonu.Shiva · Answer 2 · 2020-04-26T13:04:30.197

0

To do it simply, you could try to execute the curl command directly through python. Store the xml string to a file.xml

import os
os.system('curl -k https://192.1.1.1:90/APIController -F "reqxml=<<path to file.xml>"')

(Updated)OR

Based on your curl command output as well as the php code, it looks like the content type is being sent as multipart/form-data. Maybe you could try to using the same Content-type: application/x-www-form-urlencoded with the payload converted to "reqxml=" + "base64 encoded xml data"

edited Apr 26 '20 at 13:04

answered Apr 26 '20 at 10:10

sonu.Shiva

23
3
10

I am aware of this work around, but the task has to be done with python only. – Madamadam Apr 26 '20 at 10:28
Thank you, but changing Content-type and base64-encoding of the XML-part didn't work neither. – Madamadam Apr 26 '20 at 14:59
Did you try adding an `Accept: 'text/xml'` header and sending the data as mentioned here [https://stackoverflow.com/questions/30689393/how-to-perform-an-http-xml-authentication-with-requests](https://stackoverflow.com/questions/30689393/how-to-perform-an-http-xml-authentication-with-requests) – sonu.Shiva Apr 26 '20 at 22:19
Adding `Accept: 'text/xml` to the header doesn't help. – Madamadam Apr 27 '20 at 06:00

Converting cURL command with XML string to python request

2 Answers2