How to sanitize url string in Python?

Question

I have a python app that works with URL lists and produces bash script as an output.

How to sanitize URL so that a malicious user could not inject bash commands that will be executed on my infrastructure.

For instance:

http://www.circl.lu/a.php?rm -Rf /etc

https://stackoverflow.com/questions/205923/best-way-to-handle-security-and-avoid-xss-with-user-entered-urls/205967#205967 ? — DirtyBit, Apr 10 '19 at 13:19
Whether or not you *need* to sanitize it depends on what you are doing with it. As a simple example, `printf '%s\n' "$UNSANITIZED_URL"` is completely safe. (Assuming no buffer-overflow bugs or the like in your shell itself.) — chepner, Apr 10 '19 at 13:20
@DirtyBit the bash script is output, I need to sanitize URL in Python — iceone213, Apr 10 '19 at 13:25
@chepner the output (bash script) may be executed on different systems, so I have no control over it — iceone213, Apr 10 '19 at 13:31

score 5 · Accepted Answer · answered Apr 10 '19 at 18:43

I guess urllib is an option to parse the urls, in order to escape harmful characters. At least it looks like a good resource for your use case. See the docs of url-quoting.

from urllib.parse import quote

quote('touch foo', safe='/')
quote('rm -Rf /etc', safe='/')
quote('http://www.circl.lu/a.php?rm -Rf /etc', safe='/:?&')

#'touch%20foo'
#'rm%20-Rf%20/etc'
#'http://www.circl.lu/a.php?rm%20-Rf%20/etc'

How to sanitize url string in Python?

1 Answers1