BeautifulSoup is HTML escaping strings which have escaped characters

Question

I am reading a string from file:

a = '<script>closedSign: \'<img src="/static/images/drop-down.png" style="margin-top: -3px;"  />\'</script>'

Now, when I run

BeautifulSoup(a)

<script>closedSign: '&lt;img src="/static/images/drop-down.png" style="margin-top: -3px;"   /&gt;'</script>

Thus, <img is being HTML escaped into <img

How can I avoid this?

Just edited the Question to explain it better. Also, just noticed this problem happens only on BeautifulSoup3.2.1 and not on 3.2.0 — Jamal, Nov 08 '12 at 15:28
It sounds like you've run into this issue: https://bugs.launchpad.net/beautifulsoup/+bug/949074 — Josh Kelley, Sep 24 '14 at 15:10

score 4 · Accepted Answer · answered Nov 18 '12 at 11:55

4

Use BeautifulSoup 3.2.0 instead of 3.2.1 to fix this problem.

answered Nov 18 '12 at 11:55

Jamal

Fixed the problem for me too. Thanks a lot for the solution and -1 for BeautifulSoup breaking such a thing after a minor update. – Ponytech Oct 21 '13 at 19:17

score 2 · Answer 2 · answered Nov 08 '12 at 15:35

2

Look at the "Entity Conversion" section of the Beautiful Soup Documentation.

soup = BeautifulSoup(html, convertEntities=BeautifulSoup.HTML_ENTITIES)

answered Nov 08 '12 at 15:35

Paulo Scardine

Paulo, convertEntities=BeautifulSoup.HTML_ENTITIES works on strings which are already HTML escaped. Example: to return < img from parsing & lt;img But here the problem is, parsing itself is returning a HTML Escaped string. I hope I was able to clearly explain my problem. – Jamal Nov 09 '12 at 13:52
What happens if you run BeautifulSoup a second time over the script tag HTML? – Paulo Scardine Nov 09 '12 at 16:02
3

For possile `bs4` users: `convertEntities` [doesn't](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#entities) exist anymore in bs4. – arash Dec 04 '20 at 12:31

2 Answers2