Dealing with a colon in BeautifulSoup CSS selectors

Question

Input HTML:

<div style="display: flex">
    <div class="half" style="font-size: 0.8em;width: 33%;"> apple </div>
    <div class="half" style="font-size: 0.8em;text-align: center;width: 28%;"> peach </div>
    <div class="half" style="font-size: 0.8em;text-align: right;width: 33%;" title="nofruit"> cucumber </div>
</div>

The desired output: all div elements exactly under <div style="display: flex">.

I'm trying to locate the parent div with a CSS selector:

div[style="display: flex"]

This throws an error:

>>> soup.select('div[style="display: flex"]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/.virtualenvs/so/lib/python2.7/site-packages/bs4/element.py", line 1400, in select
    'Only the following pseudo-classes are implemented: nth-of-type.')
NotImplementedError: Only the following pseudo-classes are implemented: nth-of-type.

It looks like BeautifulSoup tries to interpret the colon as a pseudo-class syntax.

I've tried to follow the advices suggested at Handling a colon in an element ID in a CSS selector, but it still throws errors:

>>> soup.select('div[style="display\: flex"]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/.virtualenvs/so/lib/python2.7/site-packages/bs4/element.py", line 1400, in select
    'Only the following pseudo-classes are implemented: nth-of-type.')
NotImplementedError: Only the following pseudo-classes are implemented: nth-of-type.
>>> soup.select('div[style="display\3A flex"]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/.virtualenvs/so/lib/python2.7/site-packages/bs4/element.py", line 1426, in select
    'Unsupported or invalid CSS selector: "%s"' % token)
ValueError: Unsupported or invalid CSS selector: "div[style="displayA"

The Question:

What is the correct way to use/escape a colon in attribute values in BeautifulSoup CSS selectors?

Note that I can workaround it with a partial attribute match:

soup.select("div[style$=flex]")

Or, with a find_all():

soup.find_all("div", style="display: flex")

Also note that I understand that using style to locate elements is far from being a good location technique, but the question itself is meant to be generic and the provided HTML is just an example.

I assume you have also tried two backslashes? `soup.select('div[style="display\\: flex"]')` — Josh Crozier, Jan 01 '16 at 05:01
@JoshCrozier happy new year, yup, and different combinations with raw strings and regular strings. Thanks. Still think I'm just missing smth. — alecxe, Jan 01 '16 at 05:03

alecxe · Accepted Answer · 2016-07-20T12:57:48.007

2

Update: the issue is now fixed in BeautifulSoup 4.5.0, upgrade if needed:

pip install --upgrade beautifulsoup4

Old answer:

Created an issue at the BeautifulSoup issue tracker:

Dealing with a colon in BeautifulSoup CSS selectors

Will update the answer in case of any updates in the launchpad issue.

edited Jul 20 '16 at 12:57

answered Feb 01 '16 at 21:43

alecxe

462,703
120
1,088
1,195

score 1 · Answer 2 · answered Jan 16 '16 at 01:59

Not sure this constitutes an answer exactly, as it is most definitely broken. However, strangely the error is triggered not by the : itself, but by the : followed by a space. The error suggests it's trying to use whatever is after the space as a CSS selector.

For example, editing the HTML to drop the space makes the block selectable again:

>>> from bs4 import BeautifulSoup
>>> html = """
... <div style="display:flex">
...     <div class="half" style="font-size: 0.8em;width: 33%;"> apple </div>
...     <div class="half" style="font-size: 0.8em;text-align: center;width: 28%;"> peach </div>
...     <div class="half" style="font-size: 0.8em;text-align: right;width: 33%;" title="nofruit"> cucumber </div>
... </div>
... """

>>> soup = BeautifulSoup(html)
>>> soup.select('div[style="display: flex"]')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/dist-packages/bs4/element.py", line 1313, in select
    'Unsupported or invalid CSS selector: "%s"' % token)
ValueError: Unsupported or invalid CSS selector: "flex"]"

>>> soup.select('div[style="display:flex"]')
[<div style="display:flex">
<div class="half" style="font-size: 0.8em;width: 33%;"> apple </div>
<div class="half" style="font-size: 0.8em;text-align: center;width: 28%;"> peach </div>
<div class="half" style="font-size: 0.8em;text-align: right;width: 33%;" title="nofruit"> cucumber </div>
</div>]

Unfortunately, with the space is the usual style so this probably won't get you very far at all!

Interesting observation! Thanks. – alecxe Jan 16 '16 at 02:09 — alecxe, Jan 16 '16 at 02:09

Dealing with a colon in BeautifulSoup CSS selectors

2 Answers2

Linked