Unable to get the full content using selector

Question

I've written some selector used within python to get some items and it's value. I wish to scrape the items not to style. However, when I run my script, It only gets the items but can't reach the value of those items which are separated by "br" tag. How can I grab them? I do not with to use xpath in this very case to serve the purpose. Thanks in advance.

Here are the elements:

html = '''
<div class="elems"><br>
    <ul>
    <li><b>Item Name:</b><br>
            titan
                </li>
        <li><b>Item No:</b><br>
                23003400
                    </li>
        <li><b>Item Sl:</b><br>
            2760400
                </li>
        </ul>
    </div>
'''

Here is my script with css selectors in it:

from lxml import html as e

root = e.fromstring(html)
for items in root.cssselect(".elems li"):
    item = items.cssselect("b")[0].text_content()
    print(item)

Upon execution, the result I'm having:

Item Name:
Item No:
Item Sl:

The result I'm after:

Item Name: titan
Item No: 23003400
Item Sl: 2760400

Possible duplicate of [How to fetch data inside tag which is inside
I'd say it's a duplicate of [Get all text inside a tag in lxml](https://stackoverflow.com/questions/4624062/get-all-text-inside-a-tag-in-lxml). — skovorodkin, Sep 03 '17 at 22:40
I followed both the links to see if my question is really duplicate or identical. The answer is clearly: no. Nowhere in those two links I could see "css-selectors" tag. Moreover, I insisted to have an answer only based on selectors. So, how can it be duplicate? — SIM, Sep 03 '17 at 22:53
@Topto, `.cssselect` is no more than just a wrapper around `.xpath` method. Look here https://github.com/scrapy/cssselect: "cssselect parses CSS3 Selectors and translate them to XPath 1.0 expressions". So there's actually no difference if you use XPath or CSS selectors here. — skovorodkin, Sep 04 '17 at 18:57
@skovorodkin, yeah that's right. However, my intention here was to accomplish the task using css selector and that is what I tried to show in the answer. Thanks. — SIM, Sep 04 '17 at 21:19

score 0 · Answer 1 · answered Sep 03 '17 at 22:38

Generally I use .itertext method to extract text:

from lxml.html import fromstring

def extract_text(el, sep=' '):
    return sep.join(s.strip() for s in li.itertext() if s.strip())

tree = fromstring(html)
for li in tree.cssselect('.elems li'):
    print(extract_text(li))

score 0 · Accepted Answer · answered Sep 04 '17 at 12:18

0

The easiest solution ever. Values are within "li" tag not "b".

from lxml import html as e

root = e.fromstring(html)
for items in root.cssselect(".elems"):
    item = [item.text_content() for item in items.cssselect("li")]
    print(''.join(item))

answered Sep 04 '17 at 12:18

SIM

21,997
5
37
109

Unable to get the full content using selector

2 Answers2

Linked