1

Let's say I have the following page structure:

enter image description here

Within that structure I have the following nested structure:

enter image description here

I wrote

offers = response.xpath('//li[@class = "search-page__result"]')

to try and create an inerrable list. This part is successful, I believe, because if I say

for offer in offers:
    print(offer.get())

then it will print out the HTML subsection as a string.

However if I say

for offer in offers:
    features = offer.xpath('//ul[@class = "listing-key-specs"]')

the content of features looks as if I would have used response instead of offer.

Does anyone have any guidance on how I can use xpath iteratively on each subsection of the structure at a time?

EDIT: I wanted to add this alternative solution that I found before my question was answered. As suggested here by YohanObadia, the below will work too:

for offer in offers:
    features = Selector(text = offer.extract()).xpath('//ul[@class = "listing-key-specs"]')
CristianCapsuna
  • 392
  • 3
  • 14
  • 1
    // means your context is the root again. Just put a dot in front of the slashes and the context is offer. – Siebe Jongebloed May 31 '21 at 23:18
  • Hi @SiebeJongebloed ! Thank you for the help. That worked. Can you please post it as an answer so I can mark it as accepted? I have found one more solution on the internet and will append it to your answer. – CristianCapsuna Jun 01 '21 at 18:38

1 Answers1

1

// means your context is the root again. Just put a dot in front of the slashes and the context is offer.

for offer in offers:
    features = offer.xpath('.//ul[@class = "listing-key-specs"]')

Here is an answer that explains it.

Siebe Jongebloed
  • 3,906
  • 2
  • 14
  • 19