Detect commentaries from webpage while web scrapping

Asked Nov 02 '22 at 20:54

Active Nov 02 '22 at 20:54

Viewed 23 times

I am trying to web scrapping sites with reviews and i get all site blocks I need to extract user's comments from scrapped web-page, how can i do it? I have two probably directions how to do this:

Somehow detect which site block contains commentaries and extract from those blocks but some sites have very strange html syntax and it hard to detect from tags - now i am trying something like recommend in this answer but it work not good enought.
Some text classification of text block on site but i cant fiud any models/apis to deal with this classification problem

asked Nov 02 '22 at 20:54

Vasily

1

are there a lot of websites and you need a unique thing to scrape them all at once? – Dmitriy Neledva Nov 02 '22 at 20:59
@DmitriyNeledva I thought of parsing them one by one and it is possible decision but i have about 30-40 most common sites and it looks too long – Vasily Nov 02 '22 at 21:00
'most common' you mean they are similar in html structure? show us their html – Dmitriy Neledva Nov 02 '22 at 21:04
most common with absolutley different html structure – Vasily Nov 02 '22 at 21:04
most common - i mean for for most requests to my app i need info from 10-20 sites – Vasily Nov 02 '22 at 21:05
1

If they are absolutely different you need to write scraper for each separately – Dmitriy Neledva Nov 02 '22 at 21:07
If I can't find a better way, I'll do it like this ty – Vasily Nov 02 '22 at 21:08
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/249258/discussion-between-dmitriy-neledva-and-vasily). – Dmitriy Neledva Nov 02 '22 at 21:09

Detect commentaries from webpage while web scrapping

0 Answers0