for an assignment I am working on a research question on wether "normal" (e.g. non climate change-related) migration is portrayed in a different tone in the media whereas compared to climate change-related migration.
I want to web scrape different media outlets and then perform a text analysis.
However, I am wondering from a methodological perspective how I could scrape articles that are mutually exclusive, whereas the keywords (migration) are so similar. I am very new to web scraping but I have the guess that it will be really hard to differentiate which articles are about climate migration and which are about normal migration.
The only solution I would have at hand is to search articles manually first and then scrape the text, does that sound reasonable?
Thank you!