0

I had Scraped Data from https://www.worldometers.info/coronavirus/ for countrywise stats using bs4. but i want to use that data to populate my django model with same fields as scraped data which i dont know how. i am also having trouble with scraping tabular data with other libraries like scrapy (celery).this is the xpath of the table i am try to scrap "//*[@id="main_table_countries_today"]". if anyone could help me how to use this scraped data to store in django models would be great. PS not using external CSV or Json Files.

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Umair Ramay
  • 3
  • 1

1 Answers1

0

You could use the Django ORM in an external script from which you are scraping the data.

# replace project_name with your own project name
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project_name.settings")
django.setup()

from models import MyModel

and then in the script, you can create an object of the model class with the scraped data.

MyModel.objects.create(recoverd=recoverd_cases, deaths=people_died)
Hasham Ghuffary
  • 68
  • 5
  • for this both the script file and django project should be in same environment? like i should create a task.py file inside my django project. – Umair Ramay Apr 11 '20 at 16:07
  • Another thing which i did i configured postgres with my script to add data to model fields directly but only string file data is being added to db columns but integers data fields are not adding due to type errors the data type and model field type are not matching even though both are set to int. – Umair Ramay Apr 11 '20 at 16:10
  • @UmairRamay What you could do is to make a copy of the same Django project with the same database credentials and you can then run the script in another environment. – Hasham Ghuffary Apr 12 '20 at 08:29
  • Take a look at this answer: https://stackoverflow.com/a/37647383/13116155 – Hasham Ghuffary Apr 12 '20 at 08:30