2

I have a web scraping script written in BeautifulSoup4. The script is supposed to retrieve information after 600 seconds. Now, my app is not starting.

I am calling the script from urls.py

from django.urls import path
from details.task import scrape

urlpatterns = [
    ...
]
scrape()

task.py

import time
from details.scrape import ScrapeFunction

def scrape():
    ScrapeFunction()
    time.sleep(600)
    scrape()

The script must be called as soon as the app started.

Musa Ibrahim
  • 23
  • 2

1 Answers1

1

Your app isn't starting because your function has no exit point. When you start the server, it is falling into that infinite loop. You can use threading here.

Your code should look like this:

urls.py

from django.urls import path
from details.task import scrape
import threading

urlpatterns = [
    ...
]

threading.Thread(target=scrape, daemon=True).start()

task.py

import time
import threading
from details.scrape import ScrapeFunction

def scrape():
    threading.Thread(target=ScrapeFunction, daemon=True).start()
    time.sleep(600)
    scrape()

You should take a look here.

Mhasan502
  • 73
  • 8