1

I have a Heroku app to launch a Scrapy spider. After the launch, after a while, the error 503 (service unavailable) appears in the browser console. The script that refreshes the page after receiving data from the spider does not work. How to fix it?

js:

$.post('/wellness', {'specialty': specialty, 'state': state, 'city': city}, (res) => { 
      $(location).attr('href', 'http://127.0.0.1:5000/wellness')
    });

flask:

if request.method == 'POST':
    specialty = request.form.getlist('specialty[]')
    state = request.form.getlist('state[]')
    city = request.form.getlist('city[]')

    settings = ''
    with open(file_json, 'r') as f:
        for line in f.read():
            settings += line

    settings = json.loads(settings)
    settings['specialty'] = specialty
    settings['state'] = state
    settings['city'] = city

    with open(file_json, 'w') as f:
        f.write(json.dumps(settings, indent=4))

    process = subprocess.Popen('python spiders/' + file_py, shell=True)
    process.wait()

return render_template(file_html)
Alexander
  • 91
  • 3
  • 13

1 Answers1

0

I'm not sure what could be triggering this problem, but you might want to test some of these approaches to run the spiders: How to integrate Flask & Scrapy?

I personally suggest those two options:

503 errors will generally occur when the server is down. It could be a problem running the server or trying to run the spider.

Geraldo Castro
  • 189
  • 5