0

I am new to the web-scraping and making an API, facing an error while scraping an e-commerce website. Below is my python code please guide me through the same, I am getting "The requested URL was not found on the server." while running on a local-host.

from flask import Flask , request , jsonify
from bs4 import BeautifulSoup
import requests

app = Flask(__name__)


@app.route('/',methods=['GET'])
def API():
    if request.method == 'GET':
        uri = 'https://www.flipkart.com'
        query = str(request.args['query'])
        print(query)
        if " " in query:
            query = str(query).replace(" ","+")
        else:
            pass

        search = '/search?q=' + query

        ready_uri = uri + search
        print(ready_uri)
        content = requests.get(ready_uri).content
        soup = BeautifulSoup(content, 'html.parser')
        quotes_links = soup.find_all('a', {'class': '_3O0U0u'})
        l = []
        for i in quotes_links:
            d = {}
            quote_url = uri + i.get('href')
            quote_content = requests.get(quote_url).content
            quote_soup = BeautifulSoup(quote_content, 'html.parser')
            d['quote'] = quote_soup.find('p', {'class': '_3wU53n'}).text
            d['author'] = quote_soup.find('p', {'class': '_1vC4OE _2rQ-NK'}).text
            l.append(d)


        return jsonify(l)


if __name__ == '__main__':
    app.run()

Error:

[33mGET /search?q=books HTTP/1.1[0m" 404 -
Salvatore
  • 65
  • 2
  • 9

1 Answers1

0

How do you get a query string on Flask?

You appear to be getting the query argument incorrectly.

query = str(request.args['query'])

When it should be:

query = str(request.args.get('query'))

Doing so returns a 200 but with blank data. I would suggest looking at the element your scraping:

quotes_links = soup.find_all('a', {'class': '_3O0U0u'})

Once you obtain the correct element with soup, you should start seeing return data.

mihuo999o
  • 96
  • 2