How to extract anchor class using beautifulsoup?

Question

Can anyone help me to resolve the error. My head is paining, I have wasted my 1 hour in solving this problem. Actually I am getting query_links as null(I should get all the classes values), but not so.

from flask import Flask, jsonify, request
# from markupsafe import escape
from bs4 import BeautifulSoup
import requests

app = Flask(__name__)

@app.route('/api/',methods=['GET'])
def API():
    if request.method == 'GET':
        url = 'https://www.brainyquote.com/'
        query = str(request.args['query'])

        if " " in query:
            query = str(query).replace(" ","+")
        else:
            pass

        search = '/search_results?q=' + query
        ready_url = url + search
        content = requests.get(ready_url).content
        soup =  BeautifulSoup(content, 'html.parser')
        quotes_links = soup.find_all("a", class_= "b-qt")
        print("hello")
        print(quotes_links)
        list = []
        for i in quotes_links:
            d = {}
            quote_url = url + i.get('href')
            quote_content = requests.get(quote_url).content
            quote_soup = BeautifulSoup(quote_content, 'html.parser')
            d['quote'] = quote_soup.find('p', class_= "b-qt").text
            d['author'] = str(quote_soup.find('p', class_= "bq-aut").text).strip()
            list.append(d)
            

        return jsonify(list)

if __name__ == "__main__":
    app.run(debug=True)

Please help me. Why I am not getting any value in json. My list is empty. And also Query_links is null. Is there any syntax mistake or anything else?

Please take a look at this https://stackoverflow.com/a/34019493/9592801 I hope this helps. — Shakeel, Sep 04 '20 at 11:15

score 1 · Accepted Answer · answered Sep 04 '20 at 11:11

Your ready_url variable ends up having a double slash in it (i.e. https://www.brainyquote.com//search_results?q=testing). If you test that in a browser or with curl, you'll see that yields no results. If you fix the definition of url so it doesn't have the trailing slash (i.e. url='https://www.brainyquote.com'), your code will work.

How to extract anchor class using beautifulsoup?

1 Answers1