0

New to coding, trying to write a spider in python.

I got an error that a variable is not defined.

I've defined it in the code. It is declared as global.

import requests

from bs4 import BeautifulSoup


def get_products():
    headers = {
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36',
        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
        'accept-encoding':'gzip, deflate, br',
        'accept-language':'zh-CN,zh;q=0.9'
    }

    for i in range(0,10):
        link = 'https://shopee.sg/miniso_singapore?page='+ str(i)
        r = requests.get(link, headers=headers)

        soup = BeautifulSoup(r.text, "lxml")
    global name,price,soldnum
    product_list=soup.find_all("div",class_="shop-search-result-view__item col-xs-2-4")  
    for each in product_list:       
        name = each.find("div",class_="_1NoI8__2gr36I")
        name=name.text 

        price = each.find("span",class_="_341bF0")
        price=price.text

        soldnum=each.find("div",class_="_18SLBt")
        soldnum=price.text

    print(name,price,soldnum)


get_products()
num3ri
  • 822
  • 16
  • 20
Poppy Bee
  • 154
  • 2
  • 3
  • 11
  • 4
    `name` is not defined if `product_list` is empty, which is probably the case here. Also, the `print` should probably go inside the loop, otherwise you will only print the values from the last iteration. – tobias_k May 09 '19 at 10:38
  • 2
    plase use `print` in `for` loop with Tab – Wang Liang May 09 '19 at 10:39
  • I've tried print(product_list) and the list is empty. do not know what errors in the lines:product_list=soup.find_all("div",class_="shop-search-result-view__item col-xs-2-4") – Poppy Bee May 10 '19 at 07:42

1 Answers1

0

First global variables are evil. So you should try not to use the globals keyword.

In python you define a variable by assigning a value to it. e.g.

variable_x = 10

You cannot say this variable exists, but don't assign a value to it, like in other programming languages.

Your problem is, that you assign a value only to the variable in the for loop, but if you never enter it, there is no value assigned.

I think you have to add the print inside the for loop.

for product in product_list:       
        name = product.find("div",class_="_1NoI8__2gr36I")
        name=name.text 

        price = product.find("span",class_="_341bF0")
        price=price.text

        soldnum=product.find("div",class_="_18SLBt")
        soldnum=price.text

        print(name, price, soldnum)
Uli Sotschok
  • 1,206
  • 1
  • 9
  • 19