-2

I have a problem when using regex. When I use below regex in regex101.com the answer is correct: 

\<div style=\"width:67px; font-weight:bold;\"\>\n(.+)\<

but when I used it in below python code it returns a empty list, my code is:

import re
import requests
from bs4 import BeautifulSoup

r = requests.get('https://www.sgcarmart.com/used_cars/listing.php?MOD=audi&PRC=0&DEP=0&RGD=0&VEH=0&AVL=2')
soup = BeautifulSoup(r.text, 'html.parser')
res = soup.find_all('td', attrs= {'style':'padding:15px 0'})
ex = str(res[1])
price = re.findall(r'\<div style=\"width:67px; font-weight:bold;\"\>\n(.+)\<', ex)
print(price)
Amin
  • 43
  • 5

2 Answers2

0
import re
import requests
from bs4 import BeautifulSoup

r = requests.get('https://www.sgcarmart.com/used_cars/listing.php?MOD=audi&PRC=0&DEP=0&RGD=0&VEH=0&AVL=2')
soup = BeautifulSoup(r.text, 'html.parser')
res = soup.find_all('td', attrs= {'style':'padding:15px 0'})

ex = str(res[1])
price = re.findall('(\<div style=\"width:67px; font-weight:bold)(.+)((?:\n.+)+)(\<\/div\>)', ex)
print(price)

output is[('<div style="width:67px; font-weight:bold', ';">\r', '\n\t\t\t\t\t\t\t\t\t\t $36,500 </div>\n</td>\n<td align="center" background="https://i.i-sgcm.com/used_cars/grey_dotted_long_1x87.jpg" width="1"><img alt="spacer" height="1" src="https://i.i-sgcm.com/images/spacer.gif" width="1"/></td>\n<td align="center" valign="top" width="101">\n<div style="width:101px;">\r\n $15,980 /yr </div>\n</td>\n<td align="center" background="https://i.i-sgcm.com/used_cars/grey_dotted_long_1x87.jpg" width="1"><img alt="spacer" height="1" src="https://i.i-sgcm.com/images/spacer.gif" width="1"/></td>\n<td align="center" valign="top" width="89">\n<div style="width:89px;">\r\n\t\t\t\t\t\t\t\t\t\t03-Jun-2010 \r\n </div>\n</td>\n<td align="center" background="https://i.i-sgcm.com/used_cars/grey_dotted_long_1x87.jpg" width="1"><img alt="spacer" height="1" src="https://i.i-sgcm.com/images/spacer.gif" width="1"/></td>\n<td align="center" valign="top" width="84">\n<div style="width:84px;">\r\n\t\t\t\t\t\t\t\t\t\t1,984 cc </div>\n</td>\n<td align="center" background="https://i.i-sgcm.com/used_cars/grey_dotted_long_1x87.jpg" width="1"><img alt="spacer" height="1" src="https://i.i-sgcm.com/images/spacer.gif" width="1"/></td>\n<td align="center" valign="top" width="83">\n<div style="width:83px;">\r\n\t\t\t\t\t\t\t\t\t\t120,918 km </div>\n</td>\n<td align="center" background="https://i.i-sgcm.com/used_cars/grey_dotted_long_1x87.jpg" width="1"><img alt="spacer" height="1" src="https://i.i-sgcm.com/images/spacer.gif" width="1"/></td>\n<td align="center" valign="top" width="89">\n<div style="width:89px;">\n<a class="link_black nounderline" href="listing.php?VEH=11">Sports</a> </div>\n</td>\n<td align="center" background="https://i.i-sgcm.com/used_cars/grey_dotted_long_1x87.jpg" width="1"><img alt="spacer" height="1" src="https://i.i-sgcm.com/images/spacer.gif" width="1"/></td>\n<td align="center" valign="top" width="82">\n<div style="width:82px;">\n<strong><font color="#009900">Available</font></strong> ', '')]

Jainil Patel
  • 1,284
  • 7
  • 16
0

Remove the control characters (\r \n \t...) first. Then your regex returns values. I got the nifty snippet to remove the control characters from this post

I did remove the \n from your regular expression as it was removed when removing control characters.

import re
import requests
from bs4 import BeautifulSoup
import unicodedata

r = requests.get('https://www.sgcarmart.com/used_cars/listing.php?MOD=audi&PRC=0&DEP=0&RGD=0&VEH=0&AVL=2')
soup = BeautifulSoup(r.text, 'html.parser')
res = soup.find_all('td', attrs= {'style':'padding:15px 0'})
ex = str(res[1])
ex = "".join(ch for ch in ex if unicodedata.category(ch)[0]!="C")
price = re.findall(r'\<div style=\"width:67px; font-weight:bold;\"\>(.+)\<', ex)
print(price)
drxl
  • 364
  • 1
  • 7