0

i am working on a code which is extracting data from web pages

# first is task.py
import requests
from bs4 import BeautifulSoup

url = ('https://www.naukri.com/job-listings-Python-Developer-Cloud-Analogy-Softech-Pvt-Ltd-Noida-Sector-63-Noida-1-to-2-years-250718003152?src=rcntSrchWithoutCount&sid=15327965116011&xp=1&px=1&qp=python%20developer&srcP 
ge=s')
response = requests.get(url)
page = response.text
soup = BeautifulSoup(page, 'html.parser')
links = soup.find_all("div", {"id":"viewContact"})
for link in links:
    print(link.text)

I want to retrieve the contact detail this page. which is in the bottom of the page "View Contact Details" the web page contains:

<div class="jDisc viewContact" id="viewContact" style="display: block;"><p> 
<em>Recruiter Name:</em><span>Malika Pathak, Himani Adhikari</span></p><p> 
<em>Contact Company:</em><span>Cloud Analogy Softech Pvt Ltd</span></p><p> 
<em>Address:</em><span>H-77, H Block, Sector 63, Noida, UP-201307NOIDA,Uttar 
Pradesh,India 201307</span></p><p><em>Email Address:</em><span><img 
title="himani.adhikari@cloudanalogy.com , malika.pathak@cloudanalogy.com" 
src="data:image/jpeg;base64,"></span></p><p><em>Website:</em><a 
target="_blank" 
rel="nofollow" href="http://cloudanalogy.com/">http://cloudanalogy.com/</a> 
</p> 
<p><em>Telephone:</em><span>9319155392</span></p></div>

I get nothing in the result

TylerH
  • 20,799
  • 66
  • 75
  • 101
VIKAS RATHEE
  • 97
  • 2
  • 13

1 Answers1

2

For the first link, you can access the information via the recSum div:

import requests, re
from bs4 import BeautifulSoup
d = soup(requests.get('https://www.naukri.com/job-listings-Python-Developer-Cloud-Analogy-Softech-Pvt-Ltd-Noida-Sector-63-Noida-1-to-2-years-250718003152?src=rcntSrchWithoutCount&sid=15327965116011&xp=1&px=1&qp=python%20developer&srcP%20ge=s').text, 'html.parser')
results = [i.text for i in d.find('div', {'class':'recSum'}).find_all(re.compile('p|span'))]
print(dict(zip(['name', 'title', 'company', 'location', 'followers'], results)))

Output:

{'name': ' Malika Pathak Senior Human Resource Executive Cloud Analogy Softech Pvt Ltd Noida ', 'title': 'Senior Human Resource Executive', 'company': 'Cloud Analogy Softech Pvt Ltd', 'location': 'Noida', 'followers': '11'}

For the second link, however, you are attempting to access a password protected mail server. For that, you will need to either send you account credentials via requests or use a mail connection client such as smtplib.

Ajax1234
  • 69,937
  • 8
  • 61
  • 102
  • thanks for this help! please tell me what is the use of [ find_all(re.compile('p|span'))] and this [i.text] in the code – VIKAS RATHEE Jul 28 '18 at 18:06