I tried everything to set up Web Scraping Libraries (installing requests, lxml and bs4) but just can't get it done

Question

I tried everything to set up Web Scraping Libraries (installing requests, lxml and bs4) but just can't get it done.

---

My specs -

• Windows 10 Pro
• Python version - Python 3.8.3
• Using Jupyter Notebook - C:\Users\LENOVO\anaconda3\Scripts\jupyter.exe
• Jupyter version - jupyter core: 4.6.3

• jupyter-notebook : 6.0.3
• qtconsole : 4.7.5
• ipython : 7.16.1
• ipykernel : 5.3.2
• jupyter client : 6.1.6
• jupyter lab : 2.1.5
• nbconvert : 5.6.1
• ipywidgets : 7.5.1
• nbformat : 5.0.7
• traitlets : 4.3.3

---

• I installed through pip install - It said requirements already satisfied (Through both Anaconda and Command Prompt)

• I restarted the system. It didn't work with Jupyter.

• I also tried installing the virtual environment package - It was also installed Shows the following error - ModuleNotFoundError

• I Googled it and found the following details -

  python -m pip install requests - (Said 'requirements already satisfied')
  
  conda install -c anaconda requests - (Said 'Collecting package metadata
  

  (current_repodata.json): done
  Solving environment: done)
  

What Should I do now?

Answer 1

• First, run your jupyter notebook and print sys.path.
• Check if your libraries are installed in the directories that your modules reside in, which can be retrieved by running conda list or pip show [module name].
• If not, prepend the path to the module to sys.path and try the import again.
  More info

More about the python import system.

Answer 2

I did the following as advised -

sys.path

['C:\Users\LENOVO', 'C:\Users\LENOVO\anaconda3\envs\Python 38\python38.zip', 'C:\Users\LENOVO\anaconda3\envs\Python 38\DLLs', 'C:\Users\LENOVO\anaconda3\envs\Python 38\lib', 'C:\Users\LENOVO\anaconda3\envs\Python 38', '', 'C:\Users\LENOVO\anaconda3\envs\Python 38\lib\site-packages', 'C:\Users\LENOVO\anaconda3\envs\Python 38\lib\site-packages\win32', 'C:\Users\LENOVO\anaconda3\envs\Python 38\lib\site-packages\win32\lib', 'C:\Users\LENOVO\anaconda3\envs\Python 38\lib\site-packages\Pythonwin', 'C:\Users\LENOVO\anaconda3\envs\Python 38\lib\site-packages\IPython\extensions', 'C:\Users\LENOVO\.ipython']

pip show [bs4]

Note: you may need to restart the kernel to use updated packages. 'C:\Users\LENOVO\anaconda3\envs\Python' is not recognized as an internal or external command, operable program or batch file.

pip show [requests]

Note: you may need to restart the kernel to use updated packages. 'C:\Users\LENOVO\anaconda3\envs\Python' is not recognized as an internal or external command, operable program or batch file.

conda list enter image description here Note: you may need to restart the kernel to use updated packages. EnvironmentLocationNotFound: Not a conda environment: C:\Users\LENOVO\anaconda3\envs\Python

I'm guessing the issue might be lying here. But I don't know how to resolve it. I'm using the one marked in Blue - Jupyter Notebook (Python38) (1).

Do I need to uninstall or delete anything ?

Screenshot from Jupyter Notebook

Answer 3

Finally it worked. I did it the following way -

I checked the path I was working on using -

import os

os.getcwd()

For me --> 'C:\Users\LENOVO'

Then from the command prompt I checked the path where 'site-packages' are installed using

pip3 install requests

(for python3) (This gives the path in cmd prompt after installing)

For me --> Requirement already satisfied: requests in c:\users\lenovo\anaconda3\lib\site-packages (2.24.0)

I moved the whole folder(site-packages) from installed path (where 'site-packages' are installed) to the destination path (where I want it to be).

There are a few ways to do it. I used -

import shutil

shutil.move('source path','destination path')

For me --> shutil.move('c:\users\lenovo\anaconda3\lib\site-packages','C:\Users\LENOVO')

After that go to the 'site-packages' folder, open a kernel and you can import requests , import bs4 and start working!