Error : ModuleNotFoundError: No module named 'packaging

Question

I am trying to run this repository: https://github.com/facebookresearch/CodeGen

I tried running it on both Windows and Ubuntu, but I am facing same error in both when I run this command present in install_env.sh.

pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

I checked and I already have the packaging library installed, but the command still says no module named 'packaging'. I tried uninstalling and reinstalling, but I still get the same error. How can I fix it?

Error

Using pip 23.1.2 from /home/meghna/miniconda3/envs/codegen_3_10/lib/python3.10/site-packages/pip (python 3.10)
DEPRECATION: --build-option and --global-option are deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use --config-settings. Discussion can be found at https://github.com/pypa/pip/issues/11859
WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option.
Processing /home/meghna/CodeGen/apex
  Running command pip subprocess to install build dependencies
  Collecting setuptools
    Using cached setuptools-68.0.0-py3-none-any.whl
  Collecting wheel
    Using cached wheel-0.40.0-py3-none-any.whl
  Installing collected packages: wheel, setuptools
  Successfully installed setuptools-68.0.0 wheel-0.40.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "/home/meghna/miniconda3/envs/codegen_3_10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/meghna/miniconda3/envs/codegen_3_10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/meghna/miniconda3/envs/codegen_3_10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-tznejqwh/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-tznejqwh/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-tznejqwh/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 4, in <module>
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/meghna/miniconda3/envs/codegen_3_10/bin/python /home/meghna/miniconda3/envs/codegen_3_10/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpnoh7u27d
  cwd: /home/meghna/CodeGen/apex
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Answer 1

Option 1:

vim /usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py

sys.path.append('/usr/local/lib/python3.10/dist-packages')

Option 2:

vim pyproject.toml

[build-system]
requires = ["packaging"]

Answer 2

You are encountering an issue with the packaging module when trying to install the 'apex' package from the FacebookResearch/CodeGen repository. This error is likely due to a compatibility problem with the specific version of packaging required by 'apex' and the version installed in your environment.

To resolve this issue, try the following steps:

Step 1: Upgrade the packaging package to the latest version:

pip install --upgrade packaging

Step 2: Reattempt the installation of the apex package:

pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./CodeGen/apex

If upgrading packaging does not solve the problem, it's possible that the 'apex' package has specific requirements for certain versions of its dependencies. In this case, consider creating a new virtual environment and installing the dependencies there.

To create a new virtual environment, follow these steps:

Step 1: Create a new virtual environment:

conda create -n codegen_venv python=3.10

Step 2: Activate the virtual environment:

conda activate codegen_venv

Step 3: Install the required dependencies and then attempt to install apex:

pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./CodeGen/apex

Using a separate virtual environment ensures that the installation process does not interfere with your system-wide packages and provides a clean environment to install and run the repository.