It seems that now it works. You should be able to achieve that:
user@host:~/ source path/to/venv/bin/activate
(venv) user@host:~/ python3 -m pip install ipython
(venv) user@host:~/ ipython
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import sys
In [2]: sys.executable
Out[2]: '/path/to/env/bin/python3'
In [3]:
Note that I am using python3 -m pip [...]
to ensure that the module are installed in the environment and not elsewhere.