I am assuming by access you mean ssh or sftp access. Here's a quick hack I tried:
Mount as sftp folder(ssh+ftp)
If your servers are already mounted then skip to the next step. You can mount it through UI (very easy for ubuntu) or use sshfs+fuse.
Open this sftp folder as a local terminal(as opposed to remote terminal
Again I did this through UI but you can consult the above link to do it through terminal.
Your command line will be something like:
nihal@thinkpad:/run/user/1000/gvfs/sftp:host=<host_ip_alias>/home/nihal$
From here navigate to the folder where your venv is situated.
nihal@thinkpad:/run/user/1000/gvfs/sftp:host=<host_ip_alias>/home/nihal$ cd /path/to/venv/
Add the kernel using ipykernel package:
$ . venv/bin/activate
(venv)$ pip install ipykernel
(venv)$ ipython kernel install --user --name=remote-kernel
(venv)$ deactivate
Now this env will be available under the name remote-kernel
in jupyterlab/notebook kernel menu. You can use this kernel whenever the remote server is mounted as sftp folder.