How do I make Poetry-based Jupyter use IRkernel installed using renv?

Question

I have the following project layout:

. my-project
|- pyproject.toml
|- poetry.lock
|- .jupyter
|  '- jupyter_notebook_config.py
|- kernels
|  '- R-renv
|     '- kernel.json
|- .renv
|- renv.lock
|- .Rprofile
'- # other stuff

IRkernel is installed in the renv using:

R -e 'install.packages("renv"); renv::activate(); renv::install("IRkernel"); IRkernel::installspec(); renv::snapshot(type = "all")'

Jupyterlab is installed through Poetry.

[tool.poetry]
name = "my-project"
version = "0.1.0"

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.dev-dependencies]
jupyterlab = "^3.2.8"
jupytext = "^1.13.6"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

In jupyter_notebook_config.py, I have:

c.NotebookApp.contents_manager_class="jupytext.TextFileContentsManager"
c.NotebookApp.notebook_dir = '/Users/X/Code/my-project/kernels' # absolute directory for debugging.
c.ContentsManager.default_jupytext_formats = ".ipynb,.Rmd"

In kernel.json, I have:

{"argv": ["R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"],
 "display_name":"R renv",
 "language":"R"
}

based on this answer.

However, Jupyter does not seem to be picking up the kernel:

❯ poetry run jupyter kernelspec list
Available kernels:
  ir         /Users/varun/Library/Jupyter/kernels/ir
  python3    /Users/varun/Library/Caches/pypoetry/virtualenvs/sg-data-analysis-ywZVJcuX-py3.9/share/jupyter/kernels/python3

If I try poetry run jupyter notebook, I run into errors when trying to open a .Rmd file, because it complains about IRkernel not being found, which is consistent with the result of kernelspec list.

Is there a way I can point the Poetry-based Jupyter to the renv-based IRkernel, ideally without modifying any files outside the project directory?