I use git hooks to make conda environment updates automatic. You can have more information on git hooks here.
The idea here is to have two git hooks:
- One which detects if a change in your local conda environment occured and if so, create a new commit with the updated env.yml file (I chose a pre-push hook for this one).
- One which detects a change in env.yml file after a pull (i.e. the remote env.yml was different than the local one and was merged, I chose a post-merge hook for this one)
As described in the documentation, when a git repository is initiated, a folder .git/hooks is created and filled with example scripts. To use one of them, you only have to edit the file, rename it to remove its extension (.sample) and make sure it is executable.
NOTE: I use zsh as shell but the script should be the same in bash (please comment if not), you would just need to change the shebang line.
pre-push hook
- Rewrite the pre-push.sample file already present in .git/hooks (replace
<ENV_NAME> by the name of your conda environment):
#!/usr/bin/env zsh
echo "\n==================== pre-push hook ===================="
# Export conda environment to yaml file
conda env export -n <ENV_NAME> > env.yml
# Check if new environment file is different from original
git diff --exit-code --quiet env.yml
# If new environment file is different, commit it
if [[ $? -eq 0 ]]; then
echo "Conda environment not changed. No additional commit."
else
echo "Conda environment changed. Commiting new env.yml"
git add env.yml
git commit -m "Updating conda environment"
echo 'You need to push again to push additional "Updating conda environment" commit.'
exit 1
fi
- Remove its extension .sample and make it executable if necessary (
chmod u+x pre-push)
post-merge hook
#!/usr/bin/env zsh
echo "\n==================== post-merge hook ===================="
changed_files="$(git diff-tree -r --name-only --no-commit-id ORIG_HEAD HEAD)"
check_run() {
echo "$changed_files" | grep --quiet "$1" && eval "$2"
}
echo "Have to update the conda environment"
check_run env.yml "conda env update --file env.yml"
- And make it executable (
chmod u+x post-merge)
What will happen now ?
- When pushing, if the conda environment changed, a message will show that you have to push again to push the commit with the updated env.yml
- When pulling, if the pulled env.yml differs from the local env.yml, conda will update the local environment with the newly pulled env.yml.
Limitations
- In case the environment changed locally, you can see that the updated env.yml is not automatically pushed to remote. I took the advice from this post git commit in pre-push hook.
- Currently the updating of the conda environment after pull is using a post-merge hook. I don't know how this will be handled in case of
rebase for example.
- No git expert here, maybe there is hooks better suited for these tasks.
- I noticed a
prefix section in the env.yml which give the path to your environment folder on your local machine. After some test, everything seems to run fine but I don't know if this could somehow create conflicts when developing on various machines.
So ... comments, corrections and ideas of improvements are more than welcome !
