How to handle GitHub Actions deleting its cache after 7 days

Question

I have a GitHub Action that runs tests for my Python/Django project. It caches the virtual environment that Pipenv creates. Here's the workflow with nearly everything but the relevant steps commented out/removed:

jobs:
  build:
    runs-on: ubuntu-latest

    services:
      # postgres:

    steps:
      #- uses: actions/checkout@v2
      #- name: Set up Python
      #- name: Install pipenv and coveralls

      - name: Cache pipenv virtualenv
        uses: actions/cache@v2
        id: pipenv-cache
        with:
          path: ~/.pipenv
          key: ${{ runner.os }}-pipenv-v4-${{ hashFiles('**/Pipfile.lock') }}
          restore-keys: |
            ${{ runner.os }}-pipenv-v4-

      - name: Install dependencies
        env:
          WORKON_HOME: ~/.pipenv/virtualenvs
          PIPENV_CACHE_DIR: ~/.pipenv/pipcache
        if: steps.pipenv-cache.outputs.cache-hit != 'true'
        run: pipenv install --dev

      # Run tests etc.

This works fine usually, but because caches are removed after 7 days, if this is run less frequently than that, it can't find the cache and the Install Dependencies step fails with:

FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/.pipenv/virtualenvs/my-project-CfczyyRI/bin/pip'

I then bump the cache key's version number (v4 above) and the action runs OK.

I thought the if: steps.pipenv-cache.outputs.cache-hit != 'true' would fix this but it doesn't. What am I missing?

Not familiar with `pipenv` but with many dependency installers they will use the cached data if it’s available (for a speed up) or download if it’s not. (eg `npm install`) So there’s no conditional necessary. Is this not true for `pipenv`? — Edward Thomson, May 10 '21 at 10:29
@EdwardThomson I think that is true. Most examples I've seen doing this have that conditional though. But then every example I've seen is different in some way... — Phil Gyford, May 10 '21 at 11:12
That's the correct expression per https://github.com/actions/cache#outputs. But isn't the problem that if the cache wasn't hit `pipenv install --dev` fails, in which case only running it on a cache miss is the exact opposite of the behaviour you want? On a miss you need to do _something else_ to set up the `.pipenv` directory so the other commands can work (and the cache can be refilled). — jonrsharpe, May 10 '21 at 15:17
@jonrsharpe Hmm, I *think* I do want to run `pipenv install --dev` on a cache miss, so it can refill the cache. But for some reason it can't find the directory to install the dependencies in. Maybe I've just rephrased what you said? — Phil Gyford, May 10 '21 at 16:23
I haven't used pipenv, but how is `.pipenv/virtualenvs/my-project-CfczyyRI/bin/pip` filled to begin with? — jonrsharpe, May 10 '21 at 17:01
@jonrsharpe I think, first time it's run, the "Install dependencies" step creates the virtualenv at that location. I am honestly hazy about *exactly* how a pipenv (or any other package manager) environment and dependencies are related to the cache. I feel there's some bit of "magic" in there that I'm not understanding. — Phil Gyford, May 10 '21 at 21:15

score 0 · Answer 1 · answered Nov 03 '22 at 07:11

First alternative: using a separate workflow, with a schedule trigger event, you can run workflows on a recurring schedule.
That way, you force a refresh of those dependencies in the workflow cache.
Second alternative: use github.rest.actions.getActionsCacheList from actions/github-script (as seen here) again in a separate workflow, just to read said cache, and check if it still disappear after 7 days.
Third alternative: check if reading the cache through the new Web UI is enough to force a refresh.

On that third point (Oct. 2022):

Manage caches in your Actions workflows from Web Interface

Caching dependencies and other commonly reused files enables developers to speed up their GitHub Actions workflows and make them more efficient.

We have now enabled Cache Management from the web interface to enable developers to get more transparency and control over their cache usage within their GitHub repositories.

Actions users who use actions/cache can now:

View a list of all cache entries for a repository.

Filter and sort the list of caches using specific metadata such as cache size, creation time, or last accessed time.

Delete a corrupt or a stale cache entry

Monitor aggregate cache usage for repositories and organizations.

In addition to the Cache Management UX that we have now enabled, you could also use our Cache APIs or install the GitHub CLI extension for Actions cache to manage your caches from your terminal.

Learn more about dependency caching to speed up your Actions workflows.

How to handle GitHub Actions deleting its cache after 7 days

1 Answers1

Manage caches in your Actions workflows from Web Interface