Gitlab CI not downloading correct submodule files

Question

I am trying to use a private git repo as a submodule in my application. Locally I can clone my main repo and pull all submodules correctly.

I am using GitLab.com for my hosting and runner. In GitLab runner, a GitHub hosted public repo pulls fine, but a private GitLab hosted repo will not pull the latest code. I am the owner of the main project as well as the private submodule.

I noticed this started after I made and pushed a change to my submodule while working on my main project.

When I attempt an automated build on the GitLab runner, the submodule will pull, but even though it shows the correct (current/latest) Git Ref SHA1, the files are shown from a commit long ago. I have attempted many iterations of ways to fetch the private repo and I have pushed insignificant changes to my submodule. Even still I am unable to pull the most recent version of the submodule code.

I have tried using the builtin GitLab runner variables to pull the submodules and I have setup the before_script to attempt a manual pull.

I have also made sure to update my local submodule and commit the changes to my main repo.

Here is using the GitLab runner variable

  variables:
    GIT_SUBMODULE_STRATEGY: recursive

Here is a sample before script, I have tried this with and without setting GIT_SUBMODULE_STRATEGY: none I have tried many variations of this removing and adding lines trying to get the most recent code.

    - git submodule sync --recursive
    - git submodule update --force --recursive --remote
    - git submodule foreach git pull
    - git submodule status
    - git submodule foreach git log -1

Expected: GitLab runner should pull the latest commit on the branch of the submodule.

Actual: GitLab pulls the submodule, but code changes after a certain commit do not show even though the shown commit is current.

score 2 · Answer 1 · answered Apr 08 '19 at 19:13

After spending all of last week trying to figure this out, I found a solution today. I found this question that assisted in my eventual solution.

Here is my gitlab-ci.yml

stages:
  - deployGAE

deploy_production:
  image: google/cloud-sdk:alpine
  stage: deployGAE
#  variables:
#    GIT_SUBMODULE_STRATEGY: recursive
  environment:
    name: Production
  only:
    - master
  before_script:
    - git submodule update --init --remote --merge
    - git submodule status
    - git submodule foreach git log -1
  script:
    # Set GCloud service account key
    - echo $SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
    # Authenticate to GCloud API
    - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
    # Globally set the GCLoud project for future commands
    - gcloud config set project $PROJECT_ID
    # Deploy the app to GCloud Build for deployment on App Engine
    - gcloud --quiet app deploy app.yaml #--verbosity=info
    # Remove versions that are no longer serving traffic
    - gcloud --quiet app versions delete $(gcloud app versions list --sort-by '~version' --format 'value(version.id)' --filter="TRAFFIC_SPLIT:0.00")
  after_script:
    # Remove the GCloud service account key
    - rm /tmp/$CI_PIPELINE_ID.json

Gitlab CI not downloading correct submodule files

1 Answers1