Gitlab-CI: set different variables/paths depending on specific runner

Question

I am using Gitlab CI to automate my unity builds. For this I use two machines. (one Mac and one windows machine. Important: build should only run on the first machine that is available. )

unity-build:
  script: "/Applications/Unity/Hub/Editor/2019.3.14f1/Unity.app/Contents/MacOS/Unity \      
  -projectPath pathXY \
  -executeMethod BuildScript.PerformBuild
  -quit"
  stage: build
  tags:
    - unity-mac-runner, unity windows runner

The problematic line is this: script: "/Applications/Unity/Hub/Editor/2019.3.14f1/Unity.app/Contents/MacOS/Unity, because on my windows runner path to unity is "S:\Unity\2019.3.14f1\Editor\Unity.exe" \

Tried to fix this with these changes:

windows job:
  stage: setpaths
  variables:
    UNITY_PATH: "S:/Unity/2019.3.14f1/Editor/Unity.exe"
  script: "set windows variables..."
  tags:
    - unity windows runner
#osx job:
#  stage: setpaths
#  variables:
#    UNITY_PATH: "S:/Unity/2019.3.14f1/Editor/Unity.exe"
#  script: "set mac variables..."
#  tags:
#    - unity-mac-runner

Questions

How can I do this? What should I use?
I tried to set path of unity installation depending on build runner, but I couldn't get this to work.

An example would be nice. Thx a lot ;)

Addition to the config.toml solution: I have this problem on Windows (Mac is running without problems.

     In C:\WINDOWS\TEMP\build_script782418789\script.ps1:185 Zeichen:15
 + ${UNITY_PATH} -batchmode -projectPath ${BUILD_PROJECT_PATH} -runTests ...
 +               ~~~~~~~~~~
 Unerwartetes Token "-batchmode" in Ausdruck oder Anweisung.
 In C:\WINDOWS\TEMP\build_script782418789\script.ps1:185 Zeichen:26
 + ${UNITY_PATH} -batchmode -projectPath ${BUILD_PROJECT_PATH} -runTests ...
 +                          ~~~~~~~~~~~~
 Unerwartetes Token "-projectPath" in Ausdruck oder Anweisung.
     + CategoryInfo          : ParserError: (:) [], ParseException
     + FullyQualifiedErrorId : UnexpectedToken
  
 ERROR: Job failed: exit status 1

Unexpected token ... in expression or statement.

This is my config.toml on windows:

  executor = "shell"
  shell = "powershell"
  environment = [
  "UNITY_PATH=S:/Unity/2019.4.1f1/Editor/Unity.exe",
  "BUILD_PROJECT_PATH=S:/Gitlab-runner/builds/QZ5_yEjt/0/FussballManager/Game",
  "UNITY_BUILD_PATH=S:/Gitlab-runner/builds/QZ5_yEjt/0/FussballManager/Game/WindowsBuild/"
  ]

Rekovni · Answer 1 · 2020-07-01T10:54:56.947

0

One option is to split out the different macOS and Windows jobs, so something along the lines of:

build:mac:
  stage: build
  script:
    - "/Applications/Unity/Hub/Editor/2019.3.14f1/Unity.app/Contents/MacOS/Unity \      
        -projectPath pathXY \
        -executeMethod BuildScript.PerformBuild
        -quit"
  tags:
    - unity-mac-runner


build:windows:
  stage: build
  script:
    - "S:/Unity/2019.3.14f1/Editor/Unity.exe \      
        -projectPath pathXY \
        -executeMethod BuildScript.PerformBuild
        -quit"
  tags:
    - unity-windows-runner

You can also use templates/anchors if the script is very similar, eg:

.unity_template:
  stage: build
  script:
    - "${UNITY_PATH} -projectPath ${PROJECT_PATH} \
        -executeMethod BuildScript.PerformBuild
        -quit"

build:mac:
  extends: .unity_template
  variables:
    UNITY_PATH: "/Applications/Unity/Hub/Editor/2019.3.14f1/Unity.app/Contents/MacOS/Unity"
    PROJECT_PATH: "pathXY"
  tags:
    - unity-mac-runner

build:windows:
  extends: .unity_template
  variables:
    UNITY_PATH: "S:/Unity/2019.3.14f1/Editor/Unity.exe"
    PROJECT_PATH: "pathXY"
  tags:
    - unity-windows-runner

This would setup two jobs for the pipeline, and require that both runners are available.

Another option, is as you have mentioned, setting the path on the runner themselves. I'm not sure how you have it setup, but for example:

config.toml file for the mac:

[[runners]]
  ...
  environment = ["UNITY_PATH=/Applications/Unity/Hub/Editor/2019.3.14f1/Unity.app/Contents/MacOS/Unity"]
  ...

config.toml file for windows:

[[runners]]
  ...
  environment = ["UNITY_PATH=S:/Unity/2019.3.14f1/Editor/Unity.exe"]
  ...

and then in the CI file, it should be something similar to option one with:

unity_build:
  stage: build
  script:
    - "${UNITY_PATH} \      
        -projectPath pathXY \
        -executeMethod BuildScript.PerformBuild
        -quit"
  tags:
    - unity

See the advanced configuration for more information on editing the config.toml.

This would only setup one job for the pipeline, and use either of the unity runners available (ensure that both runners have the unity tag).

HTH!

edited Jul 01 '20 at 10:54

answered Jun 30 '20 at 11:05

Rekovni

6,319
3
39
62

Thx for your help. I first tried your second solution: .unity_template: But on my gitlab CI/CD page, this commit is marked as stuck. If I am clicking on the build Stage that is not completed I can see that my mac runner finished but I am waiting for my windows runner (that is offline right now). Is there a way that this build goes to the next stage if one of the jobs was succesful? (I don't need windows runner and mac at the same time. ) – basti12354 Jun 30 '20 at 12:40
The first option (and template option) requires both runners to be available. I've updated the 3rd option of updating the config.toml's for both runners, let me know how that works? – Rekovni Jun 30 '20 at 13:39
If I am using this : tags: - unity-mac-runner - unity-windows-runner. It will also try to run on both machines I suppose? – basti12354 Jun 30 '20 at 13:55
Ah, gitlab will try find a machine with both tags. You'd probably want to assign another tag to both the machines like `unity`. I've updated my answer again. – Rekovni Jun 30 '20 at 14:08
I searched a couple of hours how to use this config.toml file. Right now I have no idea how to setup this correctly. Can I add these files to my existing runners and where should I put them? Do I have to remove existing runners and register them again? I searched for these files on my mac and couldnt find any of them so I suppose current runner has not such a file? – basti12354 Jul 01 '20 at 09:51
So you should already have a `config.toml` file setup for both your runners. On macOS, depending if you set it up as `root` or not, it should be in `~/.gitlab-runner/config.toml`. And then you can manually edit this file and add the `environment` variable I've mentioned into the `[[runners]]` section. You do not need to remove the existing runner and re-register them. Once you've manually made the change (using nano for example), the runner should automatically restart. If not, run `gitlab-runner restart` and that will pick up the new `config.toml` – Rekovni Jul 01 '20 at 10:51
For windows, the `config.toml` is where you set it from the [instructions](https://docs.gitlab.com/runner/install/windows.html), so from the example there, it is in `C:\GitLab-Runner\config.toml`. You might need to restart the gitlab-runner windows service to get the change however (not sure on this). – Rekovni Jul 01 '20 at 10:53
Okay your third solution is working on Mac, but on windows there is a problem (I added problems to my question text.) Unexptected token -batchmode in expression or statement. – basti12354 Jul 01 '20 at 22:02
Hmm, this might be an issue with the slashes in the Windows path? From what I can tell from the [unity cli docs](https://docs.unity3d.com/Manual/CommandLineArguments.html), it uses backslash, so you might want to try that in the `UNITY_PATH` instead? Might also be worth `echo`-ing out the command before calling it in windows so you can see what the command looks like? – Rekovni Jul 01 '20 at 22:31
changed slash to backslash on windows. Same result. Echoing result: $ echo 'Building...${UNITY_PATH} -batchmode -projectPath ${BUILD_PROJECT_PATH} -runTests -testPlatform editmode -logFile -testResults ./edit-mode-tests.xml' RESULT: Building...${UNITY_PATH} -batchmode -projectPath ${BUILD_PROJECT_PATH} -runTests -testPlatform editmode -logFile -testResults ./edit-mode-tests.xml (same result on Mac and WIndows, so I can not read my variables when using echo) – basti12354 Jul 02 '20 at 06:47
It might be worth just testing on windows for now (so just tag `unity-windows-runner`), and do you know if you're using powershell? if so you'd need to use [Write-Host](https://stackoverflow.com/questions/707646/echo-equivalent-in-powershell-for-script-testing) to print out the command? We need to see how the command on windows is being expanded as something could be funny going on here. Either that, or see if you can use `${UNITY_PATH }` with an different command to see if that works – Rekovni Jul 02 '20 at 10:06
I can read in the config.toml that I am using powershell. Now I run into the next problem. I tried to restart the runner and now he is no longer responding. I am not sure if this is worth that much work. I used teamcity in the past and this was much less pain – basti12354 Jul 03 '20 at 07:51
Can you post your windows `config.toml` file? If using backslash, make sure it's double backslashed I suppose? ie `"UNITY_PATH=S:\\Unity\\2019.3.14f1\\Editor\\Unity.exe"` Or if you can get the `gitlab-runner` error log from Windows event viewer to see what it's complaining about? – Rekovni Jul 03 '20 at 09:19
1

I will work on this next weekend (I am currently on holidays and have no access to my windows computer). I will give the award to you, but I will wait to accept this answer. (It seems that your solution is right but there is some weird windows setting we are missing right now ;) ) – basti12354 Jul 04 '20 at 22:59

Gitlab-CI: set different variables/paths depending on specific runner

1 Answers1