Install Python to Self-hosted Windows build agent

Question

I have installed Windows agent and I need to be able run Python scripts. I know I need to install Python, but I have no idea how.

I added Python files from standard installation to

$AGENT_TOOLSDIRECTORY/
    Python/
        3.8.2/
            x64/
                {tool files}
            x64.complete

Restarted agent, but what now? How to put it to Capabilities? What I'm missing?

EDIT: I need to run this YAML task

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.x'
    addToPath: true

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  displayName: 'Install dependencies'

- task: BatchScript@1
  displayName: 'Run script make.bat'
  inputs:
    filename: make.bat
    arguments: html

Answer 1

I have set up a self-hosted agent on a Windows 10 laptop, (for which I have admin access), and I'm running Azure DevOps Express 2020.

I found, downloaded and installed an agent according to the instructions at Download and configure the agent. I used vsts-agent-win-x64-2.170.1.zip and set this up to run as a service, (I guess anyone running it manually needs to double check that it's runnning at show time). I also ran the install command as admin in powershell!

To install a Python version I need to download the appropriate installer from the ftp site at Python.org, eg. for 3.7.9 I've used python-3.7.9-amd64.exe. I then run this from the command line (CMD run as administrator) without UI with: python-3.7.9-amd64.exe /quiet InstallAllUsers=0 TargetDir=$AGENT_TOOLSDIRECTORY\Python\3.7.9\x64 Include_launcher=0 (other options for install available in python docs)

Once this is complete, (and it runs in background so will take longer than the initial command), you need to create an empty {platform}.complete file (as described here), in my case this is x64.complete.

This then worked! I did restart the server for this first version, but I've added other python versions since without needing to. My pipeline task was simply:

steps:
- task: UsePythonVersion@0
  displayName: 'Use Python $(python.version)'
  inputs:
    versionSpec: '$(python.version)'

(with a variable python.version set us as a list of versions 3.7.9, 3.8.8)

One key element for me was the file structure, where the documentation says {tool files} this means the python.exe file and other common dirs such as Lib and Scripts. I initially installed these in a sub-dir which didn't work. So it should look like this:

$AGENT_TOOLSDIRECTORY/
    Python/
        3.7.9/
            x64/
                Doc/
                Lib/
                Scripts/
                python.exe
                ...etc...
            x64.complete

To be honest I'm mostly relieved that this worked without too much trouble. I gave up trying to get Artifacts to work on-prem. In my limited experience all of this is much easier, and better, on the cloud version. Haven't yet persuaded my employer to take that leap however...

Answer 2

The way I install python in my own self-hosted agent that you can refer.

parameters:
- name: projecWorkingDirectory
  displayName: 'Working directory of project'
  default: $(System.DefaultWorkingDirectory)

- name: pythonVersion
  displayName: 'Python version'
  default: '3.10.12'

steps:
- script: |
    mkdir -p $(Agent.ToolsDirectory)/Python/${{ parameters.pythonVersion }}/x64
    mkdir -p $(Agent.ToolsDirectory)/Python/${{ parameters.pythonVersion }}/x64.complete
  displayName: 'Create Python local folders'

- script: |
    curl -LO https://github.com/actions/python-versions/releases/download/3.10.12-5200619051/python-3.10.12-linux-22.04-x64.tar.gz
    tar -xvf python-3.10.12-linux-22.04-x64.tar.gz
  workingDirectory: $(Agent.ToolsDirectory)/Python/${{ parameters.pythonVersion }}/x64
  displayName: 'Downloaded'

- task: UsePythonVersion@0
  inputs:
    versionSpec: ${{ parameters.pythonVersion }}
    architecture: 'x64'
    disableDownloadFromRegistry: true
  displayName: 'Use Python ${{ parameters.pythonVersion }}'

- script: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
  workingDirectory: ${{ parameters.projecWorkingDirectory }}
  displayName: 'Install dependencies'

Answer 3

For this issue, in order to use the python version installed in your on-premise machine. You either need to point to the python.exe physical path in cmd task. Or add the python.exe path to environment variable path manually in powershell task. For example:

To use local python in powershell task:

$env:Path += ";c:\{local path to}\Python\Python38\; c:\{local path to}\Python\Python38\Scripts\"
python -V

To use python in CMD task:

c:\{local path to}\Python\Python38\python.exe -V
c:\{local path to}\Python\Python38\Scripts\pip.exe install

So, I think to run python script with private agent, just make sure python is installed locally, then point to python.exe path. You can refer to this case for details.

Answer 4

I added these 4 Tasks before being able to execute python on my pipeline with a vs2017-win2016 agent:

Use Python 3.x

steps:
- task: UsePythonVersion@0
  displayName: 'Use Python 3.x'

Use Pip Authenticate

steps:
- task: PipAuthenticate@1
  displayName: 'Pip Authenticate'

Use Commandline task

steps:
- script: |
   python -m pip install --upgrade pip setuptools wheel


  failOnStderr: true
  displayName: 'install pip for setup of python framework'

Use Commandline task

steps:
- script: 'pip install -r _python-test-harness/requirements.txt'
  failOnStderr: true
  displayName: 'install python framework project''s specific requirements'

Hope that helps