https_proxy variable is causing conflicts with Azure DevOps API

Question

We are developing Azure DevOps extension with pipeline tasks. We are using typescript/node/npm packages as technologies in this. One of these pipeline task’s responsibilities is gathering data from external source with aid from embedded API wrapper and create work items based on gathered/composed data.

High level architecture of the task is as below

We are trying to support users who uses proxies by passing https_proxy variable to the wrapper (wrapper is designed to accept https_proxy variable). Code is something similar to below.

    import * as tlib from 'azure-pipelines-task-lib/task';

    tlib.setVariable(“https_proxy”, “http://username:password@hostname:port” ,false);

With above implantation we were successfully able to access external source trough API wrapper and gather necessary data.

However, it is observed that having https_proxy in pipeline task breaks communication between pipeline task and Azure DevOps API ('azure-devops-node-api) and we are getting below error.

    Error: tunneling socket could not be established, statusCode=407

We cannot identify root cause of this issue and noticed that defining https_proxy variable is configuring proxy server in Azure (Not 100% sure whether this is same to Azure DevOps as well).

https://learn.microsoft.com/en-us/dotnet/azure/sdk/azure-sdk-configure-proxy?tabs=cmd

p.s We can resolve this issue by setting no_proxy as below after consuming http_proxy for gathering data. But there is issue in this approach as this could disable all proxies. For example, let’s assume computer is config to use proxy A, but wrapper is using proxy B. with no_proxy implementation we force Azure DevOps API to use no_proxy instead of proxy A. Hence, we must think of alternative solution

    process.env["NO_PROXY"] = "*";

Highly appreciate you can share any tips and suggestions to overcome this issue

Answer 1

The variables in Azure Pipelines are transferred from the agent to the tasks using the environment. So, any variable you define will automatically turn into an environment variable in the pipeline. This is by design and can't be changed. It's also one of the safest ways to pass variables between processes, as it prevents all kinds of character escape issues.

And the variables you're setting, are how proxy configuration works in general for Node and many unix-based tools.

There are a number of magic variables: HTTP_PROXY, HTTPS_PROXY, NO_PROXY etc. And as you've figured out when you set these most tools will automatically pick them up and will start using them to redirect traffic.

If you want to temporarily change the proxy settings, you'll need to capture the original values of these variables and then restore them when your task is finished.

If you want to only set the proxy for the current task and not the rest of the pipeline, you can use process.env['name'] = value syntax.

Ideally proxy configuration is done for the agent as a whole, not in tasks specifically. And then the task can use the built-in features of the task library to retrieve the proxy settings and pass these along.

The Azure DevOps Node API should also allow you to override the proxy for specific requests, that should allow you to chat to Azure DevOps's work item API, even when a different proxy configuration is set:

let proxy = tl.getHttpProxyConfiguration();

let option: VsoBaseInterfaces.IRequestOptions = {
  // Override your proxu config if needed:
  proxy: {
    proxyUrl: proxy.proxyUrl,
    proxyUsername: proxy.proxyUsername,
    proxyPassword: proxy.proxyPassword,
    proxyBypassHosts: proxy.proxyBypassHosts
  },
   ignoreSslError: false
};
    
// Make a Rest call to VSTS/TFS
let vsts: api.WebApi = new api.WebApi(serverUrl, authHandler, option);
let connData: lim.ConnectionData = await vsts.connect();

Answer 2

As per the response from Microsoft team, currently there is no option to achieve this.

As a work around we have used different variable name rather than https_proxy to share proxy details with API wrapper

Microsoft community post:

https://developercommunity.visualstudio.com/t/https_proxy-variable-is-causing-conflict/10306599