How to program Go to use a proxy when using a custom transport?

Question

How to write Go programs that use a proxy automatically according to the proxy environment variables?

The go get itself support the standard proxy environment variables, but i'm talking about the Go program/code itself.

This blog says,

By default http.Client checks the HTTP_PROXY and HTTPS_PROXY variables before processes any http.Request.

I tried it, but it doesn't work for my following code:

tr := &http.Transport{
    TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
resp, err := client.Get(url)

How are you creating your custom transport? Note the quoted doc says "by default", but you're overriding the defaults. — Adrian, Aug 14 '18 at 16:20
@Peter, _"ProxyFromEnvironment returns the URL of the proxy"_, that should be the same as what I specified in the terminal. I.e., the problem still holds, "_The go get itself support the standard proxy environment variables, but i'm talking about the Go program/code itself._" — xpt, Aug 14 '18 at 18:13
@xpt, but you are not using it when creating the Transport. You have to assign it to the Proxy field. — Peter, Aug 14 '18 at 18:15

score 9 · Accepted Answer · answered Nov 08 '18 at 07:02

You can use http.ProxyFromEnvironment method

  var PTransport = & http.Transport { Proxy: http.ProxyFromEnvironment }
  client: = http.Client { Transport: PTransport }

ProxyFromEnvironment returns the URL of the proxy to use for a given request, as indicated by the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions thereof). HTTPS_PROXY takes precedence over HTTP_PROXY for https requests.

I have tried below code, it works, Just add in ur proxy details in terminal.

export http_proxy='http://user:password@prox-server:3128'
export https_proxy='http://user:password@prox-server:3128'
export HTTP_PROXY='http://user:password@prox-server:3128'
export HTTPS_PROXY='http://user:password@prox-server:3128'

package main

import (
  "fmt"
  "net/http"
  "io/ioutil"
)

func main() {

  var PTransport = & http.Transport {
    Proxy: http.ProxyFromEnvironment
  }
  client: = http.Client {
    Transport: PTransport
  }
  req, err: = http.NewRequest("GET", "https://jsonplaceholder.typicode.com/todos/1", nil)
  req.Header.Add("If-None-Match", `some value`)
  resp, err: = client.Do(req)
  if err != nil {
    panic(err)
  }
  defer resp.Body.Close()

  bodyBytes, err: = ioutil.ReadAll(resp.Body)
  if err != nil {
    panic(err)
  }

  bodyString: = string(bodyBytes)
  fmt.Printf("GET Response = %s \n", string(bodyString))


}

Thanks a lot Chinmay! I wish I could give you more point than the +25. — xpt, Nov 08 '18 at 18:34
`Run code snippet` does not work for this -- it is setup for JavaScript, not Go. — Brent Bradburn, Sep 04 '19 at 21:04

Charlie Tumahai · Answer 2 · 2019-09-12T23:00:14.983

5

Set the Transport Proxy field to http.ProxyFromEnvironment to use a proxy configured from environment variables:

tr := &http.Transport{
  Proxy: ProxyFromEnvironment,
  TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}

It's usually best to clone the default transport and set your options in that transport:

tr := http.DefaultTransport.(*http.Transport).Clone()
tr.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
client := &http.Client{Transport: tr}
resp, err := client.Get(url)

The default transport includes the correct value for the Proxy field, timeouts and other useful settings.

If the application is not creating a custom transport, then none of this is required to honor the proxy related environment variables.

edited Sep 12 '19 at 23:00

answered Aug 14 '18 at 19:20

Charlie Tumahai

113,709
12
249
242

1

Note that `DefaultTransport` already includes `Proxy: ProxyFromEnvironment`, so the initial statement is kind of misleading -- if you aren't constructing a non-default transport, *you* may not need to set the proxy at all (as demonstrated in the second example). An even simpler example that will use a proxy: `client := &http.Client{}` – Brent Bradburn Sep 04 '19 at 21:01
I upvoted this approach because the second example looked smart. Unfortunately, it doesn't compile: `invalid indirect of http.DefaultTransport (type http.RoundTripper)`. Related: [net/http: add Transport.Clone](https://github.com/golang/go/issues/26013) – Brent Bradburn Sep 04 '19 at 21:35

How to program Go to use a proxy when using a custom transport?

2 Answers2

Linked