1

I would like to support requests of the form:

go get acme.com/component

where component is a git project stored in Bitbucket Server which can be retrieved via:

git clone http://acme.dev:7600/scm/project/component

The go 1.8 client does not know how to fetch the component based on the go get parameters so it does an HTTPS get (as per https://golang.org/cmd/go/#hdr-Remote_import_paths) of:

https://acme.com/component?go-get=1

In response it expects an HTTP 200 containing the following:

<head>
  <meta name="go-import" content="acme.com/component git http://acme.dev:7600/scm/project/component">
</head>

HAProxy is attractive because the Bitbucket Server SSL termination procedure uses it. It has a monitor-uri hack to return an HTTP response if it detects that the backend is responding to a specific URI but I don't think it's sufficiently flexible to handle different component names.

Does anyone know a way to handle this using HAProxy or, perhaps, another tool like NGinx?

James Fremen
  • 2,170
  • 2
  • 20
  • 29
  • There is a tool for github, with a minor change in source code, I think you can use it for bitbucket too. (since you say another tool ) https://github.com/niemeyer/gopkg – fzerorubigd May 06 '17 at 15:18
  • thanks for the suggestion. Building a custom web server would certainly work but I use a reverse proxy for a number of other tasks and i'm trying to keep the number of components down. – James Fremen May 06 '17 at 15:39
  • i see a few nginx posts i.e. http://stackoverflow.com/questions/26347516/using-go-get-on-a-personal-git-repo – James Fremen May 06 '17 at 15:58
  • This can be done solely in HAProxy, if I understand what you're looking to do... but can you give examples of a couple of different requests and their respective (different) responses? – Michael - sqlbot May 06 '17 at 21:37
  • hi Michael.. the solution requires a flexible HTTP response using variables and NGinx can do this quite easily. HAProxy really isn't intended for this.. it's best at load-balancing. Having said that, i'll try to back it up with a gist in a day or two. – James Fremen May 06 '17 at 23:26
  • Okay, @JamesFremen, have it your way. :) HAProxy solution shown below. – Michael - sqlbot May 07 '17 at 02:25
  • it looks like a viable solution.. just a minor quibble, but the 'go' client is actually sending an https request to the server by default. I find that the client also objects if it gets back a non-https path to the git repo in the meta tag. I was handling this using sni and an internal redirect but your nice magic trick is better than the monitor-uri hack :). I will try to post nginx solution shortly for comparison. – James Fremen May 07 '17 at 14:31
  • Well, I didn't have an SSL cert for example.com handy for testing, so I just wrote the whole proof of concept to work with HTTP. It sounds to me like you are sufficiently familiar with HAProxy that you could wire it up to do exactly what you need, once you had a working example of how to render a response. For static replies, another simple hack is to create a backend with no servers, set its `errorfile 503` to a raw HTTP response file that begins `HTTP/1.0 200 OK\r\n...` and contains the complete raw response. :) – Michael - sqlbot May 07 '17 at 18:23
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/143624/discussion-between-james-fremen-and-michael-sqlbot). – James Fremen May 07 '17 at 18:23

2 Answers2

1

Depending on how you need to customize the response, this is possible using only HAProxy, compiled with Lua support. Here, using HAProxy 1.6.12 with Lua 5.3.4, I'm taking the request path and dropping it into the the response as-is, to do the customization -- essentially any path is just getting copied in. You could also use string.gsub() if you needed to modify it.

A small Lua script:

-- /etc/haproxy/lua/custom-response.lua

core.register_service("magic-endpoint", "http", function(applet)
   local response =
         "<head>\r\n" ..
         "  <meta name=\"go-import\" content=\"example.com" .. applet.path ..
         " git http://example.dev:7600/scm/project" .. applet.path ..
         "\">\r\n</head>\r\n";
   applet:set_status(200);
   applet:add_header("Content-Length", string.len(response));
   applet:add_header("Content-Type", "text/whatevs");
   applet:start_response();
   applet:send(response);
end)

And the relevant lines from config. I'm just using the hostname to route the request to the backend, but you could of course trigger it using whatever logic you need.

-- /etc/haproxy.cfg 

global
    lua-load /etc/haproxy/lua/custom-response.lua

frontend main-frontend
    mode http
    bind :80
    use_backend example if { hdr(host) -i example.com }

backend example
    mode http
    http-request use-service lua.magic-endpoint

And, test:

$ curl -v http://example.com/project1
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to example.com (127.0.0.1) port 80 (#0)
> GET /project1 HTTP/1.1
> User-Agent: curl/7.35.0
> Host: example.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/whatevs
< Content-Length: 124
<
<head>
  <meta name="go-import" content="example.com/project1 git http://example.dev:7600/scm/project/project1">
</head>

Seems legit.

Michael - sqlbot
  • 169,571
  • 25
  • 353
  • 427
0

For the purpose of conversation and because flexible generation of HTTP responses is useful I gave it a try using nginx - with a switch to ssh. The non-standard part of the configuration file has a rule to point the go client to the Bitbucket server:

location ~ "^(/[^/]+)(/[^/]+)?" {
    if ($arg_go-get = "1") {
        return 200 '<html><head><meta name="go-import" content="acme.com$1$2 git ssh://acme.dev:7999$1$2"></head></html>';
    }
}
James Fremen
  • 2,170
  • 2
  • 20
  • 29
  • the original title of the question explicitly referenced HAProxy so I think Michael has a legitimate claim on the answer. That said, i think Nginx is better suited out-of-the-box for this task because it can flexibly handle multiple projects/components in a simple manner which is well within its scope. – James Fremen May 08 '17 at 12:54