1

I pass some requests to this location (this is a simplified version of the configuration for clarity). ONBOARDING_URL is set in the environment and replaced when the server is run by a URL. This URL is resolved to an IP once. It needs to be resolved for each request when the TTL has expired. Initially, the configuration looked like this:

Initial Configuration Template

server {
  listen                443 ssl;
  server_name           localhost;
  resolver ${AWS_DNS};

  location /onboarding {
    proxy_cache onboarding;
    proxy_set_header Host $proxy_host;
    proxy_cache_valid 200 10m;
    proxy_cache_key $cacheKey;

    proxy_set_header Host $proxy_host;
    proxy_pass ${ONBOARDING_URL}/api-users;
  }
}

I discovered this was only resolved one time. Changes to the IP underlying the DNS resolution had no effect; requests were routed to the IP that was resolved initially.

So following guidance in the proxy_pass documentation, and from other questions / sources (below), we modified the configuration.

from: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass

Parameter value can contain variables. In this case, if an address is specified as a domain name, the name is searched among the described server groups, and, if not found, is determined using a resolver.

The new configuration was intended to replace the template variable, populated from the environment at runtime, with a configuration variable that will always be resolved. However, it doesn't seem to be. Once an IP is resolved, it is used for the life of the process. Any ideas how to get around this? This seems a bit complex to test, too, which further complicates the fix.

New Configuration Template

server {
  listen                443 ssl;
  server_name           localhost;
  resolver ${AWS_DNS};

  set $onboardingUrl ${ONBOARDING_URL};

  location  ~ ^/onboarding/?(.*)$ {
    proxy_cache onboarding;
    proxy_set_header Host $proxy_host;
    proxy_cache_valid 200 10m;
    proxy_cache_key $cacheKey;

    proxy_set_header Host $proxy_host;
    proxy_pass $onboardingUrl/api-users/$1;
  }
}

Yet still, for the life of the process only the initially resolved IP is used.

theherk
  • 6,954
  • 3
  • 27
  • 52
  • Are you sure? Using a variable in the `proxy_pass` statement should cause Nginx to use the `resolver`. However, the resolved IP address will still be valid for the TTL. – Richard Smith Oct 07 '21 at 15:34
  • I'm confident but not certain. If I dig on the url that populates `ONBOARDING_URL`, I show an updated address and nginx is logging a different address that I believe was the first resolved address. I'm not certain, because I don't know how well my local dnsmasq setup is testing this. – theherk Oct 07 '21 at 18:26
  • Even a resolver will cache answers, so there will be a delay before Nginx notices that the DNS records have changed. You can set a `valid` time on the [`resolver` statement](http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) and make it as short as you need. You shouldn't make the valid time too short, otherwise your website may become very slow. – Richard Smith Oct 07 '21 at 18:39
  • I considered this, but I figured the resolver should honoring the ttl and since my dig resolves the update I thought I was good, but my I didn't dig using the custom resolver so maybe I'm just not seeing accurate results. I'll try the `valid` property. – theherk Oct 07 '21 at 18:46
  • I have tested this carefully, and only found that it works as we expect. It seems, however, that this is not working correctly outside the test environment though, so I'm not sure what is causing the issue that looks like this. I suppose I'll update this if I find more, otherwise viewers should know this should indeed work. – theherk Oct 12 '21 at 06:42

0 Answers0