Reroute specific outgoing url to another url - linux ubuntu 16.04

Question

I am working on building a very old program that has many outdated links to dependencies.

These links might be in other dependencies downloaded from the web which takes out the option to change the url

I am able to find all of these dependencies with other links but changing the paths has become a endless task.

Is it possible to create a list of rules for outgoing urls that map one to one. For example:

http://Oldserver.com/this/is/one/OldDependency.jar -> http://Newserver.com/this/is/one/with/other/url/NewDependency.jar

It does not matter what tool is used for the routing, iptables or something else. I am willing to set anything up for this.

This needs to happen on the OS level because the paths are inside tar files

Possible duplicate of [Using grep and sed to find and replace a string](https://stackoverflow.com/questions/6178498/using-grep-and-sed-to-find-and-replace-a-string) — jww, Jan 02 '18 at 00:24
This is not a duplicate. This needs to happen on the OS level because the paths that I would need to change are in tar.gz files downloaded from the web. — Thor, Jan 03 '18 at 15:31

score 1 · Accepted Answer · answered Jan 03 '18 at 15:28

I was able to get this working by using a local nginx. The best solution here was a dockerized nginx container.

I will use the example above

http://Oldserver.com/this/is/one/OldDependency.jar -> http://Newserver.com/this/is/one/with/other/url/NewDependency.jar

Steps:

Edit your host file to route the host to your localhost

$ sudo vim /etc/hosts

Add this line to your hosts file

127.0.0.1 Oldserver.com

Pull docker the nginx docker container

docker pull nginx

Save this nginx configuration file to some path (code tags not working, sorry)

events { worker_connections 4096; ## Default: 1024 }

http { server { listen 80; server_name Oldserver.com;

location = /this/is/one/OldDependency.jar {
  proxy_pass http://Newserver.com/this/is/one/with/other/url/NewDependency.jar;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
}

location / {
  proxy_pass http://Oldserver.com;
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
}

} }

If you have more paths, add then above the wildcard location /. The location / forwards all paths not matched to the original server with the path reserved

Set permissions on config

chmod 600 /some/path/to/nginx.conf

Start up a nginx docker container with the configuration file

docker run --name proxy -v /some/path/to/nginx.conf:/etc/nginx/nginx.conf:ro -p 80:80 -d nginx

Now, every request to the Oldserver.com will go through your nginx proxy and reroute if it matches any of your location configurations

Thanks for the answer. How should I configure if I need to redirect https request? — Andy Yi Cui, Aug 02 '23 at 16:18

score 0 · Answer 2 · answered Nov 28 '21 at 23:36

I had a similar problem to this, needing to rewrite an outgoing url. In my case it was in a docker container running on kubernetes.

In my case it was because of this issue: https://stackoverflow.com/a/63712440

The app runtime (.net core 3.1) crypto code only checks the first url in the list of certificate revocation endpoints. I was doing an SSL client certificate setup (mTLS).

The PKI cert I was issued contained an internal domain address first+second, and then a publicly addressable url third:

    X509v3 CRL Distribution Points: 

        Full Name:
          URI:http://some.domain.1/CRL/Cert.crl
          URI:http://some.domain.2/CRL/Cert.crl
          URI:http://x509.domain.com/CRL.crl

Because the domain addresses use a 'CRL' folder in the path, but the public url does not, just mapping the public IP address to the local domain host via /etc/hosts (or k8s hostAliases) didn't work.

To solve this in k8s, I added a sidecar to my pod; here's the details:

First, start with an nginx.conf:

events { }
http {
  server {
    listen 80;
    location / {
      proxy_pass http://127.0.0.1:5555;
    }
    location /CRL/ {
      proxy_pass http://x509.domain.com/;
    }
  }
}

This kind of looks like a reverse proxy, but really it just an actual proxy. My dotnet app serves on port 5555 inside the pod. 127.0.0.1 will route to the pod, not the nginx container. Note that the second proxy_pass value doesn't include the 'CRL' path, that allows the url to rewritten not just redirected.

I then built an nginx docker image called crl-rewrite-proxy:

FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf

I then added the image into my pod yaml as a sidecar:

- name: crl-rewrite-proxy
  image: $(image):crl-rewrite-proxy
  ports:
    - containerPort: 80

And then added an alias for the internal domain address, so outgoing calls to it from the app would route back into the pod:

hostAliases:
- ip: "127.0.0.1"
  hostnames:
  - "some.domain.1"

Lastly I defined an ingress in my k8s yaml, so the aliased calls will be routed to the sidecar:

- kind: Ingress
    apiVersion: networking.k8s.io/v1
    metadata:
      name: $(name)-crl-rewrite-proxy
      namespace: $(namespace)
    spec:
      rules:
        - host: $(name).$(host)
          http:
            paths:
              - path: /CRL
                pathType: ImplementationSpecific
                backend:
                  service:
                    name: $(name)
                    port:
                      number: 80

The app runtime makes a call to http://some.domain.1/CRL/Cert.crl; the host alias routes that to 127.0.0.1:80; k8s passes that to the sidecar; the sidecar passes that to nginx; nginx rewrite the host+url to a public IP on a different path; the resource then gets fetched successfully.

Thanks to thor above for the local setup, I used this to verify it would work locally before doing up the k8s bits.

Reroute specific outgoing url to another url - linux ubuntu 16.04

2 Answers2

Linked