"Waiting for localhost" when trying to connect to mapped port with docker-compose. Issue with iptables or ipv6?

Question

I have a docker-compose.yml file which maps a port like this:

ports:
  - '81:3000'

and is linked to a mongodb container like this:

links:
  - mongo

In the Dockerfile port 3000 is exposed like so:

EXPOSE 3000

(used by express.js like so: app.listen(3000)

But when I try to open up localhost:81 in Firefox-esr it doesn't load the page and just keeps loading, saying "Waiting for localhost".
How to find out what it's waiting for and how to fix this issue?

---

I'm running Debian9/KDE with Docker version 18.09.0 and I'm using iptables and apparmor. Do I need to add some iptables rules to get this working? Couldn't really find which rules I'd need to add and rules like -A INPUT -i docker0 -p tcp --dport 81 -j ACCEPT didn't work. https://docs.docker.com/network/iptables/ didn't help me so far either. Do I maybe need some FORWARD rules?

sudo iptables -L -t nat shows

Chain DOCKER (2 references)
target     prot opt source               destination         
RETURN     all  --  anywhere             anywhere            
RETURN     all  --  anywhere             anywhere            
DNAT       tcp  --  anywhere             anywhere             tcp dpt:27017 to:172.18.0.2:27017
DNAT       tcp  --  anywhere             anywhere             tcp dpt:81 to:172.18.0.3:3000

Mapping ports with docker container run works fine.

sudo lsof -i shows:

docker-pr 28466       root    4u  IPv6 6484109      0t0  TCP *:27017 (LISTEN)
docker-pr 28587       root    4u  IPv6 6483733      0t0  TCP *:81 (LISTEN)

which I found strange because I'm currently using IPv4. Do I need to add net.ipv4.ip_forward=1 to /etc/sysctl.conf?

Running sudo tcpdump -X -s0 -w ~/tcpdumps/waitingforlocalhostdocker and opening that up in wireshark shows stuff like:

Source      Destination  Protocol  Info
172.18.0.3  172.18.0.2   SSL       Continuation Data
172.18.0.3  172.18.0.2   SSL       Continuation Data
172.18.0.2  172.18.0.3   TCP       27017 -> 52120 [ACK] Seq=1 Ack=56 Win=237 Len=0
172.18.0.2  172.18.0.3   TCP       27017 -> 52120 [PSH, ACK] Seq=1 Ack=56 Win=237 Len=239 [TCP segement of a reassembled PDU]

Maybe I have to configure IP forwarding in the kernel? I already tried it with net.ipv4.conf.all.forwarding set to 1. Or maybe some other kernel configurations?

How to fix this so that I can open localhost:81? Likely I'm just missing something trivial here.