Running apt-get update didn't solve it for me because, it always seemed to read the result of this command from cache and somehow the cache seemed to be corrupted. My suspicion is that, this happens if you have multiple docker containers having the same 'base image' ( for me it was python:3.8-slim).
So, here's what worked for me:
Stopping all Docker containers in the system
docker stop $(sudo docker ps -aq)
Removing all dangling containers
docker container prune
Removing all dangling images
docker image prune
After running these commands the docker build seems to lose the corrupted cache and does a clean apt-get update which fetches the correct package info and the package installations progress as expected.