This happens because when the browser is reloaded when the angular app is in a route(Not with #) it request it from S3 where there isn't a file on that path. The Angular development handles this which is why its not appear locally. As a workaround you can do the following.
- Setup AWS CloudFront Distribution in-front of S3.
- Create an error handler if a different path is called and its not found in S3 to return the index.html.
For more details and automate the setup use the AWS Cloudformation in the article Deploying Angular/React Apps in AWS
Note: You can directly serve your S3 content using this approach using the Apache server only for the backend logic which can also be routed through CloudFront while in the same origin.