If you clone a repository using --mirror you will create a bare repository. Which implies:
" that Git just contains the version control information and no working files (no tree) and it doesn't contain the special .git sub-directory. Instead, it contains all the contents of the .git sub-directory directly in the main directory itself.".
The mirror option at clone is ment to make a "as it is" copy of your current repo, including notes and external refs.
Being that said, you are doing it on the wrong repo, you should use the original one to make changes and the mirror will change accordingly.
Answering your question:
Is there a way to remove remote branches from a mirror clone or do I have to build another clone for that?
Not that i know of, besides a bare repository shoul not be used like a non-bare one.
Maybe what you need it's a fork, maybe if you explain a little better what you need. Anyways, a year has past since you make the question.
Here's some related questions and doc: