When I have a project that requires enough customization to justify doing a separate build of the Bootstrap CSS, I just edit the LESS files directly so that none of the overridden rules would remain in the compiled version. Presuming git is being used, one can commit these edits to a separate branch from the master branch, and then when updates on Bootstrap come through, rebase those changes onto the tracking branch, then rebuild.
Your other option is to use some external tool to try and clean up the unreachable CSS afterward. (tools for that have been suggested elsewhere)
However, I also want to point out that for many Bootstrap projects, loading Bootstrap CSS from a popular CDN and then loading separate overrides from your own server works just fine. Those unused rules aren't going to kill you, and from my own experience the custom build is the exception, not the rule. I don't know your project, so it's really gotta be your call; just make sure the benefits outweigh the added work this will mean in terms of future maintenance.