If you do not have any branches (or if you have but those are redundant and do not care if those get deleted) then simply flatten your repo, check out this.
Else you can shrink the repo, check out this.
In any case I am not sure if history could be the real reason for large size of your repo, it is also possible that some code was cleaned up but there was no re-packaging so all stale things exists, if that is the case then "Reclaim space" section of shrink the repo link should help.
If nothing is helping then go to your ".git" folder of the repo (will be on your disk checkout code) and try to see the files over there, one of the files would be packed-refs
which contains information like what all branches etc. are present in your repo, suppose you have removed the branch you if its name still exist in this file then your overall GIT pack size would include that size, so what you can do is delete that particular entry from this file and then execute the step of "Reclaim space" section of shrink the repo link I have provided.
Key is that you will have to debug things at your end because we do not know exact position of your repo.