You can create another repository to host all your builds , I mean executable files . With in that repository don't add any of your code other than your builds ,
As a result of this , people can click on download Zip button at git hub , which downloads only executable ( as a zip file ).
while building you can copy the executable file in a folder just push to remote repository which is hosting only builds .
Hope this helps .
basically , GIT is just an SCM ( source code management system ) it is not meant for this purpose .
but still this how you can utilize the service of github.org amd git .
hope this helps .
EDIT : -
Git hub now has a solution for hosting releases it has been well explained by @VonC in the post below . Please use that as a solution.