You should not be deploying all the files in the bin\Release folder anyway. The vshost files are for debugging within Visual Studio, they should not be deployed. The pdb file is for debugging and does not need to be deployed. If your application doesn't use the config file for anything you don't need to deploy it. You will need to deploy any additional assemblies though, which you may be able to solve using ILMerge.
Having said that, my recommendation would be to either create an installer or create a self-executing archive (zip or 7z) over trying to use ILMerge to get any third-party assemblies deployed with your application.
UPDATE
Just to be clear, here's a list of files from a default WinForms app to which I added JSON.NET (Release build):
- Newtonsoft.Json.dll
- Newtonsoft.Json.xml
- WindowsFormsApplication1.exe
- WindowsFormsApplication1.exe.config
- WindowsFormsApplication1.pdb
- WindowsFormsApplication1.vshost.exe
- WindowsFormsApplication1.vshost.exe.config
- WindowsFormsApplication1.vshost.exe.manifest
If the app doesn't use the config file, the only files you need to deploy are Newtonsoft.Json.dll
and WindowsFormsApplication1.exe