Typically, a bin folder is intended as the main component of your deployment package. In a very simple application deployment, everything a program needs exists in the bin folder. You could copy paste that folder anywhere and it would work. Of course, that won't happen if your external assembly isn't there.
You find that those assemblies shouldn't exist in 2 places on your development machine, and I understand why you don't like that. The reason CopyLocal is set to true by default is because the typical scenario I described above is worth the cost of having assemblies existing twice in a development machine. Typically, an application will exist on many times more client machines than development machines, so the simplicity of a single deployment folder is worth the cost of disk space on dev machines.
You said yourself that you would include the lib folder whenever you'd give your exe to someone else? Wouldn't it be simpler to just give the bin folder with everything in it?
In your original question, you mention the compiler can't find the assemblies, I think that's incorrect and I tested a similar situation. The compiler found the assembly, however when I tried executing the code, the runtime told me it couldn't find it. The compiler does not insert code to make the executable find assemblies in relative paths. You can find the detailed probing behavior of .Net external assemblies here. Your exe can't magically guess that the assembly is in a lib folder in the parent folder.
You can circumvent all this by coding your own assembly resolution, but unless you are starving for a few kilobytes of disk space, let Visual Studio do its thing.