GCC records its version in the .comment section (at least on targets that support it — e.g. ELF ones). You can compile something and check with objdump -s -j .comment a.out.
-frecord-gcc-switches (see the manual and this answer)
-grecord-gcc-switches
Generally when you ask the compiler to emit debug information (the -g* family of options), plenty meta-info might be emitted. I know of DW_AT_producer (info about the compiler/assembler name and version along with command line swithes) and DW_AT_comp_dir (compilation directory) tags.
There's also -fverbose-asm, but that only emits comments in the assembly; they don't make it through even to the object files, let alone executables.
Also keep in mind, that the compiler does not produce executables. It just produces the assembly code (the gcc command is the compiler driver that invokes the compiler proper, then the assembler, then the linker). The linker can decide what to do with .comment sections and such when combining object files into an executable/library. I think, the "normal" behaviour is to catenate them, although discarding them at that stage or later (e.g. with strip) should't surprise anyone.
How can one deliberately insert metadata into an executable?
Usually, SHT_NOTE sections are used for it (if we are speaking ELF). Here concatenation and preservation through to the end are guaranteed, because they are not mere comments; they are used by the toolchain to store some properties of programs (see man elf, grep for 'note').
Another (hacky) way is defining absolute symbols. You can encode information in the symbol's name. This can be done at any stage. E.g. in the source program, or in assembly. Or at link stage with ld --defsym. Or you can modify the resulting executable with objcopy --add-symbol — oh, that works for sections, too (objcopy --add-section).
