CPUs execute what is known as machine code. Machine code (or assembly) come in different variations, the instruction set (language) depends on the CPU architecture.
Of course any programs written for a new totally new instruction set needs to be written directly in assembly, this is also the case for compilers. The initial compiler for each instruction set has to be written in assembly. When you have a working compiler for that instruction set it's possible to rewrite the compiler in C or what ever language the assembly compiler supports.
The GNU Compiler Collection(GCC) is arguably one of the most used compilers to date, it's written in C++.
In short though, the first compiler for a new instruction set is written in assembly.
EDIT: Your questions made me curious about how to implement a compiler for a new language. Turns out this is done via a process called bootstrapping, I don't fully understand it myself yet, but the general idea is writing the compiler for your new language in the language itself. I know it's a little weird. I found some pretty nice information on this.
Thank you for a really nice question by the way.