Because the specs say so:
enum-declaration:
attributesopt
enum-modifiersopt enum identifier enum-baseopt
enum-body ;opt
enum-base:
: integral-type
enum-body:
{ enum-member-declarationsopt }
{ enum-member-declarations , }
Each enum type has a corresponding integral type called the underlying type
of the enum type. This underlying type must be able to represent all the
enumerator values defined in the enumeration. An enum declaration may
explicitly declare an underlying type of byte, sbyte, short, ushort, int, uint,
long or ulong. Note that char cannot be used as an underlying type. An enum
declaration that does not explicitly declare an underlying type has an
underlying type of int.
...
integral-type is defined as,
integral-type:
sbyte
byte
short
ushort
int
uint
long
ulong
char