From the article linked by shdobxr, the most relevant part regarding the difference between KLIPS and Netkey seems to be the following:
When you apply firewall (iptables) rules, KLIPS is the easier case,
because with KLIPS, you can identify IPsec traffic, as this traffic
goes through ipsecX interfaces. You apply iptables rules to these
interfaces in the same way you apply rules to other network interfaces
(such as eth0).
When using NETKEY, applying firewall (iptables) rules is much more
complex, as the traffic does not flow through ipsecX interfaces; one
solution can be marking the packets in the Linux kernel with iptables
(with a setmark iptables rule). This mark is a member of the kernel
socket buffer structure (struct sk_buff, from the Linux kernel
networking code); decryption of the packet does not modify that mark.
Writing in 2014, all linux distros should now come with a kernel that support both KLIPS and Netkey.