Starting 2022 AWS finally provides a solution for this problem with managed prefix-lists.
You can create an inbound security rule and under source directly specify the prefix list, instead of manually providing IP-Addresses:
To make your server reachable only from Cloudfront Servers follow these steps:
Go to https://console.aws.amazon.com/vpc/home#ManagedPrefixLists
Choose your region (The region of your Load Balancer) and search for "com.amazonaws.global.cloudfront.origin-facing" and copy the id (e.g. "pl-a3a144ca" for europe-central-1)
Edit your security group for the Load-Balancer and add a new Entry with Type: HTTP and as source paste the prefix-list-id from step 2
Now your security group will automatically always use the current IP-Addresses from Cloudfront, now updating necessary. - A caveat: The prefix list counts as ~50 rules against the rules-limit for a security group. If you have a lot of other custom rules, you will likely have to create a second security group with the other rules if this one is full.