This is not an answer, as it doesn't help work out why the emails that are "Delivered" have not been received, but this is SendGrid's note on why Email messages with the “Delivered” status are not received:
Twilio SendGrid posts the Delivered event after the destination server accepts the message with a 250 OK response. Once an email is accepted by the destination server, we are unable to see what happens to the message. The receiving server could send it to the inbox, queue it for later delivery, put it into the spam folder, etc.
Often times, a recipient domain will initially accept a message for delivery, and then apply additional filtering afterwards. In this situation, we would have posted the Delivered event, but not have any insight into the additional filtering. Any additional Twilio SendGrid events for your email message would be triggered by recipient engagement (i.e. open/click events, unsubscribes, etc.).
Also note that at the time of writing, that answer had 26 downvotes (and zero upvotes) on the SendGrid website, probably indicating that many other users have experienced this problem.