You can use Event Orchestration or Event Rules at a service level to control this. What you are looking for is to create an alert but pause notifications. This will suspend the alert for a time period of your choice, allowing the alert to resolve itself. If the alert doesn't resolve within the time period an incident will open as expected.
Event Orchestration (should be available on basic+ tiers)
Once you define the conditions for the Orchestration rule you want to set an incident action to pause notifications for 300 seconds (5 minutes).
https://support.pagerduty.com/docs/event-orchestration#incident-actions
If you have the event intelligence package you can also look at the auto-pause feature which detects and pauses transient alerts for you.