rich.prompt.Confirm not working in rich.Progress context python

Question

I am working on an app that uses a rich.Progress for rendering progress bars. The problem is rich.prompt.Confirm just flashes instead of showing the message and asking for the confirmation while in the Progress context.

Demo Code

from rich.progress import Progress
from rich.prompt import Confirm
from time import sleep

with Progress() as progress:
    task = progress.add_task('Cooking')
    while not progress.finished:
        if Confirm.ask('Should I continue', default=False):
            progress.update(task, advance=0.6)
            sleep(0.4)

EDIT: I have seen git issues and researched a bit and it seems input(which the rich.Prompt uses) doesn't work on any thing that uses rich.Live(which the rich.Progress uses). So now my question is, How can you structure your code so that you don't put a prompt inside a rich.Progress context manager. Or any possible workarounds to this issue.