I was unable to do this using debase
and recommend using gem debug
instead, along with extension "VSCode rdbg Ruby Debugger".
You'll get a "rescue any exception" breakpoint which works (though I wasn't able to get it to rescue only uncaught exceptions).

See https://github.com/ruby/vscode-rdbg and https://github.com/ruby/debug
This is the launch.json for my rails application
{
"type": "rdbg",
"name": "Debug Rails with rdbg",
"rdbgPath": "bundle exec rdbg",
"request": "launch",
"cwd": "${workspaceFolder}",
"command": "${workspaceFolder}/bin/rails",
"script": "server",
"args": [
"-p",
"8080"
]
}