Try first and set terminal.integrated.defaultProfile.windows
to bash (if you want bash by default)

That way, you don't have to switch from your current default Powershell terminal.
... Provided microsoft/vscode
issue 132149 is fixed, which is not the case, yet.
This is a duplicate of issue 132150, which is now fixed by commit c765e70, by Megan Rogge.
async createTerminal(options?: ICreateTerminalOptions): Promise<ITerminalInstance> {
if (!this._availableProfiles) {
await this._refreshAvailableProfiles();
}
Update Sept. 14th (2021): this is now fixed, with VSCode 1.60.1, and its August 2021 Endgame Recovery.