It's not quite as easy as the code .
technique you get with the Remote - WSL extension, but Microsoft also provides a Remote - SSH extension that can be used to directly access the remote machine (without requiring WSL in-the-middle).
Once the Remote - SSH extension is installed in VSCode (and it may have already been installed in an extension pack with the WSL extension), and you have installed an OpenSSH client in Windows, there are several ways to access files on the remote host through SSH:
From the Command Palette (Ctrl+Shift+P), type Remote SSH to filter on those commands. From Connect to Host, you can add a new configuration, etc.
From the Activity Bar on the left, select the Remote Explorer icon, then in the dropdown at the top, select SSH Targets. You can add hosts here through the + icon.
There's a direct shortcut to Open a Remote Window at the far left of the status bar. This will give you similar filter options on the Command Palette as above.
Once a host is configured, you can browse it just as (well, almost) if it were local, open files, edit, etc. The one thing you can't do (as far as I'm aware) is any type of sudo
/su
editing on files that you don't have permissions to directly.
Full details in the VSCode docs here.