I just managed to correctly install pandas on my M1 MacBook Pro following the recommended way here : Trouble installing Pandas on new MacBook Air M1.
I also have downloaded the specific version of VSCODE for Apple M1 chip.
The issue I have now is that the integrated terminal inside VSCODE is NOT running under Rosetta 2 (equivalent to the native Terminal app on MacOS before using the option Open Using Rosetta). I know that I can use VSCODE with a external Terminal, but I'd rather find a solution to use the integrated terminal in VSCODE, if possible.