This question led me down a rabbit hole!
I had never really realized that this was a problem since I am normally using jupyter notebooks. Anyways, a new thread about this topic was started at here! The thread ended up in this PR where they have added a new config parameter "jupyter.enableExtendedKernelCompletions"
. You can find that setting by going to settings and typing "jupyter" in the search bar. CAUTION: As @wjandrea pointed out in comments, this setting is only changing the module used by the kernel to check for autocompletes instead of the Jupyter server and is linked to some kernel problems when working with large data sets.

This setting was not necessary for me, as I had autocomplete in the interactive window already, with it turned off. But it might do something for you.

So I started to look into why autocompletion for column names in *.py
files is still not supported, and investigate if there are any alternatives. AI code helpers seem to understand what column names are present through static code analysis, such as Tabnine.

This might be overkill for such a trivial use case though. It also doesn't help that it costs $12 per month.
Spyder is an alternative to VSCode. It is however made for Python, which might not be what you are after.

Dictionary keys are supported through static code analysis but do not know the state of the dictionary.

The whole process boils down to that when the DataFrame is created in a cell it becomes accessible to the jupyter server, which can keep track of the column names, values etc. Before this is done the interpreter does not know what is in the object, it just knows it's a Pandas.DataFrame. Therefore I can't see column names completion being done anytime soon in *.py
files (without AI companions).

TL;DR: Column name completion is available in the interactive window, but will probably not be available in pure python files anytime soon.
Hope gave some insight :)