column names cannot start with a number, pyspark dataframes

Question

I noticed in pyspark dataframes that if the column starts with a number it doesnt show when i call the show() clause

meta.select('7.5sig_UCL').show(1)

An error was encountered: "cannot resolve '7.5sig_UCL' given input columns:

But if I change the name of the same column, it works

df.withColumnRenamed('7.5sig_UCL', 'sevensig_UCL')
meta.select('sevensig_UCL').show(1)

Output:

+------------+
|sevensig_UCL|
+------------+
|   121.44565|
+------------+

Is this a limitation for pyspark dataframes?

Answer 1

Use `(backticks) to enclose the column name:

meta.select('`7.5sig_UCL`').show(1)

From this answer, I think the problem is with the dot, not digit.