0

I need your help with AWS glue job ERROR: column "id" does not exist The main goal is to transform data from one RDS(postgresql) to another RDS(postgresql). I created connections to RDSs. Create Crawlers. Everything works correct. After that I created a Spark 2.4, Python 3 (Glue Version 1.0) job and try to move modified data from source to target. But I get an error

org.postgresql.util.PSQLException ERROR: column "id" does not exist

In source database I have lower case columns name. But I try solution from this article - fails. I try manualy change from lower case to upper case colums name in script - fails. When I create a test tables with lower case columns in source and target everything works correctr.

Any ideas how I can resolve this issue.

Best regards.

Nishu Tayal
  • 20,106
  • 8
  • 49
  • 101
mVadim
  • 43
  • 4
  • artical it means this one https://stackoverflow.com/questions/58093109/aws-glue-spark-job-fails-to-support-upper-case-column-name-with-double-quotes – mVadim Jun 17 '20 at 08:58
  • found that this happens when we deal with Id column name in Upper case and this column is primary key. In other cases no errors. – mVadim Jun 18 '20 at 10:10
  • I have the same issue. Do you know how to solve the issue? – Nishu Tayal Dec 22 '20 at 21:31

0 Answers0