Object not callable error | Where function

Question

I am trying to run below query:

df3 = df1.join(df2, df1["DID"] == df2["JID"],'inner')\
          .select(df1["DID"],df1["amt"]-df2["amt"]\
          .where(df1["DID"]== "BIG123")).show()

I get error as shown below:

TypeError: 'Column' object is not callable TypeError
Traceback (most recent call last)

What is the issue with the query and how do I fix it?

Answer 1

There is syntax issue in your query. Closing bracket for select is coming after where block. Below is syntactically query .

df3 = df1.join(df2, df1["DID"] == df2["JID"],'inner')\
          .select(df1["DID"],df1["amt"]-df2["amt"])\
          .where(df1["DID"]== "BIG123").show()

Answer 2

Try this code :

 from pyspark.sql import functions as F    

 df3 = df1.join(df2, df1["DID"] == df2["JID"] ,how = 'inner')\
      .select("DID",df1["amt"]-df2["amt"])\
      .where(F.col("DID")== "BIG123" ).show()

Best,

Abakar