2

I get an error I don't understand when trying to save a dataframe I've been working on to a documentDB collection.

Others similar issues on stack pointed to incorrect or case sensitive names provided for the Database or the Collection But I Checked Those... What other explanations could there be ? PartitionKey ? Region ?...

On an other hand I was unable to find a complete doc on which options were eventually to be provided to the line df.write.format("com.microsoft.azure.cosmosdb.spark").mode('overwrite').options(**ddbconfig).save()

Error given by Spark on Databricks is : 

com.microsoft.azure.documentdb.DocumentClientException: Message: {'Errors':['Owner resource does not exist']}

The StackTrace gave those

Py4JJavaError: 
  An error occurred while calling o646.save. :
    com.microsoft.azure.documentdb.DocumentClientException: 
      Message: {"Errors":["Owner resource does not exist"]}

Response gave in the storereadresult

LSN: 623, GlobalCommittedLsn: 623, PartitionKeyRangeId: , IsValid: True, StatusCode: 404, IsGone: False, IsNotFound: True, IsInvalidPartition: False, RequestCharge: 1, ItemLSN: -1, SessionToken: -1#623, ResourceType: Collection, OperationType: Read

edit : different situation than in similar linked posts. This error occuring when trying to write data in a new, empty collection. Not reading existing data. And I had already clarified in my question that I have explored every path (collection/database names mismatches mainly) that I had found in those similar posts.

Vincent Chalmel
  • 590
  • 1
  • 6
  • 28
  • Possible duplicate of [Azure DocumentDB Owner resource does not exist](https://stackoverflow.com/questions/43551321/azure-documentdb-owner-resource-does-not-exist) – 10465355 Dec 15 '18 at 22:51
  • No it's not a duplicate. The question you linked, which I had reviewed previous to my message, concerns a similar error message but a really different situation. It's a reading error, mine is a writing error. Additionnaly, In my question I noted that I checked any possible mismatch in database and collection names & case – Vincent Chalmel Dec 16 '18 at 13:47

1 Answers1

0

After investigating further, it's a bug in the version of the library I was using.

Resolved by switching from azure-cosmosdb-spark_2.3.0_2.11-1.2.2-uber.jar to azure-cosmosdb-spark_2.3.0_2.11-1.2.7-uber.jar

As seen here on github https://github.com/Azure/azure-cosmosdb-spark/issues/268

Vincent Chalmel
  • 590
  • 1
  • 6
  • 28