4

I'm trying to export and import data from an old MongoDB database server to Azure CosmosDB with MongoDB API using mongodump and mongorestore. But i'm having issues with the connection to CosmosDB. I'm using a connection string with the URI flag.

My mongorestore command including connection string is the following:

mongorestore --uri="mongodb://$COSMOS_USERNAME:$COSMOS_PASSWORD@$COSMOS_HOST:$COSMOS_PORT/?maxIdleTimeMS=120000&retrywrites=false&appName=@$DB_NAME@&replicaSet=globaldb&ssl=true" --archive="$ARCHIVE_NAME"

The error message from the command is:

error restoring from archive 'testProdExport.archive': (BadValue) Retryable writes are not supported. Please disable retryable writes by specifying "retrywrites=false" in the connection string or an equivalent driver specific config.

As you can see in the connection string i'm including the retrywrites=false URI parameter, but it looks like CosmosDB doesn't recognize the parameter.

Does anyone have experience with something similar?

//Edit: I've tried and verified that the connection string is working in a mongoose connection as well as in MongoDB Compass.

D. SM
  • 13,584
  • 3
  • 12
  • 21
CMarker
  • 75
  • 8
  • What is producing this message - mongorestore or cosmosdb? Drivers do not send "retryWrites" to the server, this is a client side-only parameter. The server receives it as a transaction number. – D. SM Sep 15 '20 at 12:19
  • That is my question as well. I have no clue actually. – CMarker Sep 15 '20 at 12:31
  • What is mongorestore version? – D. SM Sep 15 '20 at 12:35
  • And what is ismaster output for the server you are restoring to. – D. SM Sep 15 '20 at 12:38
  • mongorestore version: 100.1.1 { ismaster: true, maxBsonObjectSize: 16777216, maxMessageSizeBytes: 48000000, maxWriteBatchSize: 1000, localTime: 1600175975194, logicalSessionTimeoutMinutes: 30, minWireVersion: 0, maxWireVersion: 6, readOnly: false, tags: { region: 'West Europe' }, hosts: [ '-westeurope.mongo.cosmos.azure.com:10255' ], setName: 'globaldb', setVersion: 1, primary: '-westeurope.mongo.cosmos.azure.com:10255', me: '-westeurope.mongo.cosmos.azure.com:10255', connectionId: 620417038, ok: 1 } – CMarker Sep 15 '20 at 13:21

4 Answers4

19

I found that the various mongotools (mongoimport, mongorestore, etc.) seem to ignore the retrywrites=false or retryWrites=false in the URI, but tacking on the option --writeConcern="{w:0}" allows the command to run successfully on instances that don't support retryable writes

Jthorpe
  • 9,756
  • 2
  • 49
  • 64
  • 2
    This one worked for me with mongo 4.4 tools and cosmosDB with mongo v 3.6 – Myar Feb 15 '21 at 09:33
  • This is what worked for me as well. mongoimport version: 100.3.0 cosmosDB with mongo v 3.6 – Ken Feb 24 '21 at 17:15
  • Confirming that this works too with `mongorestore v100.3.0` and cosmos DB with `mongo server v4.0` – mdev Mar 29 '21 at 17:34
3

You can try a version of mongorestore that shipped with MongoDB 3.4. This may not be able to read recent dumps though.

The ismaster output you provided includes:

logicalSessionTimeoutMinutes: 30

This advertises session support but https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support-36 says cosmosdb does not support sessions.

CosmosDB is advertising support for a feature it does not implement, the tooling then (correctly) attempts to use it. This is a bug in cosmosdb.

D. SM
  • 13,584
  • 3
  • 12
  • 21
0

On Windows, didn't have luck with appending on the connectionstring (not necessary if copied from Azure portal)

retryableWrite=false

nor --writeConcern w:0 switch But I ended up trying all previous archives and the magic combination of mongo tools 100.0.0 worked on CosmosDB with MongoServer v3.6.

Vinay
  • 954
  • 8
  • 13
0

Attention: Please take care that this can lead to missing data in the collections you restored. Tested it multiple times with different versions of mongorestore. Use the version which is in the mongoserver archive with version 4.0.27. You don't need to provide writeConcern at all in this version. It will also display how many documents are restored instead of 0 in the newer versions.

This affects CosmosDb with API v4.0.

MAGYS
  • 181
  • 1
  • 14