1

Below is the DocumentDB stored procedure I am using for bulk insert of records in document db

function bulkCustomerImport(docObject) {
var context = getContext();
var collection = context.getCollection();
var collectionLink = collection.getSelfLink();
var count = 0;
var customerList = [];
// Check input
if (!docObject.length) throw new Error("Invalid object.");
var docs = docObject;
var docsLength = docs.length;
if (docsLength == 0) {
    context.getResponse().setBody(0);
}

// Call the funct to create a document.
tryCreateOrUpdate(docs[count], callback);

// Note that there are 2 exit conditions:
// 1) The createDocument request was not accepted. 
//    In this case the callback will not be called, we just call setBody and we are done.
// 2) The callback was called docs.length times.
//    In this case all documents were created and we don't need to call tryCreate anymore. Just call setBody and we are done.
function tryCreateOrUpdate(doc, callback) {
    var isAccepted = true;
    var isFound = collection.queryDocuments(collectionLink, 'SELECT * FROM root r WHERE r.FullName = "' + doc.FullName + '" and r.Company = "' + doc.Company + '"', function (err, feed, options) {
        if (err) throw err;
        if (feed.length==0) {
            isAccepted = collection.createDocument(collectionLink, doc, callback);
        }
        else {
                 if(!feed.length > 0) throw 'Unable to find metadata document';
            callback(false, feed[0])
        }
    });

    // If the request was accepted, callback will be called.
    // Otherwise report current count back to the client, 
    // which will call the script again with remaining set of docs.
    // This condition will happen when this stored procedure has been running too long
    // and is about to get cancelled by the server. This will allow the calling client
    // to resume this batch from the point we got to before isAccepted was set to false
    if (!isFound && !isAccepted) getContext().getResponse().setBody(customerList);
}

// This is called when collection.createDocument is done and the document has been persisted.
function callback(err, doc) {
    if (err) throw err;

    // One more document has been inserted, increment the count.
    count++;
    customerList.push(doc);
    if (count >= docsLength) {
        // If we have created all documents, we are done. Just set the response.
        getContext().getResponse().setBody(customerList);
    } else {
        // Create next document.
        tryCreateOrUpdate(docs[count], callback);
    }
}
}

The above code for bulk inserts works fine for less than about one hundred records, and returns a customer list as well. But if I have more than one hundred records, I always get null value returned to my .Net code. I checked for all my records which I am inserting, there is nothing incorrect value in it. Neither is the procedure throwing any errors.

This is my C# code used to call the procedure:

public static async Task<List<Customer>> InsertBulk(string storedProcedureID, List<Customer> cl)
{
var sprocLink = UriFactory.CreateStoredProcedureUri("DatabaseName", "Collection", storedProcedureID);
        StoredProcedure sproc = await Client.ReadStoredProcedureAsync(sprocLink.ToString());
        string argsJson = JsonConvert.SerializeObject(cl.ToArray());
        var docObject = new dynamic[] { JsonConvert.DeserializeObject<dynamic[]>(argsJson) };

        try
        {
            cl = await Client.ExecuteStoredProcedureAsync<List<Customer>>(sproc.SelfLink, docObject);
        }
        catch (DocumentClientException ex)
        {
            throw;
        }

        return cl;
}

Please that this insert of records can be any number of count from hundreds to millions. Any help related to this will be highly appreciated..

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Nadeem Shaikh
  • 362
  • 5
  • 23
  • Look related/duplicate of http://stackoverflow.com/a/7207539/2265446 – Cleptus Feb 15 '17 at 13:51
  • @bradbury No Bro, it is not related to your given link.. – Nadeem Shaikh Feb 15 '17 at 13:54
  • is there not a chance your bulk update is in effect too big/timing out? I think @bradbury9 suggested the size of your json as an issue - which seems like a valid question – BugFinder Feb 15 '17 at 14:00
  • Sprocs will time out. You either need to sense that and restart the sproc with the remaining or you need to chunk it up small enough that you never hit that limit. – Larry Maccherone Feb 15 '17 at 15:54

0 Answers0