mongoexport aggregate export to a csv file

Question

I want to save the result from an aggregation into a csv file.

Using the mongo cmd line tool I can do this to get the results I want:

db.compras.aggregate({ $group : { _id : "$data.proponente", total : { $sum : "$price" } }}

How would I translate this into a mongoexport command that saves results into a csv?

Answer 1

Slightly simpler option as of 2.6+ is to now add an $out step to your aggregate to put the results into a collection:

db.collection.aggregate( [ { aggregation steps... }, { $out : "results" } ] )

Then just use mongoexport as:

mongoexport -d database -c results -f field1,field2,etc --csv > results.csv

After that you might want to delete the temporary collection from the database so that it does not keep using unnecessary resources, and also to avoid confusion later, when you have forgotten why this collection exists in your database.

db.results.drop()

Answer 2

You can export to a CSV file with the following 3 steps:

1. Assign your aggregation results to a variable (reference):

   var result = db.compras.aggregate({ $group : { _id : "$data.proponente", total : { $sum : "$price" } }}
   

2. Insert a value of the variable to a new collection:

   db.bar.insert(result.toArray());
   

3. In terminal (or command line) export this bar collection to a CSV file:

   mongoexport -d yourdbname -c bar -f _id,total --csv > results.csv
   

...and you're done :)

Answer 3

You can't run aggregate() queries through mongoexport. The mongoexport tool is intended for more basic data export with a query filter rather than full aggregation and data processing. You could easily write a short script using your favourite language driver for MongoDB, though.

Answer 4

If you don't want to store the results in a collection, you could also write directly to a CSV file from JavaScript using the print function. Save the following script to a file like exportCompras.js.

let cursor = db.compras.aggregate({ $group : 
  { _id : "$data.proponente", 
    total : { $sum : "$price" }
  }
});

if (cursor && cursor.hasNext()) {

  //header
  print('proponente,total');

  cursor.forEach(item => {
    print('"' + item._id + '",' + item.total);
    // printjson(item); -- if you need JSON
  });
}

From the command line, call >mongo server/collection exportCompras.js > comprasResults.csv --quiet

Updated from @M.Justin's comment to replace the previous while loop.

Answer 5

Take the following and save it on the mongo server somewhere:

// Export to CSV function
DBCommandCursor.prototype.toCsv = function(deliminator, textQualifier) 
{
    var count = -1;
    var headers = [];
    var data = {};

    deliminator = deliminator == null ? ',' : deliminator;
    textQualifier = textQualifier == null ? '\"' : textQualifier;

    var cursor = this;

    while (cursor.hasNext()) {

        var array = new Array(cursor.next());

        count++;

        for (var index in array[0]) {
            if (headers.indexOf(index) == -1) {
                headers.push(index);
            }
        }

        for (var i = 0; i < array.length; i++) {
            for (var index in array[i]) {
                data[count + '_' + index] = array[i][index];
            }
        }
    }

    var line = '';

    for (var index in headers) {
        line += textQualifier + headers[index] + textQualifier + deliminator;
    }

    line = line.slice(0, -1);
    print(line);

    for (var i = 0; i < count + 1; i++) {

        var line = '';
        var cell = '';
        for (var j = 0; j < headers.length; j++) {
            cell = data[i + '_' + headers[j]];
            if (cell == undefined) cell = '';
            line += textQualifier + cell + textQualifier + deliminator;
        }

        line = line.slice(0, -1);
        print(line);
    }
}

Then you can run the following commands via the shell or a GUI like Robomongo:

load('C:\\path\\to\\your\\saved\\js\\file')
db.compras.aggregate({ $group : { _id : "$data.proponente", total : { $sum : "$price" } }}.toCsv();

Answer 6

By the same logic, in my case, I want the output as JSON data into out.json

1- Create file.js on local, not on the server

date = new Date("2019-09-01");
query = {x: true, 'created': {$gte: date}};
now = new Date();
userQuery = {'user.email': {$nin: [/\.dev$|@test\.com$|@testing\.com$/]}};
data = db.companies.aggregate([
    {$match: query},
    {$sort: {x: 1, created: 1}},
    {$lookup: {as: 'user', from: 'users', localField: 'creator', foreignField: '_id'}},
    {$addFields: {user: {$arrayElemAt: ['$user', 0]}}},
    {$match: userQuery},
    {$project: {"_id": 1, "name": 1, "user.email": 1, "xyz": 1}}
]).toArray();

print(JSON.stringify(data));

2- Run

mongo server/collection file.js > out.json --quiet

3- Delete header and check out.json

[
  {
    "_id": "124564789",
    "xyz": [
      {
        "name": "x",
        "value": "1"
      },
      {
        "name": "y",
        "value": "2"
      }
    ],
    "name": "LOL",
    "user": {
      "email": "LOL@LOL.com"
    }
  },
....
....
}]

Answer 7

Try this for query-

mongoexport -d DataBase_Name -c Collection_Name --type=csv --fields name,phone,email -q '{_bank:true,"bank.ifsc":{$regex:/YESB/i}}' --out report.csv

Answer 8

I tried to run the export as suggested, but I'm getting csv flag deprecated

so here is a working snippet

mongoexport -d database_name -c collection_name -f field1,field_2 --type=csv > results.csv

Optionally, you can add a filter like the following

mongoexport -h 0.0.0.0 -d database_name -c collection_name --query '{"$or": [{"condition_one": true},{"condition_two": true}]}' --type=csv --fields "field1,field_2" > out.csv