File write operations in mongo script?

Question

Is this possible to write the query result to the file from mongo js script. I have searched a lot, but I didn't find any solution.

ex:-

cursor = db.users.find();

while(cursor.hasNext()) {
  cursor.next();
  // writing the cursor output to file ????<br/>
}

btw, if you're always iterating the whole collection than it's more efficient to use mongodump. using a script is more flexible, you can do all kinds of processing before printing results. — milan, Jan 23 '12 at 15:14
more on [mongodump](http://stackoverflow.com/questions/8991292/dump-mongo-collection-into-json-format) — averasko, Dec 30 '15 at 18:55

milan · Accepted Answer · 2012-01-23T15:13:32.797

84

You could use print and then redirect output:

script.js:

cursor = db.users.find();
while(cursor.hasNext()){
    printjson(cursor.next());
}

then run the script and redirect output to a file:

mongo --quiet script.js > result.txt

edited Jan 23 '12 at 15:13

answered Jan 23 '12 at 13:07

milan

11,872
3
42
49

1

yes, it's possible. But i am trying to run the script as daemon. So only I asked for file write operations. thanks. – sudesh Jan 23 '12 at 14:45
daemon process' output can also be redirected to a file. you could write a shell script around it, to invoke mongo on demo.js and redirect to a file. – milan Jan 23 '12 at 15:02
1

How can I specify what database the script should use? – Nelu Nov 09 '13 at 05:35
How do I import the data to another mongodb instance afterwards? :) – Per Lundberg Nov 13 '13 at 15:17
2

@NeluMalancea you simply can use any Mongo shell command line args. So to specify the database on a remote host, e.g. mongo --host MYSERVER DATABASENAME --quiet script.js > result.txt – AndrewL Dec 19 '13 at 12:52
2

@NeluMalancea You can use ``dr= new Mongo("127.0.0.1:27017").getDB("dr")`` then you prefix your collections with dr. That will enable you to work with more than one database; i.e. read from one copy to another etc. – kenzie May 02 '14 at 22:31
3

this answer doesn't fulfill the original intent. is there any way to write to a file from a mongo script? – comiventor May 22 '15 at 06:48
@comiventor JavaScript cannot write files, well with Local Storage as an exception (NodeJs has it's own file package to do this) – milan May 22 '15 at 06:56
@milan I understand the constraint from javascript perspective but isn't this too restrictive from mongo shell perspective? :( dunno why they chose JS for mongo shell – comiventor May 22 '15 at 07:16
I prefer `mongo db.example.com/live --quiet --eval "db.foo.find().forEach(function(r){ print(JSON.stringify(r)) })" -u live -p $(< config/db_secret) > /tmp/foo.ljson` because it doesn't print a bunch of useless white space. Saves time and disk space, and now line count roughly matches record count. Also note this isn't ACTUALLY JSON, because each record dict is just concatenated without separator or enclosing list. – Abram Oct 09 '16 at 08:13

score 22 · Answer 2 · edited Nov 01 '19 at 00:56

22

http://www.mongodb.org/display/DOCS/Scripting+the+shell paragraph "Differences between scripted and interactive/ Printing".

./mongo server.com/mydb --quiet --eval "db.users.find().forEach(printjson);" > 1.txt

edited Nov 01 '19 at 00:56

treecoder

43,129
22
67
91

answered Oct 11 '12 at 00:17

Roman Ivanov

2,477
3
20
31

This approach gets tricky if you want to put a query inside the find() because of the character escaping. In that case, best to use @milan 's answer above with a separate query file – AndrewL Dec 19 '13 at 12:53
@AndrewL Can you use single quotes instead? I think that's supposed to not do as much escaping or something, right? – Erhannis Dec 05 '14 at 15:18
For printing certain fields/custom-format, use: ".forEach( function(d) { print( d.id ); } );". – Curtis Yallop Dec 08 '14 at 23:39
Yes, you can use single quotes eg in the find filter. – Curtis Yallop Dec 08 '14 at 23:40

score 15 · Answer 3 · answered Aug 22 '18 at 15:39

15

Whenever I need to write the result of a mongo query to a local file I generally use the the writeFile(pathToFile, stringContents) function.

Example: let's say that you quickly need to find the email of all registered users and send it to your buddy Jim in the marketing department.

$ mongo mongodb://my-fancy-mongo-server --ssl -u fancy_username -p fancy_password 
successfully connected to my-fancy-mongo-server!
> emails = db.users.distinct('email_address')
> writeFile("jims_email_list.json", tojson(emails))

or if Jim expect's a CSV file then

$ mongo mongodb://my-fancy-mongo-server --ssl -u fancy_username -p fancy_password 
successfully connected to my-fancy-mongo-server!
> emails = db.users.distinct('email_address')
> writeFile("jims_email_list.csv", emails.join("\n"))

You can now send Jim the list of emails and save the day!

To important things to notice about the writeFile function:

The second argument must be a string.
The first argument must be a file that doesn't already exist, otherwise you will get an error.

answered Aug 22 '18 at 15:39

fernandohur

7,014
11
48
86

This sounds great, but where does `writeFile` come from? It is not defined in my mongo CLI. I also can't seem to find any documentation on it (but maybe I didn't look hard enough?). Perhaps I have too old of a mongo client? – pkaeding Mar 01 '19 at 05:27
2

Its part of the NodeJS fs API. See here: https://nodejs.org/api/fs.html#fs_fs_writefile_file_data_options_callback – fernandohur Mar 01 '19 at 16:15
2

Mongo often tends to add some more exotic functions without documenting them. ```writeFile``` is one such example. It is normal mongo shell function, just they decided not to document it. There is a lot of undocumented functions in mongo, including several more file system I/O functions, functions that allow user to input data from keyboard, functions that allow execution of OS commands from the mongo shell, and so on. NodeJS is not implemented in mongo shell, because most other functions from NodeJS do not work in mongo shell. ```writeFile``` and a few more do, but very few. – SYOB SYOT Jun 25 '19 at 08:33
It is just a coincidence that mongo shell has a couple of functions with the same name as NodeJS. – SYOB SYOT Jun 25 '19 at 08:36

score 14 · Answer 4 · edited Sep 29 '14 at 18:38

14

You can skip the while loop using forEach():

db.users.find().forEach(printjson);

edited Sep 29 '14 at 18:38

JasonMArcher

14,195
22
56
52

answered Mar 14 '13 at 15:13

Oubenal Mohcine

491
4
8

7

Even more compact: db.users.find().forEach(printjson); – AndrewL Dec 19 '13 at 12:58

Darren Newton · Answer 5 · 2012-01-23T12:39:45.687

Wouldn't it be simpler to use one of the Mongo drivers for a general purpose language (such as Python, Ruby, Java, etc) and write your results to a file that way, in a format you can use (such as CSV, etc.)?

UPDATE: According to the documentation for mongodump you can export a collection with a query:

$ ./mongodump --db blog --collection posts
-q '{"created_at" : { "$gte" : {"$date" : 1293868800000},
                      "$lt"  : {"$date" : 1296460800000}
                    }
    }'

However you would need to import that collection back into MongoDB to operate on it or use mongoexport to export as JSON or CSV using the same query flag (-q) as mongodump.

thanks for your reply, but I want this to be in plain js which run using mongo shell. — sudesh, Jan 23 '12 at 12:21

score 2 · Answer 6 · edited Jun 08 '15 at 10:59

2

For distinct you have to create script.js file with contents:

mongo = new Mongo("localhost");
doctor = mongo.getDB("doctor");
users = doctor.getCollection("users");
cities = users.distinct("address.city");
printjson(cities);

then in console run:

mongo --quiet script.js > result.txt

edited Jun 08 '15 at 10:59

josliber

43,891
12
98
133

answered Jun 08 '15 at 10:36

1nstinct

1,745
1
26
30

File write operations in mongo script?

6 Answers6

Linked