1

I have a mongoddb DB called 'CRM' and in that there is a collection "users".

There are 50,000+ users into "users" collection.

I want to export all users documents from "users" collection to separate .json file.

I ran following command which gives me all users into a single (users.json)json file as an array of json objects. 

mongoexport -d CRM -c users -o users.json --jsonArray

But, I want .JSON file for each users.

Any help would greatly be appreciated.

user54858
  • 11
  • 3
  • There is no way with MongoDB to do that. You probably should write a script that reads every document from your array and create a file. It s not really complicated ;-) – bappr May 16 '17 at 08:53
  • Is it possible to load 50000+ documents into an array and then one by one write data into file? – user54858 May 17 '17 at 05:46
  • Yes it is. mongoexport will do the export of your 50K documents, then your script will read your file. But my opinion is that you have better to do a script reading the collections and creating a file for each document... you could scale it, it would be more performant – bappr May 17 '17 at 06:08
  • Thanks. I will try it and post the status. – user54858 May 17 '17 at 09:08

1 Answers1

1

jq helped me get this done.

The output file from Mongo is not a valid JSON file, rather individual lines of respectively valid JSON content. The steps I used were:

  • iterated through each line,
  • parse it to get the ID (you can use your own field here)
  • use the ID as the file name
  • write the line/document to a file

Here's the bash code I ultimately used:

# Make newlines the only separator.
IFS=$'\n'
# Disable globbing.
set -f          

# Iterate through the export file
for doc in $(cat < export.txt); do
  # $doc is a valid JSON document. Parse it to get the filename
  # based on the ID.
  export filename=`echo $doc | jq -r '._id'`.json

  # Write document to file.
  echo "Writing $filename"
  echo $doc > $filename
done

I learnt this from these Stack Exchange answers:

I'm just not sure how well this will work for 50, 000 records.

Batandwa
  • 530
  • 4
  • 7
  • 18