I have a directory, with several sub-directories, holding a total of about a million JSON files. I need to import this entire thing into mongoDB. I imagine this is a very common problem, but I cannot find any tutorial for how to do so. Is there an easy solution here?
(Or should I write a script to iterate through the directories, read each file into a variable and then insert the content into my db?)
You're basically on the right path.
Iterate through the entire directory, read the files to create a JSON object in your code, and then just store your documents directly into MongoDB.
Take a look at this.
One big problem though: MongoDB stores your documents into Collections, with the condition that every document in your collection has the same general format/structure. Basically each document must have most, if not all of the same properties as each other. Otherwise you're going to have to use something different, or store all of the JSON as a property of an encapsulating document that you can then store.
Something like the following:
{
'_id': 'YOUR_DOCUMENT_ID',
'doc': 'JSON_OR_STRING_OF_YOUR_FILE'
}
