17

I have a DB called wordlists and a collection called rockyou.

I have a file called rockyou with the contents:

123456
12345
123456789
password
iloveyou
princess
1234567
rockyou
12345678
abc123

What I would like to do is load all of the contents of the file rockyou into the collection rockyou in my wordlists DB. I have tried searching google but the resources seem slim for mongo. I could really use some help with this and also maybe some good resources for this new to me DB application.

Thanks!

**EDIT* Here are the command I have used and some of what I have tried.

use wordlists
db.createCollection("rockyou")
db.rockyou.insert(copyFile(~/wordlists/rockyou))

I have tried some other methods to insert but now realize there were even further off than this one...

Community
  • 1
  • 1
Dylan
  • 339
  • 1
  • 3
  • 10
  • @SalvadorDali if the resources are so abundant please give me a link on how to insert a plain, non JSON format text file and I will give you all of my reputations points. – Dylan Mar 26 '15 at 22:44
  • @Dylan how exactly do you want your collection to look like? – Salvador Dali Mar 26 '15 at 22:58
  • Each item containing each line from my text file. I plan to have my DB contain multiple collections, each consisting of a line from their corresponding text file and as events happen involving an entry from my collection I will add another column with data from that event. But basically just a single column containing each line of the text file – Dylan Mar 26 '15 at 23:02
  • @Dylan I was not upset because you do not know how to solve your problem (I do not know how to solve a lot of my problems as well), I was disappointed that you have not even showed your attempt. Mostly three things have contributed to my comment: 1) you posted just a requirement (before your update), which looks like do my work for me. 2) you have not stated how do you want your collection to look like, what tools would you like to use. 3) you blamed the lack of resources which is not true. Please in the future show what have you tried and what went wrong. – Salvador Dali Mar 26 '15 at 23:23

2 Answers2

45

If you really want to use only mongoshell, you can use cat() command and do the following (txt is not necessary, it is just how my file was named):

use wordlists
var file = cat('path/to/yourFile.txt');  // read the file
var words = file.split('\n'); // create an array of words
for (var i = 0, l = words.length; i < l; i++){ // for every word insert it in the collection
    db.rockyou.insert({'word': words[i]}); 
}

This was tested on Mongo 3.0.1 and produced something like:

{ "_id" : ObjectId("551491ee909f1a779b467cca"), "word" : "123456" }
{ "_id" : ObjectId("551491ee909f1a779b467ccb"), "word" : "12345" }
...
{ "_id" : ObjectId("551491ee909f1a779b467cd3"), "word" : "abc123" }

But I would introduce an application logic here (for example with python):

import pymongo
connection = pymongo.Connection()
collection = connection.wordlists.rockyou

with open('path/to/yourFile.txt') as f:
    for word in f.readlines():
        collection.insert({'word': word.rstrip()})
ovanes
  • 5,483
  • 2
  • 34
  • 60
Salvador Dali
  • 214,103
  • 147
  • 703
  • 753
  • "if the resources are so abundant please give me a link on how to insert a plain, non JSON format text file and I will give you all of my reputations points." <-- This was not a link but I am still happy to give them to you. Besides creating a bounty in two days how do I give them to you now? – Dylan Mar 26 '15 at 23:29
  • @Dylan I added a link. The bounty is the only option I am aware of, but to tell the truth it is not compulsory. I just wrote this for fun. Your question would become way better if you would specify that you need only to use mongoshell (if you can use anything, the problem is straight forward, but with mongoshell it is harder to find the solution). – Salvador Dali Mar 26 '15 at 23:33
  • lol I will edit my question to include that info. This method wont work on my large file so I will look into those other options, I am not stuck using only the shell. Guess I will look into one of those application abstractions you talked about, like pymongo. Thanks and I'll create the bounty in 2 days and let you know so you can answer and I can award it to you ;) I said I would! – Dylan Mar 26 '15 at 23:36
  • I know javascript, so would you suggest Node.js then? – Dylan Mar 26 '15 at 23:37
  • @Dylan you can use any programming language ([node](http://docs.mongodb.org/ecosystem/drivers/node-js/) /php/C#, whatever). I included python, because I had pycharm opened and because python is installed on all linux machines by default. – Salvador Dali Mar 26 '15 at 23:40
  • Ok great indo, I am in the MongoDB Driver page so I will take it all into consideration.... Thanks again and I'm serious about the bounty. Talk to you in a couple days – Dylan Mar 26 '15 at 23:49
  • @SalvadorDali what if your queries are stored in a file. I tried to read a file of queries and in the loop, I am doing queries[i] but it is not working. A sample query is db.coll.find(). I am measuring the time taken and it takes 0 sec and this takes 14 secs in normal mongo shell. – fanbondi Jun 15 '16 at 12:18
  • cat command worked, saved my day. I can now use cat command to load json files to mongo shell scripts. – shijin Aug 09 '18 at 20:18
0

The legacy mongo shell cat method is replaced in mongosh with fs.readFileSync(filename, 'utf8' )

Viacheslav Shchupak
  • 356
  • 3
  • 4