0

I am using Mongo Version 2.6.12 on CestOS6.7. It is non-clustered i.e. installed only on one server.

My project requires final output to be stored in MongoDB. The files are CSVs stored on Linux and HDFS. Can someone please guide me how to import CSV files into mongo DB from Linux/HDFS.

Mongoimport does not work as it takes by default input as JSON files. I tried this in mongo-shell, it failed

mongoimport news-output-20170125.csv --type csv

2017-01-30T19:18:25.275-0600 SyntaxError: Unexpected identifier

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
TextShilpa
  • 21
  • 5
  • If the files are on Linux and HDFS, why does the HDFS matter? Anyways, `Unexpected identifier` seems self-explanatory. Read the docs... `The csv parser accepts that data that complies with RFC 4180` – OneCricketeer Jan 31 '17 at 01:25
  • This might help you http://stackoverflow.com/questions/27112384/moving-hdfs-data-into-mongodb – BruceWayne Jan 31 '17 at 06:32
  • thanks. my files are both on Linux and Hdfs. I need a way to import files to mongodb from either of them. the above file is on linux which is giving error when i do mongoimport. – TextShilpa Jan 31 '17 at 17:23
  • Also, the file is perfectly in csv format with below test inputs: Sally Whittaker,2018,McCarren House,312,3.75 Belinda Jameson,2017,Cushing House,148,3.52 Jeff Smith,2018,Prescott House,17-D,3.20 Sandy Allen,2019,Oliver House,108,3.48 – TextShilpa Jan 31 '17 at 18:05

1 Answers1

0

Mongoimport from Linux is working for CSV file now. I was running mongoimport from mongo-shell. hence was getting the error.

[mongoritter@**** ~]$ cat locations.csv
Name,Address,City,State,ZIP
Jane Doe,123 Main St,Whereverville,CA,90210
John Doe,555 Broadway Ave,New York,NY,10010
[mongoritter@**** ~]$ pwd
/home/mongoritter
[mongoritter@**** ~]$ mongoimport -d test -c test1 --type csv --file locations.csv –headerline
connected to: 127.0.0.1
2017-01-31T12:12:22.034-0600 imported 2 objects

for more help, you can refer How to use mongoimport to import csv

Now, I will check way for import from HDFS. I have seen a mongo document, will check if this works or not https://www.mongodb.com/blog/post/using-mongodb-hadoop-spark-part-1-introduction-setup

Thanks, SHilpa

Community
  • 1
  • 1
TextShilpa
  • 21
  • 5