How to import data to mongodb container and creating an image

Question

I spent a lot of time on a task which seems a simple one. I want to run a monogodb container. then copy csv file in to the container or execute a command from docker file to import the csv file into the mongodb database. I tried using RUN, script.sh but nothing is working. Please help. I want to automate the task and when I copy the data, I will make a new image for my automated task.

Here is the Dockerfile I have.

FROM mongo
RUN mkdir -p /app/data
WORKDIR /app/data
COPY gene.csv /app/data
COPY script.sh /app/data
EXPOSE 27017
# CMD ["/app/data/script.sh"]

The script file is

#! /bin/sh 
mongoimport -d Gene -c genes --type csv --file gene.csv --headerline

Why @DanGreen-Leipciger? I can't see any possible reason to choose node for such a ridiculously lightweight task, but I have no expertise in it so genuine question. — johnharris85, Apr 23 '17 at 19:20
It's just quick and easy, if you already use it. Runs by default in cli, `$node myscript.js`. It also takes no configuration and runs on any machine. — Dan Green-Leipciger, Apr 23 '17 at 19:22
Also, MongoDB has an amazing node interface and all of the said commands are available. Plus, the OP would have access to all of the node tools available on NPM should the needs grow more complex. — Dan Green-Leipciger, Apr 23 '17 at 19:24
This is so common thing to do, I wonder why its not already on the net. What actually I am doing is: I am creating a web app and I have written a docker-compose.yml file. If I am able to enter the data and make an image, all the users will do is docker-compose up and browse to localhost:3000/genes to see the data. — Dr. Mian, Apr 23 '17 at 19:30
@Dan Green-Leipciger could you please write in detail your answer. — Dr. Mian, Apr 23 '17 at 19:33
@DanGreen-Leipciger I guess. I mean in a mongo container I don't see any reason you'd have node, or see any reason that node's interface with mongo would give me such an unbelievable experience over bash just for running a single command, for all the extra bloat it would put in the image. Just my 2/c though. — johnharris85, Apr 23 '17 at 19:42

score 2 · Answer 1 · answered Mar 17 '19 at 21:51

2

Too late, maybe help to others.

    FROM mongo
    RUN mkdir -p /app/data
    WORKDIR /app/data
    COPY gene.csv /app/data
    COPY script.sh /app/data
    CMD ["mongod", "&&", "mongoimport", "-d", "Gene", "-c", "Genes", "--file", "gene.csv", "--headerline"]

answered Mar 17 '19 at 21:51

ishak O.

168
1
2
14

I will check it out. Thanks for the reply – Dr. Mian Mar 17 '19 at 22:54

johnharris85 · Answer 2 · 2017-04-23T20:09:41.800

1

Edit: This answer solves a problem, but not the problem, as apparently mongo has to be running for any of this to work. In that case a custom entrypoint is probably the way to go.

Using RUN is the right way, as then when you use CMD you can actually run the Mongo process. However your problem is with your COPY instructions.

COPY gene.csv /app/data
COPY script.sh /app/data

These lines need a / on the end, as right now they say 'copy this file from my host, to be this file in my container'. Whereas you're trying to say 'copy this file from my host, into this folder in my container. They should read as below (but can be simplified even further, see final Dockerfile at the end):

COPY gene.csv /app/data/
COPY script.sh /app/data/

Your script will then run if you put it in a RUN instruction and that layer will be committed into the image, then you can have a CMD at the end to run mongo (or just leave it blank and inherit the default ENTRYPOINT / CMD from the parent (mongo) image. So the full Dockerfile becomes:

FROM mongo
WORKDIR /app/data        # This will be created if it doesn't exist
COPY gene.csv .
COPY script.sh .
RUN chmod +x script.sh && sync && ./script.sh

edited Apr 23 '17 at 20:09

answered Apr 23 '17 at 19:18

johnharris85

17,264
5
48
52

Thanks for help. For the above docker file I got the following error Step 5/5 : RUN ./script.sh ---> Running in 1d385c7df879 /bin/sh: 1: ./script.sh: Permission denied The command '/bin/sh -c ./script.sh' returned a non-zero code: 126 – Dr. Mian Apr 23 '17 at 19:24
I got this error now Step 5/5 : RUN chmod +x script.sh && sync && ./script.sh ---> Running in 6de14ed40581 2017-04-23T19:45:30.083+0000 [#.......................] Gene.genes 4.00KB/57.6KB (6.9%) 2017-04-23T19:45:30.598+0000 [#.......................] Gene.genes 4.00KB/57.6KB (6.9%) 2017-04-23T19:45:30.598+0000 Failed: error connecting to db server: no reachable servers 2017-04-23T19:45:30.598+0000 imported 0 documents The command '/bin/sh -c chmod +x script.sh && sync && ./script.sh' returned a non-zero code: 1 – Dr. Mian Apr 23 '17 at 19:45
I read somewhere an hour ago that you have to restart db or something like that. Dont know why they made it so complicated. – Dr. Mian Apr 23 '17 at 19:46
I'm not that familiar with mongo. Does mongo need to be running for this to work? If so then we'll need a different solution. – johnharris85 Apr 23 '17 at 19:50
If we give FROM mongo CMD ["usr/bin/mongod", "--smallfiles"] before executing the script. Same error, – Dr. Mian Apr 23 '17 at 19:52
Right, because `CMD` is a runtime instruction. You will likely need to create a custom entrypoint that starts mongo then runs your mongoimport statement. – johnharris85 Apr 23 '17 at 20:06
Any idea, how :) – Dr. Mian Apr 23 '17 at 20:11
About to go out, but will try and edit the answer later. – johnharris85 Apr 23 '17 at 20:16
Many thanks for your help ;) Let see if I get it there :)] – Dr. Mian Apr 23 '17 at 20:24

score 0 · Answer 3 · edited May 23 '17 at 12:25

0

I was able to do that but through another container. Whose job is to insert the data in to the mongodb database container and exit. For more details check the accepted answer here. I changed the JSON file with my csv file and the command which import it, although I found that mongodb works better with json.

edited May 23 '17 at 12:25

Community

1
1

answered Apr 28 '17 at 15:51

Dr. Mian

3,334
10
45
69

How to import data to mongodb container and creating an image

3 Answers3