Dockerfile create image with both python and mysql

Question

I have two containers "web" and "db". I have an existing data file in csv format.

The problem is I can initialize the MySQL database with a schema using docker-compose or just run with parameters but how can I import the existing data? I have Python script to parse and filter the data and then insert it to db but I cannot run it in the "db" container due to the single image is MySQL.

Update1

version: '3'
services:
  web:
    container_name: web
    build: .
    restart: always
    links:
      - db
    ports:
      - "5000:5000"
  db:
    image: mysql
    container_name: db
    command: --default-authentication-plugin=mysql_native_password
    restart: always
    environment:
      MYSQL_DATABASE: "test"
      MYSQL_USER: "test"
      MYSQL_PASSWORD: "test"
      MYSQL_ROOT_PASSWORD: "root"
      MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
    ports:
      - "33061:3306"

There is a Python script for read data from a csv file and insert them to database, which works fine. Now I want to running the script once the MySQL container is set up. (I have done connection with Python and MySQL in container)

Otherwise, anyone has a better solution to import existing data?

You could probably spin up a third, transient container just to run the import once. Share your `docker-compose.yml` and we can take it from there! — Max, Mar 08 '19 at 20:57
Do you have an existing Docker Compose setup you can share? This sounds like a pretty typical setup connecting a Python script to a database in a different container. — David Maze, Mar 08 '19 at 22:11
@Max I have succeeded in the same method as you suggest. But would that be a little be redundant? Since once the data imported, there is no need for the third container existing. — Yabo Lee, Mar 08 '19 at 22:35
@DavidMaze I coonect them with Python script but I want to set up them with existing data. — Yabo Lee, Mar 08 '19 at 22:36

Mostafa Hussein · Accepted Answer · 2019-03-09T14:37:44.210

2

MySQL docker image has the ability to execute shell scripts or sql files if these script/sql files mounted under /docker-entrypoint-initdb.d for a running container as described in here and here. So I suggest you to write an SQL file that reads the CSV file (which you should mount to your container so the sql file can read it) in order to restore it to MySQL maybe something similar to this answer or write a bash script to import csv into mysql whatever works for you.

You can check Initializing a fresh instance at the official dockerhub page for mysql

edited Mar 09 '19 at 14:37

answered Mar 08 '19 at 22:45

Mostafa Hussein

11,063
3
36
61

In your solution, it is not possible to modify the data in the container before importing to the database. I am really a fresh hand on Docker. Is it the best way to make the separation of data and application? – Yabo Lee Mar 10 '19 at 00:18
Can you elaborate more ? You said that you already have the data as a CSV right ? – Mostafa Hussein Mar 10 '19 at 00:19
Yep. I have a csv file which is in someone's github. I want to git clone it and modify some formats of the data before importing to the database like converting % to decimal. – Yabo Lee Mar 10 '19 at 00:21
So in this case you can add a bash script that clone the data and modify it then importing it into MySQL – Mostafa Hussein Mar 10 '19 at 00:22
Do you mean modify the data with bash? – Yabo Lee Mar 10 '19 at 00:27
Yes instead of using the python script which requires python to be installed – Mostafa Hussein Mar 10 '19 at 00:28
Actually, that is what I asked... If I use MySQL image, I cannot own Python at the same time. – Yabo Lee Mar 10 '19 at 00:30
I suggested this way which provided officially by mysql image without the need to have a 3rd container, as bash can make the job done as possible which could be a better solution – Mostafa Hussein Mar 10 '19 at 00:32
Do you need to do it through a 3rd container ? – Mostafa Hussein Mar 10 '19 at 00:33
I used the 3rd container and would like to find a better solution. Thanks a lot! – Yabo Lee Mar 10 '19 at 00:41

score 0 · Answer 2 · answered Mar 08 '19 at 22:23

0

From Dockerfile, you can call a script (Entrypoint). In this script you can call your python script. For example:

DockerFile:

FROM php:7.2-apache

RUN apt-get update    
COPY ./entrypoint.sh /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

This will run your entrypoint script in the App container. Make sure you've depends on attribute in you app container compose description.

answered Mar 08 '19 at 22:23

heyitsvajid

1,023
1
10
19

I want to make separation of data and application. The data should be in the "db" container but I am looking for a method to import the data to MySQL. – Yabo Lee Mar 08 '19 at 22:39

Dockerfile create image with both python and mysql

2 Answers2