1

Long story short

Where should I commit the Dockerfile? In the project codebase or in the devops codebase?

Reasoning details:

Without docker and without CI

In the ancient times, when developing a complex application with multiple code-bases, one normally wanted to have one repo per project and have all the passwords, credentials and dev/test/pre/prod configurations separated from the code.

+-----------------------------------------------------------------------+
|                                                                       |
|  +---------+       +---------+       +---------+       +---------+    |
|  |  app-1  |       |  app-2  |       |  app-3  |       |  app-4  |    |
|  +---------+       +---------+       +---------+       +---------+    |
|                                                                       |
|                            +----+                                     |
|                            |    |\                                    |
|                            |    +-+                                   |
|                            | conf |                                   |
|                            | files|                                   |
|                            +------+                                   |
|                                                                       |
+-----------------------------------------------------------------------+

In the old-ancient times one sysadmin installed the software in the server and then later copied the config files. Back in the 90's usually the sysop had those files in a directory of his own, shared only with the boss.

With CI but still without docker

Later we improved the cycle: in Continuos development/integration environments, "the system" itself needs to be able to clone all those repos and be able to "build" the applications and configure them to be ready to be run. Then copy the build into the servers and configure them accordingly.

This enables all the developers to trigger deploys at production, still not compromising the secret keys.

Before containers, typically the companies had an extra "devops" (AKA CI repo) where we had all those config files organized and know by an script. The CI server (pre-docker) knows all the source-code repos, knows the destination-network-topology, has the passwords to the cloud, and copies/builds/deploys everything in its destination and also configures it, making unnecessary the human intervention provided the servers are up and running.

+-----------------------------------------------------------------------+
|                                                                       |
|  +---------+       +---------+       +---------+       +---------+    |
|  |  app-1  |       |  app-2  |       |  app-3  |       |  app-4  |    |
|  +---------+       +---------+       +---------+       +---------+    |
|                                                                       |
|                          +----------------+                           |
|                          |     devops     |                           |
|                          +----------------+                           |
|                          | config-1-devel |                           |
|                          | config-1-pre   |                           |
|                          | config-1-prod  |                           |
|                          | config-2-devel |                           |
|                          |      [...]     |                           |
|                          | config-4-prod  |                           |
|                          +----------------+                           |
|                                                                       |
+-----------------------------------------------------------------------+

CI with Docker

When it comes to make docker play a role in the equation, I wonder if the correct place to have the Dockerfile is inside the application CVS repository or in the devops repository.

Will the Dockerfile go into the app code-base?

Unless we do an open-source code that needs to run in many platforms, usually the companies establish a target platform and the coders "know" the target system will be an Ubuntu, or a CentOs or so beforehand.

On the other hand it is now that the coders themselves touch the Dockerfile as one moe source-code file. This pushes us to think that the Dockerfile fits in each code-base as the app and the system it runs in will be -probably- coupled by needing certain requirements.

+-----------------------------------------------------------------------+
|                                                                       |
| +-------------+   +-------------+   +-------------+   +-------------+ |
| |    app-1    |   |    app-2    |   |    app-3    |   |    app-4    | |
| +-------------+   +-------------+   +-------------+   +-------------+ |
| |Dockerfile-1 |   |Dockerfile-2 |   |Dockerfile-3 |   |Dockerfile-4 | |   
| +-------------+   +-------------+   +-------------+   +-------------+ |
|                                                                       |
|                          +----------------+                           |
|                          |     devops     |                           |
|                          +----------------+                           |
|                          | config-1-devel |                           |
|                          | config-1-pre   |                           |
|                          | config-1-prod  |                           |
|                          | config-2-devel |                           |
|                          |      [...]     |                           |
|                          | config-4-prod  |                           |
|                          +----------------+                           |
|                                                                       |
+-----------------------------------------------------------------------+

Or will the Dockerfile go into the devops code-base (AKA the CI server code-base)?

But also it seems the programmer should do the very same lines of code, for example if he is coding a web application, despite it is run under an apache, an nginx or a caddy server... so the "decission" of the runtime seems it should be coded into the devops code-base:

+-----------------------------------------------------------------------+
|                                                                       |
| +-------------+   +-------------+   +-------------+   +-------------+ |
| |    app-1    |   |    app-2    |   |    app-3    |   |    app-4    | |
| +-------------+   +-------------+   +-------------+   +-------------+ |
|                                                                       |
|                          +----------------+                           |
|                          |     devops     |                           |
|                          +----------------+                           |
|                          | Dockerfile-1   |                           |
|                          | Dockerfile-2   |                           |
|                          | Dockerfile-3   |                           |
|                          | Dockerfile-4   |                           |
|                          +----------------+                           |
|                          | config-1-devel |                           |
|                          | config-1-pre   |                           |
|                          | config-1-prod  |                           |
|                          | config-2-devel |                           |
|                          |      [...]     |                           |
|                          | config-4-prod  |                           |
|                          +----------------+                           |
|                                                                       |
+-----------------------------------------------------------------------+

In the team we can't clarify the proper way and I've searched but I am unable to find documentation that demonstrates if the different Dockerfiles should be committed into the app repos or in the devops repo (AKA CI repo).

Where should I commit them?

Rudi
  • 19,366
  • 3
  • 55
  • 77
Xavi Montero
  • 9,239
  • 7
  • 57
  • 79

2 Answers2

5

I would suggest to keep it with your application as it should evolve whith the code base. IMHO best practice is to keep CI code and config with your app, not in separate repo, so you don't have to manage dependencies between app code version and configurations.

Kamil
  • 431
  • 3
  • 8
  • If you do so, aren't you placing secrets in the app repo? – Xavi Montero Aug 14 '19 at 01:52
  • 1
    Unencrypted secrets should be kept in a secure manner outside of git. But if you have some way to encrypt them you can commit them with your source control. For example if your CI/CD/devops is based on ansible, you can use ansible-vault. I think every CI/CD tool has some way to keep secrets safe, depends on what you use. – Kamil Aug 15 '19 at 10:42
  • Well in this schema I'm suggesting; the secrets are outside the app repo, and inside the devops repo. So you "clone" the devops repo and it's capable of installing everything... The "devops repo" is the "vault" in this case. The question is still alive: The secrets outside. The Dockerfile? In the devops approimation: in the "dev repo" or in the "ops repo"? – Xavi Montero Aug 15 '19 at 14:17
3

Dockerfile into the app code-base

Maybe if the organization have a few and not standardized applications or there are multiple languages with different strategies in the same company, the Dockerfile should be at repository level to allow direct modifications

But what happen if we are talking about dozens/hundreds of microservices?

In that case, the developer should not modify the Dockerfile because it was developed previously by the architect, technical lead or senior developer

Let's imagine a Dockerfile, entrypoint.sh and other required files that are the base for dozens of applications with same nature like java microservices in the same organization. Here some issues to consider if the Dockerfile is into the code-base:

  • If you need to change something in this Dockerfile, you will have to change in every git repository.
  • You must put an extra effort to validate developers don't break this Dockerfile, because is the same for all the others microservices. Can we imagine dozens of microservices, with different Dockerfiles, I mean each of those with its own architecture?
  • If all microservices needs a common file for docker build, you must put this file in every git repository! Example : ssh-key, tokens, scripts, artifact download keys, etc

Dockerfile into the devops code-base

My advice, based on my dozens of applications, is just what you mentioned. Here some advantages:

  • Just one Dockerfile for all my microsevices (for example).
  • If I need to upgrade or fix something, just I need to change one Dockerfile and nothing else.
  • Developers could see and understand this Dockerfile, but never break it.

C.I Platform

If you choose to put Dockerfile into the devops code-base instead into every git repository in your organization, you must need to develop a flow something like this:

  • Developer pushes code to git repository
  • C.I Platform receive the notification
  • C.I Platform clones the app git repository
  • C.I Platform clones the devops code base, which contains all of Dockerfile of your organization
  • C.I Platform determine the nature of app, to select the correct Dockerfile and other files like entry-point.sh, etc
  • C.I Platform copy the Dockerfile into the app source code root folder.
  • C.I Platform performs a docker build ...
  • C.I Platform performs a docker push ... if you have a private docker registry (recommended)
  • Finally, C.I Platform performs an instantiation (docker run) of the docker image in any of remote server (with docker previously installed)

I could recommend you Jenkins, due to its ease of use

Config Files

I advice you, if possible, do not use complex files at build stage of your applications. Open source technologies are good to do that, but if you are using some proprietary language, you're toast :S

Anyway, if you need config files at build stage, you could use:

JRichardsz
  • 14,356
  • 6
  • 59
  • 94
  • I am agree with your Dockerfile go into the devops code-base solution, but I am wondering how developer can access Dockerfile? Is it good idea to use git submodule in order to store Dockerfile in microservice repo? – parsa2820 Aug 09 '22 at 09:07
  • 1
    Dockerfile into the devops code-base is for massive and standardized microservices/webs. In that case, the developer should not modify the Dockerfile because was developed previously by the architect or technical lead or senior developer. If the app is not standardized or there are multiple languages with different strategies in the same company, the Dockerfile should be at repository level to allow direct modifications – JRichardsz Aug 09 '22 at 14:42