git - how to deal with files which will contain sensitive data in the future?

Question

I have a file which contains database passwords db/connect.php.

I created the file and provided the structure for it without passwords:

<?php

$db_host = "localhost";
$db_user = "";
$db_pw = "";
$db_name = "";

$con = new mysqli($db_host, $db_user, $db_pw, $db_name);

/* check connection */
if ($con->connect_errno) {
    printf("Connect failed: %s\n", $con->connect_error);
    exit();
}

Then I commited the file. In the next commit I added the file to .gitignore.

But I noticed that it is tracked anyways. I figured out that git will track all files which are already in the repository, so the only way would be to remove it with git rm --cached db/connect.php, but then users who clone the repository will need to create it again...??

My question is not about rewriting the git history, there are no sensitive information in the repository. My question is how I can provide the filestructure but not track it after that point so that anything entered in the file should not getting tracked...

How to solve this problem?

Is your question how to structure your php code such that it can deal with the missing file, how to structure the php code in that file so it doesn't require secrets to be embedded within it, how to delete the file permanently from the git history (because you've previously committed secrets or similar) - or something else? Note that your question is currently only tagged `git`. Please edit your question to be clear, and tagged appropriately. — AD7six, Sep 12 '19 at 08:01
My question is how I can provide the filestructure but not track it after that point so that anything entered in the file should not getting tracked... — Black, Sep 12 '19 at 08:02
ok so that's `how to structure the php code in that file so it doesn't require secrets to be embedded within it` - the question is still not tagged php and there's very little `git` in that. Here's an example/hint https://github.com/cakephp/app/blob/master/.gitignore#L3 — AD7six, Sep 12 '19 at 08:05
@AD7six, thats because the question is not specifically about php, it could be any language. — Black, Sep 12 '19 at 08:06
There is no every-language answer, your question _is_ specifically about php, unless "don't put secrets in tracked files" is all you wish to take away from your question. Anyway, good luck :) — AD7six, Sep 12 '19 at 08:08

score 3 · Accepted Answer · answered Sep 12 '19 at 08:11

The correct approach here is to store a template file in your repository and either provide instructions or an automated way to create the actual file locally. This actual file would be the one being ignored by .gitignore, whereas the template file would be tracked as any other file.

For instance, you could create a file named connect_credentials.php.template with this content (bear in mind that I know nothing about PHP so the below is not meant to be real PHP at all, just copy-paste and slight editing from your question):

<?php

$db_host = "localhost";
$db_user = "";
$db_pw = "";
$db_name = "";

If you can provide useful default values in this template file, all the better but otherwise leave it in such a way that it will not work at all, and forces the programmer to fix something.

This file would be tracked as normal, and then you would provide instructions or an automated way to make a copy from this file to the real file, connect_credentials.php. This file, however, would be ignored in .gitignore:

connect_credentials.php

and never tracked. The programmer, after cloning or updating, would need to create this file him- or her-self, from the template, and edit to match the local development environment.

The file can be a real code file, such as a php file, it can be a configuration file that you read, like an ini file, a json or xml file, yaml, or similar, but the same approach would be useful regardless of the actual file type or content. Track a template, ignore the actual file. This allows programmers to also be able to see the format of this file and be aware of changes to it, like new configuration options.

Unfortunately, since you've already created the real file in there, I'd say you need to decide how to handle it. As you noticed, if you now remove the file to stop tracking it, anyone updating their personal repository and working folder will lose the file as well. You might want to consider creating a new and separate file for the actual credentials, as I've given an example of above, to avoid this deletion issue.

score 1 · Answer 2 · answered Sep 12 '19 at 07:53

1

You simply told Git to stop tracking a previously tracked file. So you can still access revisions of this file up to point where you told Git to ignore it.

If there were revisions with sensitive data you can rewrite the Git history with either git rebase or git filter-branch.

⚠️ Rewriting history can be a dangerous exercise if you need to keep every single commit a "working" commit (e.g. if you were to rollback to a previous commit that has been rewritten what guarantee do you have that it still "works"?). Also rewriting a shared history will potentially put your collaborators (or your own) branches at risk of non-trivial merge conflicts.

Having said that I don't think you should put credentials in source files. Use a configuration file (json, yaml, ini) that you can generate as part of your project setup or build and keep that file away from Git (i.e. ignore it).

Additionally you can have Git hooks that could reject any attempts to commit these sort of files into the Git repository.

answered Sep 12 '19 at 07:53

customcommander

17,580
5
58
84

My question is not about rewriting the git history, there are no sensitive information in the repository. My question is how I can provide the filestructure but not track it after that point so that anything entered in the file should not getting tracked... – Black Sep 12 '19 at 07:55
1

I understand. I think I provided some options though: move credentials into a configuration file that is ignored and autogenerate that file as part of your project setup. – customcommander Sep 12 '19 at 07:57
1

Thx, I think I will create a ini file which is ignored by git and not in the repository and load the data from that file into db/connect.php – Black Sep 12 '19 at 07:59
`So you can still access revisions of this file up to point where you told Git to ignore it.` @customcommander note that this is not true - adding a file/pattern to gitignore doesn't also delete it. You may already know this, but readers may not and the suggestion that gitignore affects tracked files is a common point of confusion - example https://stackoverflow.com/a/25436481/761202 – AD7six Sep 12 '19 at 08:15
@AD7six I see what you're getting at but I don't think that is what I suggested and even if the file is deleted, you _can_ still access revisions of it (unless you take it out of the entire history). I'm not sure what is misleading in that sentence. I do take your point though. – customcommander Sep 12 '19 at 08:39

score 1 · Answer 3 · answered Sep 12 '19 at 08:17

An easy solution is to create a file, e.g. database.template.php, add the content to it, but put simple default values to it (e.g. $db_host ="localhost"; $db_user="root"; ...). Add database.php to the ignore file. Instruct repo users to copy the template file to database.php and configure it locally.

Another more dangerous option is not to add the file to the ignore list and for repo users to simply ignore changes in their index when they update the file:

git update-index --skip-worktree path/to/file

This is more dangerous however because repo users are more likely to accidentally check the file in anyway.

score -2 · Answer 4 · answered Sep 12 '19 at 07:52

-2

The solution is easy: DO NOT STORE SENSITIVE CREDENTIALS IN GIT. Move your sensitive data to a configuration file or environment variables. If a file, never check that in, except an invalid "demo" file. If you use env variables, add a readme which explains which variables are needed.

use a library to read your configuration data at runtime at use that where you need the passwords.

answered Sep 12 '19 at 07:52

Christian Sauer

10,351
10
53
85

1

As you can see I do not store any sensitive information in git, thats exactly what I try to prevent and what my question is about... – Black Sep 12 '19 at 07:53

git - how to deal with files which will contain sensitive data in the future?

4 Answers4