77

What's the simplest way to store the application secrets (passwords, access tokens) for a Python script? I thought it'd be a *.yml file like in Ruby but surprisingly I found that it wasn't the case. So what is it then? What are the most simplest solutions?

I want to put them in a separate file because that way I'll be able not to push that file to a GitHub repository.

Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
Incerteza
  • 32,326
  • 47
  • 154
  • 261

3 Answers3

109

I think storing credentials inside another *py file is your safest bet. Then just import it. Example would look like this

config.py

username = "xy"
password = "abcd"

main.py

import config
login(config.username, config.password)
kecer
  • 3,031
  • 2
  • 21
  • 24
  • actually, I want to write something as well, either in this file or in another one. – Incerteza Aug 26 '14 at 12:15
  • 9
    Be aware that arbitrary code in config.py will be executed. This is a concern especially if main.py runs in a different trust zone than config.py. – jtpereyda Jul 13 '17 at 18:59
  • You can also pickle a `(username, password)` tuple to a file and load it whenever. – Eric Ed Lohmar Mar 20 '18 at 19:55
  • 6
    Also, having the *py file extension on the credential file would make it more likely to get pushed to a gh repo by mistake! – mgrollins Jun 12 '19 at 19:02
  • 1
    @jtpereyda so use `from config import username, password` to prevent anything else in config.py being executed, right? – Jonathan May 28 '20 at 13:03
  • 3
    @Jonathan In that case, all the code in config is still run. So if you don't trust `config.py`, that is still risky. If you don't trust `config.py` as much as the application code, this is a dangerous approach no matter how you tweak it. – jtpereyda May 28 '20 at 22:00
  • 1
    @jtpereyda Thanks - I've learned something today. Keep safe... – Jonathan May 29 '20 at 13:49
  • I think this is a bad idea. You credentials are stored in plain text this way. I'm looking into something like keyring https://pypi.org/project/keyring/ to fix this. – Hans de Ridder Jun 18 '21 at 10:01
  • I think this is not ideal as well. It would be very easy to accidentally upload this to a repo, as like Hans says it's also in plain text. The best way would to have stored on something like azure keyvault or something similar, but since they want to keep it in another file, I think using yaml or some other strategy not using python files is better overall. – Scott Jermaine Guyton Jun 21 '21 at 02:26
  • This is definitely not the recommended/ best approach for storing credentials. Anyone can read them in a prod environment if it is a shared server – dgor Aug 23 '21 at 06:30
  • from configsecrets import EMAIL, API_SECRET_KEY is somewhat more readable for the new colleague, who just cloned without the extra config file. – user18099 Aug 30 '22 at 08:45
17

I was dealing exactly the same question and actually ended up with the same solution as kecer suggested. Since I need to use it in dozens of scripts, I've created own library. Let me share this solution with you.

credlib.py -- universal library to handle credentials

class credential:
    def __init__(self, hostname, username, password):
        self.hostname = hostname
        self.username = username
        self.password = password

mycredentials.py -- my local file to store all credentials

from credlib import credential
sys_prod = credential("srv01", "user", "pass")
sys_stg = credential("srv02", "user", "pass")
sys_db = credential("db01", "userdb", "passdb")

mysystemlib.py -- this is a general library to access my system (both new credential system and legacy is supported)

from credlib import credential

def system_login(*args): # this is new function definition
#def system_login(hostname, username, password): # this was previous function definition

    if len(args) == 1 and isinstance(args[0], credential):
        hostname = args[0].hostname
        username = args[0].username
        password = args[0].password
    elif len(args) == 3:
        hostname = args[0]
        username = args[1]
        password = args[2]
    else:
        raise ValueError('Invalid arguments')

    do_login(hostname, username, password) # this is original system login call

main.py -- main script that combines credentials and system libs

from mycredentials import sys_stg, sys_db
import mysystemlib
...
mysystemlib.system_login(sys_stg)

Please note that the legacy hostname/username/password way still works so it does not affect old scripts:

mysystemlib.system_login("srv02", "user", "pass")

This has a lot benefits:

  • same credential system across all our python scripts
  • files with passwords are separated (files can have more strict permissions)
  • files are not stored in our git repositories (excluded via .gitignore) so that our python scripts/libs can be shared with others without exposing credentials (everyone defines their own credentials in their local files)
  • if a password needs to be changed, we do it at a single place only
Stevoisiak
  • 23,794
  • 27
  • 122
  • 225
CraZ
  • 1,669
  • 15
  • 24
  • 1
    What is the .gitignore rule to ensure that this does NOT get pushed by mistake to a public repo! – mgrollins Jun 12 '19 at 19:03
  • 2
    In our case, our rule of thumb is to name all credential files as 'cred_*.py' so this's what's we have in .gitignore. Of course, this does not prevent human mistake from saving credential files with different names. Though, we never have had a problem with this. The most important for us is to separate code and credentials in separate files. – CraZ Jun 12 '19 at 23:01
12

Personally I prefer to use yaml files, with the pyyaml library. Documentation here: https://pyyaml.org/wiki/PyYAMLDocumentation

Creating a .gitignore rule is very quick and painless and there is zero chances of making a mistake. You can added the rule with echo on Linux / UNIX like system with:

echo -e '*.yaml\n*.yml' >> .gitignore

Below is an example of retrieving the settings from a settings .yaml file in the same folder / location of the reader.

Code Snippets:

#!/usr/bin/env python3

import yaml
from pathlib import Path


def get_settings():
    full_file_path = Path(__file__).parent.joinpath('settings.yaml')
    with open(full_file_path) as settings:
        settings_data = yaml.load(settings, Loader=yaml.Loader)
    return settings_data
Scott Jermaine Guyton
  • 151
  • 2
  • 10
  • Feeling a bit uncomfortable with the inner workings of the import statement, I like this solution as the path to the yaml file can even be an absolute path outside of the project tree for local settings. – Johannes Ranke Nov 08 '21 at 10:44