Puppet Dependency file failing in local Vagrant Environmnet

Question

Below is my Puppet code for a Python Django App.

class ltp ($project_dir = '/vagrant') {

    exec {
        "install_project_dependencies":
            cwd => $project_dir,
            command => "/usr/bin/python setup.py install > /tmp/setuplog 2>&1",
            timeout => 0,
    }

    exec {
        "install_nltk_data":
            cwd => $project_dir,
            command => "su - vagrant -c \"cd /opt/vagrant; /usr/bin/python -m nltk.downloader stopwords > /tmp/nltklog 2>&1\"",
            require => Exec['install_project_dependencies']
    }

    exec {
        "install_django_db":
            cwd => $project_dir,
            user => 'vagrant',
            environment => ["ENV=DEV"],
            command => "/usr/bin/python manage.py syncdb --noinput > /tmp/syncdblog 2>&1",
            require => Exec['create_ba_database']
    }

    exec {
        "create_ba_database":
            cwd => $project_dir,
            command => "echo \"psql -c 'create database burson_clips;'\" | sudo su postgres",
            require => Exec['change_pg_password']
    }

    exec {
        "change_pg_password":
            cwd => $project_dir,
            command => "sudo -u postgres psql -c \"alter user postgres with password 'postgres';\""
    }

}

Below is the error Vagrant gives when I run vagrant provision

clips: Error: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[install_project_dependencies]/returns: change from notrun to 0 failed: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Warning: /Stage[main]/Ltp/Exec[install_nltk_data]: Skipping because of failed dependencies
==> clips: Notice: /Stage[main]/Ltp/Exec[change_pg_password]/returns: executed successfully
==> clips: Notice: /Stage[main]/Ltp/Exec[create_ba_database]/returns: ERROR:  database "burson_clips" already exists
==> clips: Notice: /Stage[main]/Ltp/Exec[install_django_db]: Dependency Exec[create_ba_database] has failures: true
==> clips: Error: echo "psql -c 'create database burson_clips;'" | sudo su postgres returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[create_ba_database]/returns: change from notrun to 0 failed: echo "psql -c 'create database burson_clips;'" | sudo su postgres returned 1 instead of one of [0]
==> clips: Warning: /Stage[main]/Ltp/Exec[install_django_db]: Skipping because of failed dependencies

Did any body received this error previously? Can any body let me know if any area of code I should look into. This happens on my local vagrant setup.

Update

So I tried Vagrant destroy and recreate the dependency now I am getting error.

==> clips: Error: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[install_project_dependencies]/returns: change from notrun to 0 failed: /usr/bin/python setup.py install > /tmp/setuplog 2>&1 returned 1 instead of one of [0]
==> clips: Warning: /Stage[main]/Ltp/Exec[install_nltk_data]: Skipping because of failed dependencies
==> clips: Notice: /Stage[main]/Ltp/Exec[change_pg_password]/returns: executed successfully
==> clips: Notice: /Stage[main]/Ltp/Exec[create_ba_database]/returns: executed successfully
==> clips: Error: /usr/bin/python manage.py syncdb --noinput > /tmp/syncdblog 2>&1 returned 1 instead of one of [0]
==> clips: Error: /Stage[main]/Ltp/Exec[install_django_db]/returns: change from notrun to 0 failed: /usr/bin/python manage.py syncdb --noinput > /tmp/syncdblog 2>&1 returned 1 instead of one of [0]

Adding setup.py

#!/usr/bin/env python
from setuptools import setup

# fix setuptools:
# wget http://bootstrap.pypa.io/ez_setup.py -O - | sudo python

install_requires = [
    'Django==1.8',
    'django-celery',
    'celery==3.1.23',
    'ipdb',
    # 'ipython',
    'django-kombu',
    'kombu',
    'billiard',
    'eventlet',
    #    'uwsgi',
    'xlutils',
    'pip-tools',
    'djangorestframework',
    'markdown',
    'django-filter',
    'django-cors-headers',
    'pyyaml',
    'sh',
    'pika==0.9.12',
    'pyparsing',
    'pyrabbit',
    'lxml==3.3.5',
    'xlrd',
    'raven',
    'six',
    'requests',
    'uritemplate',
    'twitter-text-python',  # for the ttp

    'python-dateutil==2.4.0',
    'redis',
    'python-docx',  ##for opening word documents
    'sqlparse',
    'httplib2',
    'simplejson',
    'requests==2.5.1',
    'six>=1.7.3',
    'django-organizations',
    'django_extensions',
    'feedparser==5.1.3',

    # The entry for this on pypi is f'd up.
    # pip install https://github.com/timothycrosley/RedisStore/blob/master/dist/RedisStore-0.1.tar.gz?raw=true
    # 'RedisStore',


    'xlsxwriter',
    'django-grappelli',

    # MDM: setuptools/pip can't install these
    # 'gensim',
    # 'numpy',
    # 'scikit-learn',
    # 'matplotlib',

    # MDM: only pip can install this one!
    # 'CairoSVG',

    'xmltodict',
    'twitter',
    'pygal',
    'tinycss',
    'cssselect',
    'dashboard_common==1.3.3',
    'pytz',
    'greenlet',
    'xlsxwriter',
    'pattern',
    'oauth2',

    'wordcloud',

    # MDM this is a dep-lib cluser-f
    # pip install it last so it can f up dependencies how ever it likes
    # 'python-social-auth',
    # 'newspaper', ##for article extraction

    'python-saml',
    'python-openid',
    'xmlsec',
    'paramiko',

    # has apt-get/brew deps
    'psycopg2'
]


# sudo -H pip install CairoSVG

dependency_links = [
    # fails unless vagrant box has a .ssh identity w/ git hub & trust github
    'workinggithublink',

]

A module with only exec commands? I would suggest either using a module or two (i.e. django or postgres) with some intrinsic types and providers, or using something other than Puppet since these are sequential commands (shell or Ansible comes to mind). — Matthew Schuchard, Aug 25 '16 at 12:53

score 1 · Accepted Answer · edited May 23 '17 at 12:06

1

The error is this

==> clips: Notice: /Stage[main]/Ltp/Exec[create_ba_database]/returns: ERROR:  database "burson_clips" already exists

==> clips: Error: echo "psql -c 'create database burson_clips;'" | sudo su postgres returned 1 instead of one of [0]

your db already exists as burson_clips so the command fails and the rest of the provisioning fails.

You should be able to leverage some answers from Check if database exists in PostgreSQL using shell to check if the db exists first and then create only if needed; something like

exec {
    "create_ba_database":
        cwd => $project_dir,
        command => "echo \"psql -c 'create database burson_clips;'\" | sudo su postgres",
        require => Exec['change_pg_password'],
        unless => "psql -lqt | cut -d \| -f 1 | grep -qw 'burson_clips'"
}

edited May 23 '17 at 12:06

Community

1
1

answered Aug 25 '16 at 12:26

Frederic Henri

51,761
10
113
139

But this happens when I try a fresh provision on a blank Ubuntu Box? Where should I check for burson_clips database? – Piyush Patil Aug 25 '16 at 12:28
I don't know - where is your postgres db installed ? do you install locally on the VM ? – Frederic Henri Aug 25 '16 at 12:33
No there is no postgres installed locally? – Piyush Patil Aug 25 '16 at 12:44
what the box you use ? is there any postgres install in your `setup.py` script ? (refer to the file /tmp/setuplog as there might be errors there too) – Frederic Henri Aug 25 '16 at 12:50
This is the error in setup log Installed /tmp/easy_install-mGuI6o/xmlsec-0.6.1/Cython-0.24.1-py2.7-linux-x86_64.egg In file included from src/xmlsec/constants.c:288:0: src/xmlsec.h:1:27: fatal error: xmlsec/xmlsec.h: No such file or directory #include ^ compilation terminated. error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 – Piyush Patil Aug 25 '16 at 12:56
well, I am not sure about that - is `xmlsec/xmlsec.h` present ? at this stage you might want to spin up the VM without any provisioning and run the scripts manually from the VM; once it works you can script in puppet to automate the tasks – Frederic Henri Aug 25 '16 at 13:05
did you check the `/tmp/setuplog` ? I believe you have the same issue with the `xmlsec` - you would need to show the `setup.py` script so we can support you here – Frederic Henri Aug 25 '16 at 20:33
I added my setup.py – Piyush Patil Aug 25 '16 at 20:36
note from https://github.com/mehcode/python-xmlsec _If you get any fatal errors about missing .h files, update your C_INCLUDE_PATH environment variable to include the appropriate files from the libxml2 and libxmlsec1 libraries._ also make sure to have required dependencies installed `apt-get install libxml2-dev libxmlsec1-dev` – Frederic Henri Aug 26 '16 at 11:15

Puppet Dependency file failing in local Vagrant Environmnet

1 Answers1