How do I build and install a Puppet module locally from source?

Question

I found a puppet module whose authors just added a bunch of modifications that I would otherwise have to make myself, manually.

This literally occurred 20 hours ago.

I have downloaded the master branch to a zip file, extracted it to my modules folder, and run puppet module build --verbose /etc/puppet/modules/arioch-redis, to no avail.

I understand that this is definitely not normal behavior, but I feel this question deserves some attention.

There is definitely a use case for using a module from source, especially in projects that move quickly, or rely on quick feedback.

I have done some research on this, and please feel free to ask, "have you looked into X" questions in the comments.

For instance, I looked over https://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.html#build-your-module, but it seems mainly geared toward uploading a module to the Forge, rather than installing locally from source.

Plus, my attempt fails at the "Build Your Module" section.

All three of the following seem to answer this question with "install librarian-puppet", but given that we can write our own modules, it seems silly to have to use librarian-puppet to use the source code from a publicly available git repository to use a simple module:

Changing puppet module source (I'm looking for an answer to what happens after the OP would create the private repo in that answer. )

How to install a puppet file from a local repo with puppet module install (My Excuse: I don't need to use only the puppet module install tool. I can use other puppet tools, too.)

Using puppet module straight from a cloned repo (My Excuse: Answer does not seem relevant to this question.)

Installing a puppet module from a manifest script (My Excuse: Again, the best answer, which I think is perfect for that question, is "don't use Puppet for this, find a workaround." In this case, the workaround was to have Vagrant partially use Shell Provisioning.)

If this really is a duplicate, or a summary, please say so in the comments, and maybe I can explain. Edits welcome!

Do I need to hack the metadata.json contents?

Should I clone this repository somewhere and link it into my own temporary Forge account, or something?

Summary:

How might I get the veeeeerry latest build of the module, when it's passing in Travis on its Git repository, but before the Forge makes it available?

Answer 1

Puppet modules downloaded from the forge are still folders, the puppet module command is just untarring them and downloading dependencies with the API.

So your approach was close, you just need to make sure the folder has the right directory name:

So for your example:

$ puppet module list
/Users/foo/.puppet/modules
├── fiddyspence-sysctl (v1.1.0)
├── puppetlabs-apt (v1.8.0)
├── puppetlabs-aws (v1.0.0)
├── puppetlabs-nodejs (v0.7.1)
└── puppetlabs-stdlib (v4.6.0)
$ cd /Users/foo/.puppet/modules
$ ll
total 0
drwxr-xr-x  15 foo  staff  510 Mar 17  2015 apt
drwxr-xr-x  18 foo  staff  612 Mar 26  2015 aws
drwxr-xr-x  14 foo  staff  476 Jan 22  2015 nodejs
drwxr-xr-x  17 foo  staff  578 Apr 15  2015 stdlib
drwxr-xr-x  12 foo  staff  408 Feb  9  2015 sysctl
$ wget https://github.com/arioch/puppet-redis/archive/master.zip
--2015-12-02 11:56:24--  https://github.com/arioch/puppet-redis/archive/master.zip
Resolving github.com... 192.30.252.130
Connecting to github.com|192.30.252.130|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/arioch/puppet-redis/zip/master [following]
--2015-12-02 11:56:24--  https://codeload.github.com/arioch/puppet-redis/zip/master
Resolving codeload.github.com... 192.30.252.144
Connecting to codeload.github.com|192.30.252.144|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 29509 (29K) [application/zip]
Saving to: 'master.zip'

master.zip                    100%[==================================================>]  28.82K  --.-KB/s   in 0.1s

2015-12-02 11:56:25 (257 KB/s) - 'master.zip' saved [29509/29509]

$ unzip master.zip
Archive:  master.zip
4fb33b960a09bfb459eff20ee112bfc9e0a1c096
   creating: puppet-redis-master/
  inflating: puppet-redis-master/.fixtures.yml
  inflating: puppet-redis-master/.gitignore
 extracting: puppet-redis-master/.puppet-lint.rc
  inflating: puppet-redis-master/.travis.yml
  inflating: puppet-redis-master/Gemfile
  inflating: puppet-redis-master/LICENSE
  inflating: puppet-redis-master/README.md
  inflating: puppet-redis-master/Rakefile
   creating: puppet-redis-master/manifests/
  inflating: puppet-redis-master/manifests/config.pp
  inflating: puppet-redis-master/manifests/init.pp
  inflating: puppet-redis-master/manifests/install.pp
  inflating: puppet-redis-master/manifests/params.pp
  inflating: puppet-redis-master/manifests/preinstall.pp
  inflating: puppet-redis-master/manifests/sentinel.pp
  inflating: puppet-redis-master/manifests/service.pp
  inflating: puppet-redis-master/metadata.json
   creating: puppet-redis-master/spec/
   creating: puppet-redis-master/spec/classes/
  inflating: puppet-redis-master/spec/classes/redis_sentinel_spec.rb
  inflating: puppet-redis-master/spec/classes/redis_spec.rb
 extracting: puppet-redis-master/spec/spec.opts
  inflating: puppet-redis-master/spec/spec_helper.rb
   creating: puppet-redis-master/templates/
  inflating: puppet-redis-master/templates/redis-sentinel.conf.erb
  inflating: puppet-redis-master/templates/redis-sentinel.init.erb
  inflating: puppet-redis-master/templates/redis.conf.erb
$ mv puppet-redis-master/ redis
$ ll
total 64
drwxr-xr-x  15 foo  staff    510 Mar 17  2015 apt
drwxr-xr-x  18 foo  staff    612 Mar 26  2015 aws
-rw-r--r--   1 foo  staff  29509 Dec  2 11:56 master.zip
drwxr-xr-x  14 foo  staff    476 Jan 22  2015 nodejs
drwxr-xr-x  14 foo  staff    476 Nov 30 15:10 redis
drwxr-xr-x  17 foo  staff    578 Apr 15  2015 stdlib
drwxr-xr-x  12 foo  staff    408 Feb  9  2015 sysctl

You'll notice it complains about missing dependancies.

$ puppet module list
Warning: Module 'puppetlabs-apt' (v1.8.0) fails to meet some dependencies:
  'arioch-redis' (v1.1.3) requires 'puppetlabs-apt' (>= 2.0.1 <3.0.0)
Warning: Missing dependency 'stahnma-epel':
  'arioch-redis' (v1.1.3) requires 'stahnma-epel' (>= 1.0.2 <2.0.0)
/Users/foo/.puppet/modules
├── arioch-redis (v1.1.3)
├── fiddyspence-sysctl (v1.1.0)
├── puppetlabs-apt (v1.8.0)  invalid
├── puppetlabs-aws (v1.0.0)
├── puppetlabs-nodejs (v0.7.1)
└── puppetlabs-stdlib (v4.6.0)

As far as I'm aware: There's no way to resolve dependencies from a local module using Puppet's module command.

That's where librarian-puppet is comes in:

$ cd redis/
$ librarian-puppet install --path ../. --verbose
[Librarian] Ruby Version: 2.1.2
[Librarian] Ruby Platform: x86_64-darwin14.0
[Librarian] Rubygems Version: 2.4.8
[Librarian] Librarian Version: 0.6.3
[Librarian] Librarian Adapter: puppet
[Librarian] Librarian Adapter Version: 2.2.1
[Librarian] Project: /Users/foo/.puppet/modules/redis
[Librarian] Specfile: Puppetfile
[Librarian] Lockfile: Puppetfile.lock
[Librarian] Git: /opt/boxen/homebrew/bin/git
[Librarian] Git Version: 2.4.3
[Librarian] Git Environment Variables:
[Librarian]   GIT_PS1_SHOWDIRTYSTATE=true
[Librarian]   GIT_PS1_SHOWSTASHSTATE=true
[Librarian]   GIT_PS1_SHOWUNTRACKEDFILES=true
[Librarian]   GIT_PS1_SHOWUPSTREAM=auto
[Librarian] Pre-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian] Specfile /Users/foo/.puppet/modules/redis/Puppetfile not found, using defaults
[Librarian] Post-Cached Sources:
[Librarian]   [:forge, "https://forgeapi.puppetlabs.com", {}]
[Librarian] The specfile is unchanged: nothing to do.
[Librarian] Install: dependencies resolved
[Librarian] Installing puppetlabs-stdlib/4.9.0 <https://forgeapi.puppetlabs.com>
[Librarian] Installing puppetlabs-apt/2.2.0 <https://forgeapi.puppetlabs.com>
[Librarian] Installing stahnma-epel/1.2.0 <https://forgeapi.puppetlabs.com>
$ puppet module list
/Users/petersouter/.puppet/modules
├── arioch-redis (v1.1.3)
├── fiddyspence-sysctl (v1.1.0)
├── puppetlabs-apt (v2.2.0)
├── puppetlabs-aws (v1.0.0)
├── puppetlabs-nodejs (v0.7.1)
├── puppetlabs-stdlib (v4.9.0)
└── stahnma-epel (v1.2.0)

If you don't want to use puppet-librarian, you could also just update those modules manually:

$ puppet module install puppetlabs-apt -v 2.0.1 --force
Notice: Preparing to install into /Users/petersouter/.puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/Users/foo/.puppet/modules
└── puppetlabs-apt (v2.0.1)

$ puppet module install stahnma-epel -v 1.2.0 --force
Notice: Preparing to install into /Users/petersouter/.puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/Users/foo/.puppet/modules
└── stahnma-epel (v1.2.0)