Run `apt-get update` before installing other packages with Puppet

Question

I'm trying to create puppet module which automates installation of zend server CE, this is not important here, but steps are as following

update /etc/apt/source.list
download repos key via wget
do apt-get update
do apt-get install zend-server-ce-5.2

I have init.pp file

class zendserverce {

# https://github.com/puppetlabs/puppetlabs-stdlib
file_line { 'debian_package':
    path => '/etc/apt/sources.list',
    line => 'deb http://repos.zend.com/zend-server/deb server non-free'
}

exec { "wget http://repos.zend.com/zend.key -O- |apt-key add -":
    path => ["/usr/bin", "/usr/sbin"]
}

exec { "apt-get update":
    command => "/usr/bin/apt-get update",
    onlyif  => "/bin/sh -c '[ ! -f /var/cache/apt/pkgcache.bin ] || /usr/bin/find /etc/apt/* -cnewer /var/cache/apt/pkgcache.bin | /bin/grep . > /dev/null'",
}

package { "zend-server-ce-php-5.2":
    ensure => "latest"
}

}

Seems that puppet runs commands in different order then I need. Is there any way how to for tell him to run in my desired order?

The output of such snippet is

  [0;36mnotice: /Stage[main]/Mc/Package[mc]/ensure: ensure changed 'purged' to 'latest'[0m
  [1;35merr: /Stage[main]/Zendserverce/Package[zend-server-ce-php-5.2]/ensure: change from purged to latest failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install zend-server-ce-php-5.2' returned 100: Reading package lists...
  Building dependency tree...
  Reading state information...
  E: Couldn't find package zend-server-ce-php-5.2 at /tmp/vagrant-puppet/modules 0/zendserverce/manifests/init.pp:28[0m
  [0;36mnotice: /Stage[main]/Zendserverce/Exec[wget http://repos.zend.com/zend.key -O- |apt-key add -]/returns: executed successfully[0m
  [0;36mnotice: /Stage[main]/Zendserverce/File_line[debian_package]/ensure: created[0m
  [0;36mnotice: Finished catalog run in 6.75 seconds[0m

So it says: Couldn't find package zend-server-ce-php-5.2

Can anyone guide me what is wrong?

maybe this can help http://docs.puppetlabs.com/guides/language_guide.html#run-stages — Jaro, Jun 01 '12 at 09:49
Just pointing out a side issue: your apt-get update will run only once and never execute again while the files in `/etc/apt/**` arent changed. This can lead to stale update information when installing other packages. You may also want to add this bash check that returns true if the apt-get update was executed more than 1 day ago: `[[ $(( $(date +%s) - $(stat -c %Z /var/cache/apt/pkgcache.bin) )) -gt $(( 24 * 60 * 60 )) ]]` — Bruno Penteado, Jan 19 '13 at 18:36
A quick note re your `apt-key add` invocation. Fetching something over HTTP and then passing it to `apt-key` makes you vulnerable to a variety of security attacks including MITM. It would appear zend don't offer HTTPS access to this key which is a shame. If you can manually verify the key, it would be safer to push it out via another method, perhaps `file { ... }` — jmtd, Feb 06 '13 at 20:13
Have you tried the puppetlabs-apt module to handle the management of repo and keys? This is not directly related to your question :) — Trefex, Jan 22 '15 at 11:04

DrDol · Answer 1 · 2013-07-25T16:19:26.580

99

Since Puppet 2.6.0 a new feature "relationship syntax" was introduced.

An example in Puppet 2.6.0 and above would look like this:

exec { "apt-update":
    command => "/usr/bin/apt-get update"
}

Exec["apt-update"] -> Package <| |>

Every time a package command is executed, the dependency (in our case 'apt-update') will be triggered fist. You can even define longer chains.

edited Jul 25 '13 at 16:19

answered Dec 01 '12 at 00:43

DrDol

2,220
2
19
23

1

I actually use stages for this, but I have to admit that this trick is pretty nifty! – Aktau Jun 24 '13 at 14:57
12

Thanks, this was super helpful! For other novices like me wondering about that last line: "<|" is collector notation, see http://docs.puppetlabs.com/puppet/3/reference/lang_collectors.html – yungchin Oct 15 '13 at 12:30
2

This seems to run on every puppet update whether a package install is necessary or not. Is there a way to tighten this up so that apt-get update only runs when a package is missing from the system? – r3cgm Aug 06 '14 at 16:39
1

@r3cgm What is your package 'ensure' setting? If it happens also for 'ensure' => 'installed', the relationship syntax could be changed to: Exec["apt-update"] -> Package <| ensure == latest |> But then for every non 'ensure => latest' package, apt-update will not run. – DrDol Aug 07 '14 at 09:30
1

Thanks @DrDol! We have ensure set to 'present' but may be moving toward an ensure 'latest' model soon. I think we kind of have a catch 22 issue here. If we bind an apt-get update against "ensure == installed" then it'll run the apt-get update every time before seeing if the package is there. That's the original scenario we're trying to avoid; running the apt-get update all the time. But if we bind apt-get update against "ensure == latest" only then whenever we find a missing package we'll have missed out on doing the apt-get update before doing the install and we might install something old. – r3cgm Aug 15 '14 at 17:52
1

It's actually not possible to have Puppet sync the `exec` only if the dependent package resources need syncing. Such things only work by sending events through `notify/subscribe`. This in turn would only work if the packages do synchronize before the `exec`. A middle ground may be found in a `schedule` to have the `update` run only once per night. - Generally note that the collector syntax has a drawback - it will realize **all** virtual package resources that may end up being part of the manifest. – Felix Frank Aug 27 '14 at 00:23

score 52 · Accepted Answer · answered Jun 01 '12 at 19:06

52

You need to specify the dependency relationships. The easiest/cleanest approach is to use the require parameter which is available for all resource types.

package { "zend-server-ce-php-5.2":
  ensure  => latest,
  require  => Exec['apt-get update'],
}

etc..

answered Jun 01 '12 at 19:06

czervik

2,537
23
15

5

I read http://docs.puppetlabs.com/guides/language_guide.html#run-stages and puppet provides stages so you can explicitly specify ordering of execution. Run stage were added in Puppet version 2.6.0, you now have the ability to specify any number of stages which provide another method to control the ordering of resource management in puppet. – Jaro Jun 02 '12 at 09:03
1

This may be "easiest/cleanest" but is it maintainable? If the Exec needed to change then you have to go back and replace every declaration, the point being this approach isn't referential. – Brandon Cook Jun 08 '13 at 17:42
3

@BrandonCook Then you can give your exec a new name. `exec{update: command => "apt-get update"}` then reference it as `Exec[update]`. – Kevin Cox Aug 26 '14 at 19:54
If you do this for a number of repos, then you might wind up doing more `apt-get update`s than you'd like to. Puppet's "stages" are a feature to be careful of, but this is the best use case I know of. – mc0e Feb 25 '16 at 16:21

score 14 · Answer 3 · edited Apr 25 '17 at 17:29

I tried previous variant but it doesn't work for me on Ubuntu 10.04

Finaly I prepared the following script, that runs update everytime the repository is older than one week:

exec { 'apt-get update':
    command => "/usr/bin/apt-get update",
    onlyif => "/bin/bash -c 'exit $(( $(( $(date +%s) - $(stat -c %Y /var/lib/apt/lists/$( ls /var/lib/apt/lists/ -tr1|tail -1 )) )) <= 604800 ))'"
}

Hope it helps.

score 13 · Answer 4 · edited Aug 21 '23 at 15:24

13

I prefer to put apt-upgrade into a separate stage running before the main stage, so I don't have to hard-wire any dependencies. Check here: https://www.puppet.com/docs/puppet/8/lang_run_stages.html.

A simple example would look like below. It implies you have a separate class for doing the actual apt-update:

stage { "init": before  => Stage["main"] }

class {"apt-update": 
  stage => init, 
  apt_mirror => $apt_mirror 
}

Check my sample LAMPP-box on github to see how the pieces fit together: https://github.com/joerx/vagrant-lampp

Note: be careful with apt-upgrade, as some base boxes break by things like kernel upgrades.

edited Aug 21 '23 at 15:24

akuzminsky

2,190
15
21

answered Sep 03 '13 at 03:34

joerx

2,028
1
16
18

1

Awesome. This is a pretty awesome feature I didn't know about. Thanks for sharing. – Thomas Welton Jan 22 '14 at 10:46
Link is dead like a mouse. – kaiser May 06 '14 at 19:59
Fixed the link, but the relevant part of the manifest is commented out now. You can still see it though. Nothing is static ... – joerx May 07 '14 at 04:08

Brandon Cook · Answer 5 · 2013-09-14T20:03:41.137

4

In Puppet 3 this can be done by realizing virtual resources using resource collectors

# so you don't have to fully qualify paths to binaries
Exec { path => ['/usr/bin'] }    

# virtual resource
@exec { 'sudo apt-get update':
   tag => foo_update
}

# realize resource. filter by arbitrary "foo_update"
# tag and relate it to all Package resources
Exec <| tag == foo_update |> -> Package <| |>

edited Sep 14 '13 at 20:03

answered May 19 '13 at 23:00

Brandon Cook

1,362
11
11

This doesn't work for me on 3.2.4 unless I change to fully qualified paths. `@exec { '/usr/bin/sudo /usr/bin/apt-get update`... – spuder Sep 07 '13 at 05:06
Good call I tested this having long ago added `Exec { path => ['/usr/bin'] }` – Brandon Cook Sep 14 '13 at 20:04

score 4 · Answer 6 · answered Jul 16 '14 at 16:23

4

Adding this snippet of voodoo worked for us:

  Apt::Pin <| |> -> Package <| |>
  Apt::Source <| |> -> Package <| |>

This forced an update. YMMV.

answered Jul 16 '14 at 16:23

TimP

925
1
8
13

score 2 · Answer 7 · answered Apr 29 '17 at 23:49

Package that needs updating APT lists should require Class['apt::update']

package { "zend-server-ce-php-5.2":
   ensure => "latest",
   require => Class['apt::update']
}

If you're using custom APT source, just ensure correct ordering:

Apt::Source['my_source'] 
-> Class['apt::update']

score 0 · Answer 8 · answered May 03 '19 at 15:51

You should really be using the apt module to create sources and add keys: https://forge.puppet.com/puppetlabs/apt

If you're using hiera:

apt::sources:
  'artifactory-pro-debs':
    location: 'http://repos.zend.com/zend-server/deb'
    release: 'server
    repos: 'non-free'
    key:
      source: 'http://repos.zend.com/zend.key'

Run `apt-get update` before installing other packages with Puppet

8 Answers8

Linked