3

I can't seem to figure out how to get Puppet to not run 'apt-get update' during every run.

The standard yet inefficient way:

The way I've been doing this is with the main Puppet manifest having:

exec { 'apt-get update':
  path => '/usr/bin',
}

Then each subsequent module that needs a package installed has:

package { 'nginx':
  ensure => 'present',
  require => Exec['apt-get update'],
}

The problem with this is that, every time Puppet runs, Apt gets updated. This puts unnecessary load on our systems and network.

The solution I tried, but fails:

I looked in the Puppet docs and read about subscribe and refreshonly.

Refresh: exec resources can respond to refresh events (via notify, subscribe, or the ~> arrow). The refresh behavior of execs is non-standard, and can be affected by the refresh and refreshonly attributes:

  • If refreshonly is set to true, the exec will only run when it receives an event. This is the most reliable way to use refresh with execs.

subscribe

One or more resources that this resource depends on, expressed as resource references. Multiple resources can be specified as an array of references. When this attribute is present:

  • The subscribed resource(s) will be applied before this resource.

so I tried this in the main Puppet manifest:

# Setup this exec type to be used later.
# Only gets run when needed via "subscribe" calls when installing packages.
exec { 'apt-get update':
  path => '/usr/bin',
  refreshonly => true,
}

Then this in the module manifests:

# Ensure that Nginx is installed.
package { 'nginx':
  ensure => 'present',
  subscribe => Exec['apt-get update'],
}

But this fails because apt-get update doesn't get run before installing Nginx, so Apt can't find it.

Surely this is something others have encountered? What's the best way to solve this?

svenglar
  • 157
  • 3
  • 9
  • Surely you have to run apt-get, to know if there's an update available. So it has to run every time, otherwise you wouldn't know. – Daniel Scott Sep 12 '14 at 05:34
  • 1
    @DanielScott If a package is already installed, and it's set to `ensure => 'present'`, there shouldn't be a need to update the apt database. I'm looking for a way to only run `apt-get update` if the package is not installed. You're right that, if I want the package updated (`latest`), I'd want to run apt-get update, but not if I just want it `present`. – svenglar Sep 12 '14 at 09:50
  • 1
    You can take a look to this previous answer: http://stackoverflow.com/a/13655214/1671973 – Emyl Sep 12 '14 at 09:55
  • 1
    I'm not sure if there's a way to do this using 'package'. You might be able to use 'apt' for this? There's an option 'always_apt_update': https://forge.puppetlabs.com/puppetlabs/apt – Daniel Scott Sep 12 '14 at 09:56
  • @Emyl: That previous answer always runs `apt-get update` and so is not relevant to this question. It only makes sure that `apt-get update` is run before packages are installed. – Peter V. Mørch Apr 03 '19 at 15:13

3 Answers3

3

Puppet has a hard time coping with this scenario, because all resources are synchronized in a specific order. For each resource Puppet determines whether it needs a sync, and then acts accordingly, all in one step.

What you would need is a way to implement this process:

  1. check if resource A (a package, say) needs a sync action (e.g., needs installing)
  2. if so, trigger an action on resource B first (the exec for apt-get update)
  3. once that is finished, perform the operation on resource A

And while it would be most helpful if there was such a feature, there currently is not.

It is usually the best approach to try and determine the necessity of apt-get update from changes to the configuration (new repositories added, new keys installed etc.). Changes to apt's configuration can then notify the apt-get upate resource. All packages can safely require this resource.

For the regular refreshing of the database, it is easier to rely on a daily cronjob or similar.

Felix Frank
  • 8,125
  • 1
  • 23
  • 30
  • Man... that... SUCKS! Thanks for the answer. This was the final straw. I've had nothing but headache and head-banging with puppet if I wanted to _anything_ but the _most trivial_ of operations. Puppet has got to go! Now all my `package` logic will be moved to `exec { '/my/scripts/checkPackagesInstalled.pl'}` and be done with it. – Peter V. Mørch Apr 03 '19 at 15:15
  • I guess using this approach, then using `package { sompkg: ensure => present }` is out of the question too, huh? I'll basically need to re-implement `package` or use `/some/file/that/sompkg/installs` as a proxy for whether it is installed? – Peter V. Mørch Apr 03 '19 at 18:19
  • Well `ensure => present` will try and perform the installation at that time, yes. If that's OK, sure. But if you need Puppet to do anything *before* that, but only if the package is not yet installed, then yes, you need to cobble something up that generates a change event only if the package isn't there (e.g. an `exec` of something innocuous like `echo`, with an appropriate `onlyif` condition). – Felix Frank Apr 15 '19 at 12:46
0

I run 'apt-get update' in a cron script on a daily basis, under the assumption that I don't care if it takes up to 24 hours to update OS packages via apt. Thus...

file { "/etc/cron.daily/updates":
    source => "puppet:///modules/myprog/updates",
    mode => 755
}

Where /etc/cron.daily/updates is, of course:

#!/bin/sh
apt-get -y update

Then for the applications, I just tell puppet something like:

# Ensure that Nginx is installed.
package { 'nginx':
  ensure => latest
}

And done, once apt-get update runs, nginx will get updated to the latest version within the next twenty minutes or so (the next time puppet runs its recipe). Note that this requires you to have done 'apt-get update' in the initial image via whatever process you used to install puppet into the image (for example, if this is in CloudFormation, via the UserData section of the LaunchConfiguration). That is a reasonable requirement, IMHO.

If you want to do 'apt-get update' more often, you'll need to put a cron script into /etc/cron.d with the times you want to run it. I plopped it into cron.daily because that was often enough for me.

eric.green
  • 440
  • 3
  • 8
0

This is what you need to do - create an apt-get wrapper that would do apt-get update followed by calling a real apt-get (/usr/bin/apt-get) for install. Install the wrapper into a directory that will be in a PATH before apt-get.

Modify /usr/lib/ruby/vendor_ruby/puppet/provider/package/apt.rb and locate the line:

commands :aptget => "/usr/bin/apt-get"

( it will be right below has_features :versionenable, :install_options )

replace that line with:

commands :aptget => "apt-get"

You're done. For some boneheaded reason puppet insists on calling commands with absolute path rather than using a sane PATH variable.

Alex Yuriev
  • 3
  • 2