10

I recently ran into some trouble installing some modules and discovered to my surprise that many of the modules that had been installed, have duplicated installations and versions. Trying to track where stuff goes in a standard (if there is such a thing) installation using cpanm, I found the following results very confusing.

The reports show these locations:

  • Using cpan -V:
# cpan -V

/usr/bin/cpan script version 1.672, CPAN.pm version 2.22
--------------------------------------------------
Checking install dirs...
Checking core
        + /usr/share/perl5/5.26
        + /usr/lib/perl5/5.26/x86_64-cygwin-threads
Checking vendor
        + /usr/share/perl5/vendor_perl/5.26
        + /usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads
Checking site
        + /usr/local/share/perl5/site_perl/5.26
        + /usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads
Checking PERL5LIB
        no directories for PERL5LIB
Checking PERLLIB
        no directories for PERLLIB
  • Using perl -V:.*site.*:
# perl -V:.*site.* |column -t -s "=" |sort -d -i -k 1.22

d_sitearch           'define';
usesitecustomize     'undef';
siteprefix           '/usr/local';
siteprefixexp        '/usr/local';
installsitebin       '/usr/local/bin';
installsitescript    '/usr/local/bin';
sitebin              '/usr/local/bin';
sitebinexp           '/usr/local/bin';
sitescript           '/usr/local/bin';
sitescriptexp        '/usr/local/bin';
installsitearch      '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
sitearch             '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
sitearchexp          '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
installsitehtml1dir  '/usr/local/share/doc/perl/html/html1';
sitehtml1dir         '/usr/local/share/doc/perl/html/html1';
sitehtml1direxp      '/usr/local/share/doc/perl/html/html1';
installsitehtml3dir  '/usr/local/share/doc/perl/html/html3';
sitehtml3dir         '/usr/local/share/doc/perl/html/html3';
sitehtml3direxp      '/usr/local/share/doc/perl/html/html3';
installsiteman1dir   '/usr/local/share/man/man1';
siteman1dir          '/usr/local/share/man/man1';
siteman1direxp       '/usr/local/share/man/man1';
installsiteman3dir   '/usr/local/share/man/man3';
siteman3dir          '/usr/local/share/man/man3';
siteman3direxp       '/usr/local/share/man/man3';
installsitelib       '/usr/local/share/perl5/site_perl/5.26';
sitelib              '/usr/local/share/perl5/site_perl/5.26';
sitelib_stem         '/usr/local/share/perl5/site_perl/5.26';
sitelibexp           '/usr/local/share/perl5/site_perl/5.26';
  • Using @INC:
# perl -e 'print join("\n",@INC,"")'

/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads
/usr/local/share/perl5/site_perl/5.26
/usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads
/usr/share/perl5/vendor_perl/5.26
/usr/lib/perl5/5.26/x86_64-cygwin-threads
/usr/share/perl5/5.26

The result is that cpan-outdated -p --verbose show a completely different (and shorter) list of outdated modules than what cpan -lO does. Needless to say, modules are installed all over the place and I don't see how to understand if there is a default install location and where it goes, or should go.

QUESTION:

  1. So what is the difference between the core, vendor and site type of paths?
  2. Why are there 2 paths in each type?
not2qubit
  • 14,531
  • 8
  • 95
  • 135

2 Answers2

12

The best reference for these install locations is probably the ExtUtils::MakeMaker documentation for where it installs things. In essence:

  • core (also known as privlib) - is where core modules that are installed with Perl goes. On Perls older than 5.12, updates to dual-life modules also need to be installed here over the core versions instead of into site or vendor lib, because privlib came first in @INC before 5.12. This is especially dangerous in a system Perl where the files in privlib are usually managed by a package manager.
  • vendor - is where a distribution vendor may install modules to. This is usually where system package managers install non-core modules to.
  • site - is where CPAN clients install modules to when invoked directly, barring unusual configuration like for dual-life modules as mentioned above.

(Dual-life modules are core modules which are also available separately on CPAN, which means that you can install an updated version.)

Each of these lib locations have an arch variant, which is where distributions with build-specific output files are installed to. Pure-perl distributions with no dynamic configuration are installed into the standard architecture-agnostic directory, and can usually run unmodified in other installations of Perl and architectures provided their requirements are still satisfied (though it's not a good idea unless you really know what you are doing). Distributions with any compiled XS modules or that generate modules dynamically in the build process are installed into the arch directory and are not safe to use from another Perl.

All of these locations are configured when the Perl is built and can be discovered using the perl -V option as you showed. They also each have accompanying script and bin directories (which are usually the same) and directories for manpages.

As to the discrepancy of cpan-outdated - this tool (like many tools using ExtUtils::Installed) is restricted to finding modules that have packlists, which are included when installing a module using a CPAN client, but not with core modules and they are generally stripped from vendor packages. So most likely cpan-outdated will only discover modules in sitelib, but this is usually all you need to find. I'm not sure what mechanism the cpan command uses.

not2qubit
  • 14,531
  • 8
  • 95
  • 135
Grinnz
  • 9,093
  • 11
  • 18
4

UPDATE:

Thanks to ikegami's SO link, I have since done some source browsing, where we can find a lot of info about this. To get the more specific locations:

# perl -V:'install(privlib|archlib|vendorlib|vendorarch|sitelib|sitearch)' |column -t -s "="

installarchlib     '/usr/lib/perl5/5.26/x86_64-cygwin-threads';
installprivlib     '/usr/share/perl5/5.26';
installsitearch    '/usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads';
installsitelib     '/usr/local/share/perl5/site_perl/5.26';
installvendorarch  '/usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads';
installvendorlib   '/usr/share/perl5/vendor_perl/5.26';

Then for the meaning of those, we can look (in the perl sources) in the file: ./Porting/Glossary:

installarchlib    '- is the same for modules with arch- or build-dependent components.'
installprivlib    '- contains the "pure Perl" modules that came with Perl.'
installsitearch   '- is the same for modules with arch- or build-dependent components.'
installsitelib    '- contains the "pure Perl" modules installed by you.'
installvendorarch '- is the same for modules with arch- or build-dependent components.'
installvendorlib  '- contains the "pure Perl" modules installed by your distro.'

with the additional interesting note about the installstyle option:

installstyle (installstyle.U):

This variable describes the style of the perl installation. This is intended to be useful for tools that need to manipulate entire perl distributions. Perl itself doesn't use this to find its libraries -- the library directories are stored directly in Config.pm. Currently, there are only two styles: lib and lib/perl5. The default library locations (e.g. privlib, sitelib) are either $prefix/lib or $prefix/lib/perl5. The former is useful if $prefix is a directory dedicated to perl (e.g. /opt/perl), while the latter is useful if $prefix is shared by many packages, e.g. if $prefix=/usr/local.

Unfortunately, while this "style" variable is used to set defaults for all three directory hierarchies (core, vendor, and site), there is no guarantee that the same style is actually appropriate for all those directories. For example, $prefix might be /opt/perl, but $siteprefix might be /usr/local. (Perhaps, in retrospect, the "lib" style should never have been supported, but it did seem like a nice idea at the time.)

The situation is even less clear for tools such as MakeMaker that can be used to install additional modules into non-standard places. For example, if a user intends to install a module into a private directory (perhaps by setting PREFIX on the Makefile.PL command line), then there is no reason to assume that the Configure-time $installstyle setting will be relevant for that PREFIX.

This may later be extended to include other information, so be careful with pattern-matching on the results.

For compatibility with perl5.005 and earlier, the default setting is based on whether or not $prefix contains the string "perl".

All the gritty details can then be found in the INSTALLATION file, under the Installation Directories heading.

  • Directories for the perl distribution:
    By default, Configure will use the following directories (5.28.1):

        Configure variable  Default value
    $prefixexp          /usr/local
    $binexp             $prefixexp/bin
    $scriptdirexp       $prefixexp/bin
    $privlibexp         $prefixexp/lib/perl5/$version
    $archlibexp         $prefixexp/lib/perl5/$version/$archname

  • Directories for site-specific add-on files:

       Configure        Default
   variable          value
 $siteprefixexp    $prefixexp
 $sitebinexp       $siteprefixexp/bin
 $sitescriptexp    $siteprefixexp/bin
 $sitelibexp       $siteprefixexp/lib/perl5/site_perl/$version
 $sitearchexp      $siteprefixexp/lib/perl5/site_perl/$version/$archname

  • Directories for vendor-supplied add-on files:

    If are building a binary distribution of perl for distribution, Configure can optionally set up the following directories for you to use to distribute add-on modules.

       Configure          Default
   variable            value
 $vendorprefixexp    (none)

 (The next ones are set only if vendorprefix is set.)

 $vendorbinexp       $vendorprefixexp/bin
 $vendorscriptexp    $vendorprefixexp/bin
 $vendorlibexp       $vendorprefixexp/lib/perl5/vendor_perl/$version
 $vendorarchexp      $vendorprefixexp/lib/perl5/vendor_perl/$version/$archname

  • otherlibdirs:
    As a final catch-all, Configure also offers an $otherlibdirs variable. This variable contains a colon-separated list of additional directories to add to @INC. By default, it will be empty.

  • APPLLIB_EXP:
    There is one other way of adding paths to @INC at perl build time, and that is by setting the APPLLIB_EXP C pre-processor token. The directories defined by APPLLIB_EXP get added to @INC first, ahead of any others.
    sh Configure -Accflags='-DAPPLLIB_EXP=\"/usr/libperl\"'

not2qubit
  • 14,531
  • 8
  • 95
  • 135