214

What are all the ways of affecting where Perl modules are searched for? or, How is Perl's @INC constructed?

As we know, Perl uses @INC array containing directory names to determine where to search for Perl module files.

There does not seem to be a comprehensive "@INC" FAQ-type post on StackOverflow, so this question is intended as one.

Community
  • 1
  • 1
DVK
  • 126,886
  • 32
  • 213
  • 327
  • 7
    Yeah, but there's a perfectly good one at http://search.cpan.org/perldoc/perlfaq8#How_do_I_add_a_directory_to_my_include_path_(@INC)_at_runtime? – mob Mar 26 '10 at 21:18
  • 4
    @mobrule: I don't think that's anywhere near as comprehensive - it's just telling how to add to `@INC` at runtime, not the full construction. – Cascabel Mar 26 '10 at 21:19
  • 3
    @mobrule - @Jefromi guessed right - the main problem with ANY and all references I found so far was a lack of comprehensive info on perl binary's compiled-in default @INC – DVK Mar 26 '10 at 21:31
  • 3
    Some can make that answer as comprehensive by sending me a patch. It's easy because perlfaq is in Github. :) – brian d foy Mar 26 '10 at 21:32
  • 1
    Well, the "comprehensive info on perl binary's compiled-in default @INC" is that it's the one the person who compiled it set up. If your asking about the various paths that get in there, then I think you have a different question than the one you answered. – brian d foy Mar 26 '10 at 21:34
  • @brian - If you feel the question is inappropriate for SO, you have enough rep points to moderate it out of existence. I will understand your reasons due to the comments above and will not contest that (hey! I got killed by the best sword in the Empire!), although I happen to disagree. – DVK Mar 26 '10 at 21:39
  • Also, my rep points mean nothing in terms of moderation. I don't really have any more power to do anything that anyone else who can edit. – brian d foy Mar 26 '10 at 22:07
  • Don't 10k give you full mod powers, including deletion? I had them way too briefly (1 week or so before the Great Recalc) to find out what was in scope :) – DVK Mar 26 '10 at 22:28
  • No, 10k lets you see deleted posts. The only moderators are the ones with the diamonds next to their names. – brian d foy Mar 27 '10 at 02:13
  • 1
    If you just want to see your @INC and you have GNU grep, this is a convenient command: `perl -V | grep -A20 '@INC'` (20 lines after the @INC heading should be more than enough, but you can season to taste) – Randall Oct 13 '17 at 18:52

3 Answers3

270

We will look at how the contents of this array are constructed and can be manipulated to affect where the Perl interpreter will find the module files.

  1. Default @INC

    Perl interpreter is compiled with a specific @INC default value. To find out this value, run env -i perl -V command (env -i ignores the PERL5LIB environmental variable - see #2) and in the output you will see something like this:

    $ env -i perl -V
    ...
    @INC:
     /usr/lib/perl5/site_perl/5.18.0/x86_64-linux-thread-multi-ld
     /usr/lib/perl5/site_perl/5.18.0
     /usr/lib/perl5/5.18.0/x86_64-linux-thread-multi-ld
     /usr/lib/perl5/5.18.0
     .
    

Note . at the end; this is the current directory (which is not necessarily the same as the script's directory). It is missing in Perl 5.26+, and when Perl runs with -T (taint checks enabled).

To change the default path when configuring Perl binary compilation, set the configuration option otherlibdirs:

Configure -Dotherlibdirs=/usr/lib/perl5/site_perl/5.16.3

  1. Environmental variable PERL5LIB (or PERLLIB)

    Perl pre-pends @INC with a list of directories (colon-separated) contained in PERL5LIB (if it is not defined, PERLLIB is used) environment variable of your shell. To see the contents of @INC after PERL5LIB and PERLLIB environment variables have taken effect, run perl -V.

    $ perl -V
    ...
    %ENV:
      PERL5LIB="/home/myuser/test"
    @INC:
     /home/myuser/test
     /usr/lib/perl5/site_perl/5.18.0/x86_64-linux-thread-multi-ld
     /usr/lib/perl5/site_perl/5.18.0
     /usr/lib/perl5/5.18.0/x86_64-linux-thread-multi-ld
     /usr/lib/perl5/5.18.0
     .
    
  2. -I command-line option

    Perl pre-pends @INC with a list of directories (colon-separated) passed as value of the -I command-line option. This can be done in three ways, as usual with Perl options:

    • Pass it on command line:

      perl -I /my/moduledir your_script.pl
      
    • Pass it via the first line (shebang) of your Perl script:

      #!/usr/local/bin/perl -w -I /my/moduledir
      
    • Pass it as part of PERL5OPT (or PERLOPT) environment variable (see chapter 19.02 in Programming Perl)

  3. Pass it via the lib pragma

    Perl pre-pends @INC with a list of directories passed in to it via use lib.

    In a program:

    use lib ("/dir1", "/dir2");
    

    On the command line:

    perl -Mlib=/dir1,/dir2
    

    You can also remove the directories from @INC via no lib.

  4. You can directly manipulate @INC as a regular Perl array.

    Note: Since @INC is used during the compilation phase, this must be done inside of a BEGIN {} block, which precedes the use MyModule statement.

    • Add directories to the beginning via unshift @INC, $dir.

    • Add directories to the end via push @INC, $dir.

    • Do anything else you can do with a Perl array.

Note: The directories are unshifted onto @INC in the order listed in this answer, e.g. default @INC is last in the list, preceded by PERL5LIB, preceded by -I, preceded by use lib and direct @INC manipulation, the latter two mixed in whichever order they are in Perl code.

References:

There does not seem to be a comprehensive @INC FAQ-type post on Stack Overflow, so this question is intended as one.

When to use each approach?

  • If the modules in a directory need to be used by many/all scripts on your site, especially run by multiple users, that directory should be included in the default @INC compiled into the Perl binary.

  • If the modules in the directory will be used exclusively by a specific user for all the scripts that user runs (or if recompiling Perl is not an option to change default @INC in previous use case), set the users' PERL5LIB, usually during user login.

    Note: Please be aware of the usual Unix environment variable pitfalls - e.g. in certain cases running the scripts as a particular user does not guarantee running them with that user's environment set up, e.g. via su.

  • If the modules in the directory need to be used only in specific circumstances (e.g. when the script(s) is executed in development/debug mode, you can either set PERL5LIB manually, or pass the -I option to perl.

  • If the modules need to be used only for specific scripts, by all users using them, use use lib/no lib pragmas in the program itself. It also should be used when the directory to be searched needs to be dynamically determined during runtime - e.g. from the script's command line parameters or script's path (see the FindBin module for very nice use case).

  • If the directories in @INC need to be manipulated according to some complicated logic, either impossible to too unwieldy to implement by combination of use lib/no lib pragmas, then use direct @INC manipulation inside BEGIN {} block or inside a special purpose library designated for @INC manipulation, which must be used by your script(s) before any other modules are used.

    An example of this is automatically switching between libraries in prod/uat/dev directories, with waterfall library pickup in prod if it's missing from dev and/or UAT (the last condition makes the standard "use lib + FindBin" solution fairly complicated. A detailed illustration of this scenario is in How do I use beta Perl modules from beta Perl scripts?.

  • An additional use case for directly manipulating @INC is to be able to add subroutine references or object references (yes, Virginia, @INC can contain custom Perl code and not just directory names, as explained in When is a subroutine reference in @INC called?).

ikegami
  • 367,544
  • 15
  • 269
  • 518
DVK
  • 126,886
  • 32
  • 213
  • 327
  • 1
    don't forget PERLOPT, where you can set -I. Also, things such as base.pm and local::lib use the things you've listed implicitly. – brian d foy Mar 26 '10 at 21:30
  • 1
    @brian - use::base uses it due to "require" underneath, IIRC. I'm not familiar with local::lib, will need to read more to understand what it is about – DVK Mar 26 '10 at 22:00
  • 3
    Also, to make this a really good answer, you need to tell people when they should use each one. Just giving them 10 options on how they could isn't very helpful. :) – brian d foy Mar 26 '10 at 22:01
  • P.S. Anyone, please feel free to edit the answer to include a second architecture-specific directory inclusion via -I/use lib. I planned to do so myself at a later time but need to get offline for now. – DVK Mar 26 '10 at 22:31
  • @brian - added PERLOPT. I feel like mentioning base.pm and other "when does @INC get used" items is more suited to the @INC/finding module files FAQ I posted and linked from here, so I put it there. – DVK Mar 26 '10 at 23:09
  • Your answering is looking pretty good. Just keep doing what you are doing. Writing is editing :) – brian d foy Mar 27 '10 at 00:08
  • I also added one more use case, the insertion of subroutine references into @INC – DVK Mar 30 '10 at 07:49
  • The output of perl -V depends on PERL5LIB and PERLLIB environment variables. I've modified the answer to reflect this. – Flimm Sep 20 '11 at 13:21
  • @Flimm - you're quite right. I approved the edit and added the example – DVK Sep 20 '11 at 13:51
  • Note: If you've made any modifications to `$PATH`, they will be stripped out by `env -i` -- so you may end up running a different `perl`. – Andrew McKinlay Dec 28 '16 at 03:29
  • 1
    Note that `PERL5LIB` and `use lib` also add arch dirs related to the specified dirs, while `-I` doesn't. That makes `-Mlib=...` much better than `-I ...` – ikegami Sep 12 '18 at 12:31
21

In addition to the locations listed above, the OS X version of Perl also has two more ways:

  1. The /Library/Perl/x.xx/AppendToPath file. Paths listed in this file are appended to @INC at runtime.

  2. The /Library/Perl/x.xx/PrependToPath file. Paths listed in this file are prepended to @INC at runtime.

dgatwood
  • 10,129
  • 1
  • 28
  • 49
  • When specifically are these files read during run time? When I look at the output of lsof, I have a rare case where the AppendToPath file is open for reading on fd 0 (STDIN), but in every other case I've tested, it's not there and I can't seem to make a toy example that duplicates that behavior. I'm curious why and when it's ever on STDIN. When will it be there? – hepcat72 Aug 11 '21 at 17:43
  • I dug into why I have that handle open according to lsof. Apparrently, at the top of the script that called the script that showed the open handle, there was a line of code that was closing STDIN (which was probably some misguided attempt to deal with some past issue). But apparently, if you do that, child script processed will show STDIN open to that file on macOS. Don't know why yet though. – hepcat72 Aug 11 '21 at 19:38
7

As it was said already @INC is an array and you're free to add anything you want.

My CGI REST script looks like:

#!/usr/bin/perl
use strict;
use warnings;
BEGIN {
    push @INC, 'fully_qualified_path_to_module_wiht_our_REST.pm';
}
use Modules::Rest;
gone(@_);

Subroutine gone is exported by Rest.pm.

Kacper Perschke
  • 117
  • 3
  • 4