8

I am trying to create a cache of POD6 by precompiling them using the CompUnit set of classes.

I can create, store and retrieve pod as follows:

use v6.c;
use nqp;
my $precomp-store =     CompUnit::PrecompilationStore::File.new(prefix=>'cache'.IO);
my $precomp = CompUnit::PrecompilationRepository::Default.new(store=> $precomp-store );
my $key = nqp::sha1('test.pod6');
'test.pod6'.IO.spurt(q:to/CONTENT/);
=begin pod
=TITLE More and more

Some more text

=end pod

CONTENT
$precomp.precompile('test.pod6'.IO, $key, :force);
my $handle = $precomp.load($key, )[0];
my $resurrected = nqp::atkey($handle.unit,'$=pod')[0];
say $resurrected ~~ Pod::Block::Named;

So now I change the POD, how do I use the :since flag? I thought that if :since contains a time after the compilation, then the value of the handle would be Nil. That does not seem to be the case.

my $new-handle = $precomp.load($key, :since('test.pod6'.IO.modified));
say 'I got a new handle' with $new-handle;

Output is 'I got a new handle'.

What I am doing wrong? Here is a pastebin link with code and output: https://pastebin.com/wtA9a0nP

Richard Hainsworth
  • 1,585
  • 7
  • 9

1 Answers1

4

Module loading code caches look ups and essentially start with:

$lock.protect: { 
    return %loaded{$id} if %loaded{$id}:exists; 
}

So the question becomes "how do I load a module and then unload it (so i can load it again)?" to which the answer is: you cannot unload a module. You can however change the filename, distribution longname ( via changing the name, auth, api, or version ), or precomp id -- whatever a specific CompUnit::Repository uses to uniquely identify modules -- to bypass the cache.

What seems to be overlooked is that $key is intended to represent an immutable name, such that it will always point at the same content. What version of foo.pm should be loaded for Module Used::Inside::A if foo.pm is being loaded by Module A and B at the same time, Module A loads foo first, and then Module B modifies foo? The old version Module A loaded, or the ( possibly conflicting with previous version ) Module B loaded version? And how would this differentiation work for precomp file generation themselves ( which again can happen in parallel )?

Of course if we ignore the above we could add code to make expensive .IO.modified calls for every single module load for all CompUnit::Repository types ( slowing down startup speed ) to say "hey this immutable thing changed". But the granularity of file system modified timestamps on some OS's made the check pretty fragile ( especially for multi-threaded module loading with precomp files being generated ), meaning that even more expensive calls to get a checksum would be necessary every single time.

ugexe
  • 5,297
  • 1
  • 28
  • 49
  • The question is what it is: it's trying create a cache of POD6 by using precompilation. If I understand it correctly, you are saying that we need to keep the last modification time in some other place, since every time we "install" (in our case, precompile a document module) it generates a new handle. Is that correct? – jjmerelo Oct 27 '18 at 17:50
  • "The question is what it is" -- the question has almost nothing to do with what you summarized as your interpretation of the question. If the goal was simply to precompile some pod then what is the purpose of calling `.load` ( which is indirectly referred to in the question title itself via `:since` ) ? – ugexe Oct 27 '18 at 21:39
  • Not my words. The first sentence of the OP says: "I am trying to create a cache of POD6 by precompiling them using the CompUnit set of classes." So the goal was, precisely, to precompile some Pods, as you say. `load` is called in order to use the precompiled Pod, I guess. Might be wrong, but that's the reason why he's asked the question here in the first place, to kind of find out the right way to do it. – jjmerelo Oct 28 '18 at 07:30
  • 1
    The focus of the question was on `:since`. The answer is in ugexe's comment that 'granuality of timestamps ... Fragile'. So `:since` is now Noop. I was not doing anything wrong and the result I showed is the currently expected result. Since CompUnit is intended for speed, it can't be used for code management. That's ok. – Richard Hainsworth Nov 02 '18 at 23:28
  • @RichardHainsworth It sounds like you accept ugexe's answer. If so, please consider clicking to accept it. If not, please consider adding a comment about how it falls short. – raiph May 27 '19 at 08:20
  • 1
    @ralph Sorry for delay. Moving home. Problem is v low level of documentation in CompUnit, and complexity of coding, eg., nesting of class statements. Without any comment in code, it is unclear to a newcomer like me, why this strategy is used. I wanted to find some way to track changes between compiled files - POD documentation files change rapidly. POD is compiled. Since there are hundreds of POD files, its inefficient to compile each time. So I had to subclass CompUnit to achieve my aim. And store changes separately. Can't "accept" ugexe ans as it doesn't offer a way to achieve my aim. – Richard Hainsworth Jun 01 '19 at 15:48