6

I'm playing a bit with NativeCall to get familiar with that side of Perl6. Of course, I'm trying to load libstatgrab first (what else?).

So I start with easiest part - the host information. Since no cluster support yet, it's just one result - no worries for complication.

The code:

#!/usr/bin/env perl6 

use v6;
use NativeCall;

enum sg_error (
        SG_ERROR_NONE                   => 0,
        SG_ERROR_INVALID_ARGUMENT       => 1,
        ...
);

class sg_error_details is repr('CStruct') {
        has int32 $.error;
        has int32 $.errno_value;
        has Str $.error_arg;
};

sub sg_init(int32 $ignore_errors) returns int32 is native('statgrab') { * };

enum sg_host_state (
        sg_unknown_configuration        => 0,
        sg_physical_host                => 1,
        sg_virtual_machine              => 2,
        sg_paravirtual_machine          => 3,
        sg_hardware_virtualized         => 4
);

class sg_host_info is repr('CStruct') {
        has Str $.os_name;
        has Str $.os_release;
        has Str $.os_version;
        has Str $.platform;
        has Str $.hostname;
        has uint32 $.bitwidth;
        has int32 $.host_state;
        has uint32 $.ncpus;
        has uint32 $.maxcpus;
        has uint64 $.uptime;
        has uint64 $.systime;
};

sub sg_get_host_info(size_t is rw) returns Pointer is native('statgrab') is symbol('sg_get_host_info_r') { * };
sub sg_free_host_info(Pointer) is native('statgrab') is symbol('sg_free_stats_buf') { * };

sub MAIN() {
    my int32 $ignore_errors = 0;
    my $error = sg_init($ignore_errors);
    if $error != SG_ERROR_NONE {
        say "Maeh: $error";
        exit 1;
    }

    my size_t $num_host_infos = 0;
    my $res = sg_get_host_info($num_host_infos);
    if $num_host_infos > 0 {
        my $host_info = nativecast(sg_host_info, $res);
        with $host_info {
            say "You're using ", $_.os_name, " on ", $_.hostname;
        }
    }
    sg_free_host_info($res);
}

Starting it (dumb) results in loading library error:

$ perl6 statgrab.p6
Cannot locate native library 'libstatgrab.dylib': dlopen(libstatgrab.dylib, 1): image not found
  in method setup at /Users/sno/rakudo/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 283
  in method CALL-ME at /Users/sno/rakudo/share/perl6/sources/24DD121B5B4774C04A7084827BFAD92199756E03 (NativeCall) line 570
  in sub MAIN at statgrab.p6 line 95
  in block <unit> at statgrab.p6 line 93

Okay - giving it some search path:

$ LD_LIBRARY_PATH=/opt/pkg/lib:$LD_LIBRARY_PATH perl6 statgrab.p6
Cannot locate native library 'libstatgrab.dylib': dlopen(libstatgrab.dylib, 1): image not found

Same picture when using DYLD_LIBRARY_PATH - which is supported by dlopen(3) on Darwin, too.

But changing in the directory works:

$ (cd /opt/pkg/lib && perl6 /data/Projects/OSS/p6-Unix-Statgrab/statgrab.p6 )
You're using Darwin on ernie.[...]

Is there a lack of search path passthrough in the way how moarvm is called?

Sno
  • 179
  • 1
  • 7
  • I am on Linux so I am not sure I can help here, but what is the content of the directory `/opt/pkg/lib` on a Mac? Is the file `libstatgrab.dylib`present? – Håkon Hægland Feb 20 '19 at 10:06
  • I tried running your script on my Ubuntu 18.10 laptop and got the cryptic error message: `===SORRY!=== Stub code executed`.. – Håkon Hægland Feb 20 '19 at 10:15
  • 2
    Ok, your script is working fine on Ubuntu after removing the line with the `...` from the `enum`. The output I get from running the script: `You're using Linux on hakon-Precision-7530` – Håkon Hægland Feb 20 '19 at 10:23
  • 1
    To answer your first question: Yes. Please see last code fragment using the change directory command. And yes - I shortened the code since stack overflow blamed me that my question contains to much code ;) Finally you say: Works on Linux, so probably it's a Darwin only issue? – Sno Feb 21 '19 at 12:21
  • Sorry that I do not have access to an OSX machine, but could be Darwin only issue. Could you try specify absolute path to the library: `... is native('/opt/pkg/lib/libstatgrab.dylib') ...`? – Håkon Hægland Feb 21 '19 at 12:58
  • I've just this a default install of `libstatgrab` 0.91 on Mojave: `./configure && make && make install` and then the following code works ok `perl6 -e 'use NativeCall; sub sg_init(int32 $ignore_errors) returns int32 is native("statgrab") { * }; sg_init(1)'`. So I cannot reproduce this on MacOS 10.14.3. Also, the above script (minus the `...`) works as expected, afaics. – Elizabeth Mattijsen Feb 21 '19 at 13:22
  • @ElizabethMattijsen - where does the default ./configure installs to? I expect /usr/local/lib, right? – Sno Feb 21 '19 at 14:41
  • It did install to `/usr/local/lib`. – Elizabeth Mattijsen Feb 21 '19 at 18:09
  • @ElizabethMattijsen please try with./configure --prefix=/opt(libstatgrab – Sno Feb 21 '19 at 20:45
  • 1
    I don't know Darwin or much system stuff in general but, aiui this isn't P6/MoarVM. Are these SOs relevant? [Error: dlopen() Library not loaded Reason: image not found](https://stackoverflow.com/questions/19776571/error-dlopen-library-not-loaded-reason-image-not-found)? [Alternative for the DYLD_LIBRARY_PATH-trick since Mac OS 10.11 El Capitan with System Integrity Protection](https://stackoverflow.com/questions/39927235/alternative-for-the-dyld-library-path-trick-since-mac-os-10-11-el-capitan-with-s)? Also, is `LD_DEBUG=all` helpful? – raiph Feb 22 '19 at 23:35
  • Hi raiph, I didn't had a chance to look deeper at the very moment. I discussed it with Liz on our local perl monger meeting. She detected and solved the typo ;) It's kind-of irrelevant whether the related resources show how a low-level program has to load a shared library, the user-expectation for a high level language is, that it solves those issues. Honestly, I think there're two issues - and the related resources point out, those I didn't realize. – Sno Feb 25 '19 at 06:53
  • Whatever one thinks wrt. XS, it solves the PATH issue by providing the linked library search path via RPATH, an infrastructure maintainer can modify it for the platforms where the module is compiled for. Perl 6 decided to use a high level only interface and has to deal with that issue (it's maybe at the same criticality as [CVE-2016-1238](http://cve.circl.lu/cve/CVE-2016-1238)) The second issue is, that a module maintainer should not be forced to search for the shared library before the API module is loaded. This will cause bloopers since compile time vs. runtime ... – Sno Feb 25 '19 at 06:59
  • For the records: $ otool -L /opt/pkg/lib/libstatgrab.dylib /opt/pkg/lib/libstatgrab.dylib: /opt/pkg/lib/libstatgrab.10.dylib (compatibility version 11.0.0, current version 11.0.0) /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1560.12.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5) – Sno Feb 25 '19 at 07:02
  • Taking your hints to dig a bit further shows up: env DYLD_LIBRARY_PATH=/opt/pkg/lib LD_DEBUG=all /Users/sno/rakudo/bin/moar --execname="perl6" --libpath="/Users/sno/rakudo/share/nqp/lib" --libpath="/Users/sno/rakudo/share/nqp/lib" --libpath="/Users/sno/rakudo/share/perl6/lib" --libpath="/Users/sno/rakudo/share/perl6/runtime" /Users/sno/rakudo/share/perl6/runtime/perl6.moarvm "$@" statgrab.p6 works as expected. So creating "perl6" being a shell script instead of a very tiny binary or just a softlink or whatever causing the issue. – Sno Feb 25 '19 at 07:23
  • 2
    @Sno perl6 will probably switch to use a small binary instead of the current shell scripts after the next release. – Patrick Böker Feb 26 '19 at 09:31

1 Answers1

3
doug$ perl6 -v
This is Rakudo Star version 2018.10 built on MoarVM version 2018.10
implementing Perl 6.c.

On a fairly recent Rakudo Star on MacOS High Sierra, the script worked "out of the box" for me:

  1. edited script to remove '...'.
  2. Script failed to load the library (really missing!)
  3. brew install libstatgrab
  4. Script ran successfully:
vader:learning doug$ perl6 nativecall_mac_Sno.pl6 
You're using Darwin on Vader.local

Homebrew installed the library as follows:

$ v /usr/local/lib
total 11904
-rw-r--r--  1 doug  admin  6080828 Sep 23 12:40 libmoar.dylib
lrwxr-xr-x  1 doug  admin       51 Mar 23 11:02 libstatgrab.10.dylib@ -> ../Cellar/libstatgrab/0.91/lib/libstatgrab.10.dylib
lrwxr-xr-x  1 doug  admin       44 Mar 23 11:02 libstatgrab.a@ -> ../Cellar/libstatgrab/0.91/lib/libstatgrab.a
lrwxr-xr-x  1 doug  admin       48 Mar 23 11:02 libstatgrab.dylib@ -> ../Cellar/libstatgrab/0.91/lib/libstatgrab.dylib
drwxr-xr-x  3 doug  admin      102 Mar 23 11:02 pkgconfig/

For me, the perl6 executable is indeed a shell script, but it worked (there was no need to pass any extra LD_LIBRARY_PATH=...).

doug$ file `which perl6`
/Applications/Rakudo/bin/perl6: POSIX shell script text executable, ASCII text, with very long lines
doug$ set | grep LIBRARY
doug$

I have also had issues with my nativecall scripts finding the library, but have always solved them by fixing the library install and/or supplying 'LD_LIBRARY_PATH'.

Sorry this experience was Less Than Awesome for you

dmaestro12
  • 883
  • 7
  • 15
  • 1
    See the comment above linking to the '... System Integrity Protection' SO. I recall now also running into that issue on Darwin. Painful. – dmaestro12 Mar 23 '19 at 16:42
  • 1
    "I have also had issues with my nativecall scripts finding the library, but have always solved them by fixing the library install and/or supplying 'LD_LIBRARY_PATH'." But not on Darwin, right? – raiph Mar 23 '19 at 21:42
  • 1
    Patrick Böker's comment above suggests devs are hoping to do a small binary `perl6` launcher instead of a shell script to avoid the SIP issue. – raiph Mar 23 '19 at 21:43
  • 1
    @raiph I have used that ENV variable before on Darwin, but maybe before SIP was introduced. It's also possible that I disabled SIP temporarily to fix issues with library installation. I just know that I have been able to solve similar issues on my (Darwin) systems. – dmaestro12 Apr 05 '19 at 03:19