Where is the memory leak in my Perl XS code?

Question

I have a large program written in C++ and executed from Perl using Inline::CPP. The code seems to be using a lot of memory, so I am assuming there is a leak somehow. I have written the following code that reproduces the same issue, but is much simpler. I found the leak when I looped the code 1,000,000 times to test the performance of the code. This simple Perl script uses 828MiB, and my full program uses 1.3GiB.

I have tried a lot of things, like using SvREFCNT_dec, newRV_noinc, sv_free, and sv_2mortal on various variables in the code, but I haven't been able to bring the memory usage down.

Here's my example code:

use Data::Dumper;

print Dumper test ();

use Inline 'CPP' => << 'CPP';
    #include <array>

    using namespace std;

    AV *array_to_av (const array<int,3> &v)
    {
        AV *array = newAV ();

        for (int i : v) {
            av_push (array, newSViv (i));
        }

        return array;
    }

    SV *test_leak ()
    {
        HV *hash = newHV ();

        AV *array1 = array_to_av ({1,2,3});
        AV *array2 = array_to_av ({1,2,3});
        AV *array3 = array_to_av ({1,2,3});

        SV *value1 = newRV_noinc ((SV *)(array1));
        SV *value2 = newRV_noinc ((SV *)(array2));
        SV *value3 = newRV_noinc ((SV *)(array3));

        hv_stores (hash, "Test1", value1);
        hv_stores (hash, "Test2", value2);
        hv_stores (hash, "Test3", value3);

        return newRV_noinc ((SV *)(hash));
    }

    SV *test ()
    {
        SV *hash;

        for (int i = 0; i < 1000000; i++) {
            hash = test_leak ();
        }

        return hash;
    }
CPP

sleep 10;

I'm no good with xs but in similar code I've written I have `sv_2mortal((SV*)flist);` after `AV* flist = newAV();` (then the array `flist` is populated etc). This is for C, not C++. — zdim, Sep 27 '20 at 05:10
@zdim Placing `sv_2mortal ((SV*)array);` after `AV *array = newAV ();` didn't change the memory usage, it only caused errors like "Attempt to free unreferenced scalar: SV 0x55748d5a6c68, Perl interpreter: 0x557458d6b2a0 at testleak.pl line 54." XS code is C, and you can write standard C in C++, so I don't think it really matters if I'm using C++ or not. I have also tried removing all the C++ features and using Inline::C instead, but the problem still remained. — Coffee'd Up Hacker, Sep 27 '20 at 05:32
Heh... I hoped that that could help, as i don't see what in the shown code frees memory. (But now I also see that I misread what `array_to_av()` is for). — zdim, Sep 27 '20 at 07:05

score 6 · Accepted Answer · answered Sep 27 '20 at 08:05

You need to free up the hashes that are not returned to the Perl script in the for loop. This loop:

for (int i = 0; i < 1000000; i++) {
    hash = test_leak ();
}

Should be something like:

for (int i = 0; i < 1000000; i++) {
    hash = test_leak ();
    SvREFCNT_dec(hash);  // Free the memory not returned to Perl
 }
 hash = test_leak();  // The final hashref is returned to Perl

Where is the memory leak in my Perl XS code?

1 Answers1