0

I am trying to marshal libnl's nla_parse call into C#.

iw's nla_parse extern signature is:

extern int nla_parse(struct nlattr **, int, struct nlattr *, int, struct nla_policy *);

I believe my problem is in correctly marshaling the tb nlattr double pointer (**).
The relevant C# extern signatures are (am omitting the DllImport clause for brevity - the calls work and return results)

public static extern IntPtr nlmsg_data(IntPtr nlh);
public static extern IntPtr nlmsg_hdr(IntPtr n);
public static extern int nla_parse(IntPtr tb, int maxtype, IntPtr head, int len, IntPtr policy);

My C# marshal call looks like

[StructLayout(LayoutKind.Sequential)]
public struct nlattr
{
    public ushort nla_len;
    public ushort nla_type;
}

var NL80211_ATTR_MAX = 305;
var nlAttrStructSize = Marshal.SizeOf<nlattr>();
var nlAttrStructArrayPtr = Marshal.AllocHGlobal(nlAttrStructSize * (NL80211_ATTR_MAX + 1));

var nlMsgHeaderPtr = NetlinkDemo.nlmsg_hdr(msg);
var gnlMsgHeaderPtr = NetlinkDemo.nlmsg_data(nlMsgHeaderPtr);
var gnlMsgAttrDataHeaderPtr = NetlinkDemo.genlmsg_attrdata(gnlMsgHeaderPtr, 0);
var returnCode = NetlinkDemo.nla_parse(nlAttrStructArrayPtr, NL80211_ATTR_MAX, gnlMsgAttrDataHeaderPtr, gnlMsgAttrLength, IntPtr.Zero);

I keep getting malloc(): memory corruption.
Are my marshal mappings correct, and my problem is either my msg input pointer or any of the fields in any of the subsequent pointers that work as input to nla_parse ? Or this is not the way to marshal a struct double pointer ?

Veverke
  • 9,208
  • 4
  • 51
  • 95
  • Looks like `out IntPtr` or `ref IntPtr` should work, as here: https://stackoverflow.com/questions/11131676/how-do-i-marshall-a-pointer-to-a-pointer-of-an-array-of-structures – Renat Nov 04 '21 at 09:34
  • Then it might be that `[Out]` attribute or `[In, Out]` is missing for out parameter (it's required as per https://www.mono-project.com/docs/advanced/pinvoke/#class-and-structure-marshaling ). Would this work? `public static extern int nla_parse([Out] out IntPtr tb, int maxtype, IntPtr head, int len, IntPtr policy);` – Renat Nov 04 '21 at 09:50
  • It is nlattr** because the function returns pointers the array, not structs. In other words, an [Out] IntPtr[] in C#. Use Marshal.PtrToStructure() to read the array elements after the call. – Hans Passant Nov 04 '21 at 13:21

0 Answers0