How can a SafeHandle be used in a P/Invoke signature that requires a null pointer in certain cases?

Question

Hopefully this isn't too obscure for SO, but consider the following P/Invoke signature:

[DllImport("odbc32.dll", CharSet = CharSet.Unicode)]
internal static extern OdbcResult SQLAllocHandle(
    OdbcHandleType HandleType,
    IntPtr InputHandle,
    ref IntPtr OutputHandlePtr);

I'd like to redesign this signature to use SafeHandles, as follows:

[DllImport("odbc32.dll", CharSet = CharSet.Unicode)]
internal static extern OdbcResult SQLAllocHandle(
    OdbcHandleType HandleType,
    MySafeHandle InputHandle,
    ref MySafeHandle OutputHandlePtr);

However, according to MSDN, the InputHandle argument must be a null pointer when the HandleType argument is SQL_HANDLE_ENV and a non-null pointer otherwise.

How do I capture those semantics in a single P/Invoke signature? Please include an example call-site in your answer. My current solution is to use two signatures.

score 3 · Accepted Answer · answered Dec 03 '11 at 18:50

3

SafeHandle is a class so you should be able to pass null rather than an actual SafeHandle. A null reference is marshaled as a null pointer in P/Invoke.

SafeHandle handle = new SafeHandle();
OdbcResult result= SQLAllocHandle(OdbcHandleType.SQL_HANDLE_ENV, null, ref handle);

answered Dec 03 '11 at 18:50

shf301

31,086
2
52
86

2

It is also possible to declare an `out SafeHandle` parameter. – Anton Tykhyy Dec 03 '11 at 19:08
5

Doesn't work here, I'm getting an `ArgumentNullException` from `System.StubHelpers.StubHelpers.SafeHandleAddRef(SafeHandle pHandle, Boolean& success)`. – ordag Jan 24 '12 at 11:49
Yes, @shf301, this doesn't work in P/Invoke calls. Did you even try it? – jnm2 Jul 20 '17 at 18:44

score 2 · Answer 2 · edited May 23 '17 at 12:23

The answer by shf301 passes null for the input argument InputHandle. This doesn't work on most APIs (maybe it somehow does for the OP's specific problem, given that they accepted the answer).

I use this pattern:

[SecurityPermission(SecurityAction.LinkDemand, UnmanagedCode = true)]
public class RegionHandle : SafeHandleZeroOrMinusOneIsInvalid
{
    private RegionHandle() : base(true) {}

    public static readonly RegionHandle Null = new RegionHandle();

    [ReliabilityContract(Consistency.WillNotCorruptState, Cer.MayFail)]
    override protected bool ReleaseHandle()
    {
        return Region.DeleteObject(handle);
    }
}

It means I can do this to pass a null handle:

SomeApi(RegionHandle.Null);

It's similar to how there's an IntPtr.Zero static member.

How can a SafeHandle be used in a P/Invoke signature that requires a null pointer in certain cases?

2 Answers2