7

I am calling CopyFileEx from a C# application with an anonymous delegate being passed into the LPPROGRESS_ROUTINE parameter in order to get notifications on the file copy progress.

My question is, does the anonymous delegate need to be pinned and why (or why not).

In addition, does the answer change if:

  1. CopyFileEx was not blocking.
  2. If I passed in a delegate that was not anonymous.

Thanks!

SpeksETC
  • 1,003
  • 2
  • 7
  • 13

3 Answers3

8

The delegate does not need to be pinned. A managed object is pinned if it cannot be moved by the garbage collector. If the marshalling information is correct then the marshalling layer will ensure that a pointer to something immobile is passed.

However, the comment above where you suggest that a local variable might keep the delegate alive indicates a misunderstanding of variable lifetime. I refer you to the spec, which states:

The actual lifetime of a local variable is implementation-dependent. For example, a compiler might statically determine that a local variable in a block is only used for a small portion of that block. Using this analysis, the compiler could generate code that results in the variable’s storage having a shorter lifetime than its containing block. The storage referred to by a local reference variable is reclaimed independently of the lifetime of that local reference variable

In other words, if you say:

void M()
{
    Foo foo = GetAFoo();
    UnmanagedLibrary.DoSomethingToFoo(foo);
}

then the jitter is allowed to say "you know, I see that no managed code ever uses foo again the moment after the unmanaged call is invoked; I can therefore aggressively reclaim the storage of that object from another thread at that time". Which means that the unmanaged call can be working on the object when suddenly it is deallocated on another thread.

This is particularly nasty if Foo has a destructor. The finalization code will possibly run on another thread while the object is in use by the unmanaged library, and heaven only knows what sort of disaster that will cause.

In this circumstance you are required to use a KeepAlive to keep the managed object alive. Do not rely on a local variable; local variables are specifically documented as not guaranteed to keep things alive.

See http://msdn.microsoft.com/en-us/library/system.gc.keepalive.aspx for more details.

Eric Lippert
  • 647,829
  • 179
  • 1,238
  • 2,067
  • Thanks Eric. I am a bit confused since Chris Brumme states that "PInvoke will either copy your data to fixed memory outside the GC heap, or it will pin memory in the GC heap and expose those bytes directly to unmanaged code. In either case, you don't need to explicitly pin -- so long as access to these bytes is scoped to within the duration of the PInvoke call. " in http://blogs.msdn.com/b/cbrumme/archive/2003/05/06/51385.aspx?PageIndex=1#comments - isn't this also relevant to delegates? I assumed if the pinvoke layer pins the data, GC will not collect... – SpeksETC Mar 28 '11 at 21:51
  • 2
    @SpeksETC: The article you linked to specifically states that your assumption is wrong. It says "**However, the application is responsible for somehow extending the lifetime of the delegate until no more calls will occur from unmanaged code**." – Eric Lippert Mar 28 '11 at 21:59
  • @Eric Lippert: Couldn't that statement only be referring to async unmanaged calls? Seems to negate what he writes in the comments that "If the unmanaged callee only needs access to the buffer for the life of the call, then the PInvoke marshaling layer will generally pin it for that duration" unless the marshalling layer treats delegates differently... – SpeksETC Mar 28 '11 at 22:26
  • @Eric, the KeepAlive documentation says that KeepAlive method maintains the reference until the end of the call. I would assume (wrongly?) that a P/Invoke method call would not be special in this regard and maintain the reference until the end of the call too. If this is actually true (which differs from what you posted), then KeepAlive would mainly only be needed in callback scenarios, such as the one in the example for the KeepAlive example. I know the GC is very aggressive with locals, but did not think it went as far as you wrote. Could you elaborate further on this particular point? – Jason Kresowaty Mar 28 '11 at 22:34
  • @binarycoder - GC does go "as far as you wrote" if, for example, a local variable is passed in to a method Foo(object X) which is 100 lines of code, but the first line is the only one that uses X, then after that line, X can be collected - even if Foo hasn't returned yet (assuming it is not being used by the caller again - which is what in essence KeepAlive does, uses the instance again). pInvoke seems like it could be different since as a part of marshaling it pins the parameters, but that is what I am trying to verify... – SpeksETC Mar 28 '11 at 22:43
  • @binarycoder - KeepAlive does not maintain the reference until KeepAlive returns. Like any method (that is not optimized away) it only maintains the reference until it is called, which is what the documentation says. "References the specified object, which makes it ineligible for garbage collection from the start of the current routine to the point where this method is called." This says nothing about keeping it alive "until the end of the call." – Jeffrey L Whitledge Mar 28 '11 at 22:46
  • 1
    @Jeffrey, you're right. Looks like I read the documentation too quickly... But my gut feeling is that P/Invoke automatically pins/holds references to the parameters until the end of the call, with GC.KeepAlive needed for callbacks. That basically would explain why KeepAlive seems to only be mentioned in conjunction with delegates and/or asynchronous stuff--not arrays such as `byte[]`. Any definitive info to the contrary? – Jason Kresowaty Mar 28 '11 at 22:59
  • @EricLippert Is the possible that this isn't true with mono, we have some Xamarin.Android code which crashes with GC.KeepAlive but doesn't crash with a pin. – trampster Aug 31 '22 at 22:11
  • 1
    Be very careful with GC.KeepAlive, it doesn't work with async await code. If you are using async code then you will need to keep a Normal(not pinned) GCHandle – trampster Sep 04 '22 at 23:29
3

You don't need to pin it, but you do need to keep a reference to it alive as long as the copy is in progress.

The thunk that is called by the unmanaged code is pinned, but you have to make sure the delegate is not garbage collected - hence the reference.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • If the anonymous method is a local variable is it considered "alive" as long as CopyFileEx hasn't returned? – SpeksETC Mar 28 '11 at 21:12
  • @sehe, @SpeksETC: Quite sure, are you? Where precisely in the specification does it say that? My copy of the spec says the opposite, namely "*the compiler could generate code that results in the variable’s storage having a shorter lifetime than its containing block*." – Eric Lippert Mar 28 '11 at 21:28
  • Thanks - that is exactly what I was concerned about (saw this once when an instance was GC'd and finalized while one of it's methods was in the middle of execution). I just read http://blogs.msdn.com/b/cbrumme/archive/2003/05/06/51385.aspx?PageIndex=2#comments and if I understood correctly seems that the pinvoke layer does the pinning for you for the lifetime of the call, so in this case seems like it is covered... – SpeksETC Mar 28 '11 at 21:43
  • I'm quite sure that the compiler can make good on that permission too ;) – Jon Hanna Mar 28 '11 at 21:44
  • @eric: Yes, but we are talking about a reference that is being passed as a [in] parameter to a function here. Shouldn't that normally mean that the parameter stays valid at least until the function returns? For clarity: I was responding to the comment "as long as CopyFileEx hasn't returned?" [i'm assuming absense of unsafe ptr/weakreferences] – sehe Mar 28 '11 at 22:12
  • 3
    @sehe - No. The mechanism that determines the actual lifetime of the variable may presume that the parameter itself within the called method will hold a reference to the managed object if necessary. This could be a worthwhile optimization if the parameter variable is quickly set to null (or some other value) within the called method, in which case the managed object has no references and may be safely collected, even though the called method has not yet returned. – Jeffrey L Whitledge Mar 28 '11 at 22:26
  • (continued) - Obviously, these presumptions are not necessarily valid when calling unmanged code, which is why KeepAlive is necessary in these circumstances. – Jeffrey L Whitledge Mar 28 '11 at 22:28
  • 1
    @sehe: Jeffrey is correct. Passing a reference as an argument does not *necessarily* extend the lifetime of anything. If you pass a dead object to a method that doesn't use it, it's still dead and therefore can be reclaimed at any time. From the perspective of the managed runtime, unmanaged functions use no managed objects. If you want to tell the runtime that an unmanaged function uses a managed object, *you* are responsible for keeping it alive. That's what *unmanaged* means -- if you want to go calling code that doesn't manage object lifetimes, you get to do so instead. – Eric Lippert Mar 29 '11 at 02:44
2

From the following msdn seems like both pinning and GC.KeepAlive are unneeded in this case since CopyFileEx is synchronous. Specifically it says:

"Usually, you won't have to worry about the lifetime of delegates. Whenever you are passing a delegate to unmanaged code, the CLR will make sure the delegate is alive during the call. However, if the native code keeps a copy of the pointer beyond the span of the call and intends to call back through that pointer later, you might need to use GCHandle to explicitly prevent the garbage collector from collecting the delegate."

Since CopyFileEx does not keep a pointer to the function beyond the span of the call, we shouldn't need to call KeepAlive.

SpeksETC
  • 1,003
  • 2
  • 7
  • 13
  • This is incorrect even if the chance of running into a problem in this case is "very slim". This is, as pointed out in the other answers, the GC - on another thread - *may* reclaim the delegate supplied as a parameter *at any point* after the call has started. For a short-lived synchronous call this is a "usually, you won't have to worry" but the only way to be *guaranteed* that it will work is to make sure the GC *won't* reclaim it by using KeepAlive or otherwise maintaining a strong reference. – user2864740 Feb 16 '15 at 23:10
  • Put another way: would you feel as comfortable *not* using KeepAlive if the synchronous call spans several minutes? I sure don't and have been bitten by this "usually" not being "always"; even several seconds is eons to a CPU and plenty of free time for a GC to get too hungry. Make sure to control the lifetime exactly as required. – user2864740 Feb 16 '15 at 23:12