Practically speaking, no, this doesn't happen. Overwriting memory you've just freed takes time, so there are performance penalties. "Secure" objects like SecureString are just wiping themselves, not relying on the GC.
More broadly, it depends very much on that particular implementation of that particular language. Every language that assumes the existence of a GC (like C#) specifies different rules about how and when garbage collection should happen.
To take your C# example, the C# specification does not require that objects be overwritten after being freed, and it doesn't forbid it either:
Finally, at some time after the object becomes eligible for collection, the garbage collector frees the memory associated with that object.
§3.9 C# 5.0 Language Specification
If the memory is later assigned to a reference type, you'll have a constructor that does your own custom initialization. If the memory is later assigned to a value type, it gets zeroed out before you can start reading from it:
Initialization to default values is typically done by having the memory manager or garbage collector initialize memory to all-bits-zero before it is allocated for use. For this reason, it is convenient to use all-bits-zero to represent the null reference.
§5.2 C# 5.0 Language Specification
Additionally, there's at least two implementations of C# -- Microsoft's implementation and Mono's implementation, so just saying "C#" isn't specific enough. Each implementation might decide to overwrite memory (or not).