What .NET 4.0 System.Collections.Concurrent collection added in functionality to .NET 3.0 SynchronizedCollection?

Question

.NET 4.0 introduced the System.Collections.Concurrent namespace:

"The System.Collections.Concurrent namespace provides several thread-safe collection classes that should be used in place of the corresponding types in the System.Collections and System.Collections.Generic namespaces whenever multiple threads are accessing the collection concurrently"

• BlockingCollection<T> class
• ConcurrentBag<T> class
• ConcurrentQueue<T> class
• ConcurrentDictionary<TKey, TValue> class
• OrderablePartitioner<TSource> class
• Partitioner class
• IProducerConsumerCollection<T> interface

The SynchronizedCollection<T> class (available since .NET 3.0):

"Provides a thread-safe collection that contains objects of a type specified by the generic parameter as elements"

...is in the System.Collections.Generic namespace.

So, why is the SynchronizedCollection<T> class thread-safe but not concurrent?

What specifically makes the SynchronizedCollection<T> generic class different and incompatible with collections from those in System.Collections.Concurrent?

Update: Let me rephrase the question: What is the common denominator and distinguishing new feature in all generic collections that belong to the System.Collections.Concurrent namespace, which is absent in (and impossible while using) the SynchronizedCollection<T> generic class?

I changed the title to "What .NET 4.0 System.Collections.Concurrent collection added in functionality to .NET 3.0 SynchronizedCollection?". But mostly I am interested to know what it is that made it impossible to do on the basis of .NET 3.0

Update2: Concerning the note:

"This question may already have an answer here:

What is the difference between SynchronizedCollection and the other concurrent collections?"

The answer is confusing in context of my question - are the new features evolutionary (using pre-.NET 4.0 features) or revolutionary (unavailable in pre-.NET 4.0)?

Answer 1

The previous thread-safe collection classes have a rather major flaw, they cannot be iterated in a thread-safe way. Iterating is in general very difficult to make thread-safe, there is no clean way to make code that uses an iterator aware of items that are added to or removed from a collection while the code is iterating the collection. The only truly safe way is to lock access to the collection while the iteration is taking place. That's very undesirable, such a lock will typically be held for a very long time. Next best way is to make a copy of the collection, a copy that won't be modified so can always be safely iterated. Nobody likes the O(n) storage requirement of that solution.

The Synchronized() method in the .NET 1.0 collection classes was particularly troublesome. Microsoft did not make it a secret that iterating isn't thread-safe. They even added code to detect the mishap, it throws an exception when a thread modified the collection. This exception however doesn't occur often enough and bewildered .NET programmers that failed to read the warning label and just flat-out assume that surely a thread-safe collection is safe no matter what you do. Well, it wasn't and calling a collection "thread-safe" when the most basic operation on a collection isn't thread-safe is of course a problem. What Microsoft meant was "it is thread-safe because using threads won't corrupt the collection". What programmers read was "it is thread-safe". Microsoft didn't repeat the same mistake in the .NET 2.0 generic collection classes.

This was addressed in .NET 4, iterating the new collections won't throw an exception. They will not otherwise do anything to prevent your code from misbehaving because it is reading stale data. That's an unsolvable problem that you have to solve yourself.

Answer 2

The major focus of .NET 4.0 was concurrency. Microsoft introduced the System.Threading.Tasks namespace in .NET 4.0, so it made sense to add the System.Collections.Concurrent namespace as well. As mentioned elsewhere the collections in System.Collections.Concurrent are highly performant as they are implemented in a lock-free manner. This is not an easy task and I think this is why Microsoft felt that it belonged in a new major .NET release instead of in a refresh of .NET 3.x. AFAIK the lock-free behavior is among other things implemented using the generic version of Interlocked.CompareExchange which is also new to .NET 4.0.

If you are interested in how to implement the classes of System.Collections.Concurrent, I recommend reading Joe Duffy's book "Concurrent Programming on Windows" which he wrote prior to the release of .NET 4.0. The book shows how some of the concurrent collections are implemented.

Answer 3

The linked duplicate answers the 'new functionality' part of your question. I will just speak here to this note:

what it is that that made it impossible to do on the basis of .NET 3.0

My guess is nothing made it impossible. I suspect that if you get a decompile of the new classes in .NET 4.0 and (with syntax alterations as required) create such classes yourself against .NET 3.5, they would work perfectly well.

This is the same as how, say, Enumerable.Zip can be manually implemented in .NET 3.5, if .NET 4.0 is not available. Indeed, I remember seeing somewhere an implementation of LINQ-like libraries against .NET 2.0 (without the C# language features of course!).

In general with .NET, unless libraries depend on new CLR features, there's nothing stopping the library being reimplemented against an earlier version. One notable example of the situation is generics: prior to .NET 2.0, there was no way of getting typesafe collection support without handcranking each different class, perhaps using codegen tools such as CodeSmith.

So if nothing prevented the existence of the Concurrent classes in .NET 3.5, why didn't Microsoft create them then? Simply time and budget - shipping is a feature, and some features are less important than that, so they have to wait till a later version.