5

System.Span's constructors take an int length and the length is stored internally as a 32 bit value.

However, the default padding will make the size of the struct 16 bytes anyway so having a 32 bit length doesn't save any space.

https://github.com/dotnet/corefx/blob/master/src/Common/src/CoreLib/System/Span.Fast.cs

The Span design document doesn't mention it. https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md

Eric
  • 570
  • 6
  • 19
  • I'm guessing it's because they didn't want you to be able to address more than 2 billion elements in one span? I mean, when was the last time you had a > 2 GB in a single buffer? – Etienne de Martel Oct 25 '19 at 01:08
  • Just now, actually. :) I have an 8gb MemoryMappedFile and wanted to access it with Span. – Eric Oct 25 '19 at 01:10

2 Answers2

7

For an authoritative answer, you'd have to ask the .NET team that designed and implemented Span<T>. That said…

.NET is fundamentally a 32-bit framework. It supports 64-bit processes, and can allocate from the 64-bit virtual address space. But it has a 2GB limit on objects. This leads to another design choice: collection sizes are 32-bit integers, and they are signed because CLS-compliant code doesn't use unsigned values. These two design choices go together, reinforcing each other.

So, when it comes to Span<T>, which is intended to provide subsets of existing memory in a managed context, the public interface that the type provides is compatible with CLS requirements. Thus, the Length property and the indexer parameter both are 32-bit integer types, as well are the parameters to Span<T> constructors that specify offsets and lengths within the existing memory.

This is consistent with the entire rest of the .NET API, in which arrays and other collections can have no more than 2^32 elements and can be no larger than 2GB.

Additional related reading:
64-Bit VB.NET Allocating > 2GB of RAM (.NET bug?)
How can I know the ACTUAL maximum number of elements a .net array of a given type can be allocated?
BigArray, getting around the 2GB array size limit

Peter Duniho
  • 68,759
  • 7
  • 102
  • 136
  • I see what you're saying about the 2GB limit on CLR objects. In the BigArray link it suggests that one way around this limitation is to use native memory. Now we have a new model for safely and quickly accessing native memory with Span and Memory ... but it's limited to 2GB when removing that limitation appears to be free on 64 bit systems. – Eric Oct 25 '19 at 12:23
2

The answer to the question seems to be that Span wants to match Memory which has an additional 4 byte field.

https://github.com/dotnet/corefx/issues/26603#issuecomment-370419371

The current version of Memory packs nicely into 16 bytes on x64, while Span seems to have room for replacing the int _length by IntPtr _length and still fitting into 8/16 bytes. However, increasing the Lenght property of Span requires doing the same with Memory. If I'm not mistaken, increasing the size of Memory (from 16 bytes to 24 bytes) might have consequences on the performance of the code, which would impact everyone. (Not just those of us that are playing with large regions of memory)

It is true that in the case I presented, ReadOnlySequence acts as a valid replacement for a 64 bits-enabled Memory / Span, because all I needed was to copy the data somewhere. But when you need to read/decode without copying, the API might indeed be less straightforward.

Eric
  • 570
  • 6
  • 19