Your understanding of an indexer may be slightly off. An indexer essentially doesn't have any relation to some underlying array. It merely provides a way to define a syntax similar to those used to access array elements.
In fact, the built-in Dictionary
class uses it with non-integer arguments to provide access to dictionary values via their corresponding key objects.
Also, indexers can take more than one parameter. You could, for example, define an enumeration of values representing the various arrays your class contains and return or set the appropriate value of the corresponding array, then.