How to return large vector from class without copying data

Question

I am writing a program in which a class has a data member that is a big std::vector (on the order of 100k - 1M items). Other classes need to be able to access this vector. At the moment I have a standard accessor function that returns the vector, but this returns a copy of the vector I believe. I think it would be more memory and time efficient to just return an iterator or a pointer to the first element. However, if I do this, how can one then use this pointer to run through the vector and know when to stop (i.e. where the vector ends)?

My code looks like this:

class MyClass
{
    private:
        std::vector<MyObj> objects_;
        //...

    public:
        std::vector<MyObj> getObjects() { return objects_; }
        //...
}

This question arises in another form for me when I want to run through (simulated) concatenated vectors. If I have a vector of MyClass, I'd like to be able to iterate over all of the contained object_ vectors. I know from this answer that boost::join does what I have in mind, but I think I need to return copies for it to work. Can I return a pointer to the vector and still retain the ability to iterate over it and others in succession?

R Sahu · Accepted Answer · 2016-09-11T06:18:50.973

7

To avoid the performance penalties, return references.

// Non-const version
std::vector<MyObj>& getObjects() { return objects_;}

// const version
std::vector<MyObj> const& getObjects() const { return objects_; }

However, before you make that change, you have to consider the downsides of exposing references to a member variable. It makes your class less flexible. You can't easily change objects_ to a different type of container if that made more sense without impacting all the users of the class.

edited Sep 11 '16 at 06:18

answered Sep 11 '16 at 04:10

R Sahu

204,454
14
159
270

Could you please explain further why exposing references makes it harder to change the container type? Do you mean the templated type? – marcman Sep 11 '16 at 04:13
@marcman, if your container needed to be a set instead of a list, you can't easily change `objects_` to be of type `std::set`. That would break all the clients that were depending on `objects_` to be of type `std::vector`. – R Sahu Sep 11 '16 at 04:16
@marcman See a solution for the change of container in my answer – Adrian Colomitchi Sep 11 '16 at 04:26

Adrian Colomitchi · Answer 2 · 2016-09-11T04:40:55.780

2

Make your class act as a collection by delegating to the vector data member. Of course, you may need to revisit the code that consumes MyClass but, with the getObjects() commented out, the compiler will tell you where + most of the changes are likely to be on the line of

MyClass heapsOfThem;
// ...
// just delete the `getObjects()` *and use MyClass::iterator*
// instead of std::vector::iterator.
// for(std::vector<MyObj>::iterator it=
//    heapsOfThem.getObjects().begin()...
// )
for(MyClass::iterator it=heapsOfThem.begin()...)

Delegation code goes on the line of the below - once you fixed your calling code, you can change your mind what type (vector, list, set) to use as an internal container for your objects without changes in the calling code.

class MyClass
{
    private:
        std::vector<MyObj> objects_;
        //...

    public:


        const size_t size() const {
          return objects_,size();
        }
        MyObj& operator[](size_t i) {
          return objects_[i];
        }
        const MyObj& operator[](size_t i) const {
          return objects_[i];
        }

        using iterator = std::vector<MyObj>::iterator;
        iterator begin() {
          return objects_.begin();
        }
        iterator end() {
          return objects_.end();
        }
        // TODO const iterators following the same pattern

        // *if you aren't good enough with the above*
        // uncomment it and let it return a *reference* 
        // std::vector<MyObj>& getObjects() { return objects_; }
        //...
}

edited Sep 11 '16 at 04:40

answered Sep 11 '16 at 04:25

Adrian Colomitchi

3,974
1
14
23

This doesn't really avoid the problems of changing the set type - the calling code has the same dependency on `std::vector::iterator`. – M.M Sep 11 '16 at 04:33
@M.M "This doesn't really avoid the problems " yes, it does. Thanks for showing there is a need for an extra explanation, I added it – Adrian Colomitchi Sep 11 '16 at 04:42
The calling code could still avoid any source changes when the container changes, even in the case of returning `vector`. – M.M Sep 11 '16 at 04:44
@M.M - "The calling code could still avoid any source changes when the container changes," - well, of course it could. – Adrian Colomitchi Sep 11 '16 at 04:51
Yes, so what is the benefit of returning iterators instead of vector? – M.M Sep 11 '16 at 04:52
@M.M Really? How is you question different from asking "So, what is the benefit of encapsulation?" – Adrian Colomitchi Sep 11 '16 at 05:02

score 0 · Answer 3 · answered Sep 11 '16 at 04:36

You can refactor the class to have public methods that return an element of the array, and the size of the array, so all other classes can fetch values, without any copying of the entire vector.

public:
    unsigned int getMyObjArraySize();
    MyObj getMyObjElementAt(unsigned int index);

With this approach, there is only one instance of the vector, but any collaborations can be done via the two public methods that expose the size and an accessor to values via the index.

This approach is geared to the use of for-loops and not iterators.

MyClass myClass;
// ...
MyObj myObj;

for(unsigned int i; i < myClass.getMyObjArraySize(); i++) {
    myObj = myClass.getMyObjElementAt(i);
    // do stuff
}

The issue with this is ultimately I need to run through the entire vector, and I'd like to do it without copying all the data. Your `getMyObjectElementAt` returns a copy of each element, so ultimately it's no different than just returning the whole vector and iterating through it — marcman, Sep 11 '16 at 04:38

score 0 · Answer 4 · answered Sep 11 '16 at 10:12

There's no problem returning a pointer to the vector.

std::vector<MyObj>* getObjects() { return &objects_; }

And then when want to iterate over it, just dereference:

std::vector<MyObj>* objectsPtr = getObjects();
for (auto& it : *objectsPtr)
{
   ...
}

However, make sure that you aren't writing to the vector while reading from it, since that would invalidate the iterator.

How to return large vector from class without copying data

4 Answers4