How can references be valid while iterators become invalidated in a deque

Question

I am having some difficulty grasping this concept. From this thread here it states

A deque requires that any insertion to the front or back shall keep any reference to a member element valid. It's OK for iterators to be invalidated, but the members themselves must stay in the same place in memory.

I was under the impression from this thread which states

A pointer is actually a type of iterator. In fact, for some container types, the corresponding iterator can be implemented simply as a pointer.

If we have a pointer and an iterator that each reference the same element of a container, then any operation that invalidates one will invalidate the other.

so if an iterator becomes invalidated then references also become invalidated. My question is how is that possible. If the iterator which points to a certain memory address becomes invalidated how can a reference to that address be valid ?

Update:

I understand that a deque is implemented by random chunks of memory and these chunks of memory are tracked by an independant data structure such as a dynamic array. However i am having difficulty understanding how an iterator could be invalid but a reference could be valid since essentially an iterator is a generalized pointer for the contents of the data structure. This makes me think that an iterator might be pointing to something else while a pointer points to the actual item ? Consider the following diagram of a vector .

From what i understand in the diagram above for a vector its that if content of a pointer changes the iterator also changes. How is that different for a deque .

Answer 1

Think of a deque in terms of the following:

template<typename T>
struct deque_stub {
 using Page = std::array<T, 32>; // Note: Not really, rather uninitialised memory of some size;
 std::vector<std::unique_ptr<Page>> pointers_to_pages;
 std::size_t end_insert{32};
 std::size_t start_elem{0};
 // read further
};

A deque is basically some container, storing pointers to pages which contain some elements. (The start_elem and end_insert members are to keep track of where, in terms of offset into a page, the valid range of elements starts and ends.)

Insertion eventually changes this container, when a new page is needed:

template<typename X>
void push_back(X&& element) {
 if (end_insert == 32) {
  // get a new page at the end
  pointers_to_pages.push_back(make_unique<Page>());
  end_insert = 0;
 }
 (*(pointers_to_pages.back()))[end_insert] = std::forward<X>(element);
 ++end_insert;
}

template<typename X>
void push_front(X&& element) {
 if (start_elem == 0) {
  pointers_to_pages.insert(
    pointers_to_pages.begin(), std::make_unique<Page>());
  start_elem = 32;
 }
 --start_elem;
 (*(pointers_to_pages.front()))[start_elem] = std::forward<X>(element);
}

An iterator into that deque needs to be able to "jump" across pages. The easiest way to achieve this is by having it keep an iterator to the current page it is in from the container pointers_to_pages:

struct iterator {
 std::size_t pos;
 std::vector<std::unique_ptr<Page>>::iterator page;
 // other members to detect page boundaries etc.
};

But since that page iterator, the iterator into the vector, may get invalidated when the vector gets changed (which happens when a new page is needed), the whole iterator into the deque might get invalidated upon insertion of elements. (This could be "fixed" by not using a vector as container for the pointers, though this would probably have other negative side effects.)

---

As an example, consider a deque with a single, but full page. The vector holding the pointers to pages thus holds only a single element, let's say at address 0x10, and let's further assume that its current capacity is also only 1 element. The page itself is stored at some address, let's say 0x100.

Thus the first element of the deque is actually stored at 0x100, but using the iterator into the deque means first looking at 0x10 for the address of the page.

Now if we add another element at the end, we need a new page to store that. So we allocate one, and store the pointer to that new page into the vector. Since its capacity is less than the new size (1 < 2), it needs to allocate a new larger area of memory and move its current contents there. Let's say, that new area is at 0x20. The memory where the pointers have been stored previously (0x10) is freed.

Now the very same element from above before the insertion is still at the same address (0x100), but an iterator to it would go via 0x20. The iterator from above, accessing 0x10, is thus invalid.

Since the element is at the same address, pointers and references to it remain valid, tough.

Answer 2

Because the answer you cite is wrong, and because iterators are a lot more than just pointers. For a start, a linked list iterator needs a pointer to the element but also "next" and "previous" pointers. Right there, with that simple example, your notion that "an iterator is a generalized pointer for the contents of the data structure" is completely blown out of the water.

A deque is more complicated than a totally contiguous structure (e.g. vector) and more complicated than a totally non-contiguous structure (i.e. list). When a deque grows, its overall structure moulds to fit, with a minimum of reallocations of the actual elements (often, none).

The result is that even when certain elements don't move, the "control pieces" that allow access to them may need to be updated with fresh metadata about, for example, where neighbouring elements (which maybe did move) now are.

Now, a deque cannot magically update iterators that have already been instantiated somewhere: all it can do is document that your old iterators are invalid and that you shall obtain new ones in the usual way.