19

Suppose I have 2 (or more) containers I want to iterate through simultaneously - for example, to compute the dot product of two vectors:

std::vector<double> vector1;
std::vector<double> vector2;    // identical size to vector1

What is the preferred C++11 way to specify a range-for loop over both (or all) containers simultaneously? Does it involve choosing one container/iterator to write short-hand (i.e. for ( auto i : c )) in a range-for loop, while all other containers/iterators have to be handled long-hand? Is there any reason the syntax in future could not be extended to support short-hand for both/all containers as shown below... which seems really readable:

double dotProduct( 0.0 );
for ( auto const & value1 : vector1, auto const & value2 : vector2 )  // illegal!
{
    dotProduct += value1*value2;
}
ildjarn
  • 62,044
  • 9
  • 127
  • 211
omatai
  • 3,448
  • 5
  • 47
  • 74
  • 3
    Possible duplicate of [What's the best way to iterate over two or more containers simultaneously](http://stackoverflow.com/questions/12552277/whats-the-best-way-to-iterate-over-two-or-more-containers-simultaneously) – ildjarn Jul 21 '16 at 02:50
  • 1
    On the surface, yes... but are any of the answers there (a) readable and (b) general and (c) based on standard c++? Not that I saw. From the early days of C, one could write `for ( i=0, j=0, k=0; i < N; i++, j++, k++ )` (and still can write it), which is extraordinarily succinct. Why has the comma operator not been extended to allow multiple variable declaration inside the for loop - 1 is permitted, but why only 1? – omatai Jul 21 '16 at 03:13
  • Have a look at [miterator](https://github.com/ClaasBontus/miterator). It will all become easier with ranges, which will not be before C++20. – Claas Bontus Jul 21 '16 at 10:56
  • Due to learning a lot through asking this question, yet failing to answer a central question underlying it, and not really having the space in comments to articulate that question, I have asked a new question [here](http://stackoverflow.com/questions/38601927/cake-versus-salad-is-c-heading-in-the-right-direction-with-for-loop-syntax) – omatai Jul 27 '16 at 00:31

3 Answers3

14

In other (often functional) languages this is done by using a function called zip. As an example, Python has a builtin zip that iterates over over its arguments and returns a tuple:

for i in zip( [1,2,3], (1,2,3), { 0:0, 1:1, 2:2 } ): 
    l,t,d = i 
    print("list item: %d, tuple item %d, dict item %d" % (l,t,d) )      

You can use a range library in C++ to get that functionality, e.g. Boost.Range or Eric Niebler's rangev3. Ranges were unfortunately not voted in the C++17 standard, but I would never start a project without a range library. In Boost.Range the function is called combine:

#include <boost/range/combine.hpp>
#include <boost/tuple/tuple.hpp>
#include <iostream>
#include <vector>
#include <list>

int main(int, const char*[])
{
    using namespace boost;

    std::vector<int> const v{0,1,2,3,4};
    std::list<char> const  l{'a', 'b', 'c', 'd', 'e'};

    for(auto const& i: combine(v, l))
    {
        int ti;
        char tc;
        boost::tie(ti,tc) = i;
        std::cout << '(' << ti << ',' << tc << ')' << '\n';
    }

    return 0;
}

With C++17 you can replace the std::tie with structured binding and remove the kind of unusual "initialization" with std::tie.

  for(auto const& [ti,tc] : boost::combine(v, l)) {
     std::cout << '(' << ti << ',' << tv << ')' << '\n';
  }

While I regret that ranges are not included in C++17, I think that structured bindings are a great advancement and will seriously change the way code is written. Having ranges in the standard would make them more popular and elevate them from a third-party library where many people have objections because it is something they don't know to a standard feature that C++ programmer ought to know.

Jens
  • 9,058
  • 2
  • 26
  • 43
  • A perfectly valid answer... but kinda disappointing. And shouldn't it be `std::tie(ti,tc)=i`? What I find dissatisfying about it is not your fault - it's the fault of the C++ syntax that heavily steers towards (a) combining things using (b) tricks and (c) a single iterator... rather than (simpler) multiple synchronised iterators. Given the original C syntax shown in my comment to my question, I'm surprised C++ does not support `for (int i=0,int j=0;...)` when it supports `for (int i=0,j=0;...)`. I'm curious to know if there is a compelling reason why not. – omatai Jul 21 '16 at 21:51
  • @omatai why disappointing? You can use boost::combine even in C++98 code. Regarding "tricks" - look at Python version - there is also "trick" for combining things, even Haskell has zip "trick". And I don't think this is disadvantage - why pollute language with stuff that can be easily implemented within libraries? – Evgeny Panasyuk Jul 21 '16 at 21:59
  • Readability, clutter. If I came across either my suggested (illegal) code, or this code in five years time, I would instantly know what was going on with my suggestion, and need to apply far more brainpower and leaps of faith to digest this (entirely valid) version (no offence to @Jens). So I'm disappointed not in this (entirely valid) answer, but in the failure of the C++ syntax to support something that I feel would simpler and cleaner. – omatai Jul 21 '16 at 22:05
  • 2
    You might update the answer to point out that the just-adopted-for-C++17 structured bindings would simplify this from `for(auto const& i : boost::combine(v, l)) { int ti; char tc; std::tie(ti,tc) = i; use(ti,tc); }` to just `for(auto const& [ti,tc] : boost::combine(v, l)) { use(ti,tc); }`. I haven't looked closely at `boost::combine` but if it works with `std::tie` it should be fine with structured bindings. – Herb Sutter Jul 21 '16 at 22:48
  • @omatai I think a for-each loop `for(auto const& i: combine(v,l))` is much more expressive than a loop using multiple indices. I can translate the for-each directly into its semantics, but for the C-style approach I have to scan and parse the body to figure out that it just iterates over more than one collection. I think that most C-style loops should be replaced with (STL-style) algorithms if they are not for-each loops. Consider more complicated examples like `for(auto const& i: v | filtered(pred) | transformed(f))` which is very expressive and compare it to a C-style loop. – Jens Jul 22 '16 at 06:45
  • @omatai It also a strength of a language that it has simple building blocks like a simple, well-defined for-each loop that you can compose with other library features to express very powerful functionality. I think this is far better than adding special syntax for special cases. – Jens Jul 22 '16 at 06:47
  • @Jens Agree to disagree :-) The comma operator is part of C++... but it does not extend to `for (int i=0, int j=0;...)`. Can someone offer a good reason why not? Why must j be declared outside the loop even if only used inside it? I'd love to know. Also, I wouldn't want C++ to enforce its "expressiveness" upon me and prohibit me from using the mathematical abstractions (e.g. i,j) that exist on my whiteboard. I think there is room for both - if I'm dealing with matrices and vectors, I want to express it in those natural terms. – omatai Jul 24 '16 at 23:36
  • @omatai I am not sure the comma in ` for (int i=0, j=1)` is a comma operator. Comma operators are part of expressions, but ` int i=0, j=1` is a simple declaration. Here `int` is a decl-specifier-seq and `i=0,j=1` is an init-declarator-list which is a comma-separated sequence of declarator and initializers. The comma operator can only part of the initializer expression. – Jens Jul 25 '16 at 07:09
  • @Jens - apologies: I had not seen the update based on C++ 17 syntax - thanks. And I had also not realised that `for (int i=0, j=0; ...)` does compile and does declare `j` as a secondary loop variable. My example would have been better with a different type - `for (int i=0,char j=0;...)` - to highlight the question that still puzzles me: why not keep iterators/variables within for loop contexts as independent rather than combined - salad rather than cake plus a special (and confusing) knife - `std::tie()`. But it's a philosophical question, not practical. At least C++17 will be clearer. – omatai Jul 26 '16 at 22:05
  • correcting the `cout`’s `tv` to `tc`, the example using `std::tie` [does not work](http://coliru.stacked-crooked.com/a/293e0ff0ef2bd902) and therefore, sadly, also not the range-based for loop. `boost::tie` has to be used for it [to work](http://coliru.stacked-crooked.com/a/87136d9f0f54265b). I can’t find any evidence that the implementation of `boost::combine` changed in the last 2 years. What’s up with that? – Darklighter Dec 02 '18 at 18:29
  • @Darklighter You are correct that you have to use boost::tie. But then, since ranges seem to be voted into C++2x, there will be no need for boost tuple and ranges in the future. – Jens Dec 02 '18 at 22:02
  • I get the following compile error for the version with structured binding (using `MSVC 19.13.26132.0` and `Windows SDK version 10.0.16299.0`): `error C2679: binary '<<': no operator found which takes a right-hand operand of type 'const boost::tuples::cons>,Last,true>::type>' (or there is no acceptable conversion)` – pooya13 Mar 16 '19 at 20:44
  • @Jens You are probably aware by now but just in case you're not, ranges will be in C++20! – Addy Nov 11 '19 at 13:59
4

I know that this question is quite old, but it's still the first result on google. And since the second solution in the accepted answer doesn't work as mentioned in the comments, here is a nice solution for C++17 including an example in main:

#include <tuple>
#include <type_traits>

//#define ALT2

#ifndef ALT2
template<typename T, std::size_t i = 0, std::size_t j = std::tuple_size<T>::value>
struct tuple_compare {
    static bool
    one_equal(T const& lhs, T const& rhs) {
        if constexpr(i == j) return false;
        else {
            return (std::get<i>(lhs) == std::get<i>(rhs) ||
            tuple_compare<T, i + 1, j>::one_equal(lhs, rhs));
        }
    }
};
#endif

template<typename... Conts>
struct container_ref_tuple {
    static auto constexpr get_begin{[](auto&&... args){return std::make_tuple(begin(args)...);}};

    typename std::invoke_result<decltype(&std::forward_as_tuple<Conts...>), Conts&&...>::type m_refs;

    struct iterator {
        typename std::invoke_result<decltype(get_begin), Conts&&...>::type m_iterators;

        decltype(auto)
        operator++() {
            apply([](auto&... args) {((++args), ...);}, m_iterators);
            return (*this);
        }

        #ifndef ALT2
        //Alternative 1(safe)
        //will stop when it reaches the end of the shortest container
        auto
        operator!=(iterator const& rhs) const {
            return !tuple_compare<decltype(m_iterators)>::one_equal(m_iterators, rhs.m_iterators);
        }
        #else
        //Alternative 2 (probably faster, but unsafe):
        //use only, if first container is shortest
        auto
        operator!=(iterator const& rhs) const {
            return std::get<0>(m_iterators) != std::get<0>(rhs.m_iterators);
        }
        #endif

        auto
        operator*() const {
            return apply([](auto&... args){return std::forward_as_tuple(*args...);}, m_iterators);
        }
    };

    auto
    begin() const {
        return iterator{apply(get_begin, m_refs)};
    }

    #ifndef ALT2
    //Alternative 1(safe)
    //will stop when it reaches the end of the shortest container
    static auto constexpr get_end{[](auto&&... args){return std::make_tuple(end(args)...);}};
    auto
    end() const {
        return iterator{apply(get_end, m_refs)};
    }
    #else
    //Alternative 2 (probably faster, but unsafe):
    //use only, if first container is shortest
    auto
    end() const {
        iterator ret;
        std::get<0>(ret.m_iterators) = std::end(std::get<0>(m_refs));
        return ret;
    }
    #endif
};

template<typename... Conts>
auto
make_container_ref_tuple(Conts&&... conts) {
    return container_ref_tuple<Conts...>{std::forward_as_tuple(conts...)};
}

#include <array>
#include <iostream>
#include <list>
#include <vector>

int
main(int argc, char** argv) {
    std::array integers{1, 2, 3, 4, 5, 6, 7, 8};
    std::list prime{2, 3, 5, 7, 11, 13, 17, 19, 23};
    std::vector chars{'a', 'b', 'c'};

    for(auto&& [i, p, c] : make_container_ref_tuple(integers, prime, chars)) {
        std::cout << i << ' ' << p << ' ' << c << '\n';
        std::swap(i, p);
        ++c;
    }

    std::cout << "New: \n";

    for(auto&& [i, p, c] : make_container_ref_tuple(integers, prime, chars)) {
        std::cout << i << ' ' << p << ' ' << c << '\n';
    }

    return 0;
}
Michael Mahn
  • 737
  • 4
  • 11
0

A clean alternative is to use a lambda with a function that just forwards the indices (this avoids begin/ends, etc):

template <typename Container, typename Functor>
void ForIndices (const Container & c, Functor && f)
{
    for (size_t i = 0; i < c.size(); ++i)
        f(i);
}

which you can use in your example like so:

double dotProduct = 0;
ForIndices(vector1, [&](int i){dotProduct += vector1[i]*vector2[i];});

Demo (C++11)

Elliott
  • 2,603
  • 2
  • 18
  • 35