26

I wasn't sure what to search for. I found Renaming first and second of a map iterator but it's not quite what I want to do.

Here's what I'd like to do [see below for nonsense C++ code]. Is something close to this possible? Otherwise will just have to go with the option of "adapting" the iterator as the first line inside the loop I suppose.

// what I want to do:
std::map<int, std::string> my_map;
// ... populate my_map
for(auto key, auto & value: my_map){
    // do something with integer key and string value
}

C++11 is fine, but rather avoid boost if possible.

The closest I've gotten is

// TODO, can this be templated?
struct KeyVal{
    int & id;
    std::string & info;

    template <typename P>
    KeyVal(P & p)
        : id(p.first)
        , info(p.second)
    {
    }
};

//...
for ( KeyVal kv : my_map ){
    std::cout << kv.info;
}

But that means writing an adapter class for each map :(

// slightly joke answer/"what could possibly go wrong?"
#define key first
#define value second
Community
  • 1
  • 1
JamEnergy
  • 720
  • 8
  • 21

7 Answers7

17

An approach inspired by Barry below would be to write a range adapter.

Doing this without boost or similar library support is a pain, but:

  1. Write a range template. It stores 2 class iterators and has begin() and end() methods (and whatever else you want).

  2. Write a transforming iterator adapter. It takes an iterator, and wraps it up so that its value type is transformed by some function object F.

  3. Write a to_kv transformer that takes a std::pair<K, V> cv& and returns a struct kv_t { K cv& key; V cv& value; }.

  4. Wire 3 into 2 into 1 and call it as_kv. It takes a range of pairs, and returns a range of key-values.

The syntax you end up with is:

std::map<int, std::string> m;

for (auto kv : as_kv(m)) {
  std::cout << kv.key << "->" << kv.value << "\n";
}

which is nice.

Here is a minimalist solution that doesn't actually create legal iterators, but does support for(:):

template<class Key, class Value>
struct kv_t {
  Key&& key;
  Value&& value;
};

// not a true iterator, but good enough for for(:)
template<class Key, class Value, class It>
struct kv_adapter {
  It it;
  void operator++(){ ++it; }
  kv_t<Key const, Value> operator*() {
    return {it->first, it->second};
  }
  friend bool operator!=(kv_adapter const& lhs, kv_adapter const& rhs) {
    return lhs.it != rhs.it;
  }
};
template<class It, class Container>
struct range_trick_t {
  Container container;
  range_trick_t(Container&&c):
    container(std::forward<Container>(c))
  {}
  It begin() { return {container.begin()}; }
  It end() { return {container.end()}; }
};
template<class Map>
auto as_kv( Map&& m ) {
  using std::begin;
  using iterator = decltype(begin(m)); // no extra (())s
  using key_type = decltype((begin(m)->first)); // extra (())s on purpose
  using mapped_type = decltype((begin(m)->second)); // extra (())s on purpose
  using R=range_trick_t<
    kv_adapter<key_type, mapped_type, iterator>,
    Map
  >;
  return R{std::forward<Map>(m)};
}
std::map<int, std::string> m() { return {{0, "Hello"}, {2, "World"}}; }

which is very minimal, but works. I would not generally encourage this kind of half-assed pseudo iterators for for(:) loops; using real iterators is only a modest additional cost, and doesn't surprise people later on.

live example

(Now with temporary map support. Does not support flat C arrays ... yet)

The range-trick stores a container (possibly a reference) in order to copy temporary containers into the object stored for the duration of the for(:) loop. Non-temporary containers the Container type is a Foo& of some kind, so it doesn't make a redundant copy.

On the other hand, kv_t obviously only stores references. There could be a strange case of iterators returning temporaries that break this kv_t implementation, but I'm uncertain how to avoid it in general without sacrificing performance in more common cases.


If you don't like the kv. part of the above, we can do some solutions, but they aren't as clean.

template<class Map>
struct for_map_t {
  Map&& loop;
  template<class F>
  void operator->*(F&& f)&&{
    for (auto&& x:loop) {
      f( decltype(x)(x).first, decltype(x)(x).second );
    }
  }
};
template<class Map>
for_map_t<Map> map_for( Map&& map ) { return {std::forward<Map>(map)}; }

then:

map_for(m)->*[&](auto key, auto& value) {
  std::cout << key << (value += " ") << '\n';
};

close enough?

live example

There are some proposals around first-class tuples (and hence pairs) that may give you something like that, but I do not know the status of the proposals.

The syntax you may end up if this gets into C++ would look something like this:

for( auto&& [key, value] : container )

Comments on the ->* abomination above:

So the ->* is being used sort of as an operator bind from Haskell (together with implicit tuple unpacking), and we are feeding it a lambda that takes the data contained within the map and returns void. The (Haskell-esque) return type becomes a map over void (nothing), which I elide into void.

The technique has a problem: you lose break; and continue; which suck.

A less hackey Haskell-inspired variant would expect the lambda to return something like void | std::experimental::expected<break_t|continue_t, T>, and if T is void return nothing, if T is a tuple-type return a map, and if T is a map join the returned map-type. It would also either unpack or not unpack the contained tuple depending on what the lambda wants (SFINAE-style detection).

But that is a bit much for a SO answer; this digression points out that the above style of programming isn't a complete dead-end. It is unconventional in C++ however.

Yakk - Adam Nevraumont
  • 262,606
  • 27
  • 330
  • 524
  • 2
    As far as proposals go, there's also the structured bindings - which is probably the most relevant one (`for (auto&& [key, value] : map) { ... }`) – Barry Apr 07 '16 at 16:10
  • I'm trying to decide whether I like this answer or not. After writing/being corrupted by C++ for a couple of years, I find it difficult to distinguish the Dark Side from the light... – JamEnergy Apr 07 '16 at 16:24
  • 1
    I could probably live with loss of 'break'/'continue', but I wouldn't want to inconvenience co-maintainers of the code with a strange loop where ordinary logic doesn't apply. – JamEnergy Apr 07 '16 at 16:32
  • @JamEnergy True. Sketch of a half-decent solution inspired by Barry's answer below added. – Yakk - Adam Nevraumont Apr 07 '16 at 17:07
  • Oh my god, this actually works. http://pastebin.com/FVpfRxVg It's a lot of work though, and I might not use it in real code. But it's here if anyone wants it. Should I edit it into this answer and mark it as the solution? I haven't sorted out what should be copy by val, or pass by ref in the helper functions etc. – JamEnergy Apr 08 '16 at 11:14
  • I'm also not sure about const (in)correctness. And it's a bit sad that we had to hardcode "key" and "value" names. Can't wait for structured bindings support. – JamEnergy Apr 08 '16 at 11:23
  • @jam have not read your link yet, but you'll want a cv-lr-ref-transcription helper. A `std::pair const&` becomes a `kv` under the transforming iterator. In C++14 you may be able to just return `auto` with a `make_kv` function call in `operator*` (note you cannot support `->` without extra work, but `for(:)` does not need it) – Yakk - Adam Nevraumont Apr 08 '16 at 11:39
  • I like the range adapter. And with help of Boost it's not that pain at all :) – oliora Apr 08 '16 at 23:14
  • 1
    @oliora Turns out it isn't hard even without `boost`. I exploited the fact we don't actually need iterators for `for(:)` loops under the standard to slack off. – Yakk - Adam Nevraumont Apr 09 '16 at 00:43
  • @JamEnergy I wrote a simple version that handles const maps properly and is about half the LOC by exploiting the fact that `for(:)` loops don't actually need iterators. – Yakk - Adam Nevraumont Apr 09 '16 at 00:44
  • 1
    What is the purpose of `decltype(x)(x)`? – Maxim Egorushkin Apr 09 '16 at 14:35
  • 1
    @maxim perfect forwarding with `auto&&x`. So first and second have right r/lvalue category. – Yakk - Adam Nevraumont Apr 09 '16 at 14:53
  • 1
    Is it possible to move the minimalist version "to the top" of the answer, so people don't get put off by our "historical" attempts? Then I will mark it as the solution. I don't want to mark "the rest" of the attempt as the solution really. The minimalist solution here is roughly what I went for in my attempt. And yes it works, basically renames "first" to "key" and "second" to "value". All things considered, I would probably recommend developers to just start loops with const Key& key = iter.first; though. – JamEnergy Apr 11 '16 at 11:40
  • 1
    Would not `std::forward(x)` be more appropriate? `decltype(x)(x)` ends up doing an unnecessary copy. – Maxim Egorushkin Apr 12 '16 at 16:03
  • 2
    @MaximEgorushkin if `x` is of type `auto&&`, then `decltype(x)(x)` does no copy: `x` is guaranteed to be (some) reference type, and casting to a different reference type does no copy. If `x` is of type `auto`, it does, and `std::forward(x)` is appropriate. Typically, I perfect forward with `auto&&` variables, not `auto` variables. The brevity of `decltype(x)(x)` is why I prefer it over `std::forward(x)`, especially in lambdas. The forward variation will work with `auto&&`, as `forward` works if you pass it a `int&&` or similar type as well as `int`. – Yakk - Adam Nevraumont Apr 12 '16 at 16:40
  • Is there a reason to forward `x` with `decltype(x)(x)`, when we just want to access its fields? – Mikhail Apr 21 '16 at 08:54
  • @mik yes, there is. If you want to learn about perfect forwarding and its issues and tricks, ask a SO question; there is too much information in this comment thread already. – Yakk - Adam Nevraumont Apr 21 '16 at 09:09
  • Ok, I will be more specific. Is there a reason for forwarding here in case we use it with `std::map`? – Mikhail Apr 21 '16 at 10:43
6

You could write a class template:

template <class K, class T>
struct MapElem {
    K const& key;
    T& value;

    MapElem(std::pair<K const, T>& pair)
        : key(pair.first)
        , value(pair.second)
    { }
};

with the advantage of being able to write key and value but with the disadvantage of having to specify the types:

for ( MapElem<int, std::string> kv : my_map ){
    std::cout << kv.key << " --> " << kv.value;
}

And that won't work if my_map were const either. You'd have to do something like:

template <class K, class T>
struct MapElem {
    K const& key;
    T& value;

    MapElem(std::pair<K const, T>& pair)
        : key(pair.first)
        , value(pair.second)
    { }

    MapElem(const std::pair<K const, std::remove_const_t<T>>& pair)
        : key(pair.first)
        , value(pair.second)
    { }
};

for ( MapElem<int, const std::string> kv : my_map ){
    std::cout << kv.key << " --> " << kv.value;
}

It's a mess. Best thing for now is to just get used to writing .first and .second and hope that the structured bindings proposal passes, which would allow for what you really want:

for (auto&& [key, value] : my_map) {
    std::cout << key << " --> " << value;
}
Yakk - Adam Nevraumont
  • 262,606
  • 27
  • 330
  • 524
Barry
  • 286,269
  • 29
  • 621
  • 977
  • I know it sounds lazy, but I don't really want to type out the template parameters (they might be long!). I feel like if constructors could deduce the type of a class, this would be the solution. But I don't think that's possible here (in some cases you can wrap it in a 'deducing factory function' I guess). – JamEnergy Apr 07 '16 at 16:36
  • It's easy to write an alias template that will give you the type `MapElem` from `std::map` – Jonathan Wakely Apr 07 '16 at 16:47
  • 1
    You could write a range adapter that turns ranges of pairs into ranges of your key/value type. Then you'd get `for (auto kv : as_kv(my_map))` – Yakk - Adam Nevraumont Apr 07 '16 at 17:01
  • @Yakk I like the range adapter idea. Good team effort! – Barry Apr 07 '16 at 17:39
  • Shouldn't the syntax in the last code segment be `for (auto&& {key, value} : my_map)` (i.e. braces instead of square brackets)? Implied from section 3.4 of the [structured bindings proposal](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0144r0.pdf). – Leo Heinsaar Apr 09 '16 at 06:08
  • @Leo The latest revision switched to square brackets. – Barry Apr 09 '16 at 12:51
  • @Yakk I'm amused that of all the answers I've written, this one (which is clearly not as good as your answer) got onto [isocpp](https://isocpp.org/blog/2016/04/quick-qis-it-possible-in-cpp-to-iterate-over-a-stdmap-with-unpacked-key-and)... – Barry Apr 12 '16 at 17:22
6

With modern c++17 this is now possible with structured bindings.

#include <map>
#include <string>
#include <iostream>

using namespace std;

int main() {
    map<int, string> my_map;

    my_map[0] = "hello";
    my_map[1] = "world";

    for (auto&& [key, value] : my_map) {
        cout << key << "," << value << "\n";
    }

    return 0;
}

Build it:

$ clang++ -std=c++17 test.cpp -o program

Output:

$ ./program
0,hello
1,world
Cusiman7
  • 124
  • 1
  • 6
3

The closest thing it to use std::tie:

std::map<int, std::string> my_map;
int key;
std::string value;
for(auto&& p: my_map)
{
    std::tie(key, value) = p;
    std::cout << key << ": " << value << std::endl;
}

Of course expressions can't be put in the for range loop, so instead a macro could be used to allow the expression:

#define FOREACH(var, cont) \
    for(auto && _p:cont) \
        if(bool _done = false) {} \
        else for(var = std::forward<decltype(_p)>(_p); !_done; _done = true)

So then std::tie can be used directly in the loop:

std::map<int, std::string> my_map;
int key;
std::string value;
FOREACH(std::tie(key, value), my_map)
{
    std::cout << key << ": " << value << std::endl;
}
Paul Fultz II
  • 17,682
  • 13
  • 62
  • 59
  • The problem with `tie` is that you're copying everything, which is bad not just for the overhead but also because you can't then modify the values (if that were some desired functionality). – Barry Apr 08 '16 at 21:39
  • @Barry you sure about that? The documentation says tie returns a tuple of non const l-value references, so why is it copying everything in this example? – Nicolas Holthaus Apr 09 '16 at 12:01
  • 2
    @Nicholas Yes. It creates a tuple of references, but the assignment is still a copy. – Barry Apr 09 '16 at 12:50
1

Just for the sake of providing yet another way to almost do what you want, I wrote this some time ago to avoid having .first and .second all over my code:

auto pair2params = [](auto&& f)
{
    return [f](auto&& p) {
        f(p.first, p.second);
    };
};

Now you can write something like (assuming a range-based for_each):

int main()
{
    auto values = map<int, string>{
        {0, "hello"},
        {1, "world!"}
    };

    for_each(values, pair2params([](int key, const string& value) {
        cout << key << ": " << value << "\n";
    });
}

Running example: http://ideone.com/Bs9Ctm

Bret Kuhns
  • 4,034
  • 5
  • 31
  • 43
1

I typically prefer a KISS approach for this:

template<typename KeyValuePair>
typename KeyValuePair::first_type& key(KeyValuePair& kvp)
{
    return kvp.first;
}

template<typename KeyValuePair>
const typename KeyValuePair::first_type& key(const KeyValuePair& kvp)
{
    return kvp.first;
}

template<typename KeyValuePair>
void key(const KeyValuePair&& kvp) = delete;

template<typename KeyValuePair>
typename KeyValuePair::second_type& value(KeyValuePair& kvp)
{
    return kvp.second;
}

template<typename KeyValuePair>
const typename KeyValuePair::second_type& value(const KeyValuePair& kvp)
{
    return kvp.second;
}

template<typename KeyValuePair>
void value(const KeyValuePair&& kvp) = delete;

with usage example like this:

for(auto& kvp : my_map) {
    std::cout << key(kvp) << " " << value(kvp) << "\n";
}
milleniumbug
  • 15,379
  • 3
  • 47
  • 71
1

In Apache Mesos, we use a macro called foreachpair that can be used like this:

foreachpair (const Key& key, const Value& value, elems) {
  /* ... */
}

You can of course replace Key and Value with auto, along with whatever qualifiers you'd like to use there. It also supports break and continue.

My latest implementation looks like this:

#define FOREACH_PREFIX   BOOST_PP_CAT(foreach_, __LINE__)

#define FOREACH_BODY     BOOST_PP_CAT(FOREACH_PREFIX, _body__)
#define FOREACH_BREAK    BOOST_PP_CAT(FOREACH_PREFIX, _break__)
#define FOREACH_CONTINUE BOOST_PP_CAT(FOREACH_PREFIX, _continue__)
#define FOREACH_ELEM     BOOST_PP_CAT(FOREACH_PREFIX, _elem__)
#define FOREACH_ONCE     BOOST_PP_CAT(FOREACH_PREFIX, _once__)

The macros above provide unique names to various components that are used in the foreachpair macro by including the __LINE__ number.

 1 #define foreachpair(KEY, VALUE, ELEMS)                                     \
 2   for (auto&& FOREACH_ELEM : ELEMS)                                        \
 3     if (false) FOREACH_BREAK: break; /* set up the break path */           \
 4     else if (bool FOREACH_CONTINUE = false) {} /* var decl */              \
 5     else if (true) goto FOREACH_BODY; /* skip the loop exit checks */      \
 6     else for (;;) /* determine whether we should break or continue. */     \
 7       if (!FOREACH_CONTINUE) goto FOREACH_BREAK; /* break */               \
 8       else if (true) break; /* continue */                                 \
 9       else                                                                 \
10         FOREACH_BODY:                                                      \
11         if (bool FOREACH_ONCE = false) {} /* var decl */                   \
12         else for (KEY = std::get<0>(                                       \
13                       std::forward<decltype(FOREACH_ELEM)>(FOREACH_ELEM)); \
14                   !FOREACH_ONCE; FOREACH_ONCE = true)                      \
15           for (VALUE = std::get<1>(                                        \
16                    std::forward<decltype(FOREACH_ELEM)>(FOREACH_ELEM));    \
17                !FOREACH_CONTINUE; FOREACH_CONTINUE = true)

I'll walk through this line by line.

  1. (Start of the mess).
  2. Range-based for-loop iterating over ELEMS.
  3. We set up the label FOREACH_BREAK. We jump to this label to break out of this loop.
  4. We set up the control flow flag FOREACH_CONTINUE. This is true if the current iteration exited normally, or via continue, and false if the current iteration exited via break.
  5. We always jump to the FOREACH_BODY label, below.
  6. This is where we intercept the control flow and inspect the FOREACH_CONTINUE flag to determine how we exited the current iteration.
  7. If FOREACH_CONTINUE is false, we know that we exited via break so we jump to FOREACH_BREAK.
  8. Otherwise, FOREACH_CONTINUE is true, and we break out of the for (;;) loop which takes us to the next iteration.
  9. (Half way through the mess).
  10. We always jump here from (5).
  11. Set up FOREACH_ONCE which is just used to execute the for loop that declares KEY exactly once.
  12. Declare KEY.
  13. Forward the element properly.
  14. Use FOREACH_ONCE to ensure that this loop is executed exactly once.
  15. Declare VALUE.
  16. Forward the element properly.
  17. Use FOREACH_CONTINUE to ensure that this loop is executed exactly once, and to indicate whether the loop was exited via break or not.

NOTE: The use of std::get allows support of std::tuple or std::array coming out of the sequence as well. e.g., std::vector<std::tuple<int, int>>

Ideone Demo

mpark
  • 7,574
  • 2
  • 16
  • 18