13

Let's say I want to iterate over all integers in a for loop. For the sake of discussion, assume I am calling some unknown function f(unsigned x) for each integer:

for (unsigned i = 0; i < UINT_MAX; i++) {
     f(i);
}

Of course, the above fails to iterate over all integers, because it misses one: UINT_MAX. Changing the condition to i <= UINT_MAX just results in an infinite loop, because that's a tautology.

You can do it with a do-while loop, but you lose all the niceties of the for syntax.

Can I have my cake (for loops) and eat it too (iterate over all integers)?

chqrlie
  • 131,814
  • 10
  • 121
  • 189
BeeOnRope
  • 60,350
  • 16
  • 207
  • 386

6 Answers6

10

You'll have to perform the test at the end of the loop body, much like a do-while:

for (unsigned int i = 0; /* nothing */; i++) {
    ...
    if (i == UINT_MAX) {
        break;
    }
}

For a test in the standard for loop test position to work, you would need to track the current iteration in a way that can distinguish between UINT_MAX+2 states: one for each time you enter the loop body, and one for the one time you don't. A single unsigned int can't handle that, so you'd need at least one auxiliary variable or a bigger loop counter.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • 2
    Yeah blarg, messier than `do-while` in some ways but at least scopes `i` to the body of the loop. – BeeOnRope Nov 04 '16 at 23:22
  • 1
    In comparison to Barmar's solution before, [this ends up](https://godbolt.org/g/eVPVeF) peeling out one iteration and then doing the loop with a trip count of `2^32-1`, with 32-bit counters. I need to check what would happen if the loop of the body was bigger, since then peeling would be relatively worse. – BeeOnRope Nov 04 '16 at 23:55
  • @BeeOnRope: What does the do-while do? – user2357112 Nov 04 '16 at 23:57
  • It ends up using assembly similar to Barmar's solution. [See here](https://godbolt.org/g/vNe6zP) - uses a 64-bit counter to get around the "one too many" problem. There I've expanded the "work" in the loop by calling `f(i)` four times, which shows why the loop stripping solution isn't great: it really expands the code size. I don't know what the limit is at which `gcc` will stop stripping the first iteration like that. I haven't checked other compilers yet! – BeeOnRope Nov 05 '16 at 00:05
  • 1
    FWIW, I can't off the top of my head think of a better assembly-level solution than the 64-bit counters. It wouldn't apply for 64-bit counters though, but that would take an unreasonably number of years to actually iterate through though :) – BeeOnRope Nov 05 '16 at 00:07
  • Here's a comparison showing `gcc`, `clang` and `icc`. Clang seems to produce the best code, and contradicts my comment that there isn't a better solution. It simply doesn't have the check up front in the loop and uses about the best loop possible (including allowing macro-fusion) using a check against zero. `icc` still struggles a bit - it uses an initial jump into mid-loop to skip the first increment, and uses an explicit `cmp` against `UINT_MAX`. – BeeOnRope Nov 07 '16 at 18:49
  • @BeeOnRope: Did you forget your link? – user2357112 Nov 07 '16 at 18:50
6

You can do it with a do-while loop, but you lose all the niceties of the for syntax.

It is still doable with do-while loop by using an anonymous block scope:

{
    unsigned i = 0;
    do { f(i); } while (++i != 0);
}

While this construct may not be most idiomatic, it is an obvious candidate for clear assembly code. For example, gcc -O compiles it as:

.L2:
        mov     edi, ebx   ; ebx starts with zero
        call    f
        add     rbx, 1
        cmp     rbx, rbp   ; rbp is set with 4294967296
        jne     .L2
Grzegorz Szpetkowski
  • 36,988
  • 6
  • 90
  • 137
  • True, at the cost of an an extra couple of lines and another level of indent. – BeeOnRope Nov 07 '16 at 18:37
  • 2
    Yeah, about the assembly it compiles well, about in line with the other best-compiled solutions which seem to use 64-bit registers to do the counting. `gcc` seems particularly poor at it in general though compared to `clang`. The simplest loop is just [what `clang` does](https://godbolt.org/g/twevkT) which in practice can shave off at least a cycle. – BeeOnRope Nov 07 '16 at 19:43
  • I have accepted this answer as I think it is among the clearest ways to accomplish this (the solution from user2357112 is also clear) and still effectively scopes `i` to the body of the loop (unfortunately at the cost of an extra block just for scoping). The tie-breaker is that compilers, today, seem to be better and compiling this to reasonable code than the `for` + explicit-break-at-end approach. – BeeOnRope Jun 18 '17 at 18:11
  • the gcc output is less optimized because you don't actually need to set rbp to 4294967296, the zero flag after increment can be used directly https://godbolt.org/g/BFDHSd – phuclv Apr 07 '18 at 13:07
3

You could use another variable to detect when you've looped around.

for (unsigned int i = 0, repeat = 0; !(i == 0 && repeat); i++, repeat = 1) {
    ...
}
Barmar
  • 741,623
  • 53
  • 500
  • 612
  • 3
    Yeah. I could. I _seems_ pretty terrible though - an additional variable and check executed every iteration. Kind of amazingly to me, `gcc` cuts through it all and compiles it [pretty much optimally](https://godbolt.org/g/FH0zRe). The whole `repeat` business disappears and the loop just uses 64-bit counters and compares explicitly against `2^32`. – BeeOnRope Nov 04 '16 at 23:51
  • 1
    Wow, color me amazed as well. I'm actually not proud of this code, it's not obvious what it does. I much prefer @user2357112's solution. – Barmar Nov 04 '16 at 23:53
  • In C-land it looks nicer better, but the assembly seems to slightly favor yours so far (at least on `gcc`. See my comments on user23... answer. – BeeOnRope Nov 04 '16 at 23:55
  • Normally I would say not to worry about the generated code. But if you're looping this many times, every little bit counts. A microsecond per iteration adds over an hour to the total running time. – Barmar Nov 04 '16 at 23:58
  • @BeeOnRope I think you could eliminate the additional check with a variation of this: `for (unsigned int i = 0, end = 1; i != end; i++, end = 0)` – Kelly Bundy Jul 25 '23 at 00:11
2

The classic way to implement your iteration efficiently, with a single test, is a do / while loop:

unsigned i = 0;
do { f(i); } while (i++ != UINT_MAX);

If you insist on using a for loop:

for (unsigned i = 0;; i++) {
    f(i);
    if (i == UINT_MAX)
        break;
}

Here is another variant with 2 variables where all the logic is inside the for expressions:

for (unsigned int i = 0, not_done = 1; not_done; not_done = (i++ - UINT_MAX)) {
    f(i);
}

It might produce slower code because of the extra variable, but as BeeOnRope commented, clang and icc compile it to very efficient code.

chqrlie
  • 131,814
  • 10
  • 121
  • 189
  • 1
    That version is compiled [pretty much optimally](https://godbolt.org/g/oTjTX7) by `clang` and `icc`. On the other hand, `gcc` still struggles. – BeeOnRope Nov 07 '16 at 18:44
0

An easy solution would be,

unsigned i;
for (i=0; i<UINT_MAX; i++) {
  f(i);
}
f(i);  // i will be UINT_MAX at this time.
Rishikesh Raje
  • 8,556
  • 2
  • 16
  • 31
  • still not scope the `i` variable inside. IMHO the `do while` solution is still best in this case – phuclv Nov 05 '16 at 02:56
  • 3
    Also, this looks OK when `f(i)` is a simple function call, but in the general case it may be serval lines of code that would need to be duplicated, running afoul of DRY and making it worse than solutions which don't repeat the loop body. – BeeOnRope Nov 05 '16 at 04:22
0

Use a larger integer type:

#include <limits.h>
#include <stdio.h>

int main() {
    for (unsigned long i = 0; i <= UINT_MAX; i++) {
        f(i);
    }
}

This version uses stdint for more consistency

#include <stdio.h>
#include <stdint.h>

int main() {
    for (uint_fast64_t i = 0; i <= UINT32_MAX; ++i) {
        f(i);
    }
}
KingRadical
  • 1,282
  • 11
  • 8
  • This is not always practical because the larger integer type might be slower and takes more memory. Moreover your loops still doesn't go to `UINT_MAX`. It should be `i < UINT_MAX + 1UL` – phuclv Nov 05 '16 at 02:58
  • The primary concern raised in the question was about maintaining the convenience of the `for` loop, not efficiency or performance. I've fixed the comparison, though. – KingRadical Nov 05 '16 at 07:48
  • your "fixed" version doesn't work because the comparison is still done in `unsigned long` type, making it always true – phuclv Nov 05 '16 at 11:50
  • 4
    `unsigned long` might not be a larger type: for example it is the same size as `unsigned int` on Windows 64-bit. You will have an infinite loop on such an architecture. – chqrlie Nov 05 '16 at 12:59
  • 2
    FWIW, I am concerned about both convenience and performance. Any time you are looping 4 billion times performance is probably going to matter a bit :) – BeeOnRope Nov 07 '16 at 18:26
  • @chqrlie, I compiled that on 64-bit linux and it worked as expected. But, you are correct that that behavior is not architecture-independent. I'll edit this and add an alternate version that uses fixed-size stdint types instead. – KingRadical Nov 30 '16 at 22:49
  • @BeeOnRope: No doubt, but your question was specifically about convenience. Sometimes, writing high performance code involves *not* doing it the most convenient way. – KingRadical Nov 30 '16 at 22:49
  • @KingRadical - well I am telling you exactly that I'm interested in both high performance and convenience. Those aren't opposites - if my limit was other than `UINT_MAX`, the "normal" way of iterating over a range of integers (a `for` loop) would both be the convenient and likely best performing way (granted, there are some pitfalls there). For the `UINT_MAX` case I'm ideally looking for, again, a mix of convenience and performance. If I have to trade off one against the other, then all the answers which aren't "dominated" are interesting. – BeeOnRope Nov 30 '16 at 22:52
  • Unless all you're doing is counting, it's unlikely that the loop construct is going to be even close to the biggest source of any potential performance problems. That should be the last thing on your list of things to profile and optimize. – KingRadical Nov 30 '16 at 23:58
  • For all we know, using a loop counter may not even be the most efficient or performant way to accomplish whatever your goal is here. – KingRadical Dec 01 '16 at 00:00