Detect integer overflow

Question

I am working with a large C library where some array indices are computed using int. I need to find a way to trap integer overflows at runtime in such way as to narrow to problematic line of code. Libc manual states:

FPE_INTOVF_TRAP Integer overflow (impossible in a C program unless you enable overflow trapping in a hardware-specific fashion).

however gcc option -ffpe-trap suggests that those only apply to FP numbers?
So how I do enable integer overflow trap? My system is Xeon/Core2, gcc-4.x, Linux 2.6

I have looked through similar questions but they all boil to modifying the code. I need to know however which code is problematic in the first place.
If Xeons can't trap overflows, which processors can? I have access to non-emt64 machines as well.

I have found a tool designed for llvm meanwhile: http://embed.cs.utah.edu/ioc/ There doesn't seem to be however an equivalent for gcc/icc?

As far as I know, no x86 processor supports trapping on integer overflow. Many RISC cpus do (power and sparc both do at least), as well as older mini/mainframe CPUs (such as VAX) — Chris Dodd, Apr 17 '12 at 23:18

score 3 · Accepted Answer · answered Apr 17 '12 at 23:59

3

Ok, I may have to answer my own question.

I found gcc has -ftrapv option, a quick test does confirm that at least on my system overflow is trapped. I will post more detailed info as I learn more since it seems very useful tool.

answered Apr 17 '12 at 23:59

Anycorn

50,217
42
167
261

score 2 · Answer 2 · answered Apr 17 '12 at 23:16

Unsigned integer arithmetic does not overflow, of course.

With signed integer arithmetic, overflow leads to undefined behaviour; anything could happen. And optimizers are getting aggressive about optimizing stuff that overflows. So, your best bet is to avoid the overflow, rather than trapping it when it happens. Consider using the CERT 'Secure Integer Library' (the URL referenced there seems to have gone AWOL/404; I'm not sure what's happened yet) or Google's 'Safe Integer Operation' library.

If you must trap overflow, you are going to need to specify which platform you are interested in (O/S including version, compiler including version), because the answer will be very platform specific.

I need to know *which* operations to replace (with size_t). I specified system/compiler in the question. — Anycorn, Apr 17 '12 at 23:21

Michael Dorgan · Answer 3 · 2012-04-17T23:36:54.440

2

Do you know exactly which line the overflow is occuring on? If so, you might be able to look at the assembler's Carry flag if the operation in question caused an overflow. This is the flag that the CPU uses to do large number calculation and, while not available at the C level, might help you to debug the problem - or at least give you a chance to do something.

BTW, found this link for gcc (-ftrapv) that talks about an integer trap. Might be what you are looking for.

edited Apr 17 '12 at 23:36

answered Apr 17 '12 at 23:26

Michael Dorgan

12,453
3
31
61

2

Any chance of dropping a write to memory breakpoint on the counter to help you out? – Michael Dorgan Apr 17 '12 at 23:33
I don't think so. There are multiple places where `int` are used in place of `size_t`, manual debugging is very hard. I might have found a solution though. – Anycorn Apr 17 '12 at 23:51

score 0 · Answer 4 · answered Apr 17 '12 at 23:47

You can use inline assembler in gcc to use an instruction that might generate an overflow and then test the overflow flag to see if it actually does:

int addo(int a, int b)
{
    asm goto("add %0,%1; jo %l[overflow]" : : "r"(a), "r"(b) : "cc" : overflow);
    return a+b;
overflow:
    return 0;
}

In this case, it tries to add a and b, and if it does, it goes to the overflow label. If there's no overflow, it continues, doing the add again and returning it.

This runs into the GCC limitation that an inline asm block cannot both output a value and maybe branch -- if it weren't for that, you wouldn't need a second add to actually get the result.

Detect integer overflow

4 Answers4