8

I would like to switch my application to LARGEADDRESSAWARE. One of issues to watch for is pointer arithmetic, as pointer difference can no longer be represented as signed 32b.

Is there some way how to find automatically all instances of pointer subtraction in a large C++ project?

If not, is there some "least effort" manual or semi-automatic method how to achieve this?

phuclv
  • 37,963
  • 15
  • 156
  • 475
Suma
  • 33,181
  • 16
  • 123
  • 191
  • Do you mean you are building an application for a system with a 64 bit address space yet the compiler is only using 32 bit pointers? – Martin York Jun 16 '10 at 10:05
  • 3
    "LARGEADDRESSAWARE" is a Windows32 flag in an executable header, which tells the OS that the executable can handle 32 bit addresses. Without it, you'll get 31 bit addresses (i.e. the top bit is always 0), and then the second half of the address space is reserved for the OS. – MSalters Jun 16 '10 at 10:47

5 Answers5

3

PC-Lint can find this kind of problem.

Look at http://gimpel-online.com/MsgRef.html, error code 947:

Subtract operator applied to pointers -- An expression of the form p - q was found where both p and q are pointers. This is of special importance in cases where the maximum pointer can overflow the type that holds pointer differences. For example, suppose that the maximum pointer is 3 Gigabytes -1, and that pointer differences are represented by a long, where the maximum long is 2 Gigabytes -1. Note that both of these quantities fit within a 32 bit word. Then subtracting a small pointer from a very large pointer will produce an apparent negative value in the long representing the pointer difference. Conversely, subtracting a very large pointer from a small pointer can produce a positive quantity.

Patrick
  • 23,217
  • 12
  • 67
  • 130
1

Compile the code with a 64 bit compiler and Wp64 turned on.

Because pointers are 64bit wide, but int, long, DWORD etc. stay 32 bit wide, you get warnings for shorting a ptrdiff_t to a int32_t

Christopher
  • 8,912
  • 3
  • 33
  • 38
  • 2
    This answer is not relevant, because it catches problems that may arise if you use the wrong type to store pointers, but it doesn't warn you if you are making "dangerous" pointer subtractions/comparisons. – Matteo Italia Jun 16 '10 at 12:31
  • Nice idea. It will not find cases where the pointer subtraction is assigned to ptrdiff_t, still those cases are a problem as well, as ptrdiff_t (int on Win32) is not enough to represent pointer difference - but those can be searched for by text searching for ptrdiff_t. The real problem is there are too many such warnings in the code, solving each of them seems quite a lot of work and vast majority of them is not related LARGEADDRESSAWARE at all (we have no intention to port the code to Win64). – Suma Jun 16 '10 at 12:34
  • Are you claiming that there's a compilation mode where ptrdiff_t isn't large enough to hold the difference between two pointers? That sounds… well, plain broken, given that that's the standard _definition_ of that type. (I would expect it to be of a different size in the different compilation modes.) – Donal Fellows Jun 23 '10 at 08:06
0

This is only a problem if you have 2 pointers that are more than 2000 million bytes (2GB) apart. This means that you:

  • either have very large arrays (> 2GB)
  • or you are subtracting pointers that point to totally different structures

So look for these special cases.

I think that in most cases this is not a problem.

Patrick
  • 23,217
  • 12
  • 67
  • 130
  • 1
    "or you are subtracting pointers that point to totally different structures" - this is something which can happen easily. We are sometimes using pointers to objects as secondary keys when sorting them to make sure sorting order is stable. As for "look for these special cases" - how can I look for them other then review every single piece of the code? – Suma Jun 16 '10 at 09:56
  • Comparing the pointers is not a problem, subtracting them is, and for the sort, comparing the pointers should be sufficient. Also, it's not a good idea to use pointers as the final comparison. I used to do this in the past as well, but this could give problems (problems can become hard to reproduce since the pointers can be different at every run of your application; trust me, I had this problem myself several times). Try to find another characteristic (e.g. a line number, a database key, ...) to get a stable sort. – Patrick Jun 16 '10 at 10:13
  • > "Comparing the pointers is not a problem, subtracting them is," Unfortunately, when implementing comparison function for qsort, returning difference seems to be quite natural, and I have seen it several times already. Nice comment, unfortunately the main questions is still not answered - how to find the cases? The codebase is huge, it is impossible to find all cases just by remembering them. Some systematic approach is needed, either automated, or manual (systematic code review). The solution can be compile time, a dedicated tool, or runtime, with some instrumentation. – Suma Jun 16 '10 at 11:45
  • Fair point: it's possible that you end up with `a – MSalters Jun 16 '10 at 13:14
  • 1
    Notice that if you subtract two pointers, you will get the 'offset' in number of elements in between the pointers. This works fine for pointers pointing to elements in an array, but gives strange results if the two pointers are unrelated. The result may not be integer. Suppose your pointers are pointing to a class of 1000 bytes. If ptr1 points to 50000, and ptr2 points to 60100, then a subtract will probably return 10, but that's incorrect. See https://www.securecoding.cert.org/confluence/display/seccode/ARR36-C.+Do+not+subtract+or+compare+two+pointers+that+do+not+refer+to+the+same+array. – Patrick Jun 16 '10 at 14:16
  • You can pretty much strike the first point. The OP wants to enable LARGEADDRESSAWARE, which means that it's not enabled yet... Which means that the process' address space is restricted to 2GB. – jalf Jun 23 '10 at 08:25
  • Be aware that comparing pointers could very much be a problem, at least as far as the Standard is concerned. The comparison (p < q) for pointers p and q is not specified for the general case (§5.9/2), so one must use std::less. – boycy Jun 19 '14 at 13:30
0

As our code already compiles with GCC, I think perhaps the fastest way might be:

  • build a GCC
  • create a custom modification of GCC so that it prints warning (or error) whenever pointer subtraction is detected
  • build the project and gather all warnings about pointer subtraction

Here is the outline of changes which need to be done to GCC for this:

Add your warnings into:

  • c-typeck.c (pointer_diff function)
  • cp/typeck.c (pointer_diff function).

Besides of directly detecting pointer subtraction, another thing to do can be to detect cases where you first convert pointers to integral types and then subtract them. This may be more difficult depending on how is your code structured, in out case regexp search for (.intptr_t).-.*-(.*intptr_t) has worked quite well.

Suma
  • 33,181
  • 16
  • 123
  • 191
0

For whatever it's worth, I went through the Microsoft compiler warning docs for VS2017 and searched for all warnings referring to "signed", "trunc" (truncation) and "conv" (conversion), which were higher than warning level 1. Then I explicitly enabled those warnings for all projects in our solution via a propsheet. To enable specific warnings, go to "C/C++ / Command Line / Additional Options" and add them in the format /wL####, where L is the warning level you want to assign them to and #### is the warning number.

So what I came up with is this list:

/w14365 /w14018 /w14146 /w14245 /w14092 /w14287 /w14308 /w14388 /w14389 /w14757 /w14807 /w14302 /w14305 /w14306 /w14307 /w14308 /w14309 /w14310 /w14311 /w14312 /w14051 /w14055 /w14152 /w14239 /w14223 /w14242 /w14243 /w14244 /w14254 /w14267 /w14333 /w14334 /w14367 /w14686 /w14826

Note that I used /w1 because our global warning level is already down at 1 (don't judge me, it's legacy). So some of these warnings are already enabled when you have set the default warning level of 3 or higher.

This resulted in more than 88000 warnings, most of which were about using int instead of size_t in code using the STL and about conversions regarding Windows API types like handles, WPARAMs and UINT_PTRs and such. I only found a few warnings related to actual pointer arithmetics in a 3rd party library, but those looked alright in context.

Anyway, I thought this list of related warnings might help someone.

Also, use the tools described in this answer: https://stackoverflow.com/a/22745579/9635694

Another option is to run the builtin code analysis according to the CppCoreGuideLines. Go to "Main menu / Analyze / Configure Code Analysis / For Solution" and select "C++ Core Check Raw Pointer Rules" for all the projects you want to analyze. Then "Main menu / Analyze / Run Code Analysis / For Solution". Beware: Modifies your projects, takes a long time to build and might generate a lot of warnings. You might want to concentrate on C26481 "Don't use pointer arithmetics" and perhaps C26485 "No array-to-pointer decay".

Simpleton
  • 632
  • 5
  • 22