2

Consider this code:

struct A {
    int64 member;
    int32 member2;
    virtual void f();
};

struct B {
    int16 member3;

    virtual void b();
};

struct C : A, B {
    virtual void b() override;
};

I'm interested in finding the offset of B in C. Previously with other structs with no virtual inheritance and only one base class offsetof of the first member seemed to work. I have decompiled some code (in IDA) and the base classes are nicely highlighted (hex) here:

screenshot

In a function those exact baseclass offsets are used to cast void*'s to derived classes by adding the offset to the void* (by casting to a char*). The structs A, B and C are similar to the one in the compiled code, which include classes with virtual functions and multiple base classes.

My question is how did they do that, and how can I do that? I've tried something like i32 offset = (i64)((B*)((C*)NULL)); but no luck.

ivaigult
  • 6,198
  • 5
  • 38
  • 66
  • 3
    "virtual base classes" isn't occuring here. you have bases with virtual members, not virtual bases. If you had virtual bases the answer would be "there is no unique answer" – Caleth Nov 25 '21 at 16:54
  • Who are "they", and did they have access to the debug symbols for this program? – Caleth Nov 25 '21 at 16:57
  • your right, meant virtual member functions. Its a game engine, and yes since I have the PDB if thats what you mean. The code to find the offsets would have been compile time generated since I can see the raw offsets in the data segment of the executable – Lucas Smith Nov 25 '21 at 17:09

1 Answers1

2

I tried the following, and it worked:

(char*)(B*)(C*)0x100 - (char*)(C*)0x100

It casts C* to B*; this is supposed to do the work. All the rest is support. I used an arbitrary number 0x100; it seems to work with all numbers except 0.

Why it doesn't work for 0: it sees a null-pointer of type C*; to convert it to a null-pointer of type B*, it should still be null. A special case.

Of course, this uses undefined behavior. It seems to work in Visual Studio in my short test program; no guarantee it will work anywhere else.

anatolyg
  • 26,506
  • 9
  • 60
  • 134
  • Perfect, works! Thanks a lot, frustrating that it was just because it was null, but I see since the compiler would try to optimize that since it sees its a constant of null – Lucas Smith Nov 25 '21 at 18:59
  • It's not because of the optimization. It'd horrific if casting a null pointer would give us a non-null pointer. Then it'd be impossible to handle invalid pointers after a cast. – ivaigult Nov 25 '21 at 19:08
  • Trying this with Visual Studio 2019 gets me error `Access of field '$cast' not supported` – Jonathan Aug 09 '23 at 14:32
  • @Jonathan You might want to ask a question about that. Use the "Ask Question" link. Might be interesting for some people, including myself. – anatolyg Aug 12 '23 at 18:03
  • @anatolyg https://stackoverflow.com/questions/76892949 – Jonathan Aug 13 '23 at 10:33