How do I see inside opaque data types?

Question

I know this kind of goes against the whole point of opaque data types, but for the purpose of reverse engineering, how do people go about looking into opaque data types?

C doesn't have any introspection - once the code is compiled, all you can do afterwards is very painful reverse engineering. Before the compilation is relatively easy: find the definition in the relevant source. — Amadan, Oct 09 '18 at 05:39
You'd poke around the headers on the system to see whether you can find a definition for `fpos_t`. Failing that, you have to hope you are on an open source system and look at the open source code for the C library on your platform. There's no easy, reliable way to look at the definition of a truly opaque type from the object code. (Some opaque types are not as opaque as others, though. You may find more information about a `FILE` than you'd expect for an opaque-ish type.) — Jonathan Leffler, Oct 09 '18 at 05:46
@JonathanLeffler I don't entirely understand, though; for `fpos_t` to work, the full definition has to be somewhere, otherwise it's just a useless moniker and the program will have nothing to run on. Honestly, I kind of hate when people do this. I understand the use in hiding unnecessary details or restricting access so people don't muck anything up, but I feel like we're heading in a direction where we force people to only operate on high levels. — thepufferfish, Oct 09 '18 at 06:13
In the specific case of `fpos_t`, the functions that use it are [`fgetpos()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/fgetpos.html) and [`fsetpos()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/fsetpos.html). The definition can't be wholly opaque because of the interface; the calling code has to be able to write `fpos_t pos;` and pass `&pos` to the functions. So, the type isn't opaque, even if it isn't documented what it is. That means you can find it in the system headers — the information has to be available to the compiler somehow (but it could be magic). — Jonathan Leffler, Oct 09 '18 at 06:19
See also POSIX on [File Streams](http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_05). It says: _Each wide-oriented stream has an associated `mbstate_t` object that stores the current parse state of the stream. A successful call to `fgetpos()` shall store a representation of the value of this `mbstate_t` object as part of the value of the `fpos_t` object. A later successful call to `fsetpos()` using the same stored `fpos_t` value shall restore the value of the associated `mbstate_t` object as well as the position within the controlled stream._ — Jonathan Leffler, Oct 09 '18 at 06:21
@JonathanLeffler I know I'm probably just setting myself up for even more frustration and info-craze by asking this but what would be a truly opaque type? — thepufferfish, Oct 09 '18 at 06:25
That said, I don't see much evidence of the `mbstate_t` in the macOS variation; if you track hard enough, it ends up as `__int64_t`. — Jonathan Leffler, Oct 09 '18 at 06:25
If you need to know the insides of an opaque type, then the problem is not in the opaqueness, it is in the bad interface design. If you do not need to know the inside, then opaqueness serves all the benefits it is designed for. If you then make assumptions on the inside of the opaque type (even and especially after finding out about them) then you are putting yourself into danger of making your code vulnerable against otherwise harmless changes. Yes, restricting users to the designed higher abstraction level is an advantage, not a weakness. — Yunnosch, Oct 09 '18 at 06:26
A truly opaque type would be `typedef struct Opaque Opaque;` in the public header, and the functions would only take pointers to the type, and return pointers. So: `Opaque *make_opaque(const char *str);` and `void destroy_opaque(Opaque *opaque);` and `int size_opaque(const Opaque *opaque);`, etc. In the public header, there's no information about the internals of the structure; that's completely hidden in the implementation file(s). Without the source code, you can tell nothing about the opaque type I just invented. — Jonathan Leffler, Oct 09 '18 at 06:28
Trying to understand everything on finest detail level (binary machine code probably or are you happy with the bastraction level of C?) is an understandable itch and even good for learning to some extend. But it will restrict you as the human from being an efficient productive programmer. — Yunnosch, Oct 09 '18 at 06:29
If you're curious, you can see an opaque type `AoM_Copy` (and a non-opaque type `AoM_Block`) in my [SOQ](https://github.com/jleffler/soq) (Stack Overflow Questions) repository on GitHub as files `aomcopy.c` and `aomcopy.h` in the [src/libsoq](https://github.com/jleffler/soq/tree/master/src/libsoq) sub-directory. A user of the header has no idea what's inside `AoM_Copy`; they just know it does a particular job — it implements an array of memory blocks of potentially different sizes by copying the memory passed to the functions. There are other variants there too. — Jonathan Leffler, Oct 09 '18 at 06:36
@Yunnosch Let's just agree to disagree. In retrospect, I feel like mentioning my opinion was the infotech equivalent of accidentally mentioning my political views, and this isn't the kind of forum for that. I'm sorry for that faux pas. — thepufferfish, Oct 09 '18 at 06:41
I don't know why it's being down-voted or who's down-voting (not me). Someone thinks it doesn't show any research effort, or is unclear, or doesn't think it is helpful — those are the reasons given on the down-vote button for down-voting. It isn't clear how many questions you looked at with a search such as '`[c] opaque is:q`'; I get 325 questions when I do that search on SO (you might not specify the `is:q` term, and you'd then see answers as well as questions). Pointing at a few of those and explaining why they didn't help would show some research effort. — Jonathan Leffler, Oct 09 '18 at 06:56
@JonathanLeffler Thank you for the advice. These are some of what I was looking through: [link](https://stackoverflow.com/questions/2301454/what-defines-an-opaque-type-in-c-and-when-are-they-necessary-and-or-useful) [link](https://stackoverflow.com/questions/3854113/what-is-an-opaque-value-in-c) [link](https://stackoverflow.com/questions/35245615/c-typedef-coding-style-for-opaque-structs) — thepufferfish, Oct 09 '18 at 07:11

Lundin · Accepted Answer · 2018-10-09T06:51:20.453

Unless you already have the source, there is no feasible way to "hack inside" an opaque type without having a clue of the underlying representation on a specific system. Hacking it would involve following each access to the opaque pointer in run-time and see where it goes, then start guessing from there.

Instead, you could Google for example the glibc source at Github. In stdio.h there is a conditional typedef which points either at __fpos_t or __fpos64_t in internal headers:

__fpos_t.h

typedef struct _G_fpos_t
{
  __off_t __pos;
  __mbstate_t __state;
} __fpos_t;

__fpos64_t.h

typedef struct _G_fpos64_t
{
  __off64_t __pos;
  __mbstate_t __state;
} __fpos64_t;

Not very exciting, but you can continue to trace those various types at Github from there and see what they boil down to in the end. Integers and enums, most likely.

score 1 · Answer 2 · edited Aug 10 '19 at 23:16

1

In VC++ it is defined as

 typedef __int64 fpos_t;

which in turn is

typedef long long fpos_t;

If you want to know how it looks in assembly code, then just write a sample program, run it in debug mode, and, while debugging, open the Disassembly Window to watch the code behind.

edited Aug 10 '19 at 23:16

S.S. Anne

15,171
8
38
76

answered Oct 09 '18 at 05:59

Sisir

4,584
4
26
37

That's it? That's a little underwhelming. What is the point of trying to hide an alias? – thepufferfish Oct 09 '18 at 06:18

How do I see inside opaque data types?

2 Answers2