-1

Is there a reliable way to prevent external code from calling inner functions of a lib that was compiled from C code?

I would like to deliver a static library with an API header file. The library has different modules, consisting of .c and .h files. I would like to prevent the recepients from using functions declared in the inner .h files. Is this possible?

Thanks!

Ilans
  • 95
  • 6
  • What is the concrete use case? – Basile Starynkevitch Oct 17 '17 at 17:27
  • 1
    I must misunderstand your question, because I would advise you to use static functions to fix this issue, but everyone else is saying there is no way? – Burstful Oct 17 '17 at 17:39
  • @CoreyLakey Static functions also could not be called from other translation units within the same library. But we need to know the use case to know if that's a problem. – interjay Oct 17 '17 at 17:44
  • @interjay Oh ok I understand. In that case I guess you'd have to implement the static function in every module that wanted to use it to prevent usage. But I wonder if this is what OP asks? – Burstful Oct 17 '17 at 17:49
  • Please **edit your question** to improve it and motivate it (otherwise the question will soon get closed). Explain what library are you thinking of and what you already did to handle that issue... – Basile Starynkevitch Oct 17 '17 at 17:52
  • 1
    @interjay If someone is willing to reverse engineer the library then nothing can stop them from calling its functions. All you need is the address, the calling convention, and the parameters it expects. But I agree this question needs to include the use case. How determined would people be to call these functions? – Carey Gregory Oct 17 '17 at 17:52
  • Why do you want to prevent users to use internal functions? They just need to know that they should not do that. If they bypass that rule, this is not your business. – Basile Starynkevitch Oct 18 '17 at 06:51
  • Hi,m I updated teh question, but would still like to know - if I declare a fucntion static, can it still be called if I know the adddress? Thanks! – Ilans Oct 18 '17 at 06:55
  • What kind of library are you coding? With what license? Is it free software or proprietary? In what domain? What would happen *to you* if a user breaches his obligations of not using undocumented functions? And don't comment your question, but **edit your question** once again (or more) to improve it. – Basile Starynkevitch Oct 18 '17 at 06:56
  • @Ilans As I said, if someone knows the function address, the calling convention and the expected parameters, you can't stop them from calling the function. – Carey Gregory Oct 25 '17 at 04:10

1 Answers1

1

Is there a reliable way to prevent external code from calling inner functions of a lib ?

No, there cannot be (read about Rice's theorem; detecting statically such non-trivial properties is undecidable). The library or the code might use function pointers. A malicious user could play with function pointers and pointer arithmetic to call some private function (perhaps after having reverse-engineered your code), even if it is static.

On Linux you might play with visibility tricks.

Or you could organize your library as a single translation unit (a bit like sqlite is doing its amalgamation) and have all internal functions be static ...

In general, the library should have naming conventions about its internal functions (e.g. suffix all of them with _). This could be practically helpful (but not against malicious users).

Most importantly, a library should be well documented (with naming conventions being also documented), and a serious user will only use documented functions the way they are documented to be useful.

(so I don't think you should care about internal functions being called; you do need to document which public functions can be called, and how, and when...; a user calling anything else should expect undefined behavior, that is very bad things)

I would like to deliver a static library with an APIheader file, and would like to prevent the recepients from using the structs I define and the inner functions.

I am not sure that (at least on Linux) delivering a static library is wise. I would recommend delivering a shared library, read Drepper's How to Write Shared Libraries.

And you can't prevent the recipient (assuming a malicious, clever, and determined one) to use inner functions and internal struct-s. You should just discourage them in the documentation, and document well your public functions and data types.

I would like to prevent the recepients from using functions declared in the inner .h files. Is this possible?

No, that is impossible.

It looks like you seek a technical solution to a social issue. You need to trust your users (and they need to trust you), so you should document what functions can be used (and you could even add in your documentation some sentence saying that using directly any undocumented function yields undefined behavior). You can't do much more. Perhaps (in particular if you are selling your library as a proprietary software) you need a lawyer to write a good contract.

You might consider writing your own GCC plugin (or GCC MELT extension) to detect such calls. That could take you weeks of work and is not worth the trouble (and will remain imperfect).

I am not able to guess your motivations and use case (is it some life-critical software driving a nuclear reactor, a medical drug injector, an autonomous vehicule, a missile?). Please explain what would happen to you if some (malicious but clever) user would call an internal undocumented function. And what could happen to that user?

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • Rice's theorem? What's the relevance? – John Kugelman Oct 17 '17 at 17:35
  • Undecidability to statically detect most issues – Basile Starynkevitch Oct 17 '17 at 17:35
  • 1
    Ah, I see. I read the OP's question as "is there an equivalent to Java's `private` keyword in C libraries?" and didn't see the connection to Rice's theorem. I see now you're talking about having the library detect calls. Perhaps you want to discuss how there are/aren't other access control mechanisms? – John Kugelman Oct 17 '17 at 17:37
  • Thanks for the great answer! is it still relevant after the latest edit? does it mean that the recepient can always call an internal function if he wants to? – Ilans Oct 18 '17 at 06:59
  • Please **edit your question** to improve it. The more you comment it (or my answer) the more you are confusing people (me included). In particular, explain what happens *to you* if a user calls some undocumented function? Are you against malicious users, or all of them? Then, why do you code a library? And explain what kind of library you are coding? Is it life threatening (e.g. driving a missile or a nuclear reactor or a drug injection device)? – Basile Starynkevitch Oct 18 '17 at 07:01
  • This answer is pretty baffling. OP (and I) just wants the equivalent to GCC's `visibility("hidden")`, but for static libs instead of shared libs. It's not nearly as insane as people seem to be making out here – Michael Mrozek Feb 26 '20 at 16:49