2

I am working on a software program that has to be deployed on private cloud server of a client, who has root access. I can communicate with the software through a secure port.

I want to prevent client from reverse engineering my program, or at least make it "hard enough". Below is my approach:

  1. Write code in Go and compile the software into binary code (may be with obfuscation)
  2. Make sure that program can only be initiated with secret key that can be sent through the secure port. The secret key can be changing depending on time.
  3. Every time I need to start/stop the program, I can send commands with the secret keys through the secured port.

I think this approach can prevent a root user from either:

  1. Using a debugger to reverse engineer my code

  2. Running the program repeatedly to check outputs

My question is: What are the weak spots of this design? How can a root user attack it?

Alec
  • 31,829
  • 7
  • 67
  • 114
AdamNYC
  • 19,887
  • 29
  • 98
  • 154
  • Speaking of "Secret key" does that imply a Symmetric-key algorithm? Would it be possible for the client to sniff the connection and dump the secret key? Does the key just set a flag meaning "it's ok to execute this command!" or does it implies that some parts of the code are encrypted? – Neitsa Mar 05 '15 at 11:09
  • 1) Yes. 2) I think it is hard. The key is changing so it wont help much. 3) Just a flag. – AdamNYC Mar 05 '15 at 11:22

1 Answers1

5

I want to prevent client from reverse engineering my program,

You can't prevent this fully when software runs on hardware you don't own. To run the software, the CPU must see all instructions of the program, and they will be stored in the computer memory.

https://softwareengineering.stackexchange.com/questions/46434/how-can-software-be-protected-from-piracy

Code is data. When the code is runnable, a copy of that data is un-protected code. Unprotected code can be copied.

Peppering the code with anti-piracy checks makes it slightly harder, but hackers will just use a debugger and remove them. Inserting no-ops instead of calls to "check_license" is pretty easy.

(answers in https://softwareengineering.stackexchange.com/questions/46434 may be useful for you)

Hardware owner controls the OS and memory, he can dump everything.

or at least make it "hard enough".

You can only make it a bit harder.

Write code in Go and compile the software into binary code (may be with obfuscation)

IDA will decompile any machine code. Using native machine code is a bit stronger than bytecode (java or .NET or dex)

Make sure that program can only be initiate with secret key that can be sent through the secure port. The secret key can be changing depending on time.

If the copy of the same secret key (keys) is in code or memory of the program, user may dump it and simulate your server. If part of your code, or part of data needed for code to run is stored encrypted, and deciphered with such external key, user may either eavesdrop the key (after it will be decoded from SSL but before it will be used to decrypt secret part of code), or dump decrypted code/data from the memory (It is very easy to see new executable code created in memory even with default preinstalled tools like strace in Linux, just search for all mmaps with PROT_EXEC flags)

Every time I need to start/stop the program, I can send commands with the secret keys through the secured port.

This is just a variation of online license/antipiracy check ("phone home")

I think this approach can prevent a root user to: use a debugger to reverse engineer my code, or

No, he can start debugger at any time; but you can make it a bit harder to use interactive debugger, if the program communicates with your server often (every 5 seconds). But if it communicates so often it is better to move some part of computations to your server; this part will be protected.

And he still can use non-interactive debuggers, tracing tools and memory dumping. Also he can run program in virtual machine, wait until online check is done (using tcpdump and netstat to monitor network traffic), then do live snapshot of the VM (there are several variants to enable "live migration" of VM; only short pause may be recorded by your program if it has external timing), continue to run the first copy online, and take snapshot for offline debugging (with all keys and decrypted code in it).

run the program repeatedly to check outputs

Until he cracks the communications...

Community
  • 1
  • 1
osgx
  • 90,338
  • 53
  • 357
  • 513
  • 1
    Thank you for a perfect answer! I found that PrivateCore offered vCage for this exact use case. Unfortunately they were bought by Facebook last year. :( – AdamNYC Mar 05 '15 at 19:31
  • Hmm, looks like vCage uses TPM to steal hardware from his owner ("The vCage host is packaged as a stateless live image Linux KVM on a RAM disk"). They ask end-user to boot into vCage host OS, then ask hardware TPM to check that vCage was not modified; then run protected software as one VM and possibly user's own OS as other VM. I think it will be hard to encrypt all memory (or it makes software or its access/bandwidth to memory 2-3 times slower/less). There is marketing pdf from privatecore https://forum.stanford.edu/events/2014/2014slides/plenary/Oded%20Stanford%20Annual%20Forum%202014.pdf – osgx Mar 05 '15 at 19:41
  • I don't think they sell the product anymore (can't find contact on their page, and blog is dormant since they got acquired). Isn't stealing hardware the whole purpose here :D – AdamNYC Mar 05 '15 at 19:46
  • http://security.stackexchange.com/questions/53165: "*Microsoft's 3rd immutable law: "If a bad guy has unrestricted physical access to your computer, it's not your computer anymore."*"; and vCage: "*In the case of vCage you basically only need to trust Intel and Private Core. Briefly, vCage provide a L3 resident hypervisor validated with remote attestation.*". With vCage installed and verified (only to TPM-enabled platform with Intel TXT), **user don't own the server**. Many attacks are ruled out BUT there is still JTAG http://electronics.stackexchange.com/a/8505 to dump internal mem of chip – osgx Mar 05 '15 at 19:50