You can most definitely do it, but it's hard, and it can't be done without architecting your entire software stack to be resilient against such failures. You may end up implementing half of Erlang by the time you're done :)
You must essentially run a debugger in a separate thread, attached to your own application, and use platform-specific debug APIs to get a notification of a hardware (CPU) exception happening in the watched thread(s).
Upon such notification, the monitored thread is suspended, and you will have access to the state of registers at the moment the fault happened. At that point you can disassemble the instruction to determine its length, and rewrite it with NOPs. You'd also need to identify in what kind of a memory space is the instruction that has faulted - if it's in your own code, you should definitely not touch anything and simply resume the thread, letting the native signal/exception handlers (if any) take care of it. You should only catch such issues when they happen in the non-code area (say: data area).
Alas, on most sane platforms, the data pages won't be executable anyway, so the hardware exception that you catch will indicate that code execution is attempted in a non-executable page. There's no trivial way around that short of doing stack analysis and figuring out where the code should be executing instead. It would be a very bad idea indeed to defeat data execution prevention at run-time by making the page executable!
Ideally, when faults occur, you should have a mechanism of terminating and restarting the thread in question. That's the approach taken by Erlang: let it crash and re-start the crashed thread/process.