2

Recently, I have stumbled upon an article explaining how to implement fibers in C (https://graphitemaster.github.io/fibers/). The article was eye-opening, as it showed that any contiguous memory region can be used as a call stack in place of the operating system provided one. This is really useful, and I would like to explore the possibilities.

However, the article also talks about the so called "Red Zone" as a requirement for a valid stack, i.e. the 128 bytes of stack space after (below) rsp must never be modified. This makes me think there are other obscure requirements for a valid stack that I know nothing about, so before diving in I thought about asking here if anybody knows of any such pitfalls that I may have overlooked. For example, can I store the stack on the heap, or does the OS not play nice with it? Any such tips are welcome.

I am interested specifically for the Windows operating system and the Intel x64 architecture, but I would also like to know if the requirements can be made "cross platform", as in, if I can use the lowest common denominator for all major platforms so I can get away with only one fiber implementation.

Thanks for any tips!

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
super
  • 278
  • 1
  • 11
  • 1
    x86-64 System V uses a red-zone; Windows doesn't. The kernel (and its mechanism for running user-space signal handlers when delivering signals) is what's responsible for not asynchronously clobbering the red-zone. See also [Where are the stacks for the other threads located in a process virtual address space?](https://stackoverflow.com/q/44858528) for how Linux thread stacks are allocated. The main-thread's stack is special and can grow dynamically, but thread stacks don't do that. (Instead relying on lazy allocation of physical pages via page faults to not waste physical memory.) – Peter Cordes Aug 12 '21 at 12:29
  • 1
    See also [How is Stack memory allocated when using 'push' or 'sub' x86 instructions?](https://stackoverflow.com/q/46790666) for Windows vs. Linux dynamic growth of the main-thread's stack. Also consider guard pages below the stack and anything else, to prevent stack-clash attacks: [What’s the purpose of mmap memory protection PROT\_NONE](https://stackoverflow.com/q/12916603) / [Linux process stack overrun by local variables (stack guarding)](https://stackoverflow.com/q/60058873) (and the Windows `_chkstk()` helper function). – Peter Cordes Aug 12 '21 at 12:38
  • 2
    Intel's CET and other shadow-stack based control flow limitations may interfer with any naive attempt of using arbitrary memory regions as stack. Which programming language are you targeting? Windows and C, you got `CreateFiber` ready-made for you, C++ is about to get coroutines which are also ready-made and go well beyond fibers in terms of compile time optimizations. – Ext3h Aug 12 '21 at 12:48
  • @Ext3h I don't remember all the details of CET but how so? A stack **is** an arbitrary memory region. But maybe you were referring to the shared shadow stack. – Margaret Bloom Aug 12 '21 at 15:51
  • 1
    The shadow stack will not allow to follow control flow from a different stack after arbitrary replacement of the stack by user space SP exchange. You can't do the switch between fibers without switching the shadow stack as well. Otherwise pushing frames on one, then popping frames from another is indistinguishable from a ROP attack. – Ext3h Aug 12 '21 at 17:31
  • BTW, in case I wasn't clear in my first comment, having a red-zone as part of the ABI doesn't impose any special requirements on how stack memory is *allocated*. Just like any other allocation, make sure nothing else tries to use the same memory for something else while it's already in use. – Peter Cordes Aug 12 '21 at 22:48
  • Thanks for all of the relevant info. I read the links, and although some of the stuff went a bit over my head, I understand a bit better what threads actually are. Though all of this was useful, I still don't have an actual document I can sift through to find what the x64 win32 stack ABI is. Any idea where to find it? – super Aug 13 '21 at 08:17

0 Answers0