3

I believe Apple has disabled being able to write and execute memory at the same time on the ARM64 architecture, see:

See mmap() RWX page on MacOS (ARM64 architecture)?

This makes it difficult to port implementations like jonesforth, which keeps generated code and the code to generate it (like the built-in assembler in jonesforth.f) in the same segment.

I thought I could do something like map the user space from start to HERE as 'r-x', and from here to the end as 'rw-'. Then I'd have to constantly remap memory as I compile new words, and I couldn't go and fix up previous words (I believe SCODE would make use of it).

Do you have any advice on how to handle such limitations ?

I guess I should look into other forth implementations that are running on M1 Macs.

Klapaucius Klapaucius
  • 619
  • 3
  • 13

1 Answers1

2

A Forth implementation can have a problem with write-protected segments of code only when it generates machine code that should be executable at once. There is no such a problem if it uses threaded code. So it's supposed bellow that the Forth system have to generate machine code.

Data space and code space

  1. Obviously, you have to separate code space from data space. Data space (at least mutable regions of, including regions for variables and data fields), as well as internal mutable memory regions and probably headers, should be mapped to 'rw-' segments. Code space should be mapped to 'r-x' segments.

  2. The word here ( -- addr ) returns the address of the first cell available for reservation, which is writable for a program, and it should be always in an 'rw-' segment. You can have an internal word code::here ( -- addr ) that returns address in code space, if you need.

  3. A decision for execution tokens is a compromise between speed and simplicity of implementation (an 'r-x' segment vs 'rw-'). The simplest case is that an execution token is represented by an address in an 'rw-' segment, and then execute does an additional dereferencing to get the corresponding address of code.

Code generation

In the given conditions we should generate machine code into an 'rw-' segment, but before this code is executed, this segment should be made 'r-x'.

Probably, the simplest solution is to allocate a memory block for every new definition, resize (minimize) the block on completion and make it 'r-x'. Possible disadvantages — losses due to page size (e.g. 4 KiB), and maybe memory fragmentation.

Changing protection of the main code segment starting from code::here also implies losses due to page size granularity.

Another variant is to break creating of a definition into two stages:

  1. generate intermediate representation (IR) in a separate 'rw-' segment during compilation of a definition;
  2. when the definition is completed, generate machine code in the main code segment from IR, and discard IR code.

Actually, it could be machine code on the first stage too, and then it's just relocated into another place on the second stage.

Before write to the main code segment you change it (or its part) to 'rw-', and after that revert it to 'r-x'.

A subroutine that translates IR code should be resided in another 'r-x' segment that you don't change.

Forth is agnostic to the format of generated code, and in a straightforward system only a handful of definitions "know" what format is generated. So only these definitions should be changed to generate IR code. If you relocate machine code, you probably don't need to change even these definitions.

ruvim
  • 7,151
  • 2
  • 27
  • 36
  • I won't contend that this could work. But *I* wouldn't want to call the resulting language forth anymore. It would be a huge amount of work, just to have a new language that would be much inferior in the aspects FORTH is actually good at. If that's the best I could do, I would for myself just accept that M1 is a bad architecture to implement FORTH in, and move on. I just find it sad that there is no way to turn this protection scheme off for developer that really want to, at their own risk. – Klapaucius Klapaucius Oct 23 '22 at 07:25
  • The Forth-2012 standard allows the described techniques for a standard Forth system, and some system employ some such techniques (namely, separate code space, own memory block for each definition, intermediate representation, code relocation). There is no sense to say that they don't implement the Forth language just because of that. – ruvim Oct 23 '22 at 09:19
  • I guess my sense of what forth is is different from the Forth-2012 standard. Sue me :)! Anyway, I think the amount of loopholes to jump through to get a basic forth (I'm thinking jonesforth, which is pretty barebones) to work on M1 is just not worth it - running forth on M1 is more of academic interest than any real practical implementation (for me). Maybe someone who needs forth on M1 will do the work. Again, the sad part is that MacOS does not let me get around this policy, at my own risk. My understanding is that if I run a linux VM on the same hardware, I can happily use RWX pages. – Klapaucius Klapaucius Oct 23 '22 at 18:48
  • 1
    BTW, here is an interesting link on HN about Scheme for M1. They ran into similar problems, and it's arguably a bigger problem for Scheme than for Forth to not work on M1: https://news.ycombinator.com/item?id=29707604 – Klapaucius Klapaucius Oct 23 '22 at 18:49
  • 1
    But I do want to say thanks for the suggestions - they do look sound, and this might make an excellent exercise for aspiring Operating Systems students/researchers. If the M1 is a sign of things to come, we might end up with similar protections in _all_ devices 10 years down the road, even cromulent $5 SBCs for IOT devices. – Klapaucius Klapaucius Oct 23 '22 at 18:53