How can segmentation be used to expand memory?

Question

I'm trying to understand memory models, specifically segmented and flat. Most of the reference material is too technical, it seems however, that segmentation can be used to "expand" the memory but what does that mean and how is it accomplished? Also, when using the segmented memory model, why could use only address 64KB at once?

Answer 1

why could use only address 64KB at once?

Because that's how much you can address with a 16-bit offset into a segment.

x86 segmentation in real mode is just linear address = segreg<<4 + offset, where offset is a regular addressing mode like [bx + di].

A linear address is a physical address if paging is disabled, otherwise it's a virtual address to be translated by the page tables pointed to by CR3.

With a 32-bit addressing mode, you can access 4GiB relative to a segment. Protected mode uses segments as indexes into a table of descriptors, rather than as a base value directly, so you can have large segments and large bases. (But at least with paging enabled, it's all mapped to the same 32-bit linear address space, so you can't use segmentation to have a 32-bit OS expose more than 4GiB of memory to a user-space process, even with segments with different bases and 4GiB limits.)

---

Using different values in segment registers allows accessing more memory, just not all at once which makes it really inconvenient for languages like C where "a pointer" is just a pointer. You have to introduce "far pointer" vs. "near pointer".

Answer 2

The 64 KiB limitation is not inherent in using segments, but of the context where x86 segments were born and were used the most, i.e. 16 bit x86 processors, where addresses were 16 bit wide, and thus could only index 64 KiB of memory.¹

Segments were essentially a trick to work around this limitation: each segment (roughly) contains a base address, to which you could implicitly or explicitly refer a 16 bit address. This allowed you to have simultaneously 5 64 KiB "views" inside the larger physical address space (later, the virtual address space); such views are also movable by loading a different address in the relevant segment registers.

The trick works particularly well as as different instructions work by default using different instructions - normally working on mostly segregated data - refer to different segments by default.

Code is read from CS (Code Segment), thus the 16 bit instruction pointer and the near call refer to it. Stack instructions (push, pop and stuff like call and ret) operate on the stack segment; SP and BP are normally interpreted relative to SS (Stack Segment).

Every other instruction that can operate on a memory operand works by default on DS (Data Segment) unless you use SP or BP as base register, when it's implicit that you are operating on the stack, so SS is assumed; still, you can add a prefix that allows to operate on another segment.

Finally, string instructions operate on DS on read (overridable) and ES on write. This allows easy and efficient copy of data between the data space (or another segment) and another location further away than 64 KiB. A typical arrangement was to point ES to the video memory and use string instructions to copy image data onscreen.

Thus, in general even without playing tricks by changing the segments you could have your different 64 KiB of stack, code, data and one "accessory" address spaces via ES (three since the 80386, where FS and GS were added), so you could address up to² 256 KiB (384 KiB) of memory at the same time, still keeping (mostly) just 16 bit pointers around - although you had to remember what segment they referred to.

Now, stuff gets more difficult once you need to actually address even more memory, thus changing the segment registers while your program is running.

Code is simple enough - if you need to call into a function that is in a zone outside the current CS you need to perform a "far call" passing a 32 bit far pointer; as long as far calls and far returns are matched and nobody tries to do a near call to code that is actually far everything goes smoothly.

Data is more delicate; enter far pointers, near pointers, the various bizarre DOS memory models and stuff gets messy quite fast - when programming in a higher level language such as C you had to worry about this kind of ultra-low level details. I don't really have first hand experience about writing software in a memory model with far pointers around, so I cannot share my own experiences, but everybody who talks about it pretty much agrees that it was not pretty at all, and we can all be glad to have moved to a world where a pointer is just a pointer inside a single flat address space.

---

Notes

1. Even in 32 bit flat-addressing mode you are still using segment registers, but they are all set to zero, so all segments are the same and perform no translation between the addresses you use and the actual memory accessed. FS/GS are an exception, as they are generally used by modern operating systems to refer to a block of memory containing thread-context data.
2. Remember that segments may overlap. Incidentally, this caused further complications in high level languages, as numerically comparison of far pointers gives no information about the equality of the actual pointed address.