15

So I read this interview with John Carmack in Gamasutra, in which he talks about what he calls "live C++ objects that live in memory mapped files". Here are some quotes:

JC: Yeah. And I actually get multiple benefits out of it in that... The last iOS Rage project, we shipped with some new technology that's using some clever stuff to make live C++ objects that live in memory mapped files, backed by the flash file system on here, which is how I want to structure all our future work on PCs.

...

My marching orders to myself here are, I want game loads of two seconds on our PC platform, so we can iterate that much faster. And right now, even with solid state drives, you're dominated by all the things that you do at loading times, so it takes this different discipline to be able to say "Everything is going to be decimated and used in relative addresses," so you just say, "Map the file, all my resources are right there, and it's done in 15 milliseconds."

(Full interview can be found here)

Does anybody have any idea what Carmack is talking about and how you would set up something like this? I've searched the web for a bit but I can't seem to find anything on this.

Mart
  • 1,278
  • 12
  • 12
  • I think he is deserializing C++ "immutable" objects from the flash. It's always a little complex/risky because you normally can't control memory/resource allocation by objects unless you "wrote" the code for the objects. – xanatos Aug 23 '11 at 10:07
  • Are you doing mobile development? This sounds useful predominantly when you need to be able to switch in and out of your application quickly, which isn't really an issue on an ordinary computer. – Kerrek SB Aug 23 '11 at 10:07
  • @Kerrek it's useful for games on any platform, or anything that needs to load a lot of state from disk. – Justicle Aug 23 '11 at 18:51
  • @Justicle: It's good for serialization, no doubt, but wouldn't it be relatively slow on a desktop compared to using real system memory? – Kerrek SB Aug 23 '11 at 18:52
  • That's the point, its to speed up load from disk INTO system memory. – Justicle Aug 26 '11 at 01:09

5 Answers5

7

The idea is that you have all or part of your program state serialized into a file at all times by accessing that file via memory mapping. This will require you not having usual pointers because pointers are only valid while your process lasts. Instead you have to store offsets from the mapping start so that when you restart the program and remap the file you can continue working with it. The advantage of this scheme is that you don't have separate serialization which means you don't have extra code for that and you don't need to save all the state at once - instead your (all or most of) program state is backed by the file at all times.

sharptooth
  • 167,383
  • 100
  • 513
  • 979
2

You'd use placement new, either directly or via custom allocators.

Look at EASTL for an implementation of (subset) STL that is specifically geared to working well with custom allocation schemes (such as required for games running on embedded systems or game consoles).

A free subset of EASTL is here:

sbi
  • 219,715
  • 46
  • 258
  • 445
sehe
  • 374,641
  • 47
  • 450
  • 633
2

We use for years something we call "relative pointers" which is some kind of smart pointer. It is inherently nonstandard, but works nice on most platforms. It is structured like:

template<class T>
class rptr
{
    size_t offset;
public:
    T* operator->() { return reinterpret_cast<T*>(reinterpret_cast<char*>(this)+offset); }
};

This requires that all objects are stored into the same shared memory (which can be a filemap too). It also usually requires us to only store our own compatible types in there, as well as havnig to write own allocators to manage that memory.

To always have consistent data, we use snapshots via COW mmap tricks (which work in userspace on linux, no idea about other OSs).

With the big move to 64bit we also sometimes just use fixed mappings, as the relative pointers incur some runtime overhead. With usually 48bits of address space, we chose a reserved memry area for our applications that we always map such a file to.

R. Martinho Fernandes
  • 228,013
  • 71
  • 433
  • 510
PlasmaHH
  • 15,673
  • 5
  • 44
  • 57
  • There is a danger to using fixed addresses to map your file into - the OS could change the rules at a later date and use that address range for some other purpose. For example, it could load the program code into the address range you'd reserved for the data. – Skizz Aug 23 '11 at 10:55
  • Indeed, which is why we use reserved areas that the OS has no plan in using for anythign. – PlasmaHH Aug 23 '11 at 11:01
  • @PlasmaHH How do you find such reserved areas? Where are they documented? Are they different on each OS? – fadedbee Sep 12 '13 at 10:22
  • @chrisdew: They are different on every OS, and OS configuration. Linux has the ability to check what of your stuff is mapped where via `/proc` and so you can find areas of intrest where the OS never seems to put anything. If you want to make extra sure, you read the OSs source code. – PlasmaHH Sep 29 '13 at 21:17
  • How does `reinterpret_cast(this)` work? Why not use a global_mmap_offset variable? – noɥʇʎԀʎzɐɹƆ Jan 25 '20 at 19:12
  • @noɥʇʎԀʎzɐɹƆ: It just takes the address in a way to allow for byte offset manipulation on it. The global offset variable would work for if you only ever have one thing you mmap, but you may want to do that for two or more files, which would then get relative incompatible offsets. – PlasmaHH Feb 21 '20 at 10:24
  • This is how [boost::interprocess::offset_ptr](https://www.boost.org/doc/libs/1_79_0/doc/html/interprocess/offset_ptr.html) works – Bruce Adams May 20 '22 at 08:17
1

This reminds me of a file system I came up with that loaded level files of CD in an amazingly short time (it improved the load time from 10s of seconds to near instantaneous) and it works on non-CD media as well. It consisted of three versions of a class to wrap the file IO functions, all with the same interface:

class IFile
{
public:
  IFile (class FileSystem &owner);
  virtual Seek (...);
  virtual Read (...);
  virtual GetFilePosition ();
};

and an additional class:

class FileSystem
{
public:
  BeginStreaming (filename);
  EndStreaming ();
  IFile *CreateFile ();
};

and you'd write the loading code like:

void LoadLevel (levelname)
{
  FileSystem fs;
  fs.BeginStreaming (levelname);
  IFile *file = fs.CreateFile (level_map_name);
  ReadLevelMap (fs, file);
  delete file;
  fs.EndStreaming ();
}

void ReadLevelMap (FileSystem &fs, IFile *file)
{
  read some data from fs
  get names of other files to load (like textures, object definitions, etc...)
  for each texture file
  {
    IFile *texture_file = fs.CreateFile (some other file name)
    CreateTexture (texture_file);
    delete texture_file;
  }
}

Then, you'd have three modes of operation: debug mode, stream file build mode and release mode.

In each mode, the FileSystem object would create different IFile objects.

In debug mode, the IFile object just wrapped the standard IO functions.

In stream file building, the IFile object also wrapped the standard IO but had the additional functions of writing to the stream file (the owner FileSystem opened the stream file) every byte that was read, and writing the return value of any file pointer position queries (so if anything needed to know a file size, that information is written to the stream file). This would sort of concatenate the various files into one big file, but only the data that was actually read.

The release mode would create an IFile that did not open files or seek within files, it just read from the streaming file (as opened by the owner FileSystem object).

This means that in release mode, all data is read in one sequential series of reads (the OS would buffer it nicely) rather than lots of seeks and reads. This is ideal for CDs where seek times are really slow. Needless to say, this was developed for a CD based console system.

A side effect is that the data is stripped of unnecessary meta data that would normally be skipped.

It does have drawbacks - all the data for a level is in one file. These can get quite large and the data can't be shared between files, if you had a set of textures, say, that were common across two or more levels, the data would be duplicated in each stream file. Also, the load process must be the same every time the data is loaded, you can't conditionally skip or add elements to a level.

Skizz
  • 69,698
  • 10
  • 71
  • 108
  • @Justicle: I disagree: "I want game loads of two seconds on our PC platform" and this is one way of loading game data really fast. – Skizz Aug 26 '11 at 07:53
  • The question is about serialising objects in memory mapped files using offset pointers, your example is about concatenating files (a different technique). The quote is not the question, it is reference. – Justicle Aug 27 '11 at 06:30
  • Ie Your code is cool, but it's not answering the question that was asked. – Justicle Aug 27 '11 at 06:31
0

As Carmack indicates many games (and other applications) loading code is structured lika a lot of small reads and allocations.

Instead of doing this you do a single fread (or equivalent) of say a level file into memory and just fixup the pointers afterwards.

Andreas Brinck
  • 51,293
  • 14
  • 84
  • 114