How can I manually (programmatically) place objects in my multicore project?

Question

I am developing a mutlicore project for our embedded architecture using the gnu toolchain. In this architecture, all independent cores share the same global flat memory space. Each core has its own internal memory, which is addressable from any other core through its global 32-bit address.

There is no OS implemented and we do low-level programming, but in C instead of assembly. Each core has its own executable, generated with a separate compilation. The current method we use for inter-core communication is through calculation of absolute addresses of objects in the destination core's data space. If we build the same code for all cores, then the objects are located by the linker in the same place, so accessing an object in a remote core is merely changing the high-order bits of the address of the object in the current core and making the transaction. Similar concept allows us to share objects that are located in the external DRAM.

Things start getting complicated when:

1. The code is not the same in the two cores, so objects may not be allocated in similar addresses,

2. We sometimes use a "host", which is another processor running some control code that requires access to objects in the cores, as well as shared objects in the external memory.

In order to overcome this problem, I am looking for an elegant way of placing variables in build time. I would like to avoid changing the linker script file as possible. However, it seems like in the C level, I could only control placement up to using a combination of the section attribute (which is too coarse) and the align attribute (which doesn't guarantee the exact place).

A possible hack is to use inline assembly to define the objects and explicitly place them (using the .org and .global keywords), but it seems somewhat ugly (and we did not yet actually test this idea...)

So, here's the questions:

1. Is there a semistandard way, or an elegant solution for manually placing objects in a C program?

2. Can I declare an "uber"-extarnel objects in my code and make the linker resolve their addresses using another project's executable?

This question describes a similar situation, but there the user references a pre-allocated resource (like a peripheral) whose address is known prior to build time.

Answer 1

Maybe you should try to use 'placement' tag from new operator. More exactly if you have already an allocated/shared memory you may create new objects on that. Please see: create objects in pre-allocated memory

Answer 2

You don't say exactly what sort of data you'll be sharing, but assuming it's mostly fixed-size statically allocated variables, I would place all the data in a single struct and share only that.

The key point here is that this struct must be shared code, even if the rest of the programs are not. It would be possible to append extra fields (perhaps with a version field so that the reader can interpret it correctly), but existing fields must not be removed or modifed. structs are already used as the interface between libraries everywhere, so their layout can be relied upon (although a little more care will be need in a heterogeneous environment, as long as the type sizes and alignments are the same you should be ok).

You can then share structs by either: a) putting them in a special section and using the linker script to put that in a known location; b) allocating the struct in static data, and placing a pointer to that at a known location, say in your assembly start-up files; or c) as (b), but allocate the struct on the heap, and copy the pointer to the known pointer location at run-time. The has the advantage that the pointer can be pre-adjusted for external consumers, thus avoiding a certain amount of messing about.

Hope that helps

Answer 3

1. You're well off the beaten path here - don't expect anything 'standard' for any of this :)
2. This answer suggests a method of passing a list of raw addresses to the linker. When linking the external executable, generate a linker map file, then process it to produce this raw symbol table.

You could also try linking the entire program (all cores' programs) into a single executable. Use section definitions and a linker script to put each core's program into its internal memory address space; you can build each core's program separately, incrementally link it to a single .o file, then use objcopy to rename its sections to contain the core ID for the linker script, and rename (hide) private symbols if you're duplicating the same code across multiple cores. Finally, manually supply the start address for each core to your bootstrap code instead of using the normal start symbol.

Answer 4

Response to question 1: no, there isn't.

As for the rest, it depends very much of the operating system you use. On our system at the time I was in embedded, we had only one processor's memory to handle (80186 and 68030 based), but had multi-tasking but from the same binary. Our tool chain was extended to handle the memory in a certain way. The toolchain looked like that (on 80186):

• Microsoft C 16bit or Borland-C
• Linker linking to our specific crt.o which defined some special symbols and segments.
• Microsoft linker, generating an exe and a map file with a MS-DOS address schema
• A locator that adjusted the addresses in the executable and generated a flat binary
• Address patcher.
• An EPROM burner (later a Flash loader).

In our assembly we defined a symbol that was always at the beginning of data segment and we patched the binary with a hard coded value coming from the located map file. This allowed the library to use all the remaining memory as a heap.

In fact, if you haven't the controle on the locator (the elf loader on linux or the exe/dll loader in windows) you're screwed.