How to return cJSON fragments by process parent using POSIX?

Question

tl;dr: How I can I configure cJSON to allocate memory in child processes, so that the parent can see the resulting structs?

---

I have a request that comes in that lists multiple products for which summaries are to be generated and returned as JSON. Currently, my code is single-threaded and uses the cJSON library for composing and marshalling the JSON.

As the summaries are computationally expensive (user requests certain calculations to be performed as part of the summaries), I'd like to fork(2) for each requested product, then have it fetch, process, and summarise into cJSON_Object( technically cJSON* but the "constructor" is cJSON_CreateObject), and then have the parent thread wait for all children to return their cJSON_Object objects to join together, perform some post processing, and then finally marshall into a string to return. Because of post-processing, I'd like to return to the parent a cJSON_Object rather than have the child threads return strings.

Now, I see cJSON.h:144 CJSON_PUBLIC(void) cJSON_InitHooks(cJSON_Hooks* hooks) which accepts a cJSON.h:125 struct internal_hooks which allows you to specify custom malloc() and free() implementations... which would be peachy if I could find a versions that allocate shared memory... and share the same type. The closest I found was shmalloc/shfree but that's from the OpenMPI library and just seems like overkill for what seems like it should be simple worker threads...

Now, this is where I am stuck, how do I return a graph of cJSON structs from child processes to the parent?

I've attached the best (albeit untested) solution I've thought of as an answer to not clutter this question further.

PS - Preferably the solution would restrict itself to the POSIX API, but Linux-only is acceptable, and additional libraries as a last resort.

Answer 1

I haven't tested this, and it seems hackish, but is the best I've thought of so far:

I did find shmget(3). The type signature is not compatible, so I thought I could wrap it. Within each child I could generate a unique integer to use as the key_t, and allocate enough memory sufficient to contain the response. Then provide my own malloc() that just allocates from the block allocated by shmget. Finally, return the key_t to the parent via the exit(3) call. The parent thread would get the value of key_t from wait(3), and then be able to fetch the cJSON_Object and combine it with the other fragments.

So the answer would be to have the on child start: shmget some memory, and then write a malloc()-compatible function my_malloc() that allocates off of that shmgeted memory, and have a pointer to my_malloc() passed to cJSON_InitHooks via struct internal_hooks.

Answer 2

Unfortunately, it's not that as simple as allocating some object in shared memory segment. cJSON object is a linked data structure, which is not surprising taking into account that JSON objects can be composed of nested lists and hashes.

typedef struct cJSON
{
    /* next/prev allow you to walk array/object chains. Alternatively, use GetArraySize/GetArrayItem/GetObjectItem */
    struct cJSON *next;
    struct cJSON *prev;
    /* An array or object item will have a child pointer pointing to a chain of the items in the array/object. */
    struct cJSON *child;
    ...
}

This structure pointers, and storing pointers inside shared memory area may not be good idea. Each process has its own virtual memory space, and shared memory segment may not be mapped to the same address in both processes. If that's the case, the same pointer value would point to valid object in one address space, but in other address space it may point to garbage.