1

I am currently working on a part where a linked list, one linked list node has multiple variable data, is to be saved in a shared memory segment so that another program can read that list and do operations accordingly.

I have previously worked on socket programming, but sending a stream of data does not fulfill my purpose, as I have to do validation based on reading one node/element at a time. So, of all the IPC's, I think shared memory would be the best as it also has good performance than others(in this case, not generally).

The following is the struct that I have made:

    struct DNode {
    char *polname;
    char *devname;
    char *status;
    char *srczone;
    char *dstzone;
    char *srcaddr;
    char *dstaddr;
    char *srcuser;
    char *app;
    char *service;
    char *urlcategory;
    char *action;
    char *vulnerability;

    char *value;
    struct DNode *next;
        };
    struct DNode *head = NULL;

struct DList {
    DNode pool[MAX_DNODE];      // fixed-size space for nodes
    size_t npool;               // used space in pool
    size_t pfree;               // pointer to re-use freed nodes
    size_t head;                // global list head
};

DList *dlist;
    DNode *dnode_alloc(void)
{
    if (dlist->pfree != DNULL) {
        DNode *node = dlist->pool + dlist->pfree;

        dlist->pfree = dlist->pool[dlist->pfree].next;
        return node;
    } else {
        if (dlist->npool < MAX_DNODE) return &dlist->pool[dlist->npool++];
    }

    return NULL;
}
void dnode_free(DNode *node)
{
    if (node) {
        node->next = dlist->pfree;
        dlist->pfree = node - dlist->pool;
    }
}

DNode *dnode(size_t index)
{
    return (index == DNULL) ? NULL : dlist->pool + index;
}

DNode *dnode_next(const DNode *node)
{
    return dnode(node->next);
}

DNode *dnode_push(size_t *head, const char *str)
{
    DNode *node = dnode_alloc();

    if (node) {
        strncpy(node->polname, str, sizeof(node->polname));
        node->next = *head;
        *head = node - dlist->pool;
    }

    return node;
}

    void dnode_pop(size_t *head)
{
    if (*head != DNULL) {
        size_t next = dlist->pool[*head].next;

        dnode_free(&dlist->pool[*head]);
        *head = next;
    }
}
int list_insert_front(struct node* new_node) {
        struct node *temp;
        temp = malloc(sizeof *temp);
        if (temp && new_node) {
            memcpy(temp, new_node, sizeof(struct node));
            temp->next = head;
            head = temp;
            return 1;
        }
        return 0;
    }

       int main(int argc, char **argv)
{
   struct Dnode *iter = head;
    int shmid;
    xmlDocPtr doc;
    xmlNode *root_element = NULL;


    if (argc != 2)
    {
        printf("\nInvalid argument\n");
        return(1);
    }

    doc = xmlReadFile(argv[1], NULL, XML_PARSE_NOBLANKS | XML_PARSE_NOERROR | XML_PARSE_NOWARNING | XML_PARSE_NONET);
    if (doc == NULL)
    {
        fprintf(stderr, "Document not parsed successfully.\n");
        return 0;
    }

    root_element = xmlDocGetRootElement(doc);

    if (root_element == NULL)
 {
        fprintf(stderr, "empty document\n");
        xmlFreeDoc(doc);
        return 0;
    }

    printf("Root Node is %s\n", root_element->name);
    traverse_dom_trees(root_element);

    shmid = shmget(IPC_PRIVATE, sizeof(DList), IPC_CREAT | 0660);
    if (shmid < 0) exit (1);

    dlist = shmat(shmid, NULL, 0);
    if (dlist == (void *) (-1)) exit(1);

    dlist->head = DNULL;
    dlist->pfree = DNULL;
    dlist->npool = 0;

    while(iter != NULL){
dnode_push(&dlist->head, head->polname);
    dnode_pop(&dlist->head);
    iter = head->next;
    }


    shmdt(dlist);
    xmlFreeDoc(doc);       // free document
    xmlCleanupParser();    // Free globals
    return 0;
}

As you can see, I have also included an XML parser part in the main function so as to give you an idea of what I am taking as an input. But the part where I am stuck is how to save/use this struct inside a shared memory, and making it easy for the other program to access it.

Please can somebody provide me with some pseudo-code for the same as I have never used such C functionalities before and am absolutely clueless on how to approach this. Any and all suggestions are welcome and am thankful in advance.

Edit 1

Using Centos7 on a virtual machine since somebody pointed out that mentioning the platform would be fruitful.

Edit 2 Just added some code to implement shared memory segments, and it does not give me any such errors. What my concern is:

  • Is it doing what I intended?
  • Is the approach correct?
  • I know I am currently just pushing one element but this sure is right, right?
  • Am I wasting time and efforts trying to work it out using shared memory?
Devanshu Misra
  • 773
  • 1
  • 9
  • 28
  • maybe you should clarify your platform. although there are cross-platforms shared mem / IPC libraries, usually you have some platform in mind. – Ezequiel Garcia Dec 12 '17 at 06:01
  • @EzequielGarcia I am using CentOS 7 on a Virtual Machine – Devanshu Misra Dec 12 '17 at 06:04
  • Is the model you are trying to implement a producer-consumer one? In other words, one process will put jobs into the list/queue, and another process will read them and act accordingly? – Yaniv Shaked Dec 12 '17 at 06:40
  • @YanivShaked Yes sir. It is exactly like what you said. – Devanshu Misra Dec 12 '17 at 06:42
  • 2
    I'm afraid that shared memory is not really the best tool here. To use it, you must first build a custom memory allocator able to alloc and free memory in the shared memory segment, and consistently copy in that shared dynamic memory not only the node itself but also all the char arrays pointed from the node. Long story made short, you are simply writing a custom implementation of a queue... – Serge Ballesta Dec 12 '17 at 06:49
  • @SergeBallesta does that mean, I should not use shared memory, or I should but the implementation will be something like a queue? – Devanshu Misra Dec 12 '17 at 06:54
  • What I mean is that you certainly *can* use shared memory, but you will have to write a lot of boiler plate code, including a mutex to make sure that only one process can modify the shared segment at the same time. And effectively the resulting code will in the end behave like a queue if one process add elements there and another one retrieves them. It think that a queue will be simpler here only because of this requirement. Think of what you need exactly and use the tool that requires **you** to write less code... – Serge Ballesta Dec 12 '17 at 06:58
  • 1
    Shared memory is not good for linked list. It is good for array. For your case, (in which you use linked list), shared memory is more or less like memory pool, but shared. The most work is not gonna be creating shared memory or locking/unlocking shared memory. The heaviest work is how to manage(alloc, free, etc) memory pool. Compare to how to manage memory pool, how to make memory pool shared is much easier. So if your data can be re-organized to an array, and adding is more than deleting, then you can basically forget memory pool, and shared memory is easy and fit for you. – Bruce Dec 12 '17 at 07:17
  • @Bruce I just edited my piece of code, and would like you to review it. I did not convert it into an array as you asked me to, but I think I made a workaround for working with linked list – Devanshu Misra Dec 12 '17 at 10:46
  • 1
    Yes, you did, and looks nice. The idea -- you use field `next` as pointer in local process space, and as offset in shared memory space -- is brilliant. But in fact, I think you know that, you just implement an array in shared memory space, though it looks like a linked list, right :). Now let's go back to the shared memory. First, `IPC_PRIVATE` is not good, it will result in private memory, not shared. Use other key (see `ftok`). And next you need to start to think about protecting(lock/unlock) shared memory (see `semget`). – Bruce Dec 13 '17 at 01:40
  • 1
    And I see you are using System V style (`shmget`/`shmat`), but I would like to suggest you use POSIX style (`shm_open`/`mmap`), and correspondingly the lock/unlock mechanism is `sem_open`. One reason is POSIX style is preferred by many docs (if I don't remember wrong), another is POSIX shared memory can change size after creating, via `ftruncate`. (System V style is fine, good enough, just use it if you don't want to introduce more unknown issues). – Bruce Dec 13 '17 at 01:47
  • @DevanshuMisra Keep in mind that you can not use/store pointers in shared memory. Pointers are local to a process, and you can not guarantee that memory addresses are the same in the 2 processes you communicate with, even if you copy the data the pointers point to into your shared memory. To use shared memory you need to 1. Copy everything you need to share into the shared memory. 2. Convert all pointers into integer offsets, and re-write the code to work on offsets instead of pointers 3. Introduce some kind of protection and notification for reading/writing the data , a semaphore or mutex. – nos Dec 14 '17 at 08:42
  • @nos he is not using/storing pointer in shared memory, you need read code more carefully. – Bruce Dec 31 '17 at 15:26
  • @Bruce So do you. Keep in mind that the question is how to store the existing struct , which is comprised of no less than 16 pointers, in shared memory. There is even some code that stores the `dlist` variable in shared memory, a type containing a pool of `DNode` structs. I hope it is clear why storing a struct that contains C pointers in shared memory is normally not the the proper approach,. – nos Dec 31 '17 at 16:50
  • It just looks like pointer, his code is a little tricky, don’t be confused by the name. – Bruce Dec 31 '17 at 17:56
  • @Bruce I'm referring to `dlist = shmat(shmid, NULL, 0);` now e.g. `dlist->pool[0]` resides in the shared memory segment. This means in the shared memory segment, there are `dlist->pool[0].polname` , `dlist->pool[0].devname` and all the other members of a `DNode` - which are all pointers. However you need to read the actual question, and not just the code. The question is about how to store a struct (a linked list), which the OP has shown as the `struct DNode`, this struct has a lot of pointers. Are you suggesting that it is fine for the OP to store his struct DNode in shared memory ? – nos Jan 03 '18 at 08:27

2 Answers2

1

In general you cannot warrant that a shared memory segment will occupy the same virtual address range in one process than in other. So you'll have a problem when trying to interpret the values in the pointer fields, as they represent the address of the pointed object in the virtual address space of the process that wrote there the pointer value and this can be different if both processes map the shared memory segment at different places.

You can pass the mmap call a pointer to tell the system where in your virtual address space you want the segment to be mapped, so the shared pointers point to the same place in both virtual address spaces. But that pointer is only a hint, and the operating system is not forced to follow your pointer.

There are two solutions for this. The first is to offset the pointer values, so you construct your virtual address space pointer from the one you see in the shared segment. The second is to ensure your memory segments both are mapped to the same address. This has to be coordinated between both processes (but it has to be done only once, at memory mapping) as the place that is good for one can be forbidden for the other (because it has mapped some other thing there)

In 64bit architectures this is easy, as you have a zillion virtual addresses to map the segment to, and probably you can select an address without clashing with other subsystem. Think that in 32bit systems, normally shared libraries consume a bunch of addresses for the data segments of the modules herein, and the stacks makes provision for large amounts of memory, and the heap also... so you have to plan the thing before an attempt of putting both segments in a shared, identical address.

NOTE

In your particular case, that almost all the fields of the structure are pointers, this applies to all the fields, and not only the list linking ones. Not only all the list nodes must lie in the shared segment... also all the strings, and everything you access that is shared.

Community
  • 1
  • 1
Luis Colorado
  • 10,974
  • 1
  • 16
  • 31
0

You will need to:

  • Setup shared memory on your platform - see here.
  • In your program, call shm_open to open the shared memory and use mmap to access it using a pointer - see here.
  • Accessing shared memory from different processes/thread must use some arbitration/mutual exclusion mechanism - see here.
Yaniv Shaked
  • 698
  • 6
  • 12