161

I have a bit of an issue with one of my projects.

I have been trying to find a well documented example of using shared memory with fork() but to no success.

Basically the scenario is that when the user starts the program, I need to store two values in shared memory: current_path which is a char* and a file_name which is also char*.

Depending on the command arguments, a new process is kicked off with fork() and that process needs to read and modify the current_path variable stored in shared memory while the file_name variable is read only.

Is there a good tutorial on shared memory with example code (if possible) that you can direct me to?

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
bleepzter
  • 9,607
  • 11
  • 41
  • 64
  • 6
    You may consider using threads instead of processes. Then the whole memory is shared with no further tricks. – elomage Feb 04 '14 at 12:22
  • 1
    The answers below discuss both the System V IPC mechanism, [`shmget()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/shmget.html) et al. and also the pure [`mmap()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html) approach with `MAP_ANON` (aka `MAP_ANONYMOUS`) — though `MAP_ANON` is not defined by POSIX. There is also POSIX [`shm_open()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html) and [`shm_close()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_close.html) for managing shared memory objects. _[…continued…]_ – Jonathan Leffler Mar 15 '20 at 03:59
  • 2
    _[…continuation…]_ These have the same advantage that the System V IPC shared memory has — the shared memory object can persist beyond the lifetime of the process that creates it (until some process executes [`shm_unlink()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_unlink.html)), whereas mechanisms using `mmap()` require a file and `MAP_SHARED` to persist the data (and `MAP_ANON` precludes persistence). There's a complete example in the Rationale section of the specification of `shm_open()`. – Jonathan Leffler Mar 15 '20 at 04:01

6 Answers6

237

There are two approaches: shmget and mmap. I'll talk about mmap, since it's more modern and flexible, but you can take a look at man shmget (or this tutorial) if you'd rather use the old-style tools.

The mmap() function can be used to allocate memory buffers with highly customizable parameters to control access and permissions, and to back them with file-system storage if necessary.

The following function creates an in-memory buffer that a process can share with its children:

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>

void* create_shared_memory(size_t size) {
  // Our memory buffer will be readable and writable:
  int protection = PROT_READ | PROT_WRITE;

  // The buffer will be shared (meaning other processes can access it), but
  // anonymous (meaning third-party processes cannot obtain an address for it),
  // so only this process and its children will be able to use it:
  int visibility = MAP_SHARED | MAP_ANONYMOUS;

  // The remaining parameters to `mmap()` are not important for this use case,
  // but the manpage for `mmap` explains their purpose.
  return mmap(NULL, size, protection, visibility, -1, 0);
}

The following is an example program that uses the function defined above to allocate a buffer. The parent process will write a message, fork, and then wait for its child to modify the buffer. Both processes can read and write the shared memory.

#include <string.h>
#include <unistd.h>

int main() {
  char parent_message[] = "hello";  // parent process will write this message
  char child_message[] = "goodbye"; // child process will then write this one

  void* shmem = create_shared_memory(128);

  memcpy(shmem, parent_message, sizeof(parent_message));

  int pid = fork();

  if (pid == 0) {
    printf("Child read: %s\n", shmem);
    memcpy(shmem, child_message, sizeof(child_message));
    printf("Child wrote: %s\n", shmem);

  } else {
    printf("Parent read: %s\n", shmem);
    sleep(1);
    printf("After 1s, parent read: %s\n", shmem);
  }
}
Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
salezica
  • 74,081
  • 25
  • 105
  • 166
  • 75
    This is why Linux is so frustrating for inexperienced devs. The man page doesn't explain how to actually use it, and there is no sample code. :( – bleepzter Apr 13 '11 at 22:46
  • 61
    Haha I know what you mean, but it's actually because we're not used to reading manpages. When I learned to read them and got used to them, they became even more useful than lousy tutorials with particular demonstrations. I remember I got a 10/10 in my Operating Systems course using nothing but manpages for reference during the exam. – salezica Apr 13 '11 at 22:51
  • 21
    `shmget` is a really old-fashioned, and some would say deprecated, way to do shared memory... Better to use `mmap` and `shm_open`, plain files, or simply `MAP_ANONYMOUS`. – R.. GitHub STOP HELPING ICE Apr 13 '11 at 23:29
  • 2
    SysV shared memory (shmget) is extremely crufty and unnecessary, as mmap() works properly on Linux. – MarkR Apr 14 '11 at 05:39
  • 4
    @Mark @R You guys are right, I'll point that out in the answer for future reference. – salezica Apr 14 '11 at 21:16
  • 2
    The man page for `shmget` does *not* have everything you need to get started. I can't even figure out how to properly remove shared memory. – Brian McCutchon Mar 05 '17 at 19:15
  • 5
    Well, this answer became popular for some reason, so I decided to make it worth the read. It only took 4 years – salezica May 26 '17 at 20:16
  • 1
    @BrianMcCutchon I disagree. I didn't know anything about how to use shared memory, but after reading `man shmget` I was halfway to understanding; the second half came after reading `man shmat`, which is listed at the bottom of `man shmget` as a `SEE ALSO`. As described on that page, `shmdt` is how you announce that your process no longer needs access to the shared memory region. – amalloy Jun 01 '17 at 19:02
  • 1
    Note that this is not POSIX, so you might need to `#define _GNU_SOURCE` to have the `MAP_ANONYMOUS` constant available. – Yann TM Nov 23 '17 at 10:53
  • 4
    The way you put `0` as the argument of `fd` in `mmap()` is not the bast way. From the man page of `mmap()` on Linux: `MAP_ANONYMOUS The mapping is not backed by any file; its contents are initial‐ized to zero. The fd and offset arguments are ignored; however, some implementations require fd to be -1 if MAP_ANONYMOUS (or MAP_ANON) is specified, and portable applications should ensure this. The use of MAP_ANONYMOUS in conjunction with MAP_SHARED is supported on Linux only since kernel 2.4.` ` – Galaxy May 03 '19 at 02:39
  • So the better way to write the code would be: `return mmap(NULL, size, protection, visibility, -1, 0);` – Galaxy May 03 '19 at 02:42
  • I duplicate your work but discover everytime I compile will cause error on every line needs format shmem, and the reason is they expect char* instead of void, so what I done is to modify the declare line to "char* shmem = create_shared_memory(128);" – Brady Huang May 29 '19 at 16:35
  • A question: is it possible to do something similar, but `if` and `else` code blocks residing in separate `main` functions in separate files? I.e. to do this type of memory sharing having two completely unrelated processes without child/parent hierarchy? – mercury0114 Oct 12 '20 at 18:09
  • Do I need to use lock on shared memory access? – Грузчик Jul 26 '22 at 13:28
  • @slezica Does third-party processes cannot obtain an address for it means the same as the mapping is not backed by any file (whis is stated in mmap manpage)? – lifang Aug 16 '22 at 05:48
34

Here is an example for shared memory :

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>

#define SHM_SIZE 1024  /* make it a 1K shared memory segment */

int main(int argc, char *argv[])
{
    key_t key;
    int shmid;
    char *data;
    int mode;

    if (argc > 2) {
        fprintf(stderr, "usage: shmdemo [data_to_write]\n");
        exit(1);
    }

    /* make the key: */
    if ((key = ftok("hello.txt", 'R')) == -1) /*Here the file must exist */ 
{
        perror("ftok");
        exit(1);
    }

    /*  create the segment: */
    if ((shmid = shmget(key, SHM_SIZE, 0644 | IPC_CREAT)) == -1) {
        perror("shmget");
        exit(1);
    }

    /* attach to the segment to get a pointer to it: */
    if ((data = shmat(shmid, NULL, 0)) == (void *)-1) {
        perror("shmat");
        exit(1);
    }

    /* read or modify the segment, based on the command line: */
    if (argc == 2) {
        printf("writing to segment: \"%s\"\n", argv[1]);
        strncpy(data, argv[1], SHM_SIZE);
    } else
        printf("segment contains: \"%s\"\n", data);

    /* detach from the segment: */
    if (shmdt(data) == -1) {
        perror("shmdt");
        exit(1);
    }

    return 0;
}

Steps :

  1. Use ftok to convert a pathname and a project identifier to a System V IPC key

  2. Use shmget which allocates a shared memory segment

  3. Use shmat to attache the shared memory segment identified by shmid to the address space of the calling process

  4. Do the operations on the memory area

  5. Detach using shmdt

yugr
  • 19,769
  • 3
  • 51
  • 96
Mayank
  • 2,150
  • 1
  • 15
  • 13
14

These are includes for using shared memory

#include<sys/ipc.h>
#include<sys/shm.h>

int shmid;
int shmkey = 12222;//u can choose it as your choice

int main()
{
  //now your main starting
  shmid = shmget(shmkey,1024,IPC_CREAT);
  // 1024 = your preferred size for share memory
  // IPC_CREAT  its a flag to create shared memory

  //now attach a memory to this share memory
  char *shmpointer = shmat(shmid,NULL);

  //do your work with the shared memory 
  //read -write will be done with the *shmppointer
  //after your work is done deattach the pointer

  shmdt(&shmpointer, NULL);
Mat
  • 202,337
  • 40
  • 393
  • 406
Bharat
  • 135
  • 1
  • 3
8

try this code sample, I tested it, source: http://www.makelinux.net/alp/035

#include <stdio.h> 
#include <sys/shm.h> 
#include <sys/stat.h> 

int main () 
{
  int segment_id; 
  char* shared_memory; 
  struct shmid_ds shmbuffer; 
  int segment_size; 
  const int shared_segment_size = 0x6400; 

  /* Allocate a shared memory segment.  */ 
  segment_id = shmget (IPC_PRIVATE, shared_segment_size, 
                 IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR); 
  /* Attach the shared memory segment.  */ 
  shared_memory = (char*) shmat (segment_id, 0, 0); 
  printf ("shared memory attached at address %p\n", shared_memory); 
  /* Determine the segment's size. */ 
  shmctl (segment_id, IPC_STAT, &shmbuffer); 
  segment_size  =               shmbuffer.shm_segsz; 
  printf ("segment size: %d\n", segment_size); 
  /* Write a string to the shared memory segment.  */ 
  sprintf (shared_memory, "Hello, world."); 
  /* Detach the shared memory segment.  */ 
  shmdt (shared_memory); 

  /* Reattach the shared memory segment, at a different address.  */ 
  shared_memory = (char*) shmat (segment_id, (void*) 0x5000000, 0); 
  printf ("shared memory reattached at address %p\n", shared_memory); 
  /* Print out the string from shared memory.  */ 
  printf ("%s\n", shared_memory); 
  /* Detach the shared memory segment.  */ 
  shmdt (shared_memory); 

  /* Deallocate the shared memory segment.  */ 
  shmctl (segment_id, IPC_RMID, 0); 

  return 0; 
} 
shakram02
  • 10,812
  • 4
  • 22
  • 21
  • 3
    This is good code, except I don't think it shows how to access the shared-memory segment by a client (by using `shmget` and `shmat` from a different process), which is kind of the whole point of shared memory... =( – étale-cohomology Aug 28 '17 at 02:44
8

Here's a mmap example:

#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

/*
 * pvtmMmapAlloc - creates a memory mapped file area.  
 * The return value is a page-aligned memory value, or NULL if there is a failure.
 * Here's the list of arguments:
 * @mmapFileName - the name of the memory mapped file
 * @size - the size of the memory mapped file (should be a multiple of the system page for best performance)
 * @create - determines whether or not the area should be created.
 */
void* pvtmMmapAlloc (char * mmapFileName, size_t size, char create)  
{      
  void * retv = NULL;                                                                                              
  if (create)                                                                                         
  {                                                                                                   
    mode_t origMask = umask(0);                                                                       
    int mmapFd = open(mmapFileName, O_CREAT|O_RDWR, 00666);                                           
    umask(origMask);                                                                                  
    if (mmapFd < 0)                                                                                   
    {                                                                                                 
      perror("open mmapFd failed");                                                                   
      return NULL;                                                                                    
    }                                                                                                 
    if ((ftruncate(mmapFd, size) == 0))               
    {                                                                                                 
      int result = lseek(mmapFd, size - 1, SEEK_SET);               
      if (result == -1)                                                                               
      {                                                                                               
        perror("lseek mmapFd failed");                                                                
        close(mmapFd);                                                                                
        return NULL;                                                                                  
      }                                                                                               

      /* Something needs to be written at the end of the file to                                      
       * have the file actually have the new size.                                                    
       * Just writing an empty string at the current file position will do.                           
       * Note:                                                                                        
       *  - The current position in the file is at the end of the stretched                           
       *    file due to the call to lseek().  
              *  - The current position in the file is at the end of the stretched                    
       *    file due to the call to lseek().                                                          
       *  - An empty string is actually a single '\0' character, so a zero-byte                       
       *    will be written at the last byte of the file.                                             
       */                                                                                             
      result = write(mmapFd, "", 1);                                                                  
      if (result != 1)                                                                                
      {                                                                                               
        perror("write mmapFd failed");                                                                
        close(mmapFd);                                                                                
        return NULL;                                                                                  
      }                                                                                               
      retv  =  mmap(NULL, size,   
                  PROT_READ | PROT_WRITE, MAP_SHARED, mmapFd, 0);                                     

      if (retv == MAP_FAILED || retv == NULL)                                                         
      {                                                                                               
        perror("mmap");                                                                               
        close(mmapFd);                                                                                
        return NULL;                                                                                  
      }                                                                                               
    }                                                                                                 
  }                                                                                                   
  else                                                                                                
  {                                                                                                   
    int mmapFd = open(mmapFileName, O_RDWR, 00666);                                                   
    if (mmapFd < 0)                                                                                   
    {                                                                                                 
      return NULL;                                                                                    
    }                                                                                                 
    int result = lseek(mmapFd, 0, SEEK_END);                                                          
    if (result == -1)                                                                                 
    {                                                                                                 
      perror("lseek mmapFd failed");                  
      close(mmapFd);                                                                                  
      return NULL;                                                                                    
    }                                                                                                 
    if (result == 0)                                                                                  
    {                                                                                                 
      perror("The file has 0 bytes");                           
      close(mmapFd);                                                                                  
      return NULL;                                                                                    
    }                                                                                              
    retv  =  mmap(NULL, size,     
                PROT_READ | PROT_WRITE, MAP_SHARED, mmapFd, 0);                                       

    if (retv == MAP_FAILED || retv == NULL)                                                           
    {                                                                                                 
      perror("mmap");                                                                                 
      close(mmapFd);                                                                                  
      return NULL;                                                                                    
    }                                                                                                 

    close(mmapFd);                                                                                    

  }                                                                                                   
  return retv;                                                                                        
}                                                                                                     
Leo
  • 1,160
  • 18
  • 31
  • `open` adds file I/O overhead. Use `shm_open` instead. – osvein Apr 07 '18 at 21:02
  • 1
    @Spookbuster, in some implementations of shm_open, open() is called under the covers, so I'll have to disagree with your assessment; here's an example: https://code.woboq.org/userspace/glibc/sysdeps/posix/shm_open.c.html – Leo Apr 09 '18 at 19:29
  • 1
    while some shm_open() implementations use open() under the hood, POSIX has lower requirements for the file descriptors produced by shm_open(). For instance, implementations aren't required to support I/O functions like read() and write() for shm_open() file descriptors, allowing certain implementations to make optimizations for shm_open() that can't be made for open(). If all you're going to do with it is mmap(), you should use shm_open(). – osvein Apr 09 '18 at 20:46
  • 1
    Most Linux-glibc setups make one such optimization by using tmpfs to back shm_open(). While the same tmpfs can usually be accessed through open(), there is no portable way to know its path. shm_open() let's you use that optimization in a portable manner. POSIX gives shm_open() potential to perform better than open(). Not all implementations are going to make use of that potential, but it's not going to perform worse than open(). But I agree that my claim that open() always adds overhead is too broad. – osvein Apr 09 '18 at 20:50
0

I simplified @salezica's answer to read/write a shared int between parent/child processes.

Also rename the wrapping function to shared_malloc.

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>

#include <string.h>
#include <unistd.h>

void* shared_malloc(size_t size) {
  int prot = PROT_READ | PROT_WRITE;
  int flags = MAP_SHARED | MAP_ANONYMOUS;
  return mmap(NULL, size, prot, flags, -1, 0);
}

int main() {
  int* shared_x = (int *)shared_malloc(sizeof(int));
  * shared_x = 3;
  int pid = fork();
  if (pid == 0) {
    printf("Child read: %d\n", *shared_x);
    * shared_x = 4;
    printf("Child wrote: %d\n", *shared_x);
  } else {
    printf("Parent read: %d\n", *shared_x);
    while(* shared_x == 3);
    printf("After sync, parent read: %d\n", *shared_x);
  }
}
Richard
  • 14,642
  • 18
  • 56
  • 77