Python multiprocessing: Permission denied

Question

I'm getting an error when trying to execute python program that uses multiprocessing package:

  File "/usr/local/lib/python2.6/multiprocessing/__init__.py", line 178, in RLock
    return RLock()
  File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 142, in __init__
    SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1)
  File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 49, in __init__
    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 13] Permission denied

It looks like the user doesn't have permission to access shared memory. When executing with root privileges it works fine.

Is there any solution to run it as normal user(not root)?

Python version 2.6.2 , OS is Linux 2.6.18 (CentOS release 5.4) and it's VPS machine.

codeape · Accepted Answer · 2011-10-31T15:21:30.323

79

For POSIX semaphores to work, the users need r/w access to shared memory (/dev/shm).

Check the permissions to /dev/shm. On my laptop (Ubuntu) it looks like this:

$ ls -ld /dev/shm
drwxrwxrwt 2 root root          40 2010-01-05 20:34 shm

To permanently set the correct permissions (even after a reboot), add the following to your /etc/fstab:

none /dev/shm tmpfs rw,nosuid,nodev,noexec 0 0

Haven't tried this, just copied from a forum post.

edited Oct 31 '11 at 15:21

answered Jan 05 '10 at 22:15

codeape

97,830
24
159
188

12

Using `none /dev/shm tmpfs rw,nosuid,nodev,noexec 0 0` in `/etc/fstab` still works but is more secure. See e.g. https://bugzilla.redhat.com/show_bug.cgi?id=664457 – Day Apr 14 '11 at 17:33
I was getting the same error when I tried to run Minecraft-Overviewer on Fedora 14. This question and @Day's comment were a great help! – Tim Bielawa May 13 '11 at 06:36
@Day's comment resolved OSError: [Error 38] Function not implemented for me when trying to get celery working with django on an Ubuntu 10.10 VPS – cerberos Jun 21 '11 at 16:24
@sherpya Thanks, I didn't know the ``-d`` flag. I've updated the answer. – codeape Oct 31 '11 at 15:22
Under CentOS 7.3.1611, permissions on /dev/shm of `drwxr-xr-t. 2 root root` were causing Ansible running as nonroot to fail with `OSError: [Errno 13] Permission denied`. Running `chmod go+w /dev/shm` remediated this Ansible error. I found that these errors weren't related to mount status (reported `rw`). Instead they are related to directory permissions on `/dev/shm` mount point. The only permanent remedy I've found so far is to add this to /etc/rc.local (and be sure to `chmod +x /etc/rc.local`): `/usr/bin/chmod go+w /dev/shm` – CODE-REaD Apr 27 '17 at 16:51
1

I'm having trouble on Ubuntu 18.04, file permissions look okay, see https://askubuntu.com/questions/1040724/how-to-fix-repo-forall-command-on-ubuntu-18-04 – satur9nine May 27 '18 at 03:56
1

Just checking by in 2022 to say that this post from 2010, referencing a forum post from 2006, that itself references a solution from 1997(!) has saved my day. This should be in a museum or something.. :D – voiDnyx Dec 19 '22 at 15:51
1

Also for 2023! Thanks! – stolenmoment Apr 11 '23 at 17:30
Just added this fix to mine so uvicorn would run under python3. Works great. Thanks – raddevus Aug 08 '23 at 18:45

score 5 · Answer 2 · answered Nov 06 '14 at 10:14

5

In my OVH VPS Classic, this error was caused by a loop in /dev/shm and /run/shm. Both were symlinks linking to each other. So as root here is what I did:

# rm /dev/shm
# mkdir /dev/shm
# chmod 777 /dev/shm
# nano /etc/fstab

Then I modified the shm line from:

none /dev/shm tmpfs rw 0 0

To:

none /dev/shm tmpfs rw,nosuid,nodev,noexec 0 0

Restarted the server... And that fixed the problem! Alternatively you can mount shm manually:

# mount /dev/shm

Hope this helps :-)

answered Nov 06 '14 at 10:14

Gabriel Hautclocq

3,230
2
26
31

Thanks a lot, this worked for me while other solutions didn't. Only difference was that I had to use sudo. – Vahid S. Bokharaie Oct 11 '20 at 23:43

Qin Heyang · Answer 3 · 2018-12-16T14:23:47.230

2

One simple solution without rebooting is

sudo chmod 777 /dev/shm

That solved my problem.

edited Dec 16 '18 at 14:23

answered Dec 12 '18 at 11:33

Qin Heyang

1,456
1
16
18

score 0 · Answer 4 · edited Aug 28 '21 at 12:44

I tried all the recommendations related to chmod and shm, but in my case the solution was:

Using conda navigator:

In base-environment run (in order to see the navigator):
```
$ anaconda-navigator
```
Create a new conda environment: from the button CREATE in the navigator
Select the new environment with your mouse
Install "notebook": Install it from anaconda-navigator in the new environment

Using command line:

Create a new anaconda enviroment (enviroment name "my_new_env"):
```
$ conda create --name my_new_env
```
Enter to my_new_env:
```
$ conda activate my_new_env
```

Install Jupyter notebook:

$ conda install jupyter-core (OR $ conda install notebook)

As a summary, don't use snap to install Jupyter notebook.

这个回答，用中文俗语，叫 `牛头不对马嘴` 这显然是权限问题 — fishshrimp鱼虾爆栈, Mar 18 '22 at 08:35

Python multiprocessing: Permission denied

4 Answers4

Using conda navigator:

Using command line:

Linked

Related