Run Python script from browser using virtual environment on EC2 instance

Question

TLDR;

I created a virtual environment on my EC2 instance. How can I access this from the browser?

---

Hey everyone,

I created a virtual environment, following this tutorial, on my EC2 instance to run a simple Python script. Within the terminal, it works without errors. However, I have made a web application and I would like to activate this script from the browser using the virtual environment. When I try this, I get a "Permission denied" error.

PHP

$output=shell_exec('bash /var/app/current/scripts/script.sh');
echo "<pre>$output</pre>";

script.sh

#!/bin/bash
source /home/ec2-user/venv/python3/bin/activate
python3 /var/app/current/scripts/test.py

test.py

from datetime import datetime
from bs4 import BeautifulSoup
import requests

print('hello')
print(datetime.now())

url = "https://www.stackoverflow.com/"
website = requests.get(url).text
soup = BeautifulSoup(website, "html.parser")
print(soup.title)

error

/var/app/current/scripts/script.sh: line 2: /home/ec2-user/venv/python3/bin/activate: Permission denied
Traceback (most recent call last):
  File "/var/app/current/scripts/test.py", line 2, in <module>
    from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'

What I have tried:

• I tried to change the permissions on the virtual environment using the following:

  chmod a+x /home/ec2-user/venv

This should give all users access to the virtual environment folder: /home/ec2-user/venv

However, I am still getting the error:

/home/ec2-user/venv/python3/bin/activate: Permission denied

• I have also tried to give all users the possibility of executing the activation script (/home/ec2-user/venv/python3/bin/activate):

  chmod 665 /home/ec2-user/venv/python3/bin/activate

Which results in:

-rw-rw-r-x 1 ec2-user ec2-user  /home/ec2-user/venv/python3/bin/activate

However, I still get the same error:

/home/ec2-user/venv/python3/bin/activate: Permission denied

Note:

• Note that if I only import datetime and I comment out bs4 and requests (along with everything else regarding BeautifulSoup), then the script works great as it does not have to access the virtual environment to pull in the packages.

*Virtual environment tutorial

Answer 1

You get this error because you have not added libraries that are used in the python script to the virtual env.

In the tutorial you mentioned only boto library is installed.

You need to install libraries you use. Run this from the command line:

source /home/ec2-user/venv/python3/bin/activate
pip install beautifulsoup4
pip install requests

Alternatively you can create a file and name it for example /home/ec2-user/requirements.txt for example and list all requirements your script use:

beautifulsoup4
requests

Then you can use this file to install all requirements into virtual env:

source /home/ec2-user/venv/python3/bin/activate
pip install -r /home/ec2-user/requirements.txt

Answer 2

Solved!

I got some help from this post, however, needed to modify a few things.

Let's dive into what his answer was:

sudo chown -R your_username:your_username path/to/virtuaelenv/

Okay, this is great, but I needed a bit of information.

For me, the web application's username is webapp.

Then, one thing that isn't very clear above is the path. So, my path is:

/home/ec2-user/venv/python3/bin/activate

as mentioned above. Here, you need to change permissions to the /home/ec2-user and NOT to /home/ec2-user/venv

So, to give my application permission to my virtual environment, I needed ran:

sudo chown -R webapp:webapp /home/ec2-user

That worked in the browser! However, this took away my ability to work with it on the server. To do so, I would have to switch it back to:

sudo chown -R ec2-user:ec2-user /home/ec2-user

Being far from ideal to switch back and forth, I tried to change the permissions with chmod instead.

sudo chmod 711 /home/ec2-user

Now I have read, write, and execution permissions whereas everyone else, including the web app, can only execute.

Now it all works