How to connect to Redis instance (memorystore) from Google's Standard App Engine (Python 3.7)

Question

I've been trying to get a connect to a Redis instance from an App Engine instance without any luck. I get a connection time out.

I've set the host and port in my app.yaml:

env_variables:
  REDIS_HOST: '10.0.0.3'
  REDIS_PORT: '6379'

And try to connect as specified in the examples:

redis_host = os.environ.get('REDIS_HOST', 'localhost')
redis_port = int(os.environ.get('REDIS_PORT', 6379))
redis_client = redis.StrictRedis(host=redis_host, port=redis_port)

But it's not working, some documentation seem to want en vpc access connector, but those are only available in the us-cental1 and I got both the App Engine instances & Memorystore (Redis) running in asia-northeast1.

Has anyone had any luck getting this working? In general google's pretty good at documenting their stuff but this seems lacking to me, there's really no clear documentation on how to get it working.

Answer 1

Connecting to a Redis instance from an App Engine standard environment application and App Engine Flex application 2020

1.Download and install Cloud SDK.

2.If you already have the Cloud SDK installed, update it by running the following command:

gcloud components update

3.Create a new project:

gcloud projects create [YOUR_PROJECT_ID] --set-as-default

4.Verify the project was created:

gcloud projects describe [YOUR_PROJECT_ID]     

5.Set the project you have just created:

gcloud config set core/project PROJECT_ID

6.Initialize your App Engine app with your project and choose its region:

gcloud app create --project=[YOUR_PROJECT_ID]

7.When prompted, select the region where you want your App Engine application located.Remember the region, we will create the Redis Instance and VPC connector in the same region. My App Engine Application in on europe-west2

8.Make sure billing is enabled for your project. A billing account needs to be linked to your project in order for the application to be deployed to App Engine.

9.Install the following prerequisites:

a.Download and install Git

b.Run the following command to install the gcloud component that includes the App Engine extension for Python 3.7:

gcloud components install app-engine-python

c.Prepare your environment for Python development

10.Create the Memorystore for Redis instance.Enter the following command to create a 2 GiB Basic Tier Redis instance in the 'europe-west2' region

gcloud redis instances create myinstance --size=2 --region=europe-west2 /
--redis-version=redis_4_0

11.After the instance is created, enter the describe command to get the IP address and port of the instance.You'll use the host and port values to connect to the instance.Also find your Redis instance's authorized network

gcloud redis instances describe myinstance --region=europe-west2

12.Configuring Serverless VPC Access.Make sure you create the connector in the same region as your app and your Redis instance, and make sure the connector is attached to the Redis instance's authorized VPC network. Remember the name of the connector.

13.Ensure the Serverless VPC Access API is enabled for your project:

gcloud services enable vpcaccess.googleapis.com

14.Create a connector with the command:

gcloud compute networks vpc-access connectors create connector --network default --region europe-west2 --range 10.10.0.0/28

15.Verify that your connector is in the READY state before using it, the output should contain the line state: READY.

16.Clone the repository for python.

git clone https://github.com/GoogleCloudPlatform/python-docs-samples
cd python-docs-samples/memorystore/redis

17.Preparing the application for deployment App Engine Standard.Update the app's configuration to specify your Serverless VPC Access connector and the IP address and port of your Redis instance:

 cd gae_standard_deployment/
 cat app.yaml

runtime: python37
entrypoint: gunicorn -b :$PORT main:app

# Update with Redis instance details
env_variables:
  REDISHOST: '<REDIS_IP>'
  REDISPORT: '6379'

# Update with Serverless VPC Access connector details
vpc_access_connector:
  name: 'projects/<PROJECT_ID>/locations/<REGION>/connectors/<CONNECTOR_NAME>'

18.Deploying the application to the App Engine standard environment

cd ..
cp gae_standard_deployment/app.yaml .

19.Run the deploy command

gcloud beta app deploy

20.After the deployment is complete, visit your app at the following URL, replacing [PROJECT_ID] with your Google Cloud project ID.

Visitor number: 4

SUCCESS!

21.Preparing the application for deployment App Engine Flex.Update the app's configuration to specify the IP address and port of your Redis instance:

cd gae_flex_deployment/
cat app.yaml

cp gae_flex_deployment/app.yaml .

runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app

runtime_config:
  python_version: 3

# Update with Redis instance IP and port
env_variables:
  REDISHOST: '<REDIS_IP>'
  REDISPORT: '6379'

# Update with Redis instance network name
network:
  name: default

22.Deploying the application to the App Engine standard environment

cd ..
cp gae_flex_deployment/app.yaml .

23.Run the deploy command

gcloud beta app deploy

24.After the deployment is complete, visit your app at the following URL, replacing [PROJECT_ID] with your Google Cloud project ID.

Visitor number: 4

SUCCESS!

Answer 2

At the moment (Sept 2019), you can only create a VPC conector in the us-central1 region indeed. Take into account that it is currently a beta feature. Availability will be progressively added to other regions while moving towards General Availability.

The reason why you won't be able to connect to MemoryStore without a VCP conector is that App Engine Standard apps run in a Google-owned network, not in one of your networks. A VPC conector is essentially a Compute Engine instance running in the network of your choice in your project and is set up to proxy requests to MemoryStore via internal IP.

Answer 3

You can use VPC connector in other region, but only with the gcloud command line. On the GUI, only us-central1 is available.