How do you configure Spinnaker to use an ElastiCache redis cluster?

Question

Following the instructions here, I am attempting to configure Spinnaker to use an ElastiCache redis cluster, with cluster mode enabled.

Spinnaker: 1.40.2
Halyard: 0.35
Cloud driver provider: k8s (1.7.4)
Cloud: AWS

I have an ElastiCache Redis cluster configured in cluster mode, it has its configuration endpoint:

redis-spinnaker.1234.clustercfg.use1.cache.amazonaws.com:6379

It has one node:

I have created the following file: halyard/default/service-settings/redis.yml

overrideBaseUrl: redis://redis-spinnaker.1234.clustercfg.use1.cache.amazonaws.com:6379

I also tried

overrideBaseUrl: redis://redis-spinnaker-0001-001.1234.0001.use1.cache.amazonaws.com:6379

Note that both of these addresses are accessible from the cluster.

When I run halyard deploy apply, spin-gate times out

/ Deploy spin-gate
  Timed out

And the whole deploy times out.
Problems in Global:
! ERROR Unexpected exception:
  DaemonTaskInterrupted(interruptedTime=1507614194943, message=Task interrupted at
  Tue Oct 10 05:43:14 UTC 2017 with message: Interrupted during reap by exception:
  null)

- Failed to deploy Spinnaker. Task killed because it was taking too
  long.

At this point, I'm not sure what to do to make this work?

score 2 · Answer 1 · answered Oct 14 '17 at 21:05

I'm making some progress with elasticache redis like so:

$ cat .hal/default/service-settings/redis.yml 
overrideBaseUrl: redis://xxx.use1.cache.amazonaws.com:6379

$ cat .hal/default/profiles/gate-local.yml 
redis:
  configuration:
    secure: true

the gate-local.yml change came from https://github.com/spinnaker/spinnaker/issues/1309#issuecomment-260971228

not 100% sure it works but at least all the spinnaker services start up and it is superficially working even after I kill off the redis replica set that halyard starts up

I wonder about adding `targetSize: 0` to prevent creation of `redis` and `redisBootstrap`, or even the right way to do that. https://www.spinnaker.io/reference/halyard/custom/ — kross, Oct 16 '17 at 18:34

score 2 · Answer 2 · answered Oct 17 '17 at 14:49

Here's a set of instructions for anyone else that wants to use an external redis (based on my GH issue):

# ~/.hal/default/service-settings/redis.yml
enabled: false
overrideBaseUrl: redis://:password@c1.us-central1-2.gce.cloud.redislabs.com:6379

# ~/.hal/default/service-settings/redis-bootstrap.yml
enabled: false
overrideBaseUrl: redis://:password@c1.us-central1-2.gce.cloud.redislabs.com:6379

Specific to elasticache as @DavidBlack said, you also need

# ~/.hal/default/profiles/gate-local.yml 
redis:
  configuration:
    secure: true

Follow this by deleting replica sets, services etc, or perhaps more simply (depending on your situation): hal deploy clean && hal deploy apply

If you hal deploy clean, don't forget to re-apply any additional configs (like external IP to kubernetes services if you have set them up).

score 1 · Accepted Answer · answered Oct 24 '17 at 00:20

1

I cannot make spinnaker work with clustered / replicated elasticache redis, so what I do is use only a single ElastiCache redis node and then set service.redis.host in spinnaker-local.yml to the appropriate redis endpoint and set redis.configuration.secure in gate-local.yml to true

answered Oct 24 '17 at 00:20

Salvian Reynaldi

102
5

It was a combination of things, but the key was that I was using the `clustered` mode. So this felt like the key piece of information that solved my issue. I also set `overrideBaseUrl` in `service-settings/redis.yml` and `redis.configuration.secure: true` in `profiles/gate-local.yml` – GreenKiwi Nov 07 '17 at 06:37

score 0 · Answer 4 · answered Jul 02 '21 at 13:08

For anyone using the Spinnaker operator, I got elasticsearch redis working with the following settings:

Redis Version: 5.0.6
Cluster Mode enabled: No
Encryption in-transit: Yes
Encryption at-rest: Yes
Redis AUTH: Yes

k8s secret: rediss://:AUTH_TOKEN@master.spinnaker-test.XXX.YYY.cache.amazonaws.com:6379

profiles:
  gate:
    redis:
      configuration:
        secure: true

service-settings:  
  redis:
    overrideBaseUrl: encrypted:k8s!n:spinnaker-secrets!k:redis-url-dev
    skipLifeCycleManagement: true

Once this is deployed you need to manually delete the Redis deployment the operator created previously.

How do you configure Spinnaker to use an ElastiCache redis cluster?

4 Answers4