1

I am having some issues with persistent actors which use the Postgres plugin where it seems the actors never manage to reconnect to the database after a database outage.

The persistent actors are stopped after 1 minute of inactivity so I am getting new actors all the time, but they never seem to be able to reconnect.

Restarting the pod the actor system is running on fixes the problem.

I can kind of replicate this locally by :

  • Stopping the database
  • Starting the actor system
  • Send a message which should force recovery
  • Recovery fails because of no database connection

I then start the database without restarting the actor system and send a new message which spawns a new persistent actor which fails with the same database error.

Is there some way of forcing Akka.Persistence to reconnect?

Jonas Røineslien
  • 705
  • 8
  • 18

1 Answers1

0

There are two ways you can go about solving this problem:

  1. Recreate the entity actors using a BackoffSupervisor - which will recreate the actor according to an exponential backoff schedule.
  2. Or recreate the entity actors using the third party Akka.Persistence.Extras NuGet package which works similarly to the BackoffSupervisor but is able to save messages that weren't successfully persisted provided that you implement the messaging protocol it expects. The documentation for that feature is here: https://devops.petabridge.com/articles/state-management/akkadotnet-persistence-failure-handling.html
Aaronontheweb
  • 8,224
  • 6
  • 32
  • 61