3

I'm trying to restore data in EFS from recovery points managed by AWS Backup. It seems AWS Backup does not support destructive restores and will always restore to a directory in the target EFS file system, even when creating a new one.

I would like to sync the data extracted from such a recovery point to another volume, but right now I can only do this manually as I need to lookup the directory name that is used by the start-restore-job operation (e.g. aws-backup-restore_2022-05-16T11-01-17-599Z), as stated in the docs:

You can restore those items to either a new or existing file system. Either way, AWS Backup creates a new Amazon EFS directory (aws-backup-restore_datetime) off of the root directory to contain the items.

Further looking through the documentation I can't find either of:

  • an option to set the name of the directory used
  • the value of directory name returned in any call (either start-restore-job or describe-restore-job)

I have also checked how the datetime portion of the directory name maps to the creationDate and completionDate of the restore job but it seems neither match (completionDate is very close, but it's not the exact same timestamp).

Is there any way for me to do one of these two things? Both of them missing make restoring a file system from a recovery point in an automated fashion very hard.

m90
  • 11,434
  • 13
  • 62
  • 112

1 Answers1

1

Is there any way for me to do one of these two things?

As it stands, no.

However, since we know that the directory will always be in the root, doing find . -type d -name "aws-backup-restore_*" should return the directory name to you. You could also further filter this down based on the year, month, day, hour & minute.

You could have something polling the job status on the machine that has the EFS file system mounted, finding the correct directory and then pushing that to AWS Systems Manager Parameter Store for later retrieval. If restoring to a new file system, this of course becomes more difficult but still doable in an automated fashion.

If you're not mounting this on an EC2 instance, for example, running a Lambda with the EFS file system mounted, will let you obtain the directory & then push it to Parameter Store for retrieval elsewhere. The Lambda service mounts EFS file systems when the execution environment is prepared - in other words, during the 'cold start' duration so there are no extra costs here for extra invocation time & as such, would be the cheapest option.

There's no built-in way via the APIs however to obtain the directory or configure it so you're stuck there.

It's an AWS failure that neither do they return the filename that they use in any way nor does any of the metadata returned - creationDate/completionData - exactly match the timestamp they use to name the file.

If you're an enterprise customer, suggest this as a missing feature to your TAM or SA.

Ermiya Eskandary
  • 15,323
  • 3
  • 31
  • 44
  • 1
    Trouble is I cannot really do this from the context of a program/script using the AWS SDK unless I am running this from a machine that is able to mount the file system. I could run all of this in a Lambda that mounts the file system, but then the procedure is subject to Lambda timeout limitations which is a real problem when restoring a lot of data. Not making the name of the directory available / configurable really seems like an insane oversight here. Unfortunately we're not on a support plan that would allow me to suggest this to anyone at AWS. – m90 May 22 '22 at 09:40
  • @m90 I understand your frustration however you should be able to mount the EFS drive with a Lambda, grab the filename and then store it for another process to pick up? Or does mounting it take a lot of time for you? As you say - if using just the AWS SDK, there's no way sadly. I agree it's an oversight. – Ermiya Eskandary May 22 '22 at 09:44
  • 1
    Running a Lambda just to grab the directory name should work, you're correct. I've never tested that yet though, so I don't know if mounting is slow. I'd hope it's not and even if it is, I would assume it's not billed as part of the invocation time as it's not happening at runtime. – m90 May 22 '22 at 09:53
  • 1
    @m90 Yes correct, it is probably the cheapest option - I've updated my answer to note that. Let me know if you hit any roadblocks & I'll be more than happy to help. – Ermiya Eskandary May 22 '22 at 10:11