Delete old files based on customized date filters

Question

Retention policy is "keep the last 7 backups (past week), and then keep Friday backups for 3 months. And then keep every month's last Friday backup for one year".

Below is the script which covers two retention conditions, but unable to complete the third condition, which is "keep every month's last Friday backup for one year". Need help in making IF block for same.

EDIT: Solution to move files around in different folders and then apply separate retention scripts on separate folders will not possible as there are lot of sub-folders inside. As this is all for backup files, hence moving files will make our dependent restore process more complicated also as otherwise that code will then require tweaking on all servers too.

#----- define folder where files are located ----#
$TargetFolder = "C:\Users\chsa\Desktop\Retention"

#----- define extension ----#
$Extension = "*.txt"

#----- define LastWriteTime parameter based on $Days ---#
$LastWrite = $Now.AddDays(-$Days)

#----- get files based on lastwrite filter and specified folder ---#
$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse 

foreach ($File in $Files) 
{
  if ($File -ne $NULL -and $File.LastWriteTime -lt (Get-Date).AddYears(-1))
  {
    Remove-Item $File.FullName | Out-Null
    Write-Host $File
  }

  if ($File -ne $NULL -and $File.LastWriteTime -ge (Get-Date).AddMonths(-3) -and $File.LastWriteTime -lt (Get-Date).AddDays(-7) -and $File.LastWriteTime.DayOfWeek -ne "Friday")
  {
    Remove-Item $File.FullName | Out-Null
    Write-Host $File
  }
}

As for implementing your retention policy: move the backup(s) you want to retain to a different folder before the cleanup, i.e. move the Friday backup from the "daily" folder to a "weekly" folder with a separate cleanup task. Do the same for the monthly backups. — Ansgar Wiechers, Aug 08 '17 at 16:49
Thanks Ansgar, but this will not be possible as there are lot of sub-folders and it might be risky to setup this way. — Channdeep Singh, Aug 08 '17 at 18:39
`$File -eq $Null` is unnecessary. `$File` will return `$True` if it exists, or `$False` if it doesn't by default. (blank strings and zero are also interpreted as `$False`). Additionally, `Out-Null` is one of the slowest operations you can use; I'd recommend `[Void]()`, `$Null=` or `>$Null` — Maximilian Burszley, Aug 08 '17 at 19:08
I agree you can certainly optimize and do away with the $null test but I actually just got done writing up detailed testing once again debunking this whole "out-null is slower" crap that for years anyone seasoned has known is actually USING THE PIPELINE is slower, nothing to do with out-null in fact clear proof I just posted here: https://stackoverflow.com/questions/5260125/whats-the-better-cleaner-way-to-ignore-output-in-powershell/45577369#45577369 and running the last 3 tests on all my systems actually showed out-null as FASTER than casting to void or using =$void. — Collin Chaffin, Aug 08 '17 at 20:29
As for the file filtering, you're off to a great start can you perhaps edit your question with a bit more detail on the RULES like cannot pre-move/stage folders, etc. just so we are all clear on the requirements? — Collin Chaffin, Aug 08 '17 at 20:29
Question is updated now. Kindly give inputs as I am still struggling with this requirement. Thanks. — Channdeep Singh, Aug 09 '17 at 02:33
If you can't use different folders because your backup files are scattered across various subfolders I strongly recommend you re-evaluate your backup strategy. — Ansgar Wiechers, Aug 09 '17 at 08:03

kpogue · Answer 1 · 2017-08-09T22:31:41.447

Would add a comment, but I don't have the reputation for it.

First, I defined a function to get the "Last friday of month" piece, which seems impossible to do on a single line without a jumbled mess.

function LastFridayOfMonth([DateTime] $DateIn) 
{
    $numDaysArr = {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}
    return ($numDaysArr[$DateIn.Month] - $DateIn.Day) -lt 7 -and $DateIn.DayOfWeek -eq [System.DayOfWeek]::Friday
}

Then, it was easier for me to think of this with logic inverted, DeMorgans theorem could probably simplify the logic in my if statement, but it should work.

if (-not ($file.LastWriteTime -lt (Get-Date).AddYears(-1) -and $file.LastWriteTime -gt (Get-Date).AddMonths(-3) -and (LastFridayOfMonth $file.LastWriteTime)))
{
    Remove-Item $File.FullName | Out-Null
    Write-Host $File
}

Note, it doesn't account for leap years and I only tested the LastFridayOfMonth function, not the if statement logic. Check it out before you start removing files... The general idea is there for you though.

Channdeep Singh · Accepted Answer · 2017-08-10T12:17:13.247

Final script below.

The solution is not ideal as the script uses multiple loops, which sounds unnecessary, however, the requirement seems accomplished; and the script is tested fine.

Also the script has additional provision that, if in case, the backup file of Last-Friday-Of-Month is not available, then the script will keep the backup file of Second-last-Friday-of-moth still. Same way, if second-last is also not available, then it will keep third-last; and same for fourth and fifth week respectively. Thanks a lot to all of you.

#----- calculate start and end of current month ----#
$startofmonth = Get-Date -day 1 -hour 0 -minute 0 -second 0
$endofmonth = (($startofmonth).AddMonths(1).AddSeconds(-1))

#----- define folder where files are located ----#
$TargetFolder = "C:\Users\chsa\Desktop\Retention"

#----- define extension ----#
$Extension = "*.txt"

#----- get files based on lastwrite filter and specified folder ---#

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -lt $startofmonth.AddMonths(-13) } 

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-13) -and $_.LastWriteTime -le $endofmonth.AddMonths(-13) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-13) -and $_.LastWriteTime -le $endofmonth.AddMonths(-13) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-12) -and $_.LastWriteTime -le $endofmonth.AddMonths(-12) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-12) -and $_.LastWriteTime -le $endofmonth.AddMonths(-12) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-11) -and $_.LastWriteTime -le $endofmonth.AddMonths(-11) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-11) -and $_.LastWriteTime -le $endofmonth.AddMonths(-11) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-10) -and $_.LastWriteTime -le $endofmonth.AddMonths(-10) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-10) -and $_.LastWriteTime -le $endofmonth.AddMonths(-10) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-9) -and $_.LastWriteTime -le $endofmonth.AddMonths(-9) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-9) -and $_.LastWriteTime -le $endofmonth.AddMonths(-9) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-8) -and $_.LastWriteTime -le $endofmonth.AddMonths(-8) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-8) -and $_.LastWriteTime -le $endofmonth.AddMonths(-8) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-7) -and $_.LastWriteTime -le $endofmonth.AddMonths(-7) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-7) -and $_.LastWriteTime -le $endofmonth.AddMonths(-7) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-6) -and $_.LastWriteTime -le $endofmonth.AddMonths(-6) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-6) -and $_.LastWriteTime -le $endofmonth.AddMonths(-6) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-5) -and $_.LastWriteTime -le $endofmonth.AddMonths(-5) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-5) -and $_.LastWriteTime -le $endofmonth.AddMonths(-5) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-4) -and $_.LastWriteTime -le $endofmonth.AddMonths(-4) -and $_.LastWriteTime.DayofWeek -eq "Friday"} | Sort LastWriteTime -Desc | Select -Skip 1

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-4) -and $_.LastWriteTime -le $endofmonth.AddMonths(-4) -and $_.LastWriteTime.DayofWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-3) -and $_.LastWriteTime -le $endofmonth.AddMonths(-3) -and $_.LastWriteTime.DayOfWeek -ne "Friday"}


foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-2) -and $_.LastWriteTime -le $endofmonth.AddMonths(-2) -and $_.LastWriteTime.DayOfWeek -ne "Friday"}


foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth.AddMonths(-1) -and $_.LastWriteTime -le $endofmonth.AddMonths(-1) -and $_.LastWriteTime.DayOfWeek -ne "Friday"}


foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

$Files = Get-Childitem $TargetFolder -Include $Extension -Recurse | where {$_.LastWriteTime -ge $startofmonth -and $_.LastWriteTime -lt (Get-Date).AddDays(-8) -and $_.LastWriteTime.DayOfWeek -ne "Friday"}

foreach ($File in $Files) 
{
   if ($File -ne $NULL)
   {
     Remove-Item $File.FullName | out-null
     #write-host $File
   }

}

Script deployed in production and worked as intended. Thank you all. — Channdeep Singh, Aug 10 '17 at 12:18

Delete old files based on customized date filters

2 Answers2