PowerShell - Test Multiple URLs from TXT/CSV file and record HTTP Code into a CSV file for reporting

Question

Hoping someone can help with this.

I built a script based on this link

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

$urlStr = Read-Host "Please enter URL to check"
[uri]$urlStr

# First we create the request.
$HTTPS_Request = [System.Net.WebRequest]::Create("$urlStr")

# We then get a response from the site.
$HTTPS_Response = $HTTPS_Request.GetResponse()

# We then get the HTTP code as an integer.
$HTTPS_Status = [int]$HTTPS_Response.StatusCode
$HTTPS_StatusDesc = [string]$HTTPS_Response.StatusDescription

#Write-Host "HTTP CODE: $HTTPS_Status"

do { 
If ($HTTPS_Status -eq 301) {
    Write-Host "HTTP CODE: $HTTPS_Status"
    Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
    Write-Host "Landing page moved permanently and redirects to another URL."
    Write-Host "Please update Landing page to new URL"
}
ElseIf ($HTTPS_Status -eq 302) {
    Write-Host "HTTP CODE: $HTTPS_Status"
    Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
    Write-Host "If this occurs once, then no issues"
    Write-Host "If this occurs more than once, please update Landing page to new URL"
}
} while ($HTTPS_Status -ge 300 -and $HTTPS_Status -lt 400)

If ($HTTPS_Status -eq 200) {
    Write-Host "HTTP CODE: $HTTPS_Status"
    Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
    Write-Host "Landed on page"
}
ElseIf ($HTTPS_Status -gt 400) {
    Write-Host "HTTP CODE: $HTTPS_Status"
    Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
    Write-Host "Error - issue with Landing page. Please investigate."
}


# Finally, we clean up the http request by closing it.
$HTTPS_Response.Close()
$HTTPS_Response.Dispose()
#Read-Host -Prompt “Press Enter to exit”

Currently, the above is built to handle one URL at a time, which was fine for me as the amount of links and usage of the script was low overall so I left it as is.

But as the usage and URLs are increasing for the above script, I am hoping to have the code further modified into running multiple URLs at the same time.

My idea is to save the URLs into a TXT or CSV file and have it read, line-by-line, and run the script per line. It would then record the response and output the HTTP code (e.g. 200, 404, etc...) into a CSV (or the original CSV file) and input the data there as well.

If possible, I'd like to record the output from "$HTTP_Response" and add that in as well, but this would be a secondary objective.

Any help would be much appreciated.

Thanks.
Rajiv.

Mathias R. Jessen · Accepted Answer · 2020-06-19T14:43:20.337

First thing you wanna do is turn your script into a function, with the URL as a parameter!

function Get-HTTPResponseCode
{
  param(
    [Parameter(Mandatory = $true, Position = 0)]
    [uri]$Url,

    [switch]$Quiet
  )

  [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
  try {
    # First we create the request.
    $HTTPS_Request = [System.Net.WebRequest]::Create("$Url")

    # We then get a response from the site.
    $HTTPS_Response = $HTTPS_Request.GetResponse()

    # We then get the HTTP code as an integer.
    $HTTPS_Status = [int]$HTTPS_Response.StatusCode
    $HTTPS_StatusDesc = [string]$HTTPS_Response.StatusDescription
  }
  finally {
    # Finally, we clean up the http request by closing it.
    $HTTPS_Response.Close()
    $HTTPS_Response.Dispose()
  }

  #Write-Host "HTTP CODE: $HTTPS_Status"

  if(-not $Quiet){
    if ($HTTPS_Status -eq 301) {
      Write-Host "HTTP CODE: $HTTPS_Status"
      Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
      Write-Host "Landing page moved permanently and redirects to another URL."
      Write-Host "Please update Landing page to new URL"
    }
    elseif ($HTTPS_Status -eq 302) {
      Write-Host "HTTP CODE: $HTTPS_Status"
      Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
      Write-Host "If this occurs once, then no issues"
      Write-Host "If this occurs more than once, please update Landing page to new URL"
    }
    elseif ($HTTPS_Status -eq 200) {
      Write-Host "HTTP CODE: $HTTPS_Status"
      Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
      Write-Host "Landed on page"
    }
    elseif ($HTTPS_Status -gt 400) {
      Write-Host "HTTP CODE: $HTTPS_Status"
      Write-Host "HTTP CODE DESCRIPTION: $HTTPS_StatusDesc"
      Write-Host "Error - issue with Landing page. Please investigate."
    }
  }

  # return the response code
  return $HTTPS_Status
}

Now that we have an easily reusable function, we can do interesting things, like using it in a calculated property for example:

$URLs = @(
  'https://www.stackoverflow.com'
  'https://www.stackexchange.com'
)

$URLs |Select-Object @{Name='URL';Expression={$_}},@{Name='Status'; Expression={Get-HTTPResponseCode -Url $_}} |Export-Csv .\path\to\result.csv -NoTypeInformation

If you want to store the input URLs seperately, simply put them in a file, one per line, and then use Get-Content to read them from disk:

$URLs = Get-Content .\path\to\file\with\urls.txt

Hi Mathias, is there a way for me to add the URLs into a TXT file and have your modified script to read it line-by-line, and output to the CSV file? — Rajiv Ahmed, Jun 19 '20 at 13:44
@RajivAhmed Absolutely, added example of reading URLs from file with `Get-Content` — Mathias R. Jessen, Jun 19 '20 at 13:54
Hi @Mathias, I've tested the above and although the script runs, but the output file is only showing the inputted URLs from either method, and the status code column is blank. Am I doing something wrong? — Rajiv Ahmed, Jun 19 '20 at 14:18
Thanks for that. I missed that as well. Thought it was a more complicated problem... Would it be possible to get the output of this line: "$HTTPS_Response = $HTTPS_Request.GetResponse()" and add it to the CSV? I tried modifying the code but it doesn't give what I want - states this instead "System.Net.HttpWebResponse" in the CSV. — Rajiv Ahmed, Jun 19 '20 at 15:04

PowerShell - Test Multiple URLs from TXT/CSV file and record HTTP Code into a CSV file for reporting

1 Answers1