1

I am using Jenkins, in a PowerShell command step, to call a Web API service. I am calling the service using Invoke-WebRequest.

This service call has to be in Jenkins, because I need to do it only if other Jenkins jobs have completed successfully.

The service can run for several hours. If I don't have the TimeoutSec parameter, then the step completes with a timeout, before the Web API has completed. If I have the parameter with a large value, such as 36000, then the Web API completes normally, but the step goes on for 10 hours. The other parameters have no relation to this issue.

I am seeking a good way to have the Jenkins step complete as soon as the Web API completes, not earlier and not later.

try
{ 
    $url = "https://ourserver.com/modules/OurService"
    $response = Invoke-WebRequest -Uri $url -UseDefaultCredentials -Method Get -TimeoutSec 36000 -UseBasicParsing  
}
catch 
{    
    $err=$_.Exception  
    Write-Host '-----------------------'
    Write-Host $err        
    Write-Host '-----------------------'  
    exit -1
}
exit 0
MikeDev
  • 357
  • 1
  • 2
  • 20
  • Can you add more information about your job definition in Jenkins? Maybe try setting the `MaxServicePointIdleTime`? https://stackoverflow.com/a/34139528/7411885 – Cpt.Whale Nov 19 '21 at 19:11
  • How do you determine that the "Web API completes normally"? Maybe it just does its job, but doesn't actually return a response? – zett42 Nov 19 '21 at 20:19
  • Cpt.Whale - this job only contains two steps: Build Trigger and a Build. I have no problem defining a fixed period after which a timeout happens. I just don't know how to exit exactly when the Web API is complete (and not before). – MikeDev Nov 19 '21 at 20:51
  • zett42 - my Web API is returning a 200 (OK) when success (I am pretty confident is doing that since PostMan indicates this). Should I be checking for a 200 value in the $response somehow? It seems to be a sync call to Invoke-WebRequest that never gets to the 'exit 0' when complete. – MikeDev Nov 19 '21 at 20:57
  • If `$response` is getting set to 200, the very next thing *should* already be `exit`. If the request literally took the whole timeout time, then the command should throw a timeout error instead of exiting normally. – Cpt.Whale Nov 19 '21 at 21:16
  • Isn't this a bug in `Invoke-WebRequest`? Because there are 3 options: 1) end successfully, 2) end with an error 3) end after timeout if the request is still [_pending_](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-7.2#parameters). But, if the request is or has been processed already the 3rd option should be completely out of play, shouldn't it? – Gerold Broser Nov 20 '21 at 22:25
  • @MikeDev Why don't you just use the [HTTP Request Plugin](https://www.jenkins.io/doc/pipeline/steps/http_request/)? – Noam Helmer Nov 21 '21 at 07:54
  • Cpt. Whale - the response is definitely getting set to 200 within the web-api code. The request (Invoke-WebRequest) is taking the whole time period and is throwing a timeout error when TimeoutSec is elapsed. – MikeDev Nov 22 '21 at 15:06
  • Gerold Broser - I agree, I think this is a bug in Invoke-WebRequest. Noah Helmer - I will look into Http Request Plug-In as an option. Thanks all. – MikeDev Nov 22 '21 at 15:08
  • @GeroldBroser that is correct, but the Http request plugin offers an easy, more readable alternative for the Powershell step, and is much easier to integrate into his Jenkins pipeline - and therefore should be considered as an efficient alternative. – Noam Helmer Nov 23 '21 at 08:59
  • @NoamHelmer You are too fast. :) I removed my previous comment less than a minute after I wrote it since I wanted to check if there is a HTTP Request step on a project's config page. – Gerold Broser Nov 23 '21 at 09:11
  • The webapi that is being called is very long running (multiple hours). I think that adds a layer of complexity. I tried HTTP Request step but that step never completed (after running overnight). Also, it took a very long time to be able to cancel the very long running job. I am back to working with Invoke-WebRequest (for now). One thing I am doing differently: checking "Stop On Errors" on the Build Step. – MikeDev Nov 23 '21 at 16:05
  • @NoamHelmer And there is: https://plugins.jenkins.io/http_request/#configure-build-step-in-your-jenkins-job. – Gerold Broser Nov 23 '21 at 19:53
  • Is using an **Exceute shell** build step with [`curl`](https://www.mit.edu/afs.new/sipb/user/ssen/src/curl-7.11.1/docs/curl.html) or [`wget`](https://linux.die.net/man/1/wget) an option? – Gerold Broser Nov 23 '21 at 19:59
  • Powershell, Invoke-WebRequest: the code within the webapi completed, but Jenkins job never did, but I was at least able to cancel the job. HTTP Request: never completed and Jenkins had to be restarted to cancel the job. I think the issue is that my webapi is so long running (1-2 hours) that it is in the realm of the unusual. I am going to next look at options such as curl, wget, Groovy or something in an Execute Shell step. Thanks all for your feedback and suggestions thus far. – MikeDev Nov 24 '21 at 18:08

1 Answers1

0

I tried a different approach, and I think I am closer to where I want to be now...

I decided to go with Invoke-RestMethod

A strange issue remains: when my long running service is called and that service completes, I fall down into the catch, but the Exception has nothing in it. Per the logs for my service, I know the service completed successfully. So, at this point I am treating it as successful.

try
{
  $url = "https://ourserver.com/modules/OurService"
  $response = Invoke-RestMethod -Uri $url -UseDefaultCredentials -Method Get -TimeoutSec 18000
  $response  
  Write-Host 'Exit with success'
  exit 0
}
catch
{
  if (($_.Exception) -and ($_.Exception.Response))
  {
    $respStream = $_.Exception.Response.GetResponseStream()
    $reader = New-Object System.IO.StreamReader($respStream)
    $reader.BaseStream.Position = 0
    $responseBody = $reader.ReadToEnd() | ConvertFrom-Json
    '------------------------------------------------------------------------------------------------------------------------' 
    'Error Message:'
    $responseBody.Message
    $_.Exception
    '------------------------------------------------------------------------------------------------------------------------'
    exit -1
  }
  else
  {
    '------------------------------------------------------------------------------------------------------------------------' 
    'Got to catch, but no exception (we should not have gotten to "catch" area, possible PowerShell bug)'  
    '------------------------------------------------------------------------------------------------------------------------'
    exit 0
  }
  
}
MikeDev
  • 357
  • 1
  • 2
  • 20