I am trying to download a file from an API, which works perfectly well in chrome using the following Url:
However, when I try to download this in a c# script in SSIS using either HTTPWebRequest or Webclient.Downloadfile it fails (404 response error).
After many hours scouring Stackoverflow, trying numerous approaches, I have found that both approaches do work if I remove part of the Url:
Why does the inclusion of ".CP_EUR_HAB.NSA.B1GQ." make it fail in c# but not in chrome?
The code I am using:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(sourceFilePath));
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (Stream stream = response.GetResponseStream())
using (Stream file = File.Create(savePathAndName))
{
CopyStream(stream, file);
}
public static void CopyStream(Stream input, Stream output)
{
byte[] buffer = new byte[8 * 1024];
int len;
while ((len = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, len);
}
}
UPDATE Originally, SSIS was omitted, as I thought it would be irrelevant, however, after the comment saying that they could get it working, I tested what the Uri is returning, and it is removing the last dot before the question mark:
There are many documented difficulties in getting these dot segments included: just a couple include:
Is there a way to keep dot segments in url using Uri class? and A url resource that is a dot (%2E)
The API returns a failure if that dot isn't there. And microsoft confirms this:
https://learn.microsoft.com/en-us/dotnet/api/system.uri?view=net-5.0
After some investigation, it seems that this might be to do with the framework, and updating it might be the answer, which might suggest why others could get this working, but not me in SSIS. The framework that SSIS 2017 is using is bound to 4.5. I'm thinking that I might be at a dead-end with this.
