0

I'm trying to download .webp image, and it always returns 404, but if I hardcode URI or smply open it in browser, it returns 200 with image.

Details:

string uri1 = "https://some.site/static/.../.../0.webp";
string uri2 = parsedApiResponse;
var response1 = client.GetAsync(uri1).Result;
var response2 = client.GetAsync(uri2).Result;
Log($"{response1.StatusCode}\n", ConsoleColor.Magenta);
Log($"{response2.StatusCode}\n", ConsoleColor.Yellow);

StatusCode

parsedApiResponse contains string reply from API (it saves image on server and returns it's location) with full path leading to .webp image.

uri1 contains hardcoded parsedApiResponse with handly copied path to other image from exactly same earlier call.

response1 returns exactly what it should when response2, from absoletely same request with not hardcoded URI, returns... this (Fiddler screenshots:)

raw inspection of successful request

raw inspection of fail request

It's absolutely same! There's no custom headers, no content, but response is different. And if I'll log and copy path of uri2 and then will paste it on place of uri1, it will turn the same way.

Content of response2 is a full cloudflare 404 page, but main part says this:

<section id="text">
  <div>
    <h1>Error 404</h1>
    <h3>This object could not be viewed</h3>
  </div>

  <div>
    <p id="error-title">**You are not authorized to view this object**</p>
    <p>
            This object does not exist or is not publicly accessible at this
            URL. Check the URL of the object that you're looking for or contact
            the owner to enable Public access.
    </p>
  </div>

  <div>
    <p id="footer-title">Is this your bucket?</p>
    <p>
      Learn how to enable
      <a
        href="https://developers.cloudflare.com/r2/data-access/public-buckets/">Public Access
      </a>
    </p>
  </div>
</section>

I've already tried to play with some HttpClientHandler params, tried to use other clients like RestClient and WebClient, tried to pass all possible headers to mimic my browser and many-many other stuff - no result. What can it be? Why does it require some authorize? How does hardcoded URI get's these credentials?..

Palle Due
  • 5,929
  • 4
  • 17
  • 32
Arc
  • 43
  • 5
  • 1
    `"https://some.site/static/.../.../0.webp";` You need character escapes `"https:////some.site//static//...//...//0.webp";` or use `@` like this `@"https://some.site/static/.../.../0.webp";`. Your string gets modified and that URL doesn't exist so you get `404` not found. – ThisQRequiresASpecialist Dec 20 '22 at 11:42
  • Also using `.Result` is a very bad practice you should `await` the response otherwise there could also be issues. They're called `Deadlocks` you can read about it in this SO [post](https://stackoverflow.com/questions/17248680/await-works-but-calling-task-result-hangs-deadlocks). – ThisQRequiresASpecialist Dec 20 '22 at 11:45
  • Oh, you miss the poing. Hardcoded string returns 200, without any escaping. It just works. And it crashes with exception with `await` - that's why I use `.Result` for demonstration. – Arc Dec 20 '22 at 11:47
  • Yours successful 200 result has ETag header, failed response with no ETag. Seems like some of CORS or Access-Control-Allow-Origin response with ETag issue in source host with cross origin resource access protected. – Andrew Dec 20 '22 at 11:49
  • Can you possibly post the entire class / entire method so we can take a look closer and give the best possible answer with different code which uses best practice? Would be really nice. – ThisQRequiresASpecialist Dec 20 '22 at 11:51
  • @ThisQRequiresASpecialist Well, there's actually nothing outside of this code block that could relate to the issue... – Arc Dec 20 '22 at 12:13
  • From design, method should look like this: `public static byte[] DownloadImg(string url)` `{` `HttpClient client = new();` `byte[] img = client.GetByteArrayAsync(url).Result;` `return img;` `}` And it should be called in async Task so it could be processed further: `string imgPath = ...; // gets param from deserialized json` `using Stream mc = new MemoryStream(DownloadImg(imgPath));` `using var tempImg = File.Create("some_image.webp");` `mc.CopyTo(tempImg);` – Arc Dec 20 '22 at 12:13
  • 1
    @Andrew yeah, you was right. It was server side problem. I absolutely forgot that sometimes images need time to be fully uploaded on server. Just looped request call in try-catch loop with a little Thread.Sleep, and after ~ten seconds of waiting I got my 200. Thx for tip✌️ – Arc Dec 20 '22 at 13:07
  • @Arc consider posting your solution as an answer. There may be value for the community. – Kit Dec 20 '22 at 17:28

1 Answers1

0

It was server side problem.

For some reason, server created image path and responded with it, before image itself was fully uploaded. That's the reason why it was Not found. If you encounter similar problem, first of all try to make multiple requests with some delay in time:

public static async Task<byte[]?> DownloadImg(string url)
{
    HttpClient client = new();

    for (int i = 0; i < 10; i++)
    {
        try { return await client.GetByteArrayAsync(url); }
        catch { Thread.Sleep(2000); }
    }

    return null;
}
Arc
  • 43
  • 5