1

I have a Discord bot that makes some API calls. When I run the bot from my computer, it all works fine, however when I push my code to Heroku (where I am hosting my bot) the API call returns this error:

UnhandledPromiseRejectionWarning: FetchError: invalid json response body at https://sky.shiiyu.moe/api/v2/profile/SirArchibald97 reason: Unexpected token < in JSON at position 0
2021-01-07T11:45:02.931529+00:00 app[worker.1]:     at /app/node_modules/node-fetch/lib/index.js:272:32
2021-01-07T11:45:02.931532+00:00 app[worker.1]:     at processTicksAndRejections (internal/process/task_queues.js:97:5)
2021-01-07T11:45:02.931532+00:00 app[worker.1]:     at async Object.execute (/app/methods/link.js:93:20)
2021-01-07T11:45:02.931532+00:00 app[worker.1]:     at async Client.<anonymous> (/app/index.js:47:28)
2021-01-07T11:45:02.931832+00:00 app[worker.1]: (node:4) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
2021-01-07T11:45:02.931970+00:00 app[worker.1]: (node:4) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Edit: I am requesting two different APIs in my code, using the node-fetch npm module. This is the one which returns the error:

let res = await fetch(`https://sky.shiiyu.moe/api/v2/profile/${player.displayname}`);
let data = await res.json();

And this is the one which works fine:

let p_response = await fetch(`https://api.hypixel.net/player?key=${api_key}&name=${args[1]}`);
let { player } = await p_response.json();

Edit 2: I have switched from node-fetch to axios in the hope that it might work, and I am now recieving a 503 status error instead. Once again, it works fine on my machine, but returns the error when I deploy to Heroku. Here is the new code:

const axios = require("axios");
let res = await axios.get(`https://sky.shiiyu.moe/api/v2/profile/${player.displayname}`);
let data = res.data;
Mohammad Yaser Ahmadi
  • 4,664
  • 3
  • 17
  • 39
SirArchibald
  • 476
  • 2
  • 7
  • 23
  • `invalid json response body at https://sky.shiiyu.moe/api/v2/profile/SirArchibald97 reason: Unexpected token < in JSON at position 0`—it looks like you're getting HTML or XML or something instead of JSON. Take a look at the response without trying to parse it as JSON. It may be an error page or something. Also, check the HTTP response code you get. – ChrisGPT was on strike Jan 07 '21 at 20:59
  • I know it’s returning HTML, but I’m not sure why, as the request returns JSON when I run the bot from my system. I have logged the response to the console and it returns the same thing as the JSON but as one long string of text rather than an object. – SirArchibald Jan 08 '21 at 08:35
  • Can you provide a [mcve] of an API request? Are you setting an accept header? – ChrisGPT was on strike Jan 08 '21 at 13:27
  • @Chris I have just edited my question to include the examples you wanted. And no, I am not sending an accept header as I am not entirely sure how. – SirArchibald Jan 08 '21 at 13:38
  • have you checked which request headers are being set in both instances? – QHarr Jan 09 '21 at 23:45
  • To set an accept header see the `accept` line in the answer [here](https://stackoverflow.com/questions/29775797/fetch-post-json-data/29823632#29823632). The HTML you're receiving may be the 503 status page if it's behind something like Nginx or IIS. – Zac Anger Jan 10 '21 at 00:12
  • I set an accept header and a content-type header (both to `applicaton/json`) but I am still getting the error – SirArchibald Jan 10 '21 at 16:05
  • Are you authenticating somehow? – ChrisGPT was on strike Jan 10 '21 at 17:46
  • @Chris I am not sure what you mean by authenticating, I'm rather new to APIs. – SirArchibald Jan 10 '21 at 19:52
  • Well, the API example you give that's working asks for an API key. The one that's failing doesn't. Are you authenticating some other way, e.g. with a bearer token? Or is this a public API that anybody is able to freely use, without limit? – ChrisGPT was on strike Jan 10 '21 at 20:17
  • @Chris No, the API that isn't working doesn't need a key to use. – SirArchibald Jan 10 '21 at 20:54
  • Can you try to do the associated curl request from your computer and from your heroku server ? as heroku share some ip it might be simply an ip block/ban from the remote server – Daphoque Jan 12 '21 at 10:27
  • @Daphoque sorry, I'm quite new to this sort of stuff, I'm not sure what a curl request is? – SirArchibald Jan 13 '21 at 09:28
  • curl is just an http requester, but if you don't know it we will do in another way. Can you display the raw text response receive from fetch/axios, with fetch: console.log(res.text()), with axios: res.headers['content-type']; console.log(res.data). Try to display the two if possible :) – Daphoque Jan 13 '21 at 11:13
  • do you have somewhere the code base to take a look? – LastM4N Jan 13 '21 at 11:43
  • Please check that request/response in your browser. What is the client sending and receiving? Can you post that here? Anyways: I always handle errors as a general rule of thumb. Look at this, the checked answer give you a pretty nice solution: https://stackoverflow.com/questions/40370399/catching-errors-in-typescript-promises – Janos Vinceller Jan 15 '21 at 20:31
  • You can easily check your requests and responses for example in Google Chrome. Just push F12 and open the Developer Tools and switch to the Network Tab. Then you'll click on the request to the https://sky.shiiyu.moe/api/v2/profile/ URL and look at differences in your local setup and the heroku setup. – Janos Vinceller Jan 15 '21 at 20:33

3 Answers3

1

I checked the API you are using, using CURL.

curl "https://sky.shiiyu.moe/api/v2/profile/SirArchibald97"

HTTP/1.1 503 Service Temporarily Unavailable
Date: Sat, 16 Jan 2021 13:27:30 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Set-Cookie: __cfduid=dfef31ed9a3326383a42315e646837f531610803650; expires=Mon, 15-Feb-21 13:27:30 GMT; path=/; domain=.shiiyu.moe; HttpOnly; SameSite=Lax; Secure
X-Frame-Options: SAMEORIGIN
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 01 Jan 1970 00:00:01 GMT
cf-request-id: 07acfa3991000041f79a8d8000000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=DfiyBY1Zc4DC%2Bxzh2Ug61Brj2X5zQ4Ae5VlhYj8Ci7bRKlaJGGTO9PFZuXqXYyOEIiWKdM2n8ykJlwf3Y79zCUxIire2LUy%2BzAtDGAxqcg%3D%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 61282ca2893341f7-MRS

I got an HTML response. It was CloudFlare protection. It is probably working in your local machine due to CloudFlare cookies.

This is a issue with the API configuration. You need to reach them and ask to whitelist you IP or you would have to use some sort of browser emulator like selenium to bypass CloudFlare firewall.

Abkarino
  • 1,426
  • 1
  • 12
  • 19
  • 1
    Ahh ok, thank you for figuring it out. I have been stuck with this issue for a while. – SirArchibald Jan 16 '21 at 16:09
  • 1
    Great, glad to help. – Abkarino Jan 16 '21 at 23:43
  • Thanks @Abkarino for the direction. I’m facing a similar Cloudflare. It shows a captcha page when i do a GET reques to a website from Zapier. Quick question: By “you need to reach them” you mean whom? Cloudflare? – AnupamChugh Jan 06 '22 at 21:11
  • 1
    @AnupamChugh you need to contact whom setup cloudflare on the host. If it is automatic, they you should find a place to setup whitelist in your Heroku dashboard or contact Heroku customer support. Zapier has an ip range which you are told to use for whitelisting. – Abkarino Jan 06 '22 at 23:30
0

Sending request with default axios config works well, so your issue related to:

  • axios config
  • heroku config

const player = { displayname: 'voice' };

const sendReq = async () => {
  let res = await axios.get(`https://sky.shiiyu.moe/api/v2/profile/${player.displayname}`);
  console.log(res.data)
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.21.1/axios.min.js"></script>
<button onclick="sendReq()">Send request</button>
Alexandr Tovmach
  • 3,071
  • 1
  • 14
  • 31
0

I hope the workaround here solves the issue GET request returns index.html doc instead of json data

Let me know!

LastM4N
  • 1,890
  • 1
  • 16
  • 22