0

Here is my code:

import fetch from "node-fetch";
    
    
    class BlueNileAPI {
        constructor() {
        }
    
        fetchDiamonds() {
            return fetch(`https://www.bluenile.com/api/public/diamond-search-grid/v2?country=USA&language=en-us&currency=USD&startIndex=0&pageSize=243&shape=RD&sortColumn=price&sortDirection=asc&_=1468875090671`)
            .then((response) => response.json())
        }
    }
    
       
    export default BlueNileAPI;
    
    const api = new BlueNileAPI()
    const data = await api.fetchDiamonds()
    console.log(await data)

When using that API directly in the browser it works. However, it Node it returns:

<!DOCTYPE html><html><head></head><body><script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/ips.js?bn_uuid=0NiTEI3uwRCgPHHeG2qvD3dHsjIUrQKSGN2NMm785Vujeni4eslOsJuC5fLRIBsbETiUc1FpHWtvPzoysdhVFdJvm6Bb4ROeSb7Yrd6zlABTkWa8XUMiMNen2NUYHrTuIC0vAcYfhPotxAjJrZbZxGD9i"></script></body></html>
^

SyntaxError: Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at Response.json (file:.....body.js:149:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async file:.....BlueNileApi.js:18:14

Any pointers appreciated.

boioboi
  • 45
  • 7
  • Different user agent or HTTP headers triggering different (HTML) response? – jarmod Jul 28 '22 at 14:59
  • I presume there is a setting I need to change or something to add to code with the fetch. Do you have any idea about where I should like to try and see why the API doesn't like my request? – boioboi Jul 28 '22 at 15:40
  • First, I would send `Accept: application/json` HTTP header with your node request. See if that triggers a JSON response. Then I would debug the HTML response to your node request. It might be an error page, for example, that explains the problem. Otherwise debug the network request from your browser, see if you can determine which headers are being sent, and add those one-by-one to your node request in the hope of finding what is triggering the JSON response. – jarmod Jul 28 '22 at 15:47
  • Tried the accept, didn't work. I have just looked at the HTML response which returns: status: 429, statusText: 'Too Many Requests', headers: { 'cache-control': 'no-cache, no-store, must-revalidate', connection: 'Close', 'content-encoding': 'br', 'content-length': '253', 'content-type': 'text/html; charset=utf-8', – boioboi Jul 28 '22 at 15:50
  • Related: https://stackoverflow.com/questions/51638468/web-scraping-with-python-using-beautifulsoup-429-error – jarmod Jul 28 '22 at 16:05

0 Answers0