4

I am trying to fetch the list of torrent links from nyaa.si RSS files. I am using Axios to fetch data from nyaa.si. I am trying to host the app on replit it works on other places but it doesn't work on replit. I wonder if replit has blacklisted nyaa.si.

My sample code is as below

import axios from 'axios'
const query="Kimetsu no yaiba 1080p"
function getData(query){
axios.get('https://nyaa.si/',{
            params: {
                page: 'rss',
                q:query}
            }).then(resp => {

    console.log(resp.data)
});
}

getData(query)

It throws error as below

node:internal/process/promises:246
          triggerUncaughtException(err, true /* fromPromise */);
          ^
[AxiosError: connect ECONNREFUSED 0.0.0.0:443] {
  port: 443,
  address: '0.0.0.0',
  syscall: 'connect',
  code: 'ECONNREFUSED',
  errno: -111,
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [Function: httpAdapter],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 0,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    env: {
      FormData: [Function: FormData] {
        LINE_BREAK: '\r\n',
        DEFAULT_CONTENT_TYPE: 'application/octet-stream'
      }
    },
    validateStatus: [Function: validateStatus],
    headers: {
      Accept: 'application/json, text/plain, */*',
      'User-Agent': 'axios/0.27.2'
    },
    params: { page: 'rss', q: 'Kimetsu no yaiba 1080p' },
    method: 'get',
    url: 'https://nyaa.si/',
    data: undefined
  },
  request: <ref *3> Writable {
    _writableState: WritableState {
      objectMode: false,
      highWaterMark: 16384,
      finalCalled: false,
      needDrain: false,
      ending: false,
      ended: false,
      finished: false,
      destroyed: false,
      decodeStrings: true,
      defaultEncoding: 'utf8',
      length: 0,
      writing: false,
      corked: 0,
      sync: true,
      bufferProcessing: false,
      onwrite: [Function: bound onwrite],
      writecb: null,
      writelen: 0,
      afterWriteTickInfo: null,
      buffered: [],
      bufferedIndex: 0,
      allBuffers: true,
      allNoop: true,
      pendingcb: 0,
      constructed: true,
      prefinished: false,
      errorEmitted: false,
      emitClose: true,
      autoDestroy: true,
      errored: null,
      closed: false,
      closeEmitted: false,
      [Symbol(kOnFinished)]: []
    },
    _events: [Object: null prototype] {
      response: [Function: handleResponse],
      error: [Function: handleRequestError],
      socket: [Function: handleRequestSocket]
    },
    _eventsCount: 3,
    _maxListeners: undefined,
    _options: {
      maxRedirects: 21,
      maxBodyLength: 10485760,
      protocol: 'https:',
      path: '/?page=rss&q=Kimetsu+no+yaiba+1080p',
      method: 'GET',
      headers: {
        Accept: 'application/json, text/plain, */*',
        'User-Agent': 'axios/0.27.2'
      },
      agent: undefined,
      agents: { http: undefined, https: undefined },
      auth: undefined,
      hostname: 'nyaa.si',
      port: null,
      nativeProtocols: {
        'http:': {
          _connectionListener: [Function: connectionListener],
          METHODS: [
            'ACL',         'BIND',       'CHECKOUT',
            'CONNECT',     'COPY',       'DELETE',
            'GET',         'HEAD',       'LINK',
            'LOCK',        'M-SEARCH',   'MERGE',
            'MKACTIVITY',  'MKCALENDAR', 'MKCOL',
            'MOVE',        'NOTIFY',     'OPTIONS',
            'PATCH',       'POST',       'PROPFIND',
            'PROPPATCH',   'PURGE',      'PUT',
            'REBIND',      'REPORT',     'SEARCH',
            'SOURCE',      'SUBSCRIBE',  'TRACE',
            'UNBIND',      'UNLINK',     'UNLOCK',
            'UNSUBSCRIBE'
          ],
          STATUS_CODES: {
            '100': 'Continue',
            '101': 'Switching Protocols',
            '102': 'Processing',
            '103': 'Early Hints',
            '200': 'OK',
            '201': 'Created',
            '202': 'Accepted',
            '203': 'Non-Authoritative Information',
            '204': 'No Content',
            '205': 'Reset Content',
            '206': 'Partial Content',
            '207': 'Multi-Status',
            '208': 'Already Reported',
            '226': 'IM Used',
            '300': 'Multiple Choices',
            '301': 'Moved Permanently',
            '302': 'Found',
            '303': 'See Other',
            '304': 'Not Modified',
            '305': 'Use Proxy',
            '307': 'Temporary Redirect',
            '308': 'Permanent Redirect',
            '400': 'Bad Request',
            '401': 'Unauthorized',
            '402': 'Payment Required',
            '403': 'Forbidden',
            '404': 'Not Found',
            '405': 'Method Not Allowed',
            '406': 'Not Acceptable',
            '407': 'Proxy Authentication Required',
            '408': 'Request Timeout',
            '409': 'Conflict',
            '410': 'Gone',
            '411': 'Length Required',
            '412': 'Precondition Failed',
            '413': 'Payload Too Large',
            '414': 'URI Too Long',
            '415': 'Unsupported Media Type',
            '416': 'Range Not Satisfiable',
            '417': 'Expectation Failed',
            '418': "I'm a Teapot",
            '421': 'Misdirected Request',
            '422': 'Unprocessable Entity',
            '423': 'Locked',
            '424': 'Failed Dependency',
            '425': 'Too Early',
            '426': 'Upgrade Required',
            '428': 'Precondition Required',
            '429': 'Too Many Requests',
            '431': 'Request Header Fields Too Large',
            '451': 'Unavailable For Legal Reasons',
            '500': 'Internal Server Error',
            '501': 'Not Implemented',
            '502': 'Bad Gateway',
            '503': 'Service Unavailable',
            '504': 'Gateway Timeout',
            '505': 'HTTP Version Not Supported',
            '506': 'Variant Also Negotiates',
            '507': 'Insufficient Storage',
            '508': 'Loop Detected',
            '509': 'Bandwidth Limit Exceeded',
            '510': 'Not Extended',
            '511': 'Network Authentication Required'
          },
          Agent: [Function: Agent] { defaultMaxSockets: Infinity },
          ClientRequest: [Function: ClientRequest],
          IncomingMessage: [Function: IncomingMessage],
          OutgoingMessage: [Function: OutgoingMessage],
          Server: [Function: Server],
          ServerResponse: [Function: ServerResponse],
          createServer: [Function: createServer],
          validateHeaderName: [Function: __node_internal_],
          validateHeaderValue: [Function: __node_internal_],
          get: [Function: get],
          request: [Function: request],
          maxHeaderSize: [Getter],
          globalAgent: [Getter/Setter]
        },
        'https:': {
          Agent: [Function: Agent],
          globalAgent: Agent {
            _events: [Object: null prototype],
            _eventsCount: 2,
            _maxListeners: undefined,
            defaultPort: 443,
            protocol: 'https:',
            options: [Object: null prototype],
            requests: [Object: null prototype] {},
            sockets: [Object: null prototype],
            freeSockets: [Object: null prototype] {},
            keepAliveMsecs: 1000,
            keepAlive: false,
            maxSockets: Infinity,
            maxFreeSockets: 256,
            scheduling: 'lifo',
            maxTotalSockets: Infinity,
            totalSocketCount: 1,
            maxCachedSessions: 100,
            _sessionCache: [Object],
            [Symbol(kCapture)]: false
          },
          Server: [Function: Server],
          createServer: [Function: createServer],
          get: [Function: get],
          request: [Function: request]
        }
      },
      pathname: '/',
      search: '?page=rss&q=Kimetsu+no+yaiba+1080p'
    },
    _ended: true,
    _ending: true,
    _redirectCount: 0,
    _redirects: [],
    _requestBodyLength: 0,
    _requestBodyBuffers: [],
    _onNativeResponse: [Function (anonymous)],
    _currentRequest: <ref *1> ClientRequest {
      _events: [Object: null prototype] {
        response: [Function: bound onceWrapper] {
          listener: [Function (anonymous)]
        },
        abort: [Function (anonymous)],
        aborted: [Function (anonymous)],
        connect: [Function (anonymous)],
        error: [Function (anonymous)],
        socket: [Function (anonymous)],
        timeout: [Function (anonymous)]
      },
      _eventsCount: 7,
      _maxListeners: undefined,
      outputData: [],
      outputSize: 0,
      writable: true,
      destroyed: false,
      _last: true,
      chunkedEncoding: false,
      shouldKeepAlive: false,
      maxRequestsOnConnectionReached: false,
      _defaultKeepAlive: true,
      useChunkedEncodingByDefault: false,
      sendDate: false,
      _removedConnection: false,
      _removedContLen: false,
      _removedTE: false,
      _contentLength: 0,
      _hasBody: true,
      _trailer: '',
      finished: true,
      _headerSent: true,
      _closed: false,
      socket: <ref *2> TLSSocket {
        _tlsOptions: {
          allowHalfOpen: undefined,
          pipe: false,
          secureContext: SecureContext { context: SecureContext {} },
          isServer: false,
          requestCert: true,
          rejectUnauthorized: true,
          session: undefined,
          ALPNProtocols: undefined,
          requestOCSP: undefined,
          enableTrace: undefined,
          pskCallback: undefined,
          highWaterMark: undefined,
          onread: undefined,
          signal: undefined
        },
        _secureEstablished: false,
        _securePending: false,
        _newSessionPending: false,
        _controlReleased: true,
        secureConnecting: true,
        _SNICallback: null,
        servername: null,
        alpnProtocol: null,
        authorized: false,
        authorizationError: null,
        encrypted: true,
        _events: [Object: null prototype] {
          close: [
            [Function: onSocketCloseDestroySSL],
            [Function],
            [Function: onClose],
            [Function: socketCloseListener]
          ],
          end: [ [Function: onConnectEnd], [Function: onReadableStreamEnd] ],
          newListener: [Function: keylogNewListener],
          connect: [ [Function], [Function], [Function] ],
          secure: [Function: onConnectSecure],
          session: [Function (anonymous)],
          free: [Function: onFree],
          timeout: [Function: onTimeout],
          agentRemove: [Function: onRemove],
          error: [Function: socketErrorListener],
          drain: [Function: ondrain]
        },
        _eventsCount: 11,
        connecting: false,
        _hadError: true,
        _parent: null,
        _host: 'nyaa.si',
        _readableState: ReadableState {
          objectMode: false,
          highWaterMark: 16384,
          buffer: BufferList { head: null, tail: null, length: 0 },
          length: 0,
          pipes: [],
          flowing: true,
          ended: false,
          endEmitted: false,
          reading: true,
          constructed: true,
          sync: false,
          needReadable: true,
          emittedReadable: false,
          readableListening: false,
          resumeScheduled: false,
          errorEmitted: true,
          emitClose: false,
          autoDestroy: true,
          destroyed: true,
          errored: [Error: connect ECONNREFUSED 0.0.0.0:443] {
            errno: -111,
            code: 'ECONNREFUSED',
            syscall: 'connect',
            address: '0.0.0.0',
            port: 443
          },
          closed: true,
          closeEmitted: true,
          defaultEncoding: 'utf8',
          awaitDrainWriters: null,
          multiAwaitDrain: false,
          readingMore: false,
          dataEmitted: false,
          decoder: null,
          encoding: null,
          [Symbol(kPaused)]: false
        },
        _maxListeners: undefined,
        _writableState: WritableState {
          objectMode: false,
          highWaterMark: 16384,
          finalCalled: false,
          needDrain: false,
          ending: false,
          ended: false,
          finished: false,
          destroyed: true,
          decodeStrings: false,
          defaultEncoding: 'utf8',
          length: 155,
          writing: true,
          corked: 0,
          sync: false,
          bufferProcessing: false,
          onwrite: [Function: bound onwrite],
          writecb: [Function: bound onFinish],
          writelen: 155,
          afterWriteTickInfo: null,
          buffered: [],
          bufferedIndex: 0,
          allBuffers: true,
          allNoop: true,
          pendingcb: 1,
          constructed: true,
          prefinished: false,
          errorEmitted: true,
          emitClose: false,
          autoDestroy: true,
          errored: [Error: connect ECONNREFUSED 0.0.0.0:443] {
            errno: -111,
            code: 'ECONNREFUSED',
            syscall: 'connect',
            address: '0.0.0.0',
            port: 443
          },
          closed: true,
          closeEmitted: true,
          [Symbol(kOnFinished)]: []
        },
        allowHalfOpen: false,
        _sockname: null,
        _pendingData: 'GET /?page=rss&q=Kimetsu+no+yaiba+1080p HTTP/1.1\r\n' +
          'Accept: application/json, text/plain, */*\r\n' +
          'User-Agent: axios/0.27.2\r\n' +
          'Host: nyaa.si\r\n' +
          'Connection: close\r\n' +
          '\r\n',
        _pendingEncoding: 'latin1',
        server: undefined,
        _server: null,
        ssl: null,
        _requestCert: true,
        _rejectUnauthorized: true,
        parser: null,
        _httpMessage: [Circular *1],
        [Symbol(res)]: TLSWrap {
          _parent: TCP {
            reading: [Getter/Setter],
            onconnection: null,
            [Symbol(owner_symbol)]: [Circular *2],
            [Symbol(handle_onclose)]: [Function: done]
          },
          _parentWrap: undefined,
          _secureContext: SecureContext { context: SecureContext {} },
          reading: false,
          onkeylog: [Function: onkeylog],
          onhandshakestart: {},
          onhandshakedone: [Function (anonymous)],
          onocspresponse: [Function: onocspresponse],
          onnewsession: [Function: onnewsessionclient],
          onerror: [Function: onerror],
          [Symbol(owner_symbol)]: [Circular *2]
        },
        [Symbol(verified)]: false,
        [Symbol(pendingSession)]: null,
        [Symbol(async_id_symbol)]: 14,
        [Symbol(kHandle)]: null,
        [Symbol(kSetNoDelay)]: false,
        [Symbol(lastWriteQueueSize)]: 0,
        [Symbol(timeout)]: null,
        [Symbol(kBuffer)]: null,
        [Symbol(kBufferCb)]: null,
        [Symbol(kBufferGen)]: null,
        [Symbol(kCapture)]: false,
        [Symbol(kBytesRead)]: 0,
        [Symbol(kBytesWritten)]: 0,
        [Symbol(connect-options)]: {
          rejectUnauthorized: true,
          ciphers: 'TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!SRP:!CAMELLIA',
          checkServerIdentity: [Function: checkServerIdentity],
          minDHSize: 1024,
          maxRedirects: 21,
          maxBodyLength: 10485760,
          protocol: 'https:',
          path: null,
          method: 'GET',
          headers: {
            Accept: 'application/json, text/plain, */*',
            'User-Agent': 'axios/0.27.2'
          },
          agent: undefined,
          agents: { http: undefined, https: undefined },
          auth: undefined,
          hostname: 'nyaa.si',
          port: 443,
          nativeProtocols: { 'http:': [Object], 'https:': [Object] },
          pathname: '/',
          search: '?page=rss&q=Kimetsu+no+yaiba+1080p',
          _defaultAgent: Agent {
            _events: [Object: null prototype],
            _eventsCount: 2,
            _maxListeners: undefined,
            defaultPort: 443,
            protocol: 'https:',
            options: [Object: null prototype],
            requests: [Object: null prototype] {},
            sockets: [Object: null prototype],
            freeSockets: [Object: null prototype] {},
            keepAliveMsecs: 1000,
            keepAlive: false,
            maxSockets: Infinity,
            maxFreeSockets: 256,
            scheduling: 'lifo',
            maxTotalSockets: Infinity,
            totalSocketCount: 1,
            maxCachedSessions: 100,
            _sessionCache: [Object],
            [Symbol(kCapture)]: false
          },
          host: 'nyaa.si',
          servername: 'nyaa.si',
          _agentKey: 'nyaa.si:443:::::::::::::::::::::',
          encoding: null,
          singleUse: true
        }
      },
      _header: 'GET /?page=rss&q=Kimetsu+no+yaiba+1080p HTTP/1.1\r\n' +
        'Accept: application/json, text/plain, */*\r\n' +
        'User-Agent: axios/0.27.2\r\n' +
        'Host: nyaa.si\r\n' +
        'Connection: close\r\n' +
        '\r\n',
      _keepAliveTimeout: 0,
      _onPendingData: [Function: nop],
      agent: Agent {
        _events: [Object: null prototype] {
          free: [Function (anonymous)],
          newListener: [Function: maybeEnableKeylog]
        },
        _eventsCount: 2,
        _maxListeners: undefined,
        defaultPort: 443,
        protocol: 'https:',
        options: [Object: null prototype] { path: null },
        requests: [Object: null prototype] {},
        sockets: [Object: null prototype] {
          'nyaa.si:443:::::::::::::::::::::': [ [TLSSocket] ]
        },
        freeSockets: [Object: null prototype] {},
        keepAliveMsecs: 1000,
        keepAlive: false,
        maxSockets: Infinity,
        maxFreeSockets: 256,
        scheduling: 'lifo',
        maxTotalSockets: Infinity,
        totalSocketCount: 1,
        maxCachedSessions: 100,
        _sessionCache: { map: {}, list: [] },
        [Symbol(kCapture)]: false
      },
      socketPath: undefined,
      method: 'GET',
      maxHeaderSize: undefined,
      insecureHTTPParser: undefined,
      path: '/?page=rss&q=Kimetsu+no+yaiba+1080p',
      _ended: false,
      res: null,
      aborted: false,
      timeoutCb: null,
      upgradeOrConnect: false,
      parser: null,
      maxHeadersCount: null,
      reusedSocket: false,
      host: 'nyaa.si',
      protocol: 'https:',
      _redirectable: [Circular *3],
      [Symbol(kCapture)]: false,
      [Symbol(kNeedDrain)]: false,
      [Symbol(corked)]: 0,
      [Symbol(kOutHeaders)]: [Object: null prototype] {
        accept: [ 'Accept', 'application/json, text/plain, */*' ],
        'user-agent': [ 'User-Agent', 'axios/0.27.2' ],
        host: [ 'Host', 'nyaa.si' ]
      }
    },
    _currentUrl: 'https://nyaa.si/?page=rss&q=Kimetsu+no+yaiba+1080p',
    [Symbol(kCapture)]: false
  }
}

Any help will be very much appericiated.

1 Answers1

2

ECONNREFUSED means the other end (https://nyaa.si) or the site you're hosting the code on (replit) is rejecting your connection. It's a network error, not an Axios error.

It maybe a firewall, or (if you've been making lots of connections) a rate limiter.

Update: as discussed, you're using Replit, replit doesn't allow outbound connections.

If you've been making lots of connections successfully, and it's suddenly started refusing you, wait and slow down, or use a VPN.

Rewriting the code to be a little simpler, it works here:

import axios from "axios";

async function getData(query) {
  const query = "Kimetsu no yaiba 1080p";
  try {
    const response = axios.get("https://nyaa.si/", {
      params: {
        page: "rss",
        q: query,
      },
    });
    // Axios uses 'response.data' for the response body
    return response.data;
  } catch (error) {
    console.log(error);
  }
}

const data = await getData(query);

console.log("Finished");


Running it:

node deleteme.js

Returns:

<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:nyaa="https://nyaa.si/xmlns/nyaa" version="2.0">
        <channel>
                <title>Nyaa - &#34;Kimetsu no yaiba 1080p&#34; - Torrent File RSS</title>
                <description>RSS Feed for &#34;Kimetsu no yaiba 1080p&#34;</description>
                <link>https://nyaa.si/</link>
                <atom:link href="https://nyaa.si/?page=rss" rel="self" type="application/rss+xml" />
                <item>
                        <title>[LlPnF] Kimetsu no Ya

So I think you've likely been making too many requests! Install a VPN and retry, it should work.

mikemaccana
  • 110,530
  • 99
  • 389
  • 494
  • 1
    It used to work just fine but suddenly stopped working. I will accept your answer if it starts working again. Thank you. – noicenoicenoice Sep 12 '22 at 09:50
  • @noicenoicenoice install a VPN (a simple one like Tunnelbear) and retry, I bet it will work. – mikemaccana Sep 12 '22 at 09:51
  • It didn't work. I tried from my ubuntu terminal it did work as intended even without VPN. It just doesn't work from replit. – noicenoicenoice Sep 12 '22 at 10:16
  • @noicenoicenoice ah! There you go, replit probably doesn't allow outbound connections. (it sounds like you're scraping, and replit probably doesn't like that, so they have a firewall on outbound connections). I've edited my answer to mention that. Can you vote up / mark my answer as accepted if it's helped you? – mikemaccana Sep 12 '22 at 10:34
  • 1
    I see I thought so too. As it worked on other place but not only on replit. It used to work till end of last month, it has logs of it. But it seems it doesn't anymore. Anyway thank you for your time. – noicenoicenoice Sep 12 '22 at 10:40
  • @noicenoicenoice it could be a rate limit, rather than a block. Which would explain it working in the past. If you past more than X connection in Y time, it may start blocking you. You (as someone on replit) can't tell whether its replit or the website that's blocking you. – mikemaccana Sep 12 '22 at 10:41
  • 1
    I did try the same code from two different account so I think it is more likely from replit side. Also I am also scraping the date from `jpopsuki` as same way it works just `nyaa.si` doesn't work. – noicenoicenoice Sep 12 '22 at 10:47