1

I'm trying to get some data from the OneSignal API.

If I run it straight into the terminal (removed Auth key and App ID for this question), no problem:

$ curl --include --header "Authorization: Basic Yzzzzzzzzzzzzzzzz" 
 "https://onesignal.com/api/v1/players?app_id={XXXXXXXXXXX}&limit=300&offset=0"

Returns:

{"total_count":1,"offset":0,"limit":300,"players": [{"id":"XXXXXX","identifier":"XXXXXXX","session_count":3,"language":"en","timezone":-14400,"game_version":"3","device_os":"6.0.1","device_type":1,"device_model":"LG-K240","ad_id":"XXXXXXXX","tags":{},"last_active":1498938033,"playtime":0,"amount_spent":0.0,"created_at":1498935316,"invalid_identifier":false,"badge_count":0,"sdk":"030507","test_type":null}]}

If I use what I am lead to believe using this Curl to Ruby converter is the equivalent Ruby in my rake task, instead suddenly I hit unknown errors galore:

require 'net/http'
require 'uri'

uri = URI.parse("https://onesignal.com/api/v1/players?app_id={XXXXXX}&limit=300&offset=0")
request = Net::HTTP::Get.new(uri)
request["Authorization"] = "Basic Yzzzzzzzzzzzzzzzz"

req_options = {
  use_ssl: uri.scheme == "https",
}

response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
  http.request(request)
end

puts response.code
puts response.body

Returns:

500
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>500 &bull; OneSignal</title>
  <style>
    html, body {
      margin: 0;
      padding: 0;
      background: #F1F1F1;
      font-size: 18px;
    }

    svg#logo {
      width: 240px;
      height: 102px;
    }

    .logo-head {
      margin-top: 1.3em;
    }

    svg#logo-head {
      width: 40px;
      height: 40px;
    }

    #content {
      margin: 2em auto;
      width: 600px;
      padding: 30px;
      text-align: center;
    }

    .text {
      font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
      color: rgba(0, 0, 0, 0.5);
      font-size: 1em;
      font-weight: 200;
      line-height: 1.6em;
    }

    .text.twitter {
      font-size: 0.85em;
    }

    a {
      text-decoration: none;
    }

    a:hover {
      text-decoration: underline;
    }

    a:visited {
      text-decoration: none;
      color: inherit;
    }

    #message {
      font-weight: 400;
      font-size: 1.4em;
      margin-bottom: 0;
    }

    #submessage {
      font-size: 0.8em;
      margin-bottom: 2.5em;
    }

    footer {
      margin-top: 2em;
    }

    iframe {
      border: 1px solid rgba(0, 0, 0, 0.1) !important;
      border-radius: 4px;
    }
  </style>
</head>
<body>
<div id="content">
  <svg version="1.1" id="logo" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
             x="0px" y="0px"
             viewBox="0 0 160 34" style="enable-background:new 0 0 160 34;" xml:space="preserve">
        <style type="text/css">
            .st0{fill:#E44A49;}
            .st1{fill:#515151;}
        </style>
        <g id="XMLID_1_">
            <path id="XMLID_2_" class="st0" d="M55.3,27.1c-1.1,0-2.1-0.2-3-0.7c-0.9-0.4-1.7-1-2.3-1.8c-0.6-0.7-1.1-1.6-1.5-2.5
                c-0.4-1-0.5-1.9-0.5-2.9s0.2-2,0.6-3s0.9-1.8,1.6-2.5s1.4-1.3,2.3-1.7s1.9-0.6,2.9-0.6c1.1,0,2.1,0.2,3,0.7s1.7,1.1,2.3,1.8
                c0.6,0.7,1.1,1.6,1.5,2.5s0.5,1.9,0.5,2.9s-0.2,2-0.6,3s-0.9,1.8-1.5,2.5c-0.7,0.7-1.4,1.3-2.3,1.7C57.3,26.9,56.3,27.1,55.3,27.1z
                 M49.5,19.2c0,0.9,0.1,1.7,0.4,2.5s0.7,1.5,1.2,2.1s1.1,1.1,1.8,1.5c0.7,0.4,1.5,0.5,2.4,0.5s1.7-0.2,2.4-0.6
                c0.7-0.4,1.3-0.9,1.8-1.5s0.9-1.3,1.2-2.1c0.3-0.8,0.4-1.6,0.4-2.4s-0.1-1.7-0.4-2.5s-0.7-1.5-1.2-2.1s-1.1-1.1-1.8-1.4
                c-0.7-0.4-1.5-0.5-2.3-0.5c-0.9,0-1.7,0.2-2.4,0.6c-0.7,0.4-1.3,0.9-1.8,1.5s-0.9,1.3-1.2,2.1C49.6,17.6,49.5,18.4,49.5,19.2z"/>
            <path id="XMLID_5_" class="st0" d="M75.5,27H74v-6.4c0-1.4-0.2-2.4-0.6-3s-1-0.9-1.8-0.9c-0.4,0-0.9,0.1-1.3,0.2
                c-0.4,0.2-0.9,0.4-1.2,0.7c-0.4,0.3-0.7,0.6-1,1s-0.5,0.8-0.6,1.3v7H66V15.5h1.3v2.6c0.2-0.4,0.5-0.8,0.9-1.1
                c0.3-0.3,0.7-0.6,1.2-0.9c0.4-0.2,0.9-0.4,1.4-0.6c0.5-0.1,1-0.2,1.5-0.2c0.6,0,1.2,0.1,1.6,0.4c0.4,0.2,0.8,0.6,1,1
                s0.5,0.9,0.6,1.6c0.1,0.6,0.2,1.3,0.2,2V27L75.5,27L75.5,27z"/>
            <path id="XMLID_7_" class="st0" d="M84.3,27.2c-0.8,0-1.6-0.2-2.3-0.5s-1.3-0.7-1.8-1.3s-0.9-1.2-1.2-1.9c-0.3-0.7-0.4-1.5-0.4-2.3
                s0.1-1.6,0.4-2.3s0.7-1.3,1.2-1.9c0.5-0.5,1.1-1,1.8-1.3c0.7-0.3,1.5-0.5,2.3-0.5s1.6,0.2,2.3,0.5c0.7,0.3,1.3,0.7,1.8,1.3
                c0.5,0.5,0.9,1.2,1.2,1.9s0.4,1.5,0.4,2.2c0,0.1,0,0.2,0,0.4c0,0.1,0,0.2,0,0.2h-9.8c0,0.6,0.2,1.2,0.4,1.7c0.2,0.5,0.5,1,0.9,1.4
                s0.8,0.7,1.3,0.9s1,0.3,1.6,0.3c0.4,0,0.8-0.1,1.1-0.2c0.4-0.1,0.7-0.2,1-0.4s0.6-0.4,0.8-0.6c0.2-0.3,0.4-0.5,0.6-0.8l1.3,0.4
                c-0.2,0.4-0.4,0.8-0.7,1.2c-0.3,0.3-0.7,0.6-1.1,0.9C87,26.8,86.5,27,86,27.1C85.3,27.2,84.8,27.2,84.3,27.2z M88.5,20.7
                c0-0.6-0.2-1.2-0.4-1.7c-0.2-0.5-0.5-0.9-0.9-1.3c-0.4-0.4-0.8-0.7-1.3-0.9s-1-0.3-1.6-0.3s-1.1,0.1-1.6,0.3s-0.9,0.5-1.3,0.9
                s-0.7,0.8-0.9,1.3c-0.2,0.5-0.4,1.1-0.4,1.7H88.5z"/>
            <path id="XMLID_10_" class="st1" d="M102.2,15.5c-0.1-0.1-0.3-0.2-0.6-0.4s-0.6-0.3-1-0.5s-0.8-0.3-1.3-0.4S98.4,14,97.9,14
                c-0.8,0-1.5,0.2-1.9,0.5s-0.6,0.7-0.6,1.3c0,0.3,0.1,0.6,0.2,0.8c0.2,0.2,0.4,0.4,0.7,0.6c0.3,0.2,0.7,0.3,1.1,0.4
                c0.4,0.1,1,0.3,1.6,0.4c0.8,0.2,1.5,0.4,2.1,0.7c0.6,0.2,1.2,0.5,1.6,0.9c0.4,0.4,0.8,0.8,1,1.3s0.4,1.1,0.4,1.8
                c0,0.8-0.2,1.6-0.5,2.1c-0.3,0.6-0.7,1.1-1.3,1.4c-0.5,0.4-1.1,0.6-1.8,0.8s-1.4,0.3-2.2,0.3c-1.2,0-2.3-0.2-3.5-0.5
                c-1.1-0.4-2.2-0.9-3.1-1.5l1.3-2.6c0.1,0.1,0.4,0.3,0.7,0.5s0.8,0.4,1.2,0.6c0.5,0.2,1,0.4,1.6,0.5s1.2,0.2,1.8,0.2
                c1.7,0,2.5-0.5,2.5-1.6c0-0.4-0.1-0.6-0.3-0.9c-0.2-0.2-0.5-0.4-0.8-0.6c-0.4-0.2-0.8-0.3-1.3-0.5c-0.5-0.1-1.1-0.3-1.7-0.5
                c-0.7-0.2-1.4-0.4-1.9-0.7c-0.6-0.2-1-0.5-1.4-0.8s-0.6-0.7-0.8-1.1s-0.3-1-0.3-1.6c0-0.8,0.1-1.5,0.4-2.1s0.7-1.1,1.2-1.5
                s1.1-0.7,1.8-0.9s1.4-0.3,2.2-0.3c1.1,0,2.1,0.2,3,0.5s1.7,0.8,2.4,1.2L102.2,15.5z"/>
            <path id="XMLID_12_" class="st1" d="M106.7,13.9V11h2.9v2.9H106.7z M106.7,27V15.5h2.9V27H106.7z"/>
            <path id="XMLID_15_" class="st1" d="M117.3,27.1c-0.8,0-1.5-0.2-2.1-0.5s-1.2-0.7-1.6-1.3c-0.5-0.5-0.8-1.2-1.1-1.9
                c-0.2-0.7-0.4-1.5-0.4-2.2c0-0.8,0.1-1.6,0.4-2.3s0.6-1.3,1.1-1.9c0.5-0.5,1-1,1.7-1.3s1.4-0.5,2.2-0.5c0.9,0,1.7,0.2,2.4,0.6
                s1.2,0.9,1.7,1.6v-2h2.6v11c0,0.9-0.2,1.6-0.5,2.3s-0.8,1.2-1.3,1.7s-1.2,0.8-2,1.1c-0.8,0.2-1.6,0.4-2.6,0.4
                c-1.3,0-2.3-0.2-3.2-0.6s-1.6-1-2.2-1.8l1.6-1.6c0.4,0.5,1,1,1.7,1.3s1.4,0.5,2.1,0.5c0.5,0,0.9-0.1,1.3-0.2
                c0.4-0.1,0.8-0.3,1.1-0.6s0.6-0.6,0.7-1c0.2-0.4,0.3-0.9,0.3-1.5V25c-0.4,0.7-0.9,1.2-1.6,1.5S118.1,27.1,117.3,27.1z M118.4,24.7
                c0.3,0,0.6-0.1,0.9-0.2c0.3-0.1,0.6-0.2,0.8-0.4s0.5-0.4,0.6-0.6c0.2-0.2,0.3-0.5,0.4-0.7V20c-0.3-0.7-0.7-1.2-1.2-1.6
                c-0.6-0.4-1.1-0.6-1.8-0.6c-0.5,0-0.9,0.1-1.2,0.3s-0.7,0.5-1,0.8s-0.5,0.7-0.6,1.1c-0.1,0.4-0.2,0.9-0.2,1.3
                c0,0.5,0.1,0.9,0.2,1.3c0.2,0.4,0.4,0.8,0.7,1.1s0.6,0.5,1,0.7C117.5,24.6,117.9,24.7,118.4,24.7z"/>
            <path id="XMLID_18_" class="st1" d="M138,27h-2.9v-6.5c0-0.9-0.2-1.6-0.5-2s-0.8-0.6-1.3-0.6c-0.3,0-0.6,0.1-0.9,0.2
                c-0.3,0.1-0.6,0.3-0.9,0.5c-0.3,0.2-0.5,0.5-0.7,0.8s-0.4,0.6-0.5,1v6.7h-3V15.5h2.7v2.1c0.4-0.7,1-1.3,1.8-1.7s1.7-0.6,2.7-0.6
                c0.7,0,1.3,0.1,1.8,0.4c0.5,0.3,0.8,0.6,1.1,1s0.4,0.9,0.5,1.5c0.1,0.5,0.1,1.1,0.1,1.7L138,27L138,27z"/>
            <path id="XMLID_20_" class="st1" d="M144.3,27.2c-0.6,0-1.1-0.1-1.6-0.3s-0.9-0.4-1.3-0.8c-0.4-0.3-0.6-0.7-0.8-1.2
                c-0.2-0.4-0.3-0.9-0.3-1.5c0-0.5,0.1-1,0.4-1.5c0.2-0.5,0.6-0.9,1-1.2s1-0.6,1.6-0.8s1.3-0.3,2-0.3c0.5,0,1,0,1.5,0.1
                s0.9,0.2,1.3,0.4v-0.7c0-0.8-0.2-1.3-0.6-1.8s-1.1-0.6-1.9-0.6c-0.6,0-1.2,0.1-1.8,0.3s-1.2,0.5-1.8,1l-0.9-1.9
                c1.5-1,3.1-1.5,4.8-1.5s2.9,0.4,3.9,1.2c0.9,0.8,1.4,2,1.4,3.5v3.6c0,0.3,0.1,0.5,0.2,0.7c0.1,0.1,0.3,0.2,0.6,0.2v2.5
                c-0.3,0.1-0.5,0.1-0.8,0.1s-0.5,0-0.6,0c-0.6,0-1-0.1-1.3-0.4c-0.3-0.3-0.5-0.6-0.5-1l-0.1-0.6c-0.5,0.7-1.1,1.2-1.9,1.5
                C145.9,27,145.1,27.2,144.3,27.2z M145.1,25.1c0.5,0,1-0.1,1.4-0.3s0.8-0.4,1-0.7c0.3-0.2,0.5-0.5,0.5-0.8V22
                c-0.4-0.1-0.7-0.2-1.1-0.3c-0.4-0.1-0.8-0.1-1.2-0.1c-0.8,0-1.4,0.2-1.9,0.5s-0.7,0.8-0.7,1.3s0.2,0.9,0.6,1.2
                C144.1,24.9,144.6,25.1,145.1,25.1z"/>
            <path id="XMLID_23_" class="st1" d="M154.3,10.9h2.9v12.2c0,0.9,0.4,1.3,1.2,1.3c0.2,0,0.3,0,0.6-0.1c0.2-0.1,0.4-0.1,0.6-0.2
                l0.4,2.4c-0.4,0.2-0.9,0.3-1.4,0.4s-1,0.2-1.4,0.2c-0.9,0-1.6-0.2-2.1-0.7s-0.8-1.2-0.8-2.1V10.9z"/>
        </g>
        <path id="XMLID_8438_" class="st0" d="M18.5,0C8.3,0,0,8.3,0,18.5C0,25,3.4,30.7,8.4,34c0.3-0.2,0.6-0.4,0.9-0.6
            c0.2-0.1,0.5-0.3,0.7-0.4c0.3-0.1,0.6-0.3,0.8-0.4c0.3-0.1,0.6-0.2,0.9-0.4c0.2-0.1,0.5-0.2,0.7-0.3c0.3-0.1,0.7-0.2,1-0.3
            s0.5-0.1,0.8-0.2c0.3-0.1,0.5-0.1,0.8-0.2s0.6-0.1,0.9-0.1l0,0c0.3,0,0.7-0.1,1-0.1l0,0v-1.2v-1v-4.1v-1V20h-1v-2h1h3h1v5.4v1.1v4.2
            v1v1.5l0,0c2.8,0.4,5.3,1.4,7.6,2.8l0,0c5.1-3.3,8.4-9,8.4-15.5C37,8.3,28.7,0,18.5,0z M24,18.5c0-3-2.5-5.5-5.5-5.5
            S13,15.5,13,18.5c0,2.1,1.2,4,3,4.9v1.1c-2.3-1-4-3.3-4-6c0-3.6,2.9-6.5,6.5-6.5s6.5,2.9,6.5,6.5c0,2.3-1.2,4.3-3,5.5v-1.2
            C23.2,21.7,24,20.2,24,18.5z M22,29.4v-1.1c4.1-1.4,7-5.3,7-9.9C29,12.7,24.3,8,18.5,8S8,12.7,8,18.5c0,4.9,3.4,9.1,8,10.2v1
            c-5.1-1.1-9-5.7-9-11.2C7,12.1,12.1,7,18.5,7S30,12.1,30,18.5C30,23.6,26.6,28,22,29.4z"/>
        </svg>
  <p id="message" class="text">An Unexpected Error Occurred</p>
  <p id="submessage" class="text">Error Code: 500</p>
  <p id="explanation" class="text">
    We track these errors automatically, but if the problem persists feel free to contact us.
  </p>
</div>
</body>
</html>

This is insanity. Am I not making the EXACT same request? Why does the first one return the data I want (although I don't understand how to parse it into Ruby), but the second one hits an error?

Alternatively, how do I parse the successful response from using the curl command into JSON or something my Ruby app can use?

string = `curl --include --header "Authorization: Basic Yzzzzzzzzzzzzzzzz" "https://onesignal.com/api/v1/players?app_id={XXXXXXX}&limit=300&offset=0"`
puts string
parsed = JSON.parse(string)

Returns:

JSON::ParserError: 776: unexpected token at 'HTTP/1.1 200 OK
Date: Sat, 01 Jul 2017 20:06:17 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __yyyyyyyyyyyyyyy; expires=Sun, 01-Jul-18 20:06:17 GMT; path=/; domain=.onesignal.com; HttpOnly
Status: 200 OK
Cache-Control: max-age=0, private, must-revalidate
Access-Control-Allow-Origin: *
X-XSS-Protection: 1; mode=block
X-Request-Id: yyyyyyyyyyyy
Access-Control-Allow-Headers: SDK-Version
ETag: W/"e57da9b403c50294b432635d4ce4acfa"
X-Frame-Options: SAMEORIGIN
X-Runtime: 0.065356
X-Content-Type-Options: nosniff
X-Powered-By: Phusion Passenger 5.1.4
Server: cloudflare-nginx
CF-RAY: 377bfc285ebe096f-SCL

{"total_count":1,"offset":0,"limit":300,"players":[{"id":"zzzzzzzzzz","identifier":"zzzzzzzzzz","session_count":3,"language":"en","timezone":-14400,"game_version":"3","device_os":"6.0.1","device_type":1,"device_model":"LG-K240","ad_id":"zzzzzzzzzzzzz","tags":{},"last_active":1498938033,"playtime":0,"amount_spent":0.0,"created_at":1498935316,"invalid_identifier":false,"badge_count":0,"sdk":"030507","test_type":null}]}'
/Users/Me/Code/my_project/lib/tasks/scheduler.rake:57:in `block in <top (required)>'
/Users/Me/.rvm/gems/ruby-2.2.3/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => daily_notifications
(See full trace by running task with --trace)

Edit: Curl response with --verbose

*   Trying 104.16.204.165...
* TCP_NODELAY set
* Connected to onesignal.com (104.16.204.165) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate: ssl473492.cloudflaressl.com
* Server certificate: COMODO ECC Domain Validation Secure Server CA 2
* Server certificate: COMODO ECC Certification Authority
> GET /api/v1/players?app_id=zzzzzzzzzzzzzzzzzz&limit=300&offset=0 HTTP/1.1
> Host: onesignal.com
> User-Agent: curl/7.51.0
> Accept: */*
> Authorization: Basic Yzzzzzzzzzzzzzzzzzzz
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Thu, 06 Jul 2017 13:48:54 GMT
Date: Thu, 06 Jul 2017 13:48:54 GMT
< Content-Type: application/json; charset=utf-8
Content-Type: application/json; charset=utf-8
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
< Connection: keep-alive
Connection: keep-alive
< Set-Cookie: __cfduid=d9b57682d26de2755426b575293a211d21499348933; expires=Fri, 06-Jul-18 13:48:53 GMT; path=/; domain=.onesignal.com; HttpOnly
Set-Cookie: __cfduid=d9b57682d26de2755426b575293a211d21499348933; expires=Fri, 06-Jul-18 13:48:53 GMT; path=/; domain=.onesignal.com; HttpOnly
< Status: 200 OK
Status: 200 OK
< Cache-Control: max-age=0, private, must-revalidate
Cache-Control: max-age=0, private, must-revalidate
< Access-Control-Allow-Origin: *
Access-Control-Allow-Origin: *
< X-XSS-Protection: 1; mode=block
X-XSS-Protection: 1; mode=block
< X-Request-Id: d6276562-3cee-4c20-aa4e-5d781e7a7957
X-Request-Id: d6276562-3cee-4c20-aa4e-5d781e7a7957
< Access-Control-Allow-Headers: SDK-Version
Access-Control-Allow-Headers: SDK-Version
< ETag: W/"cef2128118655721c75aa04b81b2ce26"
ETag: W/"cef2128118655721c75aa04b81b2ce26"
< X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
< X-Runtime: 0.019079
X-Runtime: 0.019079
< X-Content-Type-Options: nosniff
X-Content-Type-Options: nosniff
< X-Powered-By: Phusion Passenger 5.1.4
X-Powered-By: Phusion Passenger 5.1.4
< Server: cloudflare-nginx
Server: cloudflare-nginx
< CF-RAY: 37a306343f03162f-LIM
CF-RAY: 37a306343f03162f-LIM

< 
* Curl_http_done: called premature == 0
* Connection #0 to host onesignal.com left intact
Jeremy E
  • 463
  • 1
  • 7
  • 19
  • Can you include `--verbose` when calling `curl` to see the exact request that is being sent? It may be including some other headers that are required (https://stackoverflow.com/questions/3252851/how-to-display-request-headers-with-command-line-curl) – Adam Sheehan Jul 02 '17 at 13:10
  • @AdamSheehan Added `--verbose` response in edit above – Jeremy E Jul 06 '17 at 13:51

1 Answers1

0

The only explanation is that the requests are different. First of all, I'd recommend using Faraday gem to help you with requests in Ruby. You can easily inspect Faraday object to see what went wrong. I can also see that you are passing req_options which is not part of your curl.

So I advise you to make the simplest call with Faraday, perhaps even on a different endpoint to make sure it works and then with this endpoint. If it's still not working, updating this question with Faraday request and response details would help further debug this

P.S. You should definitely not use Curl in your Ruby app.

Maxim Fedotov
  • 1,349
  • 1
  • 18
  • 38