DateTime to RFC3339 formatting issue as URL

Question

I am trying to pass a DateTime in RFC3339 format to an API but it keeps getting rejected as being improperly formatted. Is there a different way to convert to the correct format?

require 'cgi'
require 'date'
require 'uri'

startTime=CGI.escape(DateTime.new(2016, 6, 6, 15, 47, 40).strftime("%Y-%m-%dT%H:%M:%SZ"))
endTime=CGI.escape(DateTime.now.strftime("%Y-%m-%dT%H:%M:%SZ"))
puts startTime #start=2014-06-19T15%3A47%3A40Z   me:2016-05-19T16%3A47%3A04-04%3A00
puts endTime
hist_data=getData(startTime,endTime)

def getData(startTime,endTime)
  base="https://api-fxtrade.oanda.com/v1/candles?instrument="
  curr="EUR_USD"
  granularity="H1"
  #https://api-fxtrade.oanda.com/v1/candles?instrument=EUR_USD&start=2014-06-19T15%3A47%3A40Z&end=2014-06-19T15%3A47%3A50Z
  myurl = "#{ base }#{ curr }&candleFormat=bidask&granularity=#{ granularity }&dailyAlignment=0&alignmentTimezone=America%2FNew_York&start=#{startTime}&end=#{endTime}"
  puts myurl
  response =HTTParty.get(URI::encode(myurl))
  #{"time"=>"2016-06-03T20:00:00.000000Z", "openBid"=>1.1355, "openAsk"=>1.13564, "highBid"=>1.13727, "highAsk"=>1.13752, "lowBid"=>1.13541, "lowAsk"=>1.13554, "closeBid"=>1.13651, "closeAsk"=>1.13684, "volume"=>2523, "complete"=>true}
  response
end

the ending website url is valid however when I use this code. The full output of my code gives this:

https://api-fxtrade.oanda.com/v1/candles?instrument=EUR_USD&candleFormat=bidask&granularity=H1&dailyAlignment=0&alignmentTimezone=America%2FNew_York&start=2016-06-06T15%3A47%3A40Z&end=2016-06-08T21%3A53%3A44Z

Any idea why it doesnt work when run the method, but works when I just paste the URL? I thought it was an encoding problem, but I am definitely encoding the URL in the method.

I don't really know, but since it's so easy to try, I'd suggest *not* calling `URI::encode` and see what happens. — Keith Bennett, Jun 09 '16 at 02:14
Don't construct URI query parameters with string interpolation/concatenation. That way lies only despair and madness. Since you're using HTTParty you don't need to—and shouldn't—do *any* manual encoding/escaping. Pass your query parameters to `HTTParty.get` as a Hash using the `:query` option. — Jordan Running, Jun 09 '16 at 02:37

score 4 · Accepted Answer · edited May 23 '17 at 12:22

As I said above in the comments, you're making things really difficult for yourself by trying to encode query parameters yourself and using interpolation/concatenation to build URLs.

I would guess that the problem is that you're encoding the query parameters individually using CGI.escape and then a second time en masse using URI.encode. In other words, you're double-encoding them.

Incidentally, CGI.escape and URI.encode do the same thing, more or less (and the former is deprecated). It's not clear why you're using both, but that's moot because you shouldn't be using either. You should be letting HTTParty do it for you.

Give HTTParty.get a base URL and pass it a Hash of your (raw, i.e. not encoded) query parameters with the :query option and it will do all of the encoding for you, correctly. As a side-effect, it makes it possible to write much cleaner code:

require "httparty"
require "date"

DATETIME_FMT = "%FT%TZ" # RFC 3339
BASE_URI = "https://api-fxtrade.oanda.com/v1/candles"

DEFAULT_PARAMS = {
  instrument: "EUR_USD",
  candleFormat: "bidask",
  granularity: "H1",
  dailyAlignment: 0,
  alignmentTimezone: "America/New_York",
}.freeze

def get_data(start_time, end_time)
  params = DEFAULT_PARAMS.merge(
    start: start_time.strftime(DATETIME_FMT),
    end: end_time.strftime(DATETIME_FMT)
  )

  HTTParty.get(BASE_URI, query: params)
end

start_time = DateTime.new(2016, 6, 6, 15, 47, 40)
end_time = DateTime.now

hist_data = get_data(start_time, end_time)

Thanks for that incredibly thorough answer, it works perfectly. — Rilcon42, Jun 09 '16 at 13:24

DateTime to RFC3339 formatting issue as URL

1 Answers1