1

I'm new to R. I'm looking for a script to connect to an Azure Table, and I found some useful information in this thread: Connecting to Azure Table Storage in R

However, when I run the script I get an error very similar to the one that the user posting the question had, and I cannot figure out what's wrong.

This is the code I used (credentials have been modified):

# 1B) Connects to an Azure Table based on the specifications for Shared Key: https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key

library(httr)
library(RCurl)
library(bitops)
library(xml2)

# Stores credentials in variable

Account <- "storageaccount"
Container <- "usage"
Key <- "key"


# Composes URL

URL <- paste0(
          "https://", 
          Account, 
          ".table.core.windows.net", 
          "/", 
          Container
        )

# Requests time stamp

requestdate <- format(Sys.time(), "%a, %d %b %Y %H:%M:%S %Z", tz = "UTC" )

content_lenght <- 0

# Composes signature string

signature_string <- paste0(
                     "GET", "\n",                 # HTTP Verb
                     "\n",                        # Content-MD-5
                     "text/xml", "\n",            # Content-Type
                     requestdate, "\n",           # Date
                     "/", Account, "/", Container # Canonicalized resource
                    )

# Composes header string

header_string <- add_headers(
                    Authorization=paste0(
                      "SharedKey ", 
                      Account, 
                      ":",
                      RCurl::base64(
                        digest::hmac(
                          key = RCurl::base64Decode(
                            Key, mode = "raw"
                          ),
                          object = enc2utf8(signature_string),
                          algo = "sha256", 
                          raw = TRUE
                            )
                        )
                    ),
                    'x-ms-date' = requestdate, 
                    'x-ms-version' = "2022-01-15",
                    'Content-type' = "text/xml"
                  )

# Creates request

xml_body = content(
                GET(
                  URL, 
                  config = header_string, 
                  verbose()
                  )
                )

When I run it, I get this error:

-> GET /Usage HTTP/1.1
-> Host: storageaccount.table.core.windows.net
-> User-Agent: libcurl/7.64.1 r-curl/4.3.2 httr/1.4.2
-> Accept-Encoding: deflate, gzip
-> Accept: application/json, text/xml, application/xml, */*
-> Authorization: SharedKey key
-> x-ms-date: Mon, 17 Jan 2022 15:47:30 UTC
-> x-ms-version: 2022-01-15
-> Content-type: text/xml
-> 
<- HTTP/1.1 400 The value for one of the HTTP headers is not in the correct format.
<- Content-Length: 371
<- Content-Type: application/xml
<- Server: Microsoft-HTTPAPI/2.0
<- x-ms-request-id: 0986de5d-9002-0064-1eb9-0b1166000000
<- Date: Mon, 17 Jan 2022 15:47:30 GMT
<- 
No encoding supplied: defaulting to UTF-8.

Any help would be greatly appreciated.

Also, I don't know if this relevant or not, but I've been prompted a couple of times to install the package Rtools, but when I search for this package I cannot find it.

Thanks in advance!

Alienvolm
  • 143
  • 9
  • There are a number of issues with your code: 1) You have incorrect API version. There's no storage service version `2022-01-15` 2) Table API does not support `application/xml` content type (at least with the latest versions). I would highly recommend reading https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key and https://learn.microsoft.com/en-us/rest/api/storageservices/versioning-for-the-azure-storage-services and apply necessary fixes to your code. – Gaurav Mantri Jan 17 '22 at 17:27
  • Hi Gaurav, thanks for the hints. I have changed the x-ms-version to 20202-12-06 and it worked (same thing changing the content-type... it didn't seem to be a problem). So now I receive a 200 OK but I still don't see anything in the connections pane of R studio. Sorry for the stupid question, but how is this supposed to work? Shouldn't I be seeing the Azure Table in my connections pane? Thanks again in advance! – Alienvolm Jan 18 '22 at 12:27
  • Never mind... I figured it out... It seems like I won't be seeing a "Connection" with a REST API. I have retrieved the data from the table and parsed it from JSON, but it looks like this API pages at 1000 entries. Is there any package that would help me iterate to retrieve all values? I will be working with tables that collect around 40,000 entries a day... what would be an efficient way of doing that? – Alienvolm Jan 18 '22 at 14:07
  • I am glad to hear that you are unblocked. Please do 2 things - 1) Post your solution as an answer to this question so that it will benefit other users and 2) Post a separate question regarding your last comment. – Gaurav Mantri Jan 18 '22 at 14:19

1 Answers1

1

Thank you for your help!

So this is the code that works:

library(httr)
library(RCurl)
library(bitops)
library(xml2)

# Stores credentials in variable

Account <- "storageaccount"
Container <- "Usage"
Key <- "key"


# Composes URL

URL <- paste0(
          "https://", 
          Account, 
          ".table.core.windows.net", 
          "/", 
          Container
        )

# Requests time stamp

requestdate <- format(Sys.time(), "%a, %d %b %Y %H:%M:%S %Z", tz = "GMT")


# As per Microsoft's specs, an empty line is needed for content-length

content_lenght <- 0

# Composes signature string

signature_string <- paste0(
                     "GET", "\n",                 # HTTP Verb
                     "\n",                        # Content-MD-5
                     "text/xml", "\n",            # Content-Type
                     requestdate, "\n",           # Date
                     "/", Account, "/", Container # Canonicalized resource
                    )

# Composes header string

header_string <- add_headers(
                    Authorization=paste0(
                      "SharedKey ", 
                      Account, 
                      ":",
                      RCurl::base64(
                        digest::hmac(
                          key = RCurl::base64Decode(
                            Key, mode = "raw"
                          ),
                          object = enc2utf8(signature_string),
                          algo = "sha256", 
                          raw = TRUE
                            )
                        )
                    ),
                    'x-ms-date' = requestdate, 
                    'x-ms-version' = "2020-12-06",
                    'Content-type' = "text/xml"
                  )

# Creates request

xml_body = content(
                GET(
                  URL, 
                  config = header_string, 
                  verbose()
                  ),
                "text"
                )

Get_data <- xml_body                             # Gets data as text from API  
From_JSON <-fromJSON(Get_data, flatten = TRUE)   # Parses text from JSON
Table_name <- as.data.frame(From_JSON)           # Saves data to a table

I added the final three lines to parse the data from JSON format and saved into a table.

Alienvolm
  • 143
  • 9