0

Within my Synapse notebook, not all of the Synapse "linked services" are available to use. I can connect to linked services for KeyVault and SQL and ADLS, but for some reason I cannot connect to a linked service for a REST API. Why is this connectivity unavailable for a REST API?

Background: Within the Synapse Apache Spark notebook experience, there is a way to connect to remote data by way of the Microsoft "linked service" components. In order to connect to remote data on a "linked service", we need to use the "TokenLibrary" API. See: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python

While most linked services work fine, the REST connections do not. Here is the error message that is generated, when trying to interact with this type of a linked service:

An error occurred while calling z:com.microsoft.azure.synapse.tokenlibrary.TokenLibrary.getConnectionString.

: com.microsoft.azure.synapse.tokenlibrary.TokenLibrary$NonRetryableStatusException$1: 

POST failed with 'Bad Request' (400) and message: {"result":"DependencyError","errorId":"BadRequest",
"errorMessage":"[Code=LinkedServiceTypeNotSupported, Target=ls_rest_pbi_generic, 
Message=Linked Service Type 'RestService' not supported]. 
TraceId : 16f6eda6-0002-4833-a4d5-7c969317aaaa | 
client-request-id : c889cc16-8d19-42eb-ab4b-c2fb5dfeaaaa. 
Error Component : LSR"}

Notice that it says that linked services of type "RestService" are not supported. Why are they being excluded? I'm fairly certain that the linked services for "AzureKeyVault" are using HTTP/REST internally, so it seems like we should also be able to connect to a linked service for our own custom REST api! Please help me understand the conceptual/technical reason why one of these works and not the other.

David Beavon
  • 1,141
  • 9
  • 16

2 Answers2

0

I opened an extremely long-running support case with the Synapse Spark team. The deliverable from that support case was a little bit of new documentation. The docs now explain that some of the linked services are incompatible with "tokenlibrary".

Here are the docs ... https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-secure-credentials-with-tokenlibrary?pivots=programming-language-python#linked-service-connections-supported-from-the-spark-runtime

Notice below that there is no support for REST api's. If it was supported, it would be shown in the list. (It would probably work by way of bearer tokens, generated from the linked service and the MSI of the synapse workspace.)

enter image description here

David Beavon
  • 1,141
  • 9
  • 16
-1

I created Rest linked service with swapi.dev/api/ URL.

enter image description here

I run TokenLibrary.getConnectionString("<linkedServiceName>") I got below error:

enter image description here

As per this "Rest Service " is not natively supported by Azure synapse Analytics. If you have a custom REST API that you would like to connect to, you may need to consider using a different Linked Service type that is natively supported by Azure Synapse Analytics, such as the REST API Linked Service type. For more information you can refer this.

Bhavani
  • 1,725
  • 1
  • 3
  • 6
  • Hi @Bhavani, You said that I may need to consider using a different linked service type such as REST API, but that is the one that is not working. You just demonstrated that it is not working in your screenshots.. – David Beavon May 27 '23 at 16:42
  • Also, your link may be broken, or pointed to the wrong article. There is nothing in that link that distinguishes "native" or non-native linked services in Azure Synapse Analytics. FYI, All your links are pointed at github. Did you intend to share Microsoft's authoritative links? – David Beavon May 27 '23 at 16:49
  • Accessing AEST API linked service may be not possible. To interact with REST APIs within an Azure Synapse notebook, you would typically use standard Python libraries such as requests or specialized libraries for the specific API you're working with. You can install any required libraries using the notebook's package manager. – Pratik Lad Jun 08 '23 at 09:37
  • Hi @PratikLad . Yes, we use the python requests library to call remote API's. But in addition we wanted to use the proprietary "tokenlibrary" for the sake of the MSI credentials of the synapse workspace. In theory we should be able to generate an accesstoken for the MSI. But that functionality falls over, where REST is concerned. Other types of linked services to allow us to use the MSI credentials – David Beavon Jul 19 '23 at 00:44