I'd like to preface this with the fact that I am completely new to SAP and SAP HANA, and OData.
I was tasked with pulling changes from a SAP HANA table and transfer those to Kafka.
I noticed there was a Kafka source connector already written, which can be found here.
For this task, I was given a URL, a username and a password.
The URL looks like this:
https://blablabla.companyName.com/companyName/Foo/Bar/Baz/Foo/Table/Resource.xsodata
And this is a sample of the source connector's configs:
# The first few settings are required for all connectors:
# a name, the connector class to run, and the maximum number of
# tasks to create.
name = odatav4-source-connector
connector.class = org.init.ohja.kafka.connect.odatav4.source.OData4SourceConnector
tasks.max = 1
# The remaining configs are specific to the OData v4 source connector.
# OData server host as either DNS or IP
sap.odata.host.address = services.odata.org
# OData server port
sap.odata.host.port = 443
# OData protocol (supported values are http or https)
sap.odata.host.protocol = https
# OData user name for basic authentication
# For services not requiring authentication this can be set to any value
sap.odata.user.name = anonymous
# OData user password for basic authentication
# For services not requiring authentication this can be set to any value
sap.odata.user.pwd = anonymous
# Optional list of service URL query parameters in the form of "param1=value1,param2=value2", e.g. sap-client=200
#sap.odata.query-params=
# none(default): DECIMALs will be mapped to Connect Decimal data type
# primitive: DECIMALs will be mapped to INT64(id scale = 0) anf FLOAT64
#sap.odata.decimal.mapping = none
# maximum amount of retries in case of service connection/communication errors (e.g. HTTP status codes 400-599)
#sap.odata.max.retries = 30
# The backoff strategy applied will select a random number of milliseconds
# to wait between min.retry.backoff.ms and max.retry.backoff.ms before starting
# the next retry.
#sap.odata.min.retry.backoff.ms = 20000
#sap.odata.max.retry.backoff.ms = 180000
# Timeout in milliseconds for establishing http connections
#sap.odata.connection.connect.timeout.ms=3000
# Timeout in milliseconds for reading data from a http connection
#sap.odata.connection.read.timeout.ms=10000
# Individual configurations for each OData v4 service entity.
# service and entityset build up the primary key for each OData configuration.
# OData v4 URL service path
sap.odata#00.service = /V4/Northwind/Northwind.svc/
# OData v4 entity set name
# The entity set name can be queried from the /$metadata service URL
sap.odata#00.entityset = Order_Details
# Kafka topic name the data for this OData service entity set will be pushed to
sap.odata#00.topic = Order_Details
# Execution interval in seconds for the scheduled data extractions
# Set to -1 to process subscription events only
#sap.odata#00.exec-period = 900
# If changes to entities selected by the first query should be tracked and returned as deltas in subsequent polls
# Set to 1 to enable odata delta mode
#sap.odata#00.track-changes = 0
# Paging mode (server or client) determines the type of paging
# server: use HTTP prefer-headers to request a maximum package size from the odata server
# client: use query functions skip and top (not compatible to change tracking)
#sap.odata#00.paging.mode = server
# Packaging size in count of entity set records
#sap.odata#00.paging.size = 50000
# Optional: Hierarchy level up to which recommendations for the expand.list configuration (query option $expand) will
# be shown in the Confluent Control Center
#sap.odata#00.expand.level = 1
# Optional: List of expand query options that will define the deep structure of returned entity messages
#sap.odata#00.expand.list =
# Optional: comma separated list of selected non-key fields to be extracted
#sap.odata#00.projection =
# Optional: filter query options
# Supported logical operations/options are: eq, ne, le, lt, ge, gt, bt, nb, in
#sap.odata#00.select#00.fieldname =
#sap.odata#00.select#00.option =
#sap.odata#00.select#00.low =
#sap.odata#00.select#00.high =
# If set to 1 the connector will subscribe to push-notifications issued by the corresponding OData service entity
#sap.odata#00.subscription.enable = 0
So I tried to create my own, like so:
{
"name": "sap-hana-source-connector",
"config": {
"connector.class": "org.init.ohja.kafka.connect.odatav4.source.OData4SourceConnector",
"sap.odata.user.name": "username",
"sap.odata.host.address": "blablabla.companyName.com",
"sap.odata.host.port": "443",
"sap.odata.host.protocol": "https",
"sap.odata#00.service": "/companyName/Foo/Bar/Baz/Foo/Table/Resource.xsodata",
"sap.odata#00.entityset": "Resource",
"sap.odata.user.pwd": "pwd"
}
}
The issue is that the only error I get is this:
{
"error_code": 400,
"message": "Connector configuration is invalid and contains the following 14 error(s):\nInvalid configuration sap.odata.host.address: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.host.protocol: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.host.port: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.user.name: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.user.pwd: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.max.retries: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.min.retry.backoff.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.max.retry.backoff.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.connection.connect.timeout.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.connection.read.timeout.ms: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.query-params: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.trace.mode: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.trace.path: No configured service reachable. Maybe invalid destination configuration?\nInvalid configuration sap.odata.decimal.mapping: No configured service reachable. Maybe invalid destination configuration?\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`"
}
As someone who is completely new to OData and SAP, I don't know how I'd debug this.
I noticed that, in the OData's Kafka Source Connector documentation the services end in .svc and not .xsodata; so maybe it's something to do with that?
Also, what am I supposed to be for the sap.odata#00.entityset config?
Is there a way to get a more detailed error message?
Thanks.
