7

we have an AWS cloudformation template, through which we are creating the Amazon MSK(Kafka) cluster. which is working fine.

Now we have multiple applications in our product stack which consume the Brokers endpoints which is created by the Amazon MSK. now to automate the product deployment we decided to create a Route53 recordset for the MSK broker endpoints. we are having hard time finding how we can get a broker endpoints of MSK cluster as an Outputs in AWS Cloudformation templates.

looking forward for suggestion/guidance on this.

chitender kumar
  • 394
  • 4
  • 21

2 Answers2

1

Following on @joinEffort answer, this how i did it using custom resources as the CFN resource for an MKS::Cluster does not expose the broker URL: (option 2 is using boto3 and calling AWS API

The description of the classes and mothods to use from CDK custom resource code can be found here: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Kafka.html#getBootstrapBrokers-property

Option 1: Using custom resource:

def get_bootstrap_servers(self):
    
    create_params = {
    "ClusterArn": self._platform_msk_cluster_arn    
    }
    
    get_bootstrap_brokers = custom_resources.AwsSdkCall(
            service='Kafka',
            action='getBootstrapBrokers',
            region='ap-southeast-2',
            physical_resource_id=custom_resources.PhysicalResourceId.of(f'connector-{self._environment_name}'),
            parameters = create_params
        ) 
    
    create_update_custom_plugin = custom_resources.AwsCustomResource(self,
        'getBootstrapBrokers',
        on_create=get_bootstrap_brokers,
        on_update=get_bootstrap_brokers,
        policy=custom_resources.AwsCustomResourcePolicy.from_sdk_calls(resources=custom_resources.AwsCustomResourcePolicy.ANY_RESOURCE)
    )
            
    return create_update_custom_plugin.get_response_field('BootstrapBrokerString')

Option 2: Using boto3:

 client = boto3.client('kafka', region_name='ap-southeast-2')
    response = client.get_bootstrap_brokers(
    ClusterArn='xxx')
 
  #From here u can get the broker urls:
 json_response = json.loads(json.dumps(response))

Reff: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kafka.html#Kafka.Client.get_bootstrap_brokers

D.B
  • 4,009
  • 14
  • 46
  • 83
-1

You should be able to get it from below command. More can be found here.

aws kafka get-bootstrap-brokers --region us-east-1 --cluster-arn ClusterArn

Kalava
  • 231
  • 2
  • 4
  • 18