boto3: execute_command inside python script

Question

I am trying to run a command to an ecs container managed by fargate. I can establish connection as well as execute successfully but I cannot get the response from said command inside my python script.

import boto3
import pprint as pp

client = boto3.client("ecs")
cluster = "my-mundane-cluster-name"


def main():
    task_arns = client.list_tasks(cluster=cluster, launchType="FARGATE")
    for task_arn in task_arns.get("taskArns", []):
        cmd_out = client.execute_command(
            cluster=cluster,
            command="ls",
            interactive=True,
            task=task_arn,
        )
        pp.pprint(f"{cmd_out}")


if __name__ == "__main__":
    main()

I replaced the command with ls but for all intents and purposes, the flow is the same. Here is what I get as a reposnse

{
    'clusterArn': 'arn:aws:ecs:■■■■■■■■■■■■:■■■■■■:cluster/■■■■■■',
    'containerArn': 'arn:aws:ecs:■■■■■■■■■■■■:■■■■■■:container/■■■■■■/■■■■■■/■■■■■■■■■■■■■■■■■■',
    'containerName': '■■■■■■',
    'interactive': True,
    'session': {
        'sessionId': 'ecs-execute-command-■■■■■■■■■',
        'streamUrl': '■■■■■■■■■■■■■■■■■■',
        'tokenValue': '■■■■■■■■■■■■■■■■■■'
    },
    'taskArn': 'arn:aws:ecs:■■■■■■■■■■■■:■■■■■■■■■:task/■■■■■■■■■/■■■■■■■■■■■■■■■■■■',
    'ResponseMetadata': {
        'RequestId': '■■■■■■■■■■■■■■■■■■',
        'HTTPStatusCode': 200,
        'HTTPHeaders': {
            'x-amzn-requestid': '■■■■■■■■■■■■■■■■■■',
            'content-type': 'application/x-amz-json-1.1',
            'content-length': '■■■',
            'date': 'Thu, 29 Jul 2021 02:39:24 GMT'
        },
        'RetryAttempts': 0
    }
}

I've tried running the command as non-interactive to see if it returns a response but the sdk says Interactive is the only mode supported currently. I've also tried searching online for clues as to how to do this but no luck.

Any help is greatly appreciated.

Regarding the error message which you got when you tried to run in non-interactive mode, it's because "Amazon ECS only supports initiating interactive sessions" ([source](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html)). — micmalti, Aug 05 '21 at 14:10

pygeek · Accepted Answer · 2021-08-07T23:15:29.290

The value of the command output is located within the document stream located at streamId. You must initialize a new session and pass it the sessionID to retrieve it's contents.

crude example:

import boto3
import pprint as pp

client = boto3.client("ecs")
ssm_client = boto3.client("ssm")
cluster = "my-mundane-cluster-name"


def main():
    task_arns = client.list_tasks(cluster=cluster, launchType="FARGATE")
    for task_arn in task_arns.get("taskArns", []):
        cmd_out = client.execute_command(
            cluster=cluster,
            command="ls",
            interactive=True,
            task=task_arn,
        )
        
        session_response = client.describe_sessions(
            State='Active'|'History',
            MaxResults=123,
            NextToken='string',
            Filters=[
                {
                    'key': 'InvokedAfter'|'InvokedBefore'|'Target'|'Owner'|'Status'|'SessionId',
                    'value': cmd_out["session"]["sessionId"]
                },
            ]
        )

        document_response = client.get_document(
            Name=session_response.sessions[0].document_name,
            DocumentFormat='YAML'|'JSON'|'TEXT'
        )

        pp.pprint(document_response)

References

SSM: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html

SSM #get_document: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_document

score 0 · Answer 2 · answered Aug 05 '21 at 14:20

0

A quick solution is to use logging instead of pprint:

boto3.set_stream_logger('boto3.resources', logging.INFO)

answered Aug 05 '21 at 14:20

micmalti

561
3
16

theherk · Answer 3 · 2022-01-13T19:33:38.957

I needed to accomplish a similar task, and it turns out it doesn't work as answered here as far as I can tell. Let me know if that worked for you, and how you implemented it, if so.

For me the solution, was to open a websocket connection given back in the session, and read the output. Like this:

import boto3
import json
import uuid
import construct as c
import websocket

def session_reader(session: dict) -> str:
    AgentMessageHeader = c.Struct(
        "HeaderLength" / c.Int32ub,
        "MessageType" / c.PaddedString(32, "ascii"),
    )

    AgentMessagePayload = c.Struct(
        "PayloadLength" / c.Int32ub,
        "Payload" / c.PaddedString(c.this.PayloadLength, "ascii"),
    )

    connection = websocket.create_connection(session["streamUrl"])
    try:
        init_payload = {
            "MessageSchemaVersion": "1.0",
            "RequestId": str(uuid.uuid4()),
            "TokenValue": session["tokenValue"],
        }
        connection.send(json.dumps(init_payload))
        while True:
            resp = connection.recv()
            message = AgentMessageHeader.parse(resp)
            if "channel_closed" in message.MessageType:
                raise Exception("Channel closed before command output was received")
            if "output_stream_data" in message.MessageType:
                break
    finally:
        connection.close()
    payload_message = AgentMessagePayload.parse(resp[message.HeaderLength :])
    return payload_message.Payload


exec_resp = boto3.client("ecs").execute_command(
    cluster=cluster,
    task=task,
    container=container,
    interactive=True,
    command=cmd,
)
print(session_reader(exec_resp["session"]))

This is all that to Andrey's excellent answer on my similar question.

For anybody arriving seeking a similar solution, I have created a tool for making this task simple. It is called interloper.

boto3: execute_command inside python script

3 Answers3

References