How to discard records?

Question

I send events to AWS Kinesis and these are processed by an AWS lambda function. However, if the lambda throws some error, the records are not discarded, and are processed over and over again, blocking the processing of new records.

I would like rather to skip the faulty record and digest new records. I don't understand how to do that.

The lambda function catches any exception, so it should not give any execution error.

Here below the snippet in python.

I understand that lambda should be retried during the "retention" period (24h default), but I would like to discard and eventually log any kind of errors

def ProcessKinesisRecords(event, context):
    body = "ok"   
    response_code = 200
    for record in event['Records']:
        # Kinesis data is base64 encoded so decode here
        try:
            payload = base64.b64decode(record["kinesis"]["data"])
            #...payload processing
        except Exception as e:
            body = e 

    return {"isBase64Encoded": True, "statusCode": response_code, "headers": { "x-custom-header": "headerValue" }, "body": body}

I think I followed this lambda retries...but I don't see what I am doing wrong

score 0 · Accepted Answer · answered Oct 22 '17 at 19:47

Well, actually I was not catching ALL exceptions. In python

except Exception as e:

doesn't catch all exceptions: Difference between except: and except Exception as e: in Python

It doesn't catch BaseException or the system-exiting exceptions SystemExit, KeyboardInterrupt and GeneratorExit. That's why it wasn't working.

How to discard records?

1 Answers1