AMQPException Library error: table too large for buffer

Question

I am using the messenger component in a Symfony application to process messages from rabbitmq.

When I send a 2-3mb message and an error occurs in my handler, the middleware that tries to send the message back to rabbit raises an exception of type:

I found similar error in this links :

https://github.com/vyuldashev/laravel-queue-rabbitmq/issues/10

https://github.com/alanxz/rabbitmq-c/issues/224

https://github.com/php-amqp/php-amqp/issues/131

But I don't see any solution or workaround provided!

Sending 2-3Mb messages seems a bit like an anti-pattern to me. Messages should be tiny, with the necessary information (e.g. ids) to recover the fetch data while consuming the message. Sending as much data through the message queue is rather impractical. — yivi, Aug 31 '21 at 11:42

score 2 · Answer 1 · answered Jan 14 '22 at 13:51

I found a workaround: when the message is being redirected back to queue (to retry later) one huge stamp is being added In my serializer, in method 'encode' I filter stamps:

$allStamps = [];
        foreach ($envelope->all() as $stampKey => $stamps) {
            if ($stampKey  === 'Symfony\Component\Messenger\Stamp\ErrorDetailsStamp') {
                // this header could be huge and drasticaly increase a size of a message
                continue;
            }
            $allStamps = array_merge($allStamps, $stamps);
        }

        return [
            'body' => serialize($messageBody),
            'headers' => [
                // store stamps as a header - to be read in decode()
                'stamps' => serialize($allStamps),
            ],
        ];

AMQPException Library error: table too large for buffer

1 Answers1