It seems that the request/response limit is lower when using ALB, and WebSockets are not supported:
The maximum size of the request body that you can send to a Lambda
function is 1 MB. For related size limits, see HTTP Header Limits.
The maximum size of the response JSON that the Lambda function can
send is 1 MB.
WebSockets are not supported. Upgrade requests are rejected with an
HTTP 400 code.
See: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/lambda-functions.html
Payload limit with API Gateway is discussed here: Request payload limit with AWS API Gateway
Also the article already mentioned by @matesio provides information about additional things to consider when choosing between ALB and API Gateway.
Notable tweet referenced in the mentioned article:
If you are building an API and want to leverage AuthN/Z, request
validation, rate limiting, SDK generation, direct AWS service backend,
use #APIGateway. If you want to add Lambda to an existing web app
behind ALB you can now just add it to the needed route.
(From: Dougal Ballantyne, the Head of Product for Amazon API Gateway)