How to read S3 file only once in Apache Camel

Question

My DSL start is something like this:

from("aws-s3://" + s3_bucket_name + "?amazonS3Client=#amazonS3Client&deleteAfterRead=false&fileName=myfile.csv")

after this I covert each row into a JSON file and dump into a local directory.

The problem is it keeps on doing this like its stuck in an infinite loop.

Any idea how I process the file only once and then stop?

score 1 · Answer 1 · answered Nov 26 '17 at 07:50

1

You can read this FAQ about how to stop a route from a route: http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html

Or instead of a route you use a ConsumerTemplate to poll the s3file only once.

answered Nov 26 '17 at 07:50

Claus Ibsen

56,060
7
50
65

2

Hi @claus : is it possible to provide any example/reference link which can explain use of `ConsumerTemplate` to achieve this purpose? – Vishal Zanzrukia Sep 06 '18 at 17:55

score 0 · Answer 2 · answered Nov 27 '17 at 18:17

You can use Idempotent Consumer

from("aws-s3://" + s3_bucket_name + "?amazonS3Client=#amazonS3Client&deleteAfterRead=false&fileName=myfile.csv")
.idempotentConsumer(header("CamelAwsS3Key"), idempotentRepository)

You can provide implementation of idempotentRepository based on your need as mentioned in above link.

How to read S3 file only once in Apache Camel

2 Answers2