4

As we move our workloads to AWS I am looking for an ETL tool which is widely used and has the appropriate connectors - Apache Camel appears to fit the bill. However, I am struggling to find information on how Camel can be deployed in AWS - the obvious one is on an EC2 instance, but we would like to avoid the setup and maintenance required by Virtual Machines. I don't see anyone offering it as a managed service, so the option I'd like to explore is running it as a container in ECS, as we will have numerous other containers running.

Containers don't seem to be an installation option on the Apache Camel website - perhaps it is just too limiting for a tool whose purpose is to connect to everything else?

Is it acceptable and practical to run Camel in a container, and where could I find more information about it?

Peter
  • 5,455
  • 7
  • 46
  • 68
  • Camel does containers really well. You have to decide which runtime you want to use. You are essentially deploying Java code, so you have your pick of Java runtimes (Java main, Apache Karaf, Tomcat, Spring Boot, etc..) and a starter container with JDK – Matt Pavlovich May 14 '20 at 02:00

1 Answers1

2

Apache Camel appears to fit the bill.

Indeed the Apache Camel is a great integration framework. And that's the point. It is a framework, not a product. So there are multiple ways to run the Camel flows. As a web app, as a standalone app, as a part if our own code. Camel itself is pretty agnostic to the way you run the flows and that's why you don't have very specific way enforced in the web site.

If you want an out of box product, which can generate containerized deployments with Apache Camel, you could have a look at Apache ServiceMix, Apache Karaf or it's supported RedHat Fuse.

Is it acceptable and practical to run Camel in a container, and where could I find more information about it?

It is perfectly fine.

Question: Can you (are you able) to create a docker container with your (any other) application?. Based on the question this skill is lacking and I really suggest to learn it.

You may check folowing post https://medium.com/@wkrzywiec/how-to-put-your-java-application-into-docker-container-5e0a02acdd6b

FROM java:8-jdk-alpine
COPY ./target/myapp.jar /usr/app/myapp.jar
WORKDIR /usr/app
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "myapp.jar"]

Let's assume you can run your ETL tasks as a standalone application, then just run it in the container as any other standalone application.

it we would like to avoid the setup and maintenance required by Virtual Machines

Question: how do you distribute your camel tasks? I mean - what is result of your build? A war file? A standalone app?

To build a web app you could see https://www.baeldung.com/spring-apache-camel-tutorial

The most convenient way to deploy a war file in AWS is AWS Beanstalk service.

If you build a standalone application (or use servicemix) and you can build a container, then indeed ECS or Fargate seem as natural options.

gusto2
  • 11,210
  • 2
  • 17
  • 36
  • Thanks @gusto2, this is helpful. Yes, I know containers (all our apps and APIs are deployed as containers), but I know nothing yet about Camel, and I wanted to avoid spending time on it if I can't deploy it the way I want to. I'll work through your input, thanks. – Peter May 14 '20 at 10:32
  • The latest version of Apache Mix is from 2017, so that you know. – Edenshaw Nov 30 '21 at 17:11