4

I am using Kettle for syncing an old relational database running on several clients PCs. After skimming through the book "Enterprise Integration Patterns" I got convinced that I should use a message queue (MQ) as the communication channel between the cloud and the on-premises PCs. I chose RabbitMQ because it uses an open standard (AMQP) and it seems popular.

Anyway, my kettle transformation seems to working fine when doing tests without the MQ. How do I integrate both now? I have done some research and I found hints (see link) that other people are doing the same thing, but I didn't find an out-of-the-box solution for me (considering I don't know a thing about Java).

Notes:

  1. The on-premises DB is running on PCs (Windows) and the cloud server is running Linux (Amazon Linux or Ubuntu). I want to have kettle running as a windows service in the PCs, so that the clients don't close it by accident;
  2. I don't know a thing about Java, but I am fine with shell scripts, batch files, python, ruby and a bit of javascript.
zero323
  • 322,348
  • 103
  • 959
  • 935
Giovanni Bonetti
  • 102
  • 3
  • 8
  • 1
    Many times in the kettle topics end up with "it is not built-in but you can write your own plug-in".. Of course, that requires java skills. Also surprising that no built-in step is there to ease MOM integration. – Myobis Nov 06 '13 at 10:19
  • 2
    Yeah, I ended up writing my own JAVA steps - https://gist.github.com/giovannibonetti/7878008. One file is the input step, and the other is for output. It is very rough, but it works for me. – Giovanni Bonetti Dec 09 '13 at 18:37
  • How you used rabbitmq with Pentaho Kettle? Can you please share some details with me. – Helping Hand.. Jun 16 '15 at 11:43

0 Answers0