2

I am trying to find out how to deploy a hadoop cluster using ambari by using apache big top

According to the latest release bigtop 1.5: https://blogs.apache.org/bigtop/

my understanding is that Bigtop Mpack was added as a new feature, that enables users to deploy Bigtop components via Apache Ambari.

I am able to install the Bigtop components via command line, but do not find any documentation on how to install these bigtop hadoop components via ambari.

Can someone please help redirect me into some documentation that tells me how to install various hadoop components(bigtop packages) via ambari?

Thanks,

richyu223
  • 21
  • 1
  • 2

3 Answers3

3

I'm from Bigtop community. Though I don't have a comprehensive answer. The Bigtop user mailing list had a discussion recently that has several tech details can answer your question:

https://lists.apache.org/thread.html/r8c5d8dfdee9b7d72164504ff2f2ea641ce39aa02364d60917eaa9fa5%40%3Cuser.bigtop.apache.org%3E

OTOH, you are always welcome to join the mailing list and ask questions. Our community is active and happy to answer questions.

Evans Ye
  • 116
  • 3
1
  1. Build a repo of Big Top
  2. To install that repo with Ambari, you have to register the stack/version. You will need to create a version file. I found an example of one here.
  3. Complete installation like you would with a normal build

This is highly theoretical (..haven't done this before..) I have worked with a BIGTOP Mpack before that took care of some of this work but it's not production ready yet, and works with an old version of Ambari, not the newest. (I was able to install/stop/start HDFS/Hive). These instruction above should work with any version of Ambari.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Matt Andruff
  • 4,974
  • 1
  • 5
  • 21
1

I have been able to test Matt Andruff's theory with a VM. Here was my process and where I stopped;

  1. Built a repo of Apache BigTop 1.5.0
  2. Built BigTop using Gradlew
  3. Installed Apache Ambari 2.6.1 on my system
  4. Enabled BigInsights build version xml file and modified the package version numbers to match my Bigtop build

Note: You can also build your own version file if you want as Matt mentioned

  1. Setup a webserver to host your package repo
  2. Point your xml version file repo to your local webserver for packages

From there you can complete the installation of your packages as you would normally.

I have only done this with a single VM thus far and will be trying to spin up a small cluster using AWS in the coming weeks.

Ryan St Louis
  • 97
  • 7