0

I am trying to setup Ambari + HDP from sources (since Cloudera closed off Hortonworks package repos). Can anyone share experience / howto on this? Documentation is very scarce in this regard.

alfheim
  • 107
  • 5

1 Answers1

2

@alfheim the documentation is here:

https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.7.5

And a post with all the details:

Ambari 2.7.5 installation failure on CentOS 7

Be sure to get the correct versions of npm, maven, node, etc. There are some manual changes you may need to make inside of the source files. You can find quite a few posts solving those issues here on the ambari tag. Go back to pages 2 or 3 to find most recent posts for Building Ambari from Source or just search any errors you may have during build.

steven-matison
  • 1,554
  • 1
  • 9
  • 12
  • But that only describes how to build ambari, and that I managed to do. However I am hitting wall when trying to find out how to build stacks. – alfheim Jul 20 '20 at 08:19
  • Yah, nice how they did that right? So, you have to look at an out of the box horonworks,cloudera fileset. For example if you install 2.74, you can see the stack they put in their repo. You can deliver the same stack, or you can build your own. You can also build a management pack, which will deliver whatever kind of services you want even custom third party services. – steven-matison Jul 21 '20 at 00:16
  • 1
    You can find a management pack I made for Ambari 2.7.5 here: https://github.com/steven-matison/dfhz_hdp_mpack – steven-matison Jul 21 '20 at 00:16
  • 1
    You can find several other examples of management packs I have created here: https://github.com/steven-matison?tab=repositories – steven-matison Jul 21 '20 at 00:17
  • Are you also sharing build scripts for that? Or only final packages? – alfheim Jul 22 '20 at 09:03
  • Sorry but I am not sure what you mean. – steven-matison Jul 22 '20 at 18:13
  • look in /var/lib/ambari-server/resources/ compare this folder with Build From Source 2.7.5 and Hortonworks Cloudera 2.7.4. My hdp mpack makes adding the files for the HDP 3.1.4 stack very easy. Building Your Own stack is just adding the stack files you want to your maven project before creating RPMs. – steven-matison Jul 22 '20 at 18:15
  • Ok, I think that's clearer now. I was under impression that you build stacks and ambari separately, and from what I understand you should add stack files / sources to ambari project, and then build entire thing, right? And you get both RPMs for Ambari & stacks? – alfheim Jul 28 '20 at 14:04
  • That is how 2.7.4 is done. Hortonworks makes additional adjustments as well, including views, adding their own components (example SmartSense). I personally prefer the management pack method to deliver the stack... – steven-matison Jul 28 '20 at 16:10
  • if you look, out of the box ambari 2.7.5 has some of the original stack history.... but it ends where HDP 3 picks up... – steven-matison Jul 28 '20 at 16:11
  • So, I am back to the subject after a year and a bit. I looked at your mpack - so it describes the services that are to be delivered, what about RPMs that you've built? Are they just hadoop/spark etc. wrapped in RPM, or is there more to them? – alfheim Sep 02 '21 at 10:55
  • The repos are copies of the last public stuff before they went behind paywall. I rebuilt some of them to change the expected naming convention. Otherwise they are Hadoop rpms. – steven-matison Sep 07 '21 at 16:17