0

I'm reading the Mesos Architecture docs which, ironically, don't actually specify which components are supposed to run on which VMs/physicals.

It looks like, to run Mesos in HA, you need several categories of components:

  1. Mesos Masters
  2. ZooKeeper instances (quorum)
  3. Hadoop clusters (job nodes? name nodes?)

But there's never any mention of how many you need of each type.

So I ask: How many VMs/physicals do you need to run Mesos with HA, and what components should be deployed to each?

smeeb
  • 27,777
  • 57
  • 250
  • 447

1 Answers1

1

Did you have a look at the HA docs? To run Mesos in HA, you'll need the Mesos Masters and ZooKeeper. Any Hadoop-related configurations are out of scope for Mesos HA itself.

To have a HA setup, you'll need a uneven number of nodes for the Masters and ZooKeeper (because of quorum mechanism). In our case, we're running 3 Master and 3 ZooKeeper nodes on 3 machines (one Master and one ZooKeeper instance per machine), and a number of Mesos Slaves/Agents on different machines.

Theoretically, the Slaves/Agents can run on the same machines as the Masters/ZooKeepers as well. I guess this is a matter of preferences and availability of machines, and your SLA needs.

If you want to run a large-scale production setup, it will probably make a lot of sense to even separate the Master and ZooKeeper instances.

Further references:

Community
  • 1
  • 1
Tobi
  • 31,405
  • 8
  • 58
  • 90