Improved way of scaling in saltstack

Question

I have a problem about the Jinja2 template and that problem is breaking a one line string over multiple lines when it comes to writing a state or anything in salt [my exact case refers to trying to write a list of machines one after the other,in a list,instead of just in a really long line].

What I am trying to say is that I want to achieve this:

nodegroups:
  - group: 'L@adsdasdadas' +
            'dasdasdasdas'
             .............->imagine 10.000 names coming here
             'adsasdasddsa'

Compared to the approach that I have to do now:

nodegroups:
  - group: 'L@adsdasdadas,dasdsadasdsa,dasdsadasdsa,......,asdqwe'

Is there a better way to do it?Is there a better way to handle thousands of machines?

You could say grains,and I thought about it but I was wondering if there's a better and elegant way of doing it.

Any thoughts or opinions would help me a lot

[Edit1]: I wrote a script that takes a list of hostnames and adds them to the master config file in the nodegroups section.For now it might work

Did you try **[targeting minions through pillar](https://docs.saltstack.com/en/2014.7/topics/pillar/index.html#targeting-with-pillar)**? If it works, nodegroup can be defined through it. — uvsmtid, Jul 22 '15 at 03:56
@uvsmtid I did try but somehow It doesn't work...I will try and research a little bit about setting ndoegroups from the command line — tudoricc, Jul 22 '15 at 08:16
it didn't work for me right away too, but this might be [an existing bug](https://github.com/saltstack/salt/issues/10900). — uvsmtid, Jul 22 '15 at 14:28

score 4 · Accepted Answer · edited May 23 '17 at 12:26

4

Choice of data source

I would recommend targeting with pillars because they are managed centrally from Master = convenient, rather than static custom grains (which are configured distributively on each Minion) = inconvenient - see comparison summary here.

Limitations of configuration files

The nodegroups are specified in Salt configuration file /etc/salt/master which is not a Jinja template (it has pure YAML format). So, you don't have an option to use Jinja to join external input with list of strings.

Possible solution

Why joining is even mentioned? You can turn the problem of "breaking a one line string over multiple lines" into solution of using lists right away - no need to break (and if you need "one line string" somewhere, joining list items is easy).

In other words, you could define nodegroups via pillar (avoiding long strings as in your example). Pillars, in turn, are rendered by Jinja. Therefore, using the same list of Minions defined somewhere, you could generate derived product in pillars through Jinja (be it joined string of them or list as is). There is a trick which allows reusing the same external data in multiple pillars files.

edited May 23 '17 at 12:26

Community

1
1

answered Jul 22 '15 at 15:00

uvsmtid

4,187
4
38
64

Ok,so this is actually brilliant.Thank you so much for the idea and sorry for the confusion – tudoricc Jul 22 '15 at 15:27
@tudoricc I hope it works for you! I used to experiment with `nodegroups`, but I don't use them in my use cases anymore. My current specific is complex orchestration of several < 10 nodes - single group `*`. :) So, I would appreciate if you could post a follow comment when this [or that](https://github.com/saltstack/salt/issues/10900) is solved for you. – uvsmtid Jul 22 '15 at 15:34
could you please explain what you mean with your current setup? – tudoricc Jul 22 '15 at 15:40
I looked at your answer and I realized that I thought you really needed to extract values from `nodegroups` configuration. But you achieved what you want differently. So, nevermind. – uvsmtid Jul 22 '15 at 15:45

score 2 · Answer 2 · answered Jul 22 '15 at 15:39

2

First of all I would like to thank uvsmtid for the wonderful idea.Sorry for the confusion created too

So,what I did was create a pillar with the name of each minion[which happens to be it's id] and then in a state what I did was compared the value from that list to the actual id of the minion

{%for item in salt['pillar.get']('info') %}
  {%if grains['id'] == item %}
    something:
      cmd.run:
         - name: touch something

  {%endif%}
{%endfor%}

I hope this solution will help someone the same way it helped me

answered Jul 22 '15 at 15:39

tudoricc

709
1
12
31

I also have comment for this. Normally, I define a host role and assign minions to the role **[see `assigned_hosts` here](https://github.com/uvsmtid/common-salt-states/blob/bd4c96f0b236c5e5c4153b1f1692be362ae551d5/pillars/profile/common/system_host_roles/main.sls#L65-L68)**. Then in states I can use minion id to match against those listed for the role - **[like this](https://github.com/uvsmtid/common-salt-states/blob/b819961dbb683285329084952f24dab6f30d07c6/states/common/orchestrate/wraps/jenkins_slave_role/init.sls#L3)**. This way only minions with specific role get specific configuration. – uvsmtid Jul 22 '15 at 15:53
now what I just wrote is a pretty basic thing.but yeah,what you wrote should be the next step for me.Thx a lot – tudoricc Jul 22 '15 at 21:10

Improved way of scaling in saltstack

2 Answers2

Choice of data source

Limitations of configuration files

Possible solution