0

We have two environments, A and B, for the same set of applications.

Environment A's changes (mainly modifications to JSON files storing the applications' configurations) are wholly managed by Puppet. Part of these changes need to be applied on Environment B, where no Puppet runs, by editing the JSON files by hand.

We cannot simply copy these config files from A to B, because on A the files are also modified by other Puppet modules which must not run on B.

Basically the problem here is to extract changes to JSON files made by a specific Puppet run in A, and apply them to B. Running a simple diff won't work as objects in JSON are unordered. I was therefore thinking of sorting the objects before comparing them (related question: How to compare two JSON objects with the same elements in a different order equal?). Are there other simpler options?

Community
  • 1
  • 1
dr_
  • 2,400
  • 1
  • 25
  • 39
  • 1
    When last I checked, Puppet would output file diffs into its log when it applied file changes. This may depend on your logging settings. It should be reasonably straightforward to cull those changes from the log and apply them on a different machine. – John Bollinger Apr 24 '17 at 18:12

1 Answers1

2

When Puppet runs, a JSON document called a catalog is compiled. In the case of your JSON files, these are saved as text inside the catalog. (Yes, JSON documents encoded inside a JSON catalog doc.)

Without seeing your code I can't say exactly what you need to do, but in general terms:

Firstly, write a Puppet class that includes only the classes and resources that you want applied on B, minus the "other Puppet modules which must not run on B".

Secondly, compile a catalog on A using this modified code using puppet master --compile from the command line. Or, you might be able to compile the catalog in an isolated environment on your workstation/laptop, perhaps using Rspec-puppet. You can google how to do that, and I wrote a related blog post that might be useful.

(However you do it, remember of course that you need to feed in the Hiera data by setting the facts so that the correct Hiera files for environment A are traversed.)

I assume you now have a compiled catalog. So now, finally, you need to get your JSON config file out of it. There are many ways you could do this, from writing some fairly simple Ruby code to googling for tools that will do it for you to poking the internals of Puppet itself to get it.

But I think I would probably just open the file in vim, do a global search and replace to replace the \n sequences with real line breaks and then delete everything else.

Hope that helps.

Alex Harvey
  • 14,494
  • 5
  • 61
  • 97
  • 1
    This is an informative and accurate write-up, but I am not sure it addresses the question. John's comment seems to hint at a path forward for the question. Maybe incorporate that into this answer? – Matthew Schuchard Apr 24 '17 at 20:04
  • Actually I find these two sentences describe different problems: (1) "We cannot simply copy these config files from A to B, because on A the files are also modified by other Puppet modules which must not run on B" and (2) "Basically the problem here is to extract changes to JSON files made by a specific Puppet run in A, and apply them to B." I interpreted (2) as something you will not need to do if you simply compile a modified catalog. As for how to do this using diffs, I don't see how that helps because, as the OP says, the JSON would need to be sorted before Puppet generates a diff. – Alex Harvey Apr 25 '17 at 06:42
  • Ok I think I understand what you are suggesting now. You are advising him to glean the file diffs out of the output json from the compiled catalog. Maybe `noop` would also help here? Anyway, sounds good to me. – Matthew Schuchard Apr 25 '17 at 15:33
  • Actually I am assuming that a modified catalog, with classes and resources from the "other Puppet modules which must not run on B" removed, would contain the exact JSON file content that could then be simply copied as-is onto node B. There would be no need to use diffs at all. I guess the OP will need to advise on whether my assumptions are all valid. – Alex Harvey Apr 25 '17 at 16:28