1

I'm using NodeJS to pull options from a Postgres database, a very simplified example of the database is here:

OPTIONS

label          choices
style          [{label: 'Flat'}, {label: 'J', price: 10}, {label: 'Curved', price: 20}]
color          [{label: 'Black'}, {label: 'Purple'}, {label: 'Blue'}, {label: 'Orange', price: 40}]

JSON VERSION AFTER PULLED FROM DATABASE

var options = [
{label: 'style', choices: [{label: 'Flat'}, {label: 'J', price: 10}, {label: 'Curved', price: 20}]},
{label: 'color', choices: [{label: 'Black'}, {label: 'Purple'}, {label: 'Blue'}, {label: 'Orange', price: 40}]}
];

GENERATED VARIATIONS

[
    { style: {label: 'Flat'}, color: {label: 'Black'} },
    { style: {label: 'Flat'}, color: {label: 'Purple'} },
    { style: {label: 'Flat'}, color: {label: 'Blue'} },
    { style: {label: 'Flat'}, color: {label: 'Orange', price: 40} },
    { style: {label: 'J', price: 10}, color: {label: 'Black'} },
    { style: {label: 'J', price: 10}, color: {label: 'Purple'} },
    { style: {label: 'J', price: 10}, color: {label: 'Blue'} },
    { style: {label: 'J', price: 10}, color: {label: 'Orange', price: 40}},
    { style: {label: 'Curved', price: 20}, color: {label: 'Black'} },
    { style: {label: 'Curved', price: 20}, color: {label: 'Purple'} },
    { style: {label: 'Curved', price: 20}, color: {label: 'Blue'} },
    { style: {label: 'Curved', price: 20}, color: {label: 'Orange', price: 40} },
]

I wrote a recursive function to generate the variations.

function getCombinations(options, optionIndex, results, current) {
  var maxResults = 5;
  var allKeys = Object.keys(options);
  var optionKey = allKeys[optionIndex];

  var vals = options[optionKey];

  for (var i = 0; i < vals.length; i++) {
    current[optionKey] = vals[i];

    if (optionIndex + 1 < allKeys.length) {
      getCombinations(options, optionIndex + 1, results, current);
    } else {
        // Clone the object
        var res = JSON.parse(JSON.stringify(current));
        results.push(res);
    }
  }

  return results;
}

var variations = await getCombinations(options, 0, [], {});

That recursive function does work for small sets of data, but once you start adding in more variation factors, in my case I have 20 with various sets of choices where I think there is a potential of 10's of thousands of variations, it starts crashing the computer saying that the memory heap is exhausted.

I'm trying to figure out a recursive function that can break up the process into batches so that I can generate the variations 500-1,000 at a time and then somehow iterate through them (via paging or sometype of index) as well as someway to count the total number of potential variations in a performant way.

You can find the full database example here:

https://www.db-fiddle.com/f/dLfzW8MptLhR7SxcP4J3TH/0

It needs to run in node without crashing with that full database example.

Jordash
  • 2,926
  • 8
  • 38
  • 77
  • 1
    Why is this function `async` as opposed to just a regular function? I see no asynchronous operations in it. – jfriend00 May 10 '18 at 04:16
  • 1
    I don't get the point of recursion here, you need to map each style with all the colors, right? If you could add how `options` object is structured in your code it could be useful. – Alex Michailidis May 10 '18 at 06:18
  • It is not clear what relation does this have to the database. You are not showing reading/writing data into the database. And if you are trying to insert many records into the database, you are better off using the example from [here](https://stackoverflow.com/questions/37300997/multi-row-insert-with-pg-promise). – vitaly-t May 10 '18 at 10:49
  • @jfriend00 I added that to see if it would prevent the crashing, but you're right, it makes no difference. – Jordash May 10 '18 at 17:20
  • @alex-rokabilis Good point, I added that into the answer, maybe recursion isn't needed, I can't figure out another way. – Jordash May 10 '18 at 17:23
  • Do you really need to deep clone the object? I would bet you can save a lot of heap/time without it. – bc1105 May 10 '18 at 18:32

0 Answers0