5

As a job of a tester, one of my concerns is to always ensure complete test coverage. This can get hard since sometimes the number of possible combinations are really a lot. Lets take an example for instance. A good ol' example of making tea

  • To make tea you could use black tea, green tea or white tea. (3 variables)

  • You could use milk or water (2 variables)

  • you could use sugar or honey or none (3 variables)

  • You could have it iced or hot (2 variables)

As you can see, now if i want to test all the possible ways to make tea (assuming there is a hypothetical software which allows creation of a variety of teas), then i have to test: 3x2x3x2 combinations = 36, because there are indeed 36 unique ways to make tea

What algorithm is best in such a case. I can see a nested for loop being best. Am I right?

Dunaril
  • 2,757
  • 5
  • 31
  • 53
7hacker
  • 1,928
  • 3
  • 19
  • 32
  • 1
    I think you mean combinations, not permutations. They are two different mathematical concepts. – Null Set Apr 08 '11 at 21:11
  • Yes you are right. Im sorry I get confused between the two always. Let me see if I can edit this post - im new around :) – 7hacker Apr 08 '11 at 21:14
  • This is *not* a programming language agnostic question. Various testing tools have various ways to deal with permutations. If you want to know how to do it in one, ask about *that*. – Lasse V. Karlsen Apr 08 '11 at 21:20
  • I would like to know more on the Python language, however I felt it being an algorithmic type of question, it might be useful to consider more on the ideas of algorithms. But yeah, a language specific answer is even better! – 7hacker Apr 08 '11 at 21:25

4 Answers4

2

It can be a bit programming language-dependent... but you're basically looking for the cartesian product of the set of arguments.

for example, in Python

import itertools

for args in itertools.product(
      ['black tea','green tea','white tea'],
      ['milk','water'],
      ['sugar','honey','none'],
      ['iced','hot']
   ): 
   drink_tea(*args)
Jimmy
  • 89,068
  • 17
  • 119
  • 137
  • I would like to explore more of this feature you mentioned, since I am using Python, however I am still at a learning stage. What is this feature of the language called ? – 7hacker Apr 08 '11 at 21:22
  • itertools.product calculates a cartesian product, which is a combination of one element from each sequence given. the `function(*args)` is called argument unpacking or splatting (http://stackoverflow.com/questions/2322355/proper-name-for-python-operator) and expands the given sequence of items and calls the function with them. – Jimmy Apr 08 '11 at 21:25
1

Yep. The nested for loop is best for doing all possible combinations.

It's worth noting that this isn't feasable for most production code, which often have dozens of significant inputs, each with multiple input families.

corsiKa
  • 81,495
  • 25
  • 153
  • 204
  • By infeasible do you mean, since the combinations can get really erratic and large? In my case I test a storage product. So yes, I do have to limit my variables, so I try to focus on those which matter the most while testing a certain feature and I am trying to achieve complete coverage in that feature, among the variables chosen – 7hacker Apr 08 '11 at 21:23
1

Nested looping can get pretty messy. I've written test generators recursively to avoid this and also generalize to multiple dimensions. 

fun build_test_args(possible_values, i, current_values):
     if i>len(possible_values):
         do_test(current_values)
     for value in possible_values[i]:
         cur_list = current_values + [value]
         build_test_args(possible_values, i+1, cur_list)
dfb
  • 13,133
  • 2
  • 31
  • 52
  • This is an interesting approach. So in my example of making tea, and using your Python code, how do we encapsulate all the above variables into "current_values". Can current values be a list of lists ? – 7hacker Apr 08 '11 at 21:29
  • Sure. or you could get fancy and have them be objects containing the item type and a list of possible values. The itertools approach above is equivalent to this one as well. – dfb Apr 08 '11 at 21:35
0

Basically you are forming a matrix... with each variable being another dimension. You would simply multiply this matrix by itself...

MEANING... nested loops would work well.

Create some arrays... and get cranking!

poy
  • 10,063
  • 9
  • 49
  • 74
  • 2
    Matrices are technically always two dimensional. The programming term is just multidimensional array, and the mathematical term is tensor. – Null Set Apr 08 '11 at 21:22
  • Great! So the nested loop approach is fine. Currently though I am using tuples in python with a nested for loop – 7hacker Apr 08 '11 at 21:24