4

While carrying out analysis in R or python we are only aware of feature names (their values) and use them. In Vowpal Wabbit we also have Namespaces.

I am unable to understand: a. what is meant by Namespace; b. how is it different from features; c. when is it used? And when not used? That is, can we avoid using it. d. And how is it used?

Will be grateful for one or two examples. Sorry for so many questions.

smci
  • 32,567
  • 20
  • 113
  • 146
Ashok K Harnal
  • 1,191
  • 2
  • 15
  • 28

1 Answers1

10

In vowpal wabbit name-spaces are used in order to conveniently generate interaction features on-the-fly during run-time without the need to predeclare them.

A simple example format, without a name space is:

1 | a:2 b:3

where 1 is the label, and a, b are regular input features.

Note that there's a space after the |.

Contrast the above with using two name spaces x and y (note no space between the | separator and the name-spaces):

1 |x a:2 |y b:3

This example is essentially equivalent (except for feature hash locations) to the first example. It still has two features with the same values as the original example. The difference is that now with these name-spaces, we can cross features by passing options to vw. For example:

vw -q xy

will generate additional features on-the-fly by crossing all features in name-space x with all features in name-space y. The names of the auto-generated features will be the concatenation of the names from the two name-spaces and the values will be the products of their respective values. In this particular case, it would be as if our data-set had one additional feature: ab:6 (*)

Obviously, this is a very simple example, imagine that you have an example with 3 features in a name-space:

1 |x a:2 b:3 c:5

By adding -q xx to vw you could automatically generate 6 additional interaction features: aa, ab, ac, bb, bc, cc on the fly. And if you had 3 name-spaces, say: x, y, z, you could cross any (or any wanted subset) of them: -q xx -q xy -q xz -q yz -q yy -q zz on the command-line to get all possible interactions between the separate sets of features.

That's all there is to it. It is a powerful feature allowing you to experiment and add interaction features on the fly.

There are several options which accept (1st letters of) name-spaces as arguments, among them:

-q
--cubic
--ignore
--keep
--redefine (very new)
--lrq

Check out the vw command line arguments wiki for more details.

(*) In practice, the feature names will have the name spaces prepended to them with a ^ separator in between so the actual hashed string would be x^a^y^b:6 rather than ab:6 (You may verify this by using the --audit option) but this is just a detail.

arielf
  • 5,802
  • 1
  • 36
  • 48
  • 1
    "so the actual hashed string would be x^a^y^b:6" In practice, the hash of a quadratic feature is computed in a different way (based on the hashes of the two original features and a magic constant), so it is not the same as hash("x^a^y^b"). But this is just a technical detail. – Martin Popel Feb 19 '15 at 10:59
  • 1
    Thanks Martin. Yes, if you really want to know the exact implementation details, the safe way to not miss any detail is to look at the (free and open) source code. There's also `--hash strings` vs `--hash all` which makes a difference when a feature name is numeric. The description above was mostly to explain the concept behind name-spaces while keeping the answer brief (I guess I failed at this :) In most cases the exact hash value, and address where features end up being stored in memory, makes no difference to the user. – arielf Feb 19 '15 at 19:55
  • Thanks for explaining differences between namespaces and features. Features, if I may say so, exist in real world but namespaces need to be created if I want interaction between features. You have mentioned above that new interaction feature is a product of values of two features from different name spaces. But if the two features are categorical is it possible to explain in simple terms as to how the interaction values will be developed. Thanks again. – Ashok K Harnal Feb 20 '15 at 05:18
  • Categorical values are not _directly_ supported by vowpal wabbit. However, there's a cool trick to make binary features (values in `{0,1}`) emulate categorical values: e.g.: `color=red color=blue color=green`, this is possible because the `=` is just part of the feature name string (i.e `=` is not a special separator like `:` is). This topic is orthogonal and has nothing to do do with name spaces. When features are binary interaction just works naturally. BTW: feel free to ask a separate question about how to represent different types of features to `vw`. Thanks. – arielf Feb 20 '15 at 20:00
  • @arief Thanks. Feature representation in VW does perplex me. I have raised another question. [link] (http://stackoverflow.com/questions/28640837/vowpal-wabbit-how-to-represent-features). Please help. – Ashok K Harnal Feb 21 '15 at 01:09