14

I'm juggling code branches that were partly done a few months ago, with intertwined dependencies. So the easiest way to move forward is to mark failing tests on a particular branch as pending (the rspec way) or to be skipped, and deal with them after everything has been merged in.

In its final report, behave reports the number of tests that passed, the # failed, the # skipped, and the # untested (which are non-zero when I press Ctrl-C to abort a run). So behave as a concept of skipped tests. How do I access that?

Eric
  • 2,115
  • 2
  • 20
  • 29

5 Answers5

30

If you want to control things at the command line, then you do what Tymoteusz Paul suggested in another answer. In brief, you can use whatever tag you want to mark your features and scenarios and then use --tags to select or deselect features and scenarios based on the tags you used. The documentation gives the example of marking slow scenarios with @slow and then using behave --tags=slow to run just the slow tests, or using behave --tags=-slow to exclude the slow tests. Reading the documentation is recommended to learn what syntax --tags allows.

By the method above, you could use @skip and do behave --tags=-skip. This will exclude everything marked with @skip but it is annoying to have to include the extra argument with every invocation. Couldn't @skip just by itself tell Behave to skip, without requiring any arguments on the command line?

If you want a @skip tag that will skip features and scenarios marked with it, without requiring an additional argument, then, as of Behave 1.2.5, you must build the functionality into your environment.py file. Contrarily to what this answer suggests, it is not built-in. We add the functionality like this:

def before_feature(context, feature):
    if "skip" in feature.tags:
        feature.skip("Marked with @skip")
        return

    # Whatever other things you might want to do in this hook go here.

def before_scenario(context, scenario):
    if "skip" in scenario.effective_tags:
        scenario.skip("Marked with @skip")
        return

    # Whatever other things you might want to do in this hook go here.

The argument to the .skip methods is the reason for skipping.

I always use .effective_tags to perform tag testing in scenarios. The .effective_tags field inherits tags that were set on the feature. In the case at hand here it makes no difference because if the feature had @skip then forcibly the scenarios in it will be skipped already. However, I prefer to stick to the general principle that tag checks in scenarios should use .effective_tags so that tag inheritance works.

Wait! Doesn't the tutorial say that @skip is built-in?

No, the tutorial presents things a bit misleadingly because it gives a list that shows @skip next to @wip. @wip is built-in so @skip is built-in too right? No, they are in a list is of "Predefined or often used tags". @skip is merely "often used". It is often used because using the word "skip" in an informative way to mark something as skipped. It does not entail that the tag is built into Behave.

Community
  • 1
  • 1
Louis
  • 146,715
  • 28
  • 274
  • 320
  • A slight refinement here would be to add `and not context.config.tags.check(["skip"])` to the if statements, so that an explicit `--tags=skip` will still let you run all the currently skipped tests. – ncoghlan Apr 06 '17 at 05:10
  • A correction to my previous above: the required addition is `and not (context.config.tags and context.config.tags.check(["skip"]))`, as an empty tag expression will otherwise report `True` for every tag. – ncoghlan Apr 07 '17 at 12:02
  • The behave documentation has moved. The new tutorial link (pointing to tags) is https://behave.readthedocs.io/en/stable/tutorial.html#controlling-things-with-tags – MinchinWeb May 08 '21 at 16:13
  • @Louis: Thanks for adding this, it was really helpful, To your question to skip without command can be controlled from environment , i have added my solution below to solve that. – Arpan Saini Jul 17 '21 at 20:14
4

Behave doesn't skip tests, it skips steps in scenario that has already failed. Skipping tests is not supported directly, but if you need to run only portion of your tests then you can control the execution with tags.

elke
  • 1,220
  • 2
  • 12
  • 24
Tymoteusz Paul
  • 2,732
  • 17
  • 20
2

Old question, new answer.

Behave currently supports filtering out tags as in the example below where all tests not tagged with @my_tag_name are executed:

behave --tags=~@my_tag_name path/to/my_tests.feature

Source: from behave help

# behave --tags-help
Scenarios inherit tags declared on the Feature level. The simplest
TAG_EXPRESSION is simply a tag::

    --tags @dev

You may even leave off the "@" - behave doesn't mind.

When a tag in a tag expression starts with a ~, this represents boolean NOT::

    --tags ~@dev

A tag expression can have several tags separated by a comma, which represents
logical OR::

    --tags @dev,@wip

The --tags option can be specified several times, and this represents logical
AND, for instance this represents the boolean expression
"(@foo or not @bar) and @zap"::

    --tags @foo,~@bar --tags @zap.

Beware that if you want to use several negative tags to exclude several tags
you have to use logical AND::

    --tags ~@fixme --tags ~@buggy.
ewitter
  • 23
  • 3
0

How to Skip Scenarios in Python Behave and their application with example:

All function must be declared in environment.py module:

environment.py

Declare skip_scenario = True in before_all()

Add a skip_scenarios= True to control to skip specific scenarios from various feature files., Controller is set in before_all function in environment.py**

def before_all(context):
    pass

    context.scenario_metadata_dict = {}
    context.feature_metadata = {'skip_scenario': True}

Add a Condition in before_scenario():

if any scenario in feature file contain a tag "environmentSkip" and skip_scenario = True Then skip only those scenarios.

def before_scenario(context, scenario):
    if context.feature_metadata['skip_scenario'] and "environmentSkip" in scenario.effective_tags:
        scenario.skip("Marked with skip from environment Control")
        return
    elif "skip" in scenario.effective_tags:
        scenario.skip("Marked with @skip tag in feature File")
        return

Feature File:

will have the tag @environmentSkip tag if you want the tag to be skipped for a spefic scenarios.

  @system-testing @regression @environmentSkip
  Scenario: validate  Global Exclusion Rules for PrimaryEarlyLifeCLI Strategy
   When  derive the expected Results for global Exclusion rule NEGESTAT
      | KEY             | VALUE                                                                                              |
      |test_scenario_id |TS1_GlobalExclusion_NEGESTAT|

Note: Above Example handles both skip scenario, natural way of handling skip and controlled handling skip.

Applications

Nature way : will help developers to skip scenarios for debugging purpose.

Controlled Way: will help to control scenarios for environment specific.

Same controlled can be added for before_feature() also if needed.

Arpan Saini
  • 4,623
  • 1
  • 42
  • 50
-2

You can use predefined "@skip" tag for the scenarios or features that you would like to skip and behave will automatically skip testing the scenario or the whole feature.

shakyaabiral
  • 92
  • 8
  • Untrue. I've just tried adding @skip to one of my scenarios and it was ignored by Behave 1.2.5, just as I expected. `@wip` is predefined as pointed out [here](https://pythonhosted.org/behave/tutorial.html#controlling-things-with-tags). However, I do not find mention of `@skip` anywhere. – Louis May 19 '16 at 09:58
  • 1
    Sorry about that, skip works only for scenarios. So you can put @skip tag to skip a scenario but not the whole feature. For mode details: https://jenisys.github.io/behave.example/tutorials/tutorial11.html – shakyaabiral May 23 '16 at 04:13
  • 1
    The tutorial says "predefined or often used tags". `@skip` and `@slow` are not "predefined". They are "often used". In other words, you are free to use a `@skip` tag if you want, which you can then control with the `--tags` option, *just like any other tag*, which is what the [accepted answer](http://stackoverflow.com/a/36518200/1906307) covers. You can do skip scenarios with a tag named `@skip` or `@potato` or `@asdf`. `@skip` is more readily intelligible but has no special status. – Louis May 23 '16 at 06:45