2

Sorry for the dumb question but I could not find an answer for that.

Codeship + Testim.io + Heroku.

In my Staging env - I use Testim.io to test the app once it deployed.
The following tutorial is guiding me how to invoke my tests - but I see the tests are being invoked BEFORE the new app has been deployed - so isn't it testing one version before my latest? I expected the tests to run after the deploy.
Probably I am missing here something.

enter image description here

Benjamin Gruenbaum
  • 270,886
  • 87
  • 504
  • 504
chenop
  • 4,743
  • 4
  • 41
  • 65

1 Answers1

1

In that tutorial - the tests aren't supposed to run against your deployed version they are supposed to run against the version being tested.

The flow is:

  • You set up a local environment - for example by checking out your code and running npm start. If it's containerized then do that.
  • You run the Testim CLI and point its base url to local how testim --token ... --project ... --suite ... --base-url=localhost:PORT.
  • After the tests pass, you deploy.

If you test the version after you deploy you can't be sure the version deployed actually passes your tests.

An alternative flow would be to lean into Heroku's deployment model. Note this isn't actually specific to them and there are similar alternatives in aws/azure/gcp/whatever:

  • In your CI, you set up a staging environment in heroku heroku create --remote staging-BRANCH-NAME-<COMMIT-NAME>
  • You deploy there.
  • You run the tests against that enviroment (by passing --base-url to the Testim CLI, navigating there in your test or using a config file)
  • When tests pass you deploy to master
Benjamin Gruenbaum
  • 270,886
  • 87
  • 504
  • 504
  • Thanks for answering - so I interesting in the 2nd scenario - how do I automate this flow and run the tests after deployment (with CodeShip)? Or is there other alternative frameworks more fit to my needs? – chenop Sep 11 '20 at 15:47
  • CodeShip is fine, it's as good as any other CI here. Heroku CI makes this _slightly_ easier but it's all the same. You automate this flow by running `heroku create` inside your CI (look at the link) and then set the `--base-url` in testim to the URL you got back from `heroku create`. – Benjamin Gruenbaum Sep 11 '20 at 21:08