49

I thought I understood that a smoke server was used for continous integration to measure the performance and test coverage of a project. Is this basically correct? What does it mean to smoke-test a program? Does it simply mean to continously apply the tests defined on the smoke server to be able to pinpoint performance degradation to a particular time frame?

This other question does describe the difference between sanity testing and smoke testing but does not ask to define smoke testing. This question is many years older than the reference question.

ojblass
  • 21,146
  • 22
  • 83
  • 132
  • Does this answer your question? [What is the difference between smoke testing and sanity testing?](https://stackoverflow.com/questions/28605496/what-is-the-difference-between-smoke-testing-and-sanity-testing) – Michael Freidgeim Feb 10 '21 at 21:51

11 Answers11

70

Smoke testing is a set of basic cheap to run tests that precede actual testing. It aims to verify that the build is deployed successfully and that all test env. aspects are running and ready for the actual test process. It saves you bringing the full extent of your testing wrath down a faulty build and just realizing that you have been testing on a bad env. or erroneously deployed build possibly too late.

MahdeTo
  • 11,034
  • 2
  • 27
  • 28
  • 1
    I was totally off. Cheap to run... basically an indication of fire? – ojblass Apr 13 '09 at 20:19
  • @ojblass: Yes, what you might me referring to is soak-testing - http://en.wikipedia.org/wiki/Soak_testing. – Vineet Reynolds Apr 13 '09 at 20:22
  • 13
    I'd say it doesn't nessecarily preceed the actual testing. You might use smoke test also to test if a deployment was really successful. – Jens Schauder Dec 05 '11 at 08:51
  • 3
    Also, smoke tests aren't just for checking that env is ok, but you can also check if -in th case of a desktop app for example- that the apps opens correctly and doesn't crash. – disklosr Nov 19 '15 at 10:52
  • @VineetReynolds No, it's definitely not "soak testing". That is testing for a long period of time. Smoke testing is very quick... when you turn it on, does smoke come out? That's it. Soak testing looks for more subtle problems that can only occur after a lot of time "soaking". – ErikE Feb 25 '16 at 23:55
  • I think the term actually stems from real world testing of chimneys - it's a simple safety check to ensure there's nothing preventing smoke leaving a chimney. – Phil Cooper Mar 02 '17 at 14:39
  • Wow, I was completely wrong up to now. I thought a smoke test tests a system under heavy load. Thank you for correcting me. – guettli Feb 17 '20 at 10:40
34

Turn it on, see if any smoke comes out.

andreadi
  • 1,953
  • 1
  • 20
  • 35
25

The term originates in hardware repair and has been applied to software. It's intended to be a quick test to see if the application "catches on fire" when run for the first time. As stated above it's just to make sure you don't waste a bunch of folks time by setting them loose on something that's obviously broken.

Chris Nava
  • 6,614
  • 3
  • 25
  • 31
4

The wikipedia page on smoke testing is actually quite good. It includes a pointer to Steve McConnel's ("Code Complete") paper in the best practices section of IEEE Software 13(4), 1996 titled "Daily Build and Smoke Test".

I'd say smoke testing is a precondition to unit and other forms of testing: if the smoke test fails there's no point in even starting up a unit test.

avandeursen
  • 8,458
  • 3
  • 41
  • 51
4

Smoke testing isn't performance testing or regression prevention testing.

Smoke testing is a set of your highest priority fully automated tests. Even if your regular build succeeds, it's possible for your product to be broken in very fundamental ways that make it 100% unusable. Smoke tests are designed to test that level of core functionality. A build that doesn't pass smoke tests is very likely to fail a majority of all of your automated tests.

If a smoke test fails that generally means that you're losing productivity because the build is not usable, can't be tested, etc. Smoke test failures tend to require immediate fixes, on the order of hours not days.

The important aspect about smoke tests is that they can be run quickly. You should ALWAYS be running smoke tests of some sort. Many groups have sufficient resources to run a larger suite of tests on their daily or continuous integration builds (which is goodness), but smoke tests should be considered the bare minimum.

Wedge
  • 19,513
  • 7
  • 48
  • 71
  • 2
    I don't believe smoke tests are necessarily automated - it can also refer to basic manual tests - which may cover things that cannot really be tested automatically such as layout. – Michael Borgwardt Apr 13 '09 at 20:54
  • True, but we are getting to the point where almost anything can be automated. Be lazy, automate. – Shane Stillwell Jan 17 '13 at 02:10
2

We use the term smoke test to refer to a testing that does quick basic tests to an application or product to make it qualify for further testing like integration testing or functionality testing. It is of little use to do integration testing or other adavanced testing if the application fails to do some of the elementary things required for these advanced testing. For instance we have a web based application and we do smoke test to ensure we can load all the web pages, lets users to login/logout etc. These are the basic things that should work with the application to make it testable. If the pages do not even load or if the users could not even login, we cannot do much with functionality or integration testing. We do smoke testing with every build followed by functionality and integration testing.

msvcyc
  • 2,569
  • 4
  • 24
  • 30
1

Smoke testing is the first step of integration testing. As a developer, you've put all the parts together and you're making sure your app will run before giving it to your QA team for "real" testing.

I wanted to write more, but MahdeTo beat me to a correct answer... :)

Benoit Miller
  • 399
  • 3
  • 9
0

*checking the basic and critical feature of an application before proceeding the testing is known as smoke testing. *In smoke testing we check the positive flow of basic and critical feature to check if the build is testable or not.

vimal tyagi
  • 1
0

I agree with some of the previous answers. To me, Smoke Testing is simply a check if all my X thousands of sub-pages and redirects are still working.

Working means that the status code of each sub-page is "200 OK".

I run Smoke Tests as late as possible. After I deployed my code to the server and before I take this server active.

Today there is even an online service (SaaS) doing the job for me. https://www.smest.it helps me with Smoke Testing. I just entered once my private domain on their homepage and ever since I get my website checked daily. No stress at all.

dj_thossi
  • 774
  • 1
  • 9
  • 18
0

Smoke testing can also be called build verification testing or build acceptance testing, if smoke testing fails it means build is not usable and can not be accepted , this defect needs to be fixed immediately on high priority

kenna
  • 1
-1

From Wikipedia

a smoke test may address basic questions like "does the program run?", "does the user interface open?", or "does clicking the main button do anything?" The process of smoke testing aims to determine whether the application is so badly broken as to make further immediate testing unnecessary.

Hamza AZIZ
  • 2,582
  • 1
  • 9
  • 18