Smoke and Regression Testing: Sign-in Methods
Add Mobile-ID, run smoke checks, and find two regression defects in existing sign-in methods.
- 1Regression
- 2Task & design
- 3Smoke
- 4Mobile-ID
- 5Regression
- 6Retest
- 7Release
Your first day on the team
Hi! What’s your name?
Hi! Welcome aboard!
Get to know the product
Here is our product — the account sign-in form. An existing user can sign in with a password, Smart-ID, or Google Auth. Take a look and try the different methods. This is a test environment: use the credentials shown below the fields; external services are replaced with mocks.
Once you’ve had a look, I have your first task.
One moment, I’ll take a look.
All clear, let’s move on
Let’s quickly check that you’re ready to get started with the team. What is our product about?
Take another look at the form and try again.
Correct! Which sign-in methods do we support today?
Take another look at the form and try again.
Ready!
Soon we’ll add another way to sign in. Before changing anything, we need a regression set — a list of what already works and will need to be checked again after the change. That’s your first task.
What works today? — Let’s build a regression test set
Here’s your first task! Right now, users can sign in with a password, Smart-ID, or Google Auth. Before we add a new method, please put together a regression set.
What are you including in the regression test suite now?
Not quite. The regression set should include every sign-in method the product already has — and nothing that isn’t in the product.
The new task
Hi! Next up, we’re adding a new sign-in method: Mobile-ID. It should go at the end of the list, after Google Auth.
Here’s how it works: the user selects Mobile-ID, enters their phone number, sees a verification code, and confirms the request on their phone. Once they confirm, their account opens; if they cancel, they stay on the sign-in form and can try again.
Please also make sure all the existing methods keep working just as before.
Hi Mari! Could you please show how Mobile-ID will look on the sign-in form?
Mari is working on the mockup
Here it is! I’ve added Mobile-ID to the list of methods. The phone field and verification code appear only after it’s selected, and the button stays in place.
Sign-in form: Password, Smart-ID, Mobile-ID selected, and Google Auth. Below the methods are a phone number field showing +372 51234567 and a Continue button. Dashed arrows point to the new method and the fields shown when it is selected.
{name}, before Mark starts, could you please check the mockup against the requirements? Is everything right?
Compare the order of the methods in the mockup with Krista’s task.
Oh, you’re right — thanks for catching that! I’ll fix it now.
Mari is working on the mockup
Fixed: Mobile-ID is now at the end of the list, after Google Auth.
Sign-in form: Password, Smart-ID, Google Auth, and Mobile-ID selected at the end of the list. Below the methods are a phone number field showing +372 51234567 and a Continue button. Dashed arrows point to the new method and the fields shown when it is selected.
Mark, the task and the mockup are ready — you can start.
Mark is adding Mobile-ID
Build 2.0 is on the test server. Mobile-ID is ready to test.
Quick build check
Let’s do smoke testing! Open the form and check that:
- four sign-in methods are visible
- the layout is intact
- the sign-in button is available
This is a short smoke check, without testing every authentication outcome.
What is the smoke result?
Choose a method and press “Continue” in the form to check the button.
New feature testing: Mobile-ID
This is new feature testing: check only Mobile-ID — the other sign-in methods will be covered in regression. In the form, try both outcomes: confirm the request and cancel it, then retry after cancelling. Tick each check in the list yourself once you’ve done it.
Check existing methods
Run the regression set you saved in the first step. Check valid and invalid sign-in, session persistence, and confirmed and canceled external sign-in. These paths were outside the smoke check.
Run the scenario in the form — the check then unlocks and you can mark the result yourself: PASSED or FAILED.
- Password: valid credentials open the account.
- Password: invalid credentials show an error.
- Password: “Remember me” keeps the session after reload.
- Smart-ID: confirmation opens the account.
- Smart-ID: cancellation keeps the user on the sign-in form.
- Google Auth: confirmation opens the account.
- Google Auth: cancellation keeps the user on the sign-in form.
Analyze two defects
Two scenarios were marked FAILED during regression of build 2.0. Write bug reports for them: for each one, choose the expected result — how it should work according to the requirements — and the actual result — what you saw in the form.
Please describe both defects: what should have happened and what actually happened. With the steps I can reproduce and fix them right away.
Steps: Sign in with a password with “Remember me” ticked, then reload the page.
Steps: Choose Google Auth, press “Continue”, then “Cancel”.
Compare the intended behavior with what the form showed.
Verify the fixes
BUG-201 and BUG-202 were sent to Mark. Release remains blocked.
Thanks for the bug reports! I fixed both defects and put build 2.0.1 on the test server — please check.
Mark fixed both defects and published build 2.0.1. Repeat both scenarios, then check that ordinary sign-in still works.
Run the scenario in the form — the check then unlocks and you can mark the result yourself: PASSED or FAILED.
- Password with “Remember me” survives reload.
- Canceling Google Auth keeps the user on the form.
- Ordinary password sign-in still works.
Release decision
Smoke and the new feature passed; the two regression defects were fixed and retested. Can build 2.0.1 ship?
Both reported defects were fixed and verified. No defects remain open.
Krista, testing of build 2.0.1 is complete. Mobile-ID works as required, and both regression defects were fixed and retested. There are no open defects — the build is ready for release.
Great! Let’s release.
Krista has given the go-ahead. Shall I deploy build 2.0.1 to production?
Yes, deploy it
Mark is deploying the release
Done! Build 2.0.1 is in production.
Hi everyone! Mobile-ID sign-in is now live in production.
Build ready for release
You tested Mobile-ID, found two defects in existing methods, and verified their fixes.
Choose a method
Test credentials: qa@example.com / Demo123!
Topic reference
Smoke testing
A short check of the most important functions of a new build: does the application open, do the key elements work. If the smoke test fails, the build goes back to the developer — further testing makes no sense.
Retesting
Checking a specific defect again after it has been fixed: you repeat the steps that reproduced it and make sure the defect is gone.
New feature testing
Checking that a new feature works according to the requirements: the main scenario, alternative scenarios, and error handling.
Regression testing
Checking that the changes have not broken what already worked. It helps to define the regression set in advance — before work on the new feature starts.