UXDESIGN.CC
Test smart: which automation strategy to choose for peace of mind?
Among diverse approaches, the teams can find a unique way on their test automation journey and select the strategy that matches their needs. So lets dive deeper into pyramids, diamonds, hourglasses and other interesting objects.In the massive hype around automation in the QA field, it is easy to get lost in various approaches. The multiple perspectives on automating tests coexist. There is no recipe for success but there are some common-sense patterns to follow. Responding to the feedback of the readers of Test smart: how to apply automation and stay sane?, Ive decided to leave my two cents on automation strategies that might be handy for agileteams.No automation strategySometimes having no strategy is also a strategy. As someone who lives in a Mediterranean country, I have noticed an interesting thing about harvesting olives. Some bigger olive oil makers have used electric rakes to collect the olives from the trees. At the same time, others like us, who possess only five not-so-high olive trees, have picked the yield manually.The same refers to the reality of testing. If your product is small-scale (a few quite low olive treese.g. a landing page with registration and login flows), it may be too early and pricy to automate UI tests. Instead, the manual tests could be done promptly.This works well when you are developing application prototypes or working on Proof of Concept, where the focus is more on rapid feature development than maintaining automated tests.Also, if the user flows are raw, it will be a waste of time and budget if you start automating tests. Once the rework is needed in the UI/UX, some intervention will be required for the automated tests,too.Yet, if your product matures and more olive treesflowsare added, get ready to start using electric rakes: automate the tests for the most critical flows where applicable.If you intend to make a lot out of automating your testing efforts, think strategically and involve the team members in a discussion. From my observations, the discussions might be veryhot.Test automation pyramid as the goldenstandardDo you remember the old test automation pyramid?The last time I talked about it to developers, some guys had yawning faces. Yet what if we turn the pyramid upside down to understand it better? Magically, here the pyramid turned into the bugfilter.Look how multiple bugs fly intoit: Unit tests could filter bugs faster (thats why more unit tests are needed!); Integration tests will be your safety net in the middle (you should not neglect thispart); End-to-end (UI) tests will catch the bugs at the latest stage (and more rework will be required).As Janet Gregory and Lisa Crispinnote:Teams that practice test-driven development (TDD) build up a solid base of unit- and component-level tests that help guide code design. In most contexts, teams want to have the biggest proportion of their automated tests at the lowest level of the pyramid.Furthermore, it is smart not to wait until the bugs fly into the products UI layer. Think whether your team can create more tests for the prior layers of the pyramid (unit, integration) and catch the defectsearly.Anyhow, lets be realistic: the pyramid (bug filter) is the goal that should be targeted. However, in the real world, you and your teammates might agree that your current automation approach reminds you more of otherobjects.Diamond alternativeFor instance, your current automation strategy might be reflected in a diamond, where integration tests are in favour. It makes sense for some types of projects. Once your product is based on microservices, integration tests could bring more value than unit tests or end-to-end (UI)tests.In short, integration tests check how an application behaves once it interacts with other services. Usually, the mocks represent external services so that integration tests run in isolation. It is important to note that the main advantage of integration tests is that they are not as flaky as end-to-end tests in terms of maintenance.HourglassYour approach can also look like an hourglass: there is a large suite of unit tests and there is a certain part of end-to-end (UI) tests, yet there is low number of integration tests or they are missing at all. This could work in certain cases but due to the lack of integration tests, the teams risk that more bugs may fly into the UIlayer.As a result, there could be too many end-to-end test failures. You can avoid this case by adding a suite of medium-scope testsintegration tests.Ice creamconeBesides, there might be a pattern of an inverted pyramid, or ice cream cone: many end-to-end (UI) tests are automated, and there are quite a few integration tests but very few unittests.Overall, this pattern should be avoided when planning a testing strategy. The chances of detecting bugs in later stages of development are getting higher, thus it may cost a lot to make additional adjustments. Nevertheless, according to Kristijan Kralj, there are a few exceptions where the inverted pyramid approach could beuseful.The cases where this makes senseare:Putting legacy code into test automationyou are trying to add some automated tests to your application, but UI and business logic are tightly coupled. This means that its not possible to write unit tests, but you need to start with UI and integration tests. After you decouple UI from business logic, then you might add unittests.Your app mainly calls other API-sif your applications logic consists mainly of calling other libraries and/or performing lots of network requests, then it doesnt make too much sense to write unit tests forit.Interaction with the real deviceif you are working on an application that will be deployed on a physical device, then you might benefit more from having UI tests that will check the application worksfine.Who should be in charge of automation?Traditionally, there is a debate about whether test automation should be the sole responsibility of QA Engineers or Developers. To my mind, the truth is in the middle. Janet Gregory and Lisa Crispinsuggest:Because automating tests through the UI tends to be more time-consuming, theres a temptation to hand that off to a separate automation team or have the testers on the team take full responsibility for it. We recommend that the developers, who are good at writing efficient, maintainable code, work together with the testers, who are good at specifying test cases, to automate tests through the UI as well as all other layers above the base level of the pyramid. Having testers and programmers pair or work in ensembles to automate tests saves time. Your team will enjoy better-designed tests whose results you cantrust.From my experience, the unit tests are more in the domain of Developers, whereas creating integration (e.g. API) and end-to-end tests could be done collaboratively. Surely, the QA Engineer could give a hint about tests that should be picked for automation, and together with the Developer, they could speed up the test automation endeavour. After all, lets keep in mind that quality is a teameffort.There is a thing Ive learned working in various teams: automation strategy is unique. This means that even the golden standard of the test automation pyramid might not work for every product. So gather your team and discuss the current state of your test automation efforts and where you aim to get on thisjourney.With some teams, we visualised the testing pipeline to understand the current drawbacks of our automation strategy. I remember drawing a simple image that showed which tests were used for each development stage. Here is one simplified example ofit:Once you see any gaps in your testing pipeline (e.g. certain tests are missing), make sure to discuss with the team what should be improved. Some stages involve human (manual) testing, e.g. exploratory tests or user tests, as automation has multiple constraints, and this isOK.All in all, it is handy to outline how your testing activities are distributed in the development process: note which tests are applied and at which stage. Once you get a clearer picture of how testing is functioning across the development, it will be easier to tackle the main pain points for automation strategy. There is always a space for improvement if you want toimprove.You may check my LinkedIn page if you feel like connecting with me or are curious about my background. As a QA Engineer with strong commercial experience, Im ready to communicate with teams looking for guidance and help in enhancing product quality andtesting.Illustrations: by me (Apple Pencil, iPad, and no AI:))Resources:Janet Gregory and Lisa Crispin, Test Automation Pyramid: https://agiletester.ca/test-automation-pyramid/Ritesh Kapoor, Testing Automation, What are Pyramids and Diamonds?: https://ritesh-kapoor.medium.com/testing-automation-what-are-pyramids-and-diamonds-67494fec7c55Uladzislau Ramanenka, Hourglass into Pyramid: how you can improve the structure of your tests: https://medium.com/bumble-tech/hourglass-into-pyramid-ccf4b4da7785Kristijan Kralj, How to Use Testing Pyramid to Delight Your Boss: https://methodpoet.com/testing-pyramid/Test smart: which automation strategy to choose for peace of mind? was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.
0 Σχόλια
0 Μοιράστηκε
23 Views