Test automation strategy: automating the right things
Suites get abandoned when they are slow, flaky or testing the wrong level. A deliberate strategy prevents all three.
ZooBieDoo Infotech Engineering TeamRead
Key takeaways
- Automation finds mechanical issues and creates false confidence.
- Focus order, meaningful labels and announcements need a person.
- Keyboard-only then screen-reader passes catch most real barriers.
- Testing with actual assistive technology users finds the rest.
What automation covers
Automated checks reliably find missing alternative text, insufficient contrast, missing form labels, invalid ARIA and some structural problems. They are cheap, fast and belong in the pipeline on every build.
They also produce false confidence. A clean automated report means the obvious mechanical issues are absent, not that the page is usable.
What only a person finds
- Whether keyboard focus order makes sense and is always visible.
- Whether alternative text actually describes the meaning.
- Whether a screen reader announcement is coherent in context.
- Whether error messages are announced when they appear.
- Whether a custom component behaves as its role implies.
A practical routine
Run automated checks in the pipeline. Then, for each significant feature, tab through it without a mouse and confirm every action is reachable and focus is visible. Then navigate it with a screen reader with the display off.
That third step is uncomfortable initially and is the fastest way to find issues that matter. It takes minutes per screen once you are practised.
Involve people who use assistive technology
Testing by people who rely on assistive technology daily finds issues that competent simulation misses, because expert users navigate differently from occasional testers.
For products with significant reach or legal exposure, this is worth arranging periodically rather than relying entirely on internal testing.
Building a repeatable checklist
Manual accessibility testing benefits from a short standard checklist so coverage does not depend on who is testing that day.
- Tab through the whole feature: is everything reachable and focus always visible?
- Is the visual order the same as the focus order?
- Do form errors get announced when they appear?
- Does the page have a sensible heading structure?
- Does it work at 200 percent zoom without loss of content?
Five minutes per feature against this list catches the majority of issues that automated tools miss, and it is repeatable by anyone on the team.
Frequently asked questions
NVDA on Windows and VoiceOver on macOS and iOS cover most usage. Testing with one thoroughly is far better than sampling several superficially.
Automated checks on every build; manual passes on new and significantly changed features; a full audit at launch and after major redesigns.
Partially focus order and reachability can be scripted. Whether the resulting experience makes sense still requires judgement.
