Skip to main content
What We Found

Stop Optimizing in the Dark: A Smarter Way to Test UX Before You Ship

Nicola Raimondo7 min read

Nicola Raimondo at NetComm Forum 2026, Milan

In May I closed the second day of NetComm Forum in Milan with a simple claim. Most teams optimise their digital products in the dark.

We have never had more ways to gather data, and research has never been easier to run. Yet around 70% of carts still get abandoned, and close to a fifth of that is down to website complexity alone.

The talk above runs about ten minutes. This is the written version, with the numbers and the three things I asked the room to do on Monday morning.

We have never had more data, and still ship blind

Demand for user research has grown 66% in the last year. We can ask almost any question and get an answer back quickly.

So why is demand to test with real people growing at the same time? Because the tools most teams lean on to catch usability problems catch far less than people assume.

14%

of A/B tests produce a positive, statistically significant result. A/B testing confirms what already works. It does not tell you how to design the thing in the first place.

Source: Published win rates from Microsoft, Google and Booking.com
30%

of usability issues are caught by automated scanners. The UK Government Digital Service built a site with 150 known issues and ran the four leading scanners against it. They found roughly a third.

Source: UK Government Digital Service
Of ten usability issues, automated scanners catch about three. The other seven are missed.
Automated scanners catch roughly a third of real usability issues.

Why the other 70% stays invisible

Scanners read your code against a rulebook. A/B tests tell you which version won, not why people hesitated. Neither one watches a real person try to get something done and get stuck.

And you cannot wait for users to tell you. Most of them never do.

1 in 14

frustrated users actually raise a complaint or a support ticket. The other 13 leave quietly, and you never see them.

Fourteen users. One raises the problem with you. Thirteen leave silently.
For every user who tells you something is wrong, thirteen just leave.
£17bn

is lost every year in the UK when people abandon sites they find hard to use. Three quarters of disabled people and their families have walked away from a UK business over poor usability.

Source: Click-Away Pound study

Watching real people is the only way to see it

If scanners and A/B tests miss most of the friction, the answer is to watch real people use the product, at a scale that actually informs decisions.

The way we do this at OpenScouter is to read three streams of behaviour at once during a test, then look for the moments where they line up.

Three streams, behaviour, voice and attention, converging into one strong signal.
One signal is a hunch. Three signals landing together is evidence.
Automated scannersA/B testingWatching real users
What it seesCode against a rulebookWhich version wins a metricWhere and why people get stuck
Usability issues caughtAbout 30%The what, not the whyThe friction the other two miss
Best used forA quick first baselineConfirming what already worksInforming design before you ship

Where neurodivergent testers come in

One option makes friction surface faster: including neurodivergent people on the panel. Around one in five people are neurodivergent, and they tend to notice issues that neurotypical users miss or quietly tolerate.

That makes them an unusually sharp early read on usability, not only on accessibility. It is an option you can add, not a requirement, and not the whole story.

The wider point is diversity. A panel of your own colleagues will not find what real users find, because they already know the product too well.

Where AI helps, and where it does not

AI does the fast, repetitive part. It turns raw signals into structured insight in real time, so the personal data, the video and the rest, is converted into findings rather than stored.

It does not replace real testers. Models are not trained on how a person with ADHD, low vision or autism actually moves through a page. People are unpredictable in ways a model cannot stand in for.

Scale without the Zoom calls

Traditional moderated testing means a researcher on a call walking each person through the flow. Five testers is five hours, and it gets expensive fast.

A tester instead runs the session independently through a browser extension that guides them step by step. That is what makes it possible to go from five testers to fifty or a hundred while keeping the cost down. That is the difference between a one-off study and user testing you can actually run on every release.

A digital product is not what it does. Your product is working when it is working for the user.

Nicola Raimondo
Nicola RaimondoFounder, OpenScouter, NetComm Forum 2026, Milan
Three things to do on Monday morning
  • Stop optimising in the dark. Use A/B testing to confirm what already works, not to make your first design decisions.
  • Add margin to the panel. Your colleagues know the product too well. Diversity beats convenience if you want a representative read.
  • Treat inclusion as a business decision. An accessible product is an easy product to use, and usability is what protects your conversion rate.

See what your users actually experience

Start with a free audit of your site against the four leading scanners. It catches the first 30%. Then watch real people find the rest.

Scouty mascot with celebrating expression

See what 1 in 5 of your customers actually experience

Sign up for a free audit, or book a demo to see how it works for your team. No credit card required.