← Lab Notes

Your AI Rollout Has a Measurement Problem

17 February 2026

Julie Hendry
Julie Hendry
CTO

You've rolled out ChatGPT Enterprise, GitHub Copilot, or Gemini across your teams. Or you're about to. You've probably run an employee survey: "How satisfied are you with the new AI tools?" Maybe you're tracking adoption metrics: logins, prompts per day, features used.

None of that tells you what you actually need to know.

The question nobody is asking

Satisfaction surveys measure sentiment. Adoption dashboards measure usage. Neither measures whether AI is changing how your people think.

Is your engineering team losing the ability to debug without AI assistance? Are your analysts less confident in decisions they made themselves? Is your team's collective knowledge getting shallower because everyone's relying on the same tool for the same answers?

These are cognitive questions. And right now, most organisations deploying AI have no way to answer them.

We've seen this before

In 2020, we ran a study comparing self-reported smartphone use to actual iPhone Screen Time data. A widely-used smartphone usage scale (MTUAS) showed no significant correlation with real behaviour (r = .198, ns). The tools the field relied on were measuring the wrong thing entirely.

The same pattern is playing out in enterprise AI adoption. Employee satisfaction surveys tell you how people feel about generative AI tools. They don't tell you whether those tools are changing how your workforce thinks, decides, and collaborates. That gap between perception and reality is where the risk lives.

An employee who reports being "very satisfied" with AI tools might simultaneously be experiencing skill atrophy, reduced decision confidence, and increased cognitive load. A satisfaction survey will never surface this. A cognitive impact assessment will.

What should you measure instead?

After reviewing the evidence base across cognitive psychology, organisational behaviour, and human-computer interaction research, we've identified six dimensions that capture what AI is actually doing to your workforce:

Cognitive Load
Is AI reducing mental effort or adding a new layer of complexity? Measured using NASA-TLX adapted for workplace AI.
Decision Confidence
Are people more or less confident in decisions made with AI assistance? And does that confidence match actual decision quality?
Perceived Control
Do people feel they're directing the AI, or that the AI is directing them?
Attention Quality
Is AI sharpening focus or scattering it? Measured using adapted ARCES (Cheyne, Carriere & Smilek, 2006).
Collaboration Quality
How is AI changing the way teams share knowledge and make decisions together?
Skill Trajectory
Are people developing new capabilities or experiencing skill atrophy? The question every CTO should be asking.

These aren't hypothetical concerns. Early research on AI-assisted decision making suggests that over-reliance on AI recommendations can reduce critical thinking and create automation bias. The question isn't whether this will affect your workforce. It's whether you'll notice before it becomes a problem.

Why measure now?

If you're deploying AI tools in 2026, you have a narrow window. You can measure cognitive baselines before AI adoption changes them, or you can try to reconstruct what changed after the fact. One gives you evidence. The other gives you guesswork.

A baseline taken before your AI rollout, followed by measurement at 4 and 12 weeks, gives you a longitudinal picture of what generative AI is actually doing to your team's cognition. Not what they think it's doing. What it's doing.

What this looks like

We're running this as a research study, not a consultancy engagement. Pre-registered methodology on the Open Science Framework. Published findings. Validated instruments. The same rigour you'd expect from an academic study, applied to your organisation.

For our founding cohort (3 to 5 organisations), this means bespoke pricing, a published case study, your name on the paper if you want it, and early access to cross-industry benchmarking data.

If you're planning an AI rollout and want to know what it's actually doing to your people, now is the time to measure.

Founding research cohort

We're selecting 3 to 5 organisations for our first AI workforce cognitive impact study. 30 minutes, no pitch, just a conversation about what you're seeing.

Book a conversation