The bottom-up rigour problem in commercial research

Most commercial qualitative research is built top-down: skim for themes, find quotes to fit. Academic research does it bottom-up. AI now makes that affordable at deadline.

Cover Image for The bottom-up rigour problem in commercial research
Del denne artikel:

Twenty years in consulting teaches you a specific way to handle qualitative data under deadline pressure, and it isn't the way qualitative methodology textbooks describe. Ask twenty industry experts for their views, interview forty client employees about the operating model, collect open text feedback from a hundred customers. The decisions on the table have millions at stake. The report is due Friday.

So instead of reading everything and letting the themes emerge, you skim the materials for patterns that feel right, then go back and find the quotes that support them. Most of the time, the resulting story holds up. It's also, looked at plainly, built backwards from academic standards of rigour, and something about that always felt slightly wrong.


What does academic qualitative analysis actually require?

The gap became obvious talking with Henri Schildt, then a friend and now Skimle's co-founder, about how academic researchers actually conduct thematic analysis. The contrast was stark: transcribe every interview verbatim. Code every passage by hand, not just the ones that jump out. Iterate on the category structure as new data challenges it, rather than fitting new data into categories decided in advance. Academic researchers even have dedicated software, NVivo among the best known, purpose-built for exactly this manual, bottom-up process.

This is the approach Braun and Clarke's reflexive thematic analysis and grounded theory methodology both formalise, in different ways: the theme has to emerge from systematic engagement with the full dataset, not get imposed on a subset of it that happened to get read closely.

It is also, without question, the right way to do it. And it takes hours upon hours of work that a Friday deadline simply does not accommodate.


Why does commercial research skip it?

Not out of laziness, and not because commercial researchers don't know better. It's a straightforward consequence of the economics: a 40-interview consulting engagement or market research project runs on a timeline measured in days, not the months an academic study allows for genuinely exhaustive manual coding. Something has to give, and historically, the thing that gave was systematic bottom-up coverage.

The practical workaround, top-down synthesis, has a real logic to it. An experienced researcher reading a sample of transcripts often does correctly intuit the major themes; pattern recognition built over hundreds of past projects is a genuine skill, not a shortcut taken out of laziness. The risk isn't that the intuition is usually wrong. It's that there's no way to know, on any given project, whether this is one of the times it's wrong, because the process that would catch a missed pattern, systematic coverage of the full corpus, was never run.

Recent methodology literature acknowledges this tension directly rather than pretending it doesn't exist. A 2024 consensus framework published in Implementation Science notes that rapid qualitative analysis "has increased substantially over the past decade" in applied research settings, and argues explicitly that speed and rigour don't have to be incompatible, provided the process is designed carefully rather than simply compressed. That's a meaningfully different claim from "fast and rigorous are opposites." It's closer to: the industry has been treating them as opposites mostly because nobody built the infrastructure to avoid the trade-off.


What changed with AI

The idea that became Skimle started from a specific question: what if you combined the academic bottom-up approach, transcribe everything, code every passage, let categories emerge and iterate, with the speed AI makes possible? Not a shortcut version of rigorous analysis. The actual rigorous process, run at a speed that fits a commercial deadline.

That reframing matters because it changes what AI is for in this context. AI-assisted analysis isn't a faster way to do the top-down skim-and-narrate approach. Used properly, it's what makes the bottom-up approach affordable outside a university, for the first time. Systematic coverage of the full corpus is now possible at the deadlines commercial teams actually work to: every document processed into one analytical structure, so scale no longer forces a choice between depth and delivery.

The realisation that followed was broader than any single project: qualitative analysis is everywhere in commercial life, not just in consulting interviews. Market research, customer insights, user feedback synthesis, legal document review, political speech analysis. All of it has historically been done "expert-driven," which is a polite way of saying top-down: shoot from the hip on what the themes probably are, then find supporting evidence. If rigorous bottom-up analysis becomes affordable at commercial speed, the number of places that method becomes viable expands considerably.


What this looks like in practice

The difference isn't philosophical. It shows up in specific, checkable ways.

Coverage. A bottom-up process processes every document into the analytical structure, not a representative-feeling sample. If forty interviews went into the project, the analysis reflects patterns from all forty, not the dozen a time-pressed analyst had the bandwidth to read closely.

Traceability. Every theme in a bottom-up analysis traces back to specific supporting passages across the corpus, not just the two or three quotes an analyst remembered were vivid. This is the same ground covered in why a citation isn't the same thing as an analysis: coverage, not just evidence for a single claim, is what separates the two approaches.

Emergent, not imposed, structure. In a genuinely bottom-up process, the category structure can change shape as more data comes in, because the categories were built from what the data actually contains rather than decided before reading started. A top-down process tends to lock the story early and spend the rest of the project finding support for it.

None of this requires abandoning expert judgement. The AI does the mechanical first pass; the researcher still frames the question, challenges the structure, and writes the interpretation. What changes is that the mechanical pass now covers everything, instead of covering whatever an exhausted analyst managed to get through before the deadline.


Why this matters beyond any one project

Every one of these domains, consulting synthesis, market research, customer feedback, involves decisions with real consequences riding on an inference from a pile of text. The historical excuse for skipping bottom-up rigour in commercial settings was always a real constraint: time. That constraint didn't reflect a considered judgement that speed matters more than coverage. It reflected the absence of a tool that offered both.

If that tool now exists, the interesting question isn't whether commercial research should adopt bottom-up rigour. It's how many domains currently running on expert-driven, top-down synthesis will look, in a few years, the way manual data entry looks today: a reasonable response to a constraint nobody would choose to keep once the constraint is gone.

For the practical version of this shift, see our guide on why manual interview coding is too slow, and what AI-assisted analysis changes without sacrificing rigour, and how to structure consulting interview synthesis for teams applying this at commercial deadlines.


Try Skimle for bottom-up analysis at commercial speed

If your team has been choosing between rigorous manual coding and fast top-down synthesis, it's worth testing what happens when that choice isn't necessary.

Try Skimle for free and run a full corpus through systematic bottom-up analysis on your next project.

Related reading:


About the authors

Olli Salo is a former Partner at McKinsey & Company where he spent 18 years helping clients understand the markets and themselves, develop winning strategies and improve their operating models. He has done over 1000 client interviews and published over 10 articles on McKinsey.com and beyond. LinkedIn profile


Sources

Grav dybere i dine data med Skimle

Skimle indsamler, analyserer og kategoriserer automatisk interviews, spørgeskemasvar, rapporter og andre kvalitative data. Vores moderne program til kvalitativ analyse forener en grundig og gennemsigtig arbejdsgang med AI'ens hastighed.

Upload tekst eller lyd, fjern følsomme oplysninger med Skimle Anonymise, opret kategorier og underkategorier automatisk, udforsk materialet på tværs af dokumenterne, og eksportér det, så det passer gnidningsfrit ind i din arbejdsgang. Bygget af fagfolk til fagfolk, med fuld beskyttelse af privatlivet og overholdelse af GDPR.

Gratis prøveperiode · Intet betalingskort · Fulde planer fra 20 €/måned