From writing code to understanding users

anusha-c · May 12, 2026, 12:50pm

Running user testing from the inside

Modern engineering teams are great at building systems that work. But building something that works doesn’t always mean it works well for users. Bridging that gap requires understanding how people actually experience what we build.

In this blog, I’ll share how I transitioned from focusing purely on implementation to actively running user testing, what I learned from conducting in-depth research at Canonical, and how these insights began shaping both our product decisions and my day-to-day work as an engineer.

The challenge

I’m a software engineer working on Anbox Cloud. Over time, through QA reviews and working closely with designers and other developers on my team, I began to notice something subtle but important: the biggest gaps in our product weren’t technical. They were experiential.

We shipped features that worked perfectly from a systems perspective. The logic was clean, the architecture was sound. But when users interacted with them, friction surfaced in ways we hadn’t anticipated.

That realization pushed me outside my comfort zone and into the field of in-depth user research. In digging into how we do user testing at Canonical, I got support from both a skilled Product Manager, Lidia Luna Puerta, and our lead of User Science, Alorah Harman. With their input I was able to design a robust, objective protocol.

The solution

Over multiple weeks, I led our team in conducting interviews across two different research tracks with five participants each. The goal was to deeply understand pain points, expectation mismatches, and areas where the user experience unintentionally created friction. These are the “papercuts” that can unintentionally degrade our product’s quality by negatively impacting the user experience. That shift changed how I work as an engineer. I now think of the users when building, not just when testing if something works.

Since this mindset shift was so impactful for me, I was inspired to share the principles behind it with our broader web engineering department, a team of 59 engineers. I wanted to ensure the user testing best practices that I had benefited from were shared with the wider team.

To that end, I teamed up with User Science again to design a workshop for our company-wide sprint in Gothenburg. The goal was to gain hands-on experience with fast, objective-driven user testing. Through specific exercises, engineers would be able to pick up research practices quickly at scale and feel empowered to learn more and implement them further if they deemed deeper research to be necessary. Additionally, through an in-person, department-wide, rapid iteration testing format, teams could quickly validate a few existing assumptions, gather real feedback, and prioritize improvements accordingly.

Inside the workshop

The workshop was designed around two parts:

Building a foundation for objective-driven user testing
Applying that foundation immediately through a structured live exercise

Part 1: Building the foundation

Start with the objective

At Canonical, design thinking is one of the practices we’re building up in our engineering organization as a whole, through initiatives like the Design Academy for technical authors. One of the most critical aspects of design thinking for engineering teams is a heavy emphasis on solving the right problems and early product derisking, which involves validating assumptions early so we reduce the risk of building the wrong product.

Similarly, most research goes wrong before it even begins, by jumping ahead too quickly. The problem is in the framing. When you’re planning user testing, the most important question isn’t: “What questions should I ask the user?” Instead, the question should be:

What do I want to learn?
Why do I need to learn this?
How will I use these insights?

If you cannot answer those three questions clearly, your research will fail to achieve the desired impact.

For example, during our research on Anbox, one objective was to learn about the discoverability and effectiveness of our onboarding tour –a step by step walkthrough of key user flows that guided users through the dashboard without relying on external documentation. The onboarding tour was a step-by-step walkthrough of critical userflows on the journey that guided the user in learning how to use the dashboard without relying on external resources like our documentation. We suspected it was helping users, but we didn’t actually know whether new users were noticing it, using it intentionally, or genuinely finding it helpful.

If the tour was being overlooked, the problem was discoverability. If it was being used but not reducing confusion, the problem was effectiveness. Depending on the answer, the roadmap implications would be completely different: from UX refinements to structural redesign.

Clarity at the objective stage shaped everything that followed: the questions we asked, the behaviors we observed, and how the insights translated into product decisions.

Ask better questions

Once your objective is defined, your questions should exist solely to address it. Before crafting strong questions, it helps to recognize the patterns that weaken user testing.

Here are common mistakes we discussed in the workshop:

Closed or binary questions
- This limits the response to yes or no and prevents deeper insight.
- For example: “Did you like the onboarding tour?”
Leading questions
These questions tend to make assumptions about the user and prevent them from expressing their true opinion.
For example: “How well did the tour improve your workflow?”
Overly broad questions
- These questions dilute focus and makes the feedback unusable.
- For example: “Did you like the product?”
Double-barreled questions
- These make responding difficult and confusing. If the answer is yes, which part was yes? If no, which part failed?
- For example: “Was the dashboard fast and intuitive?”

Instead, questions should be focused, neutral, and tied directly to the objective.

Use structured questioning methods

To improve the quality of our user research, I shared the TEDW framework for user testing, using our Anbox onboarding tour as an example.

T - Talk me through / Tell me
- This helps uncover expectations and prior experiences.
- For example: “Tell me about the last time you went through the onboarding tour. What do you remember about it?”
E - Explain
- This helps gather logic, motivation, or reasoning.
- For example: “Explain what you were expecting from the tour before you started it.”
D - Describe
- This reveals emotional friction, confusion, or confidence.
- For example: “Describe how you felt as you went through the different steps of the tour.”
W - Walk me through
- This exposes behavioral patterns and breakdown points.
- For example: “Walk me through what you did when you reached the end of the onboarding tour. What was the next thing you tried to do?”

Listening is a skill

Capturing effective user feedback is multi-dimensional. It involves asking structured questions and approaching the conversation skillfully. Let’s dive into some of the ways you can ensure your feedback sessions are fruitful.

The first is encouraging participants to think aloud. This can feel unnatural at first, both for them and for you. However, hearing someone verbalize their thoughts in real time reveals their decision-making process, not just their final opinion. By revealing a participant’s thought process, you can better understand their approach to your product and bridge any existing knowledge gaps.

Silence plays an equally important role. When a participant pauses, there is often a temptation to fill the gap, to clarify, to guide, or even to help. Resist that urge. In real-world usage of the product, you won’t be there to assist them. Allowing silence creates space for organic confusion, hesitation, or realization to surface and be captured.

There are also small conversational techniques that make a significant difference. One of them is the echo method. If a participant says, “This feels weird,” instead of defending the design or reframing the question, simply repeat the word back as a question: “Weird?” That subtle prompt often encourages them to expand on what they’re experiencing, leading to more effective data.

Similarly, the “Five Whys” technique helps uncover root causes beneath surface preferences. The idea is to keep asking “why” repeatedly until you reach the underlying reason behind a user’s reaction, not necessarily stopping at five, but going as deep as needed.

If someone expresses a preference or frustration, gently ask why, and then why again, based on their response. Not interrogatively, but curiously. Many users don’t immediately understand the underlying reason for their reaction, which is what you need to identify in order to make a product more effective. Peeling back those layers often reveals deeper constraints, expectations, or unmet needs.

These techniques transform surface-level feedback into meaningful insight. They shift user conversations from opinions to structured feedback loops.

Part 2: Applying user testing in practice

Running fast user research

After walking through the foundations, we moved straight into a learn-by-doing application.

Nine different teams participated in the exercise, with 25 engineers in total. Each participant began by defining one clear research objective for their assigned product, along with a few focused questions designed to directly support it.

To make this easier, we provided a simple template for outlining objectives and key research questions, inspired by our Director of Documentation Daniele Procida’s value-mapping format:

Once objectives were defined, participants were paired across teams. Each pair ran two rounds of rapid testing so that everyone experienced both roles: interviewer and participant.

Each round lasted 15 minutes:

10 minutes observing the participant use their product
5 minutes asking targeted, objective-driven questions

After the first round, the roles of interviewer and participant were reversed. After completing those two rounds, participants were reshuffled into new pairs and repeated the process again. In total, four rounds were conducted. This allowed each person to observe multiple products across teams and gather both perspectives on methodology and product insights from a broader range of participants.

This compressed format in the workshop was intentional. As a “first taste” of user testing, it removed the barrier of heavy preparation and demonstrated that even a short, structured session can generate meaningful product insight. This goes to show that when user testing becomes part of the day-to-day for engineering teams, our findings over time become more and more rigorous and objective. However, running the sessions was only part of the exercise. Making sense of what we observed and turning it into actionable insights was just as important.

Capture insights systematically

Collecting user observations is easy, but turning them into actionable insights is harder. This can be one of the biggest bottlenecks for effective user feedback loops on an engineering team. To avoid scattered notes and forgotten conversations during the exercise, we provided a highly structured template to document findings. Each entry required participants to connect their observation back to their specific objective and categorize it clearly.

When insights are labeled into clear categories such as UX oversights, expectation mismatches, visual improvements, or potential new features, that’s when patterns begin to emerge across interviews.

Here’s an example of how insights could be captured:

Capturing insights systematically makes pattern recognition easier and prevents valuable observations from being forgotten. This is how structure takes us beyond the realm of opinions and into rigorous practice.

What changed

What began as an uncomfortable step I initially saw as “outside” my role became a lens I now apply daily.

The in-depth user testing on Anbox shaped roadmap priorities. The workshop enabled other teams to adopt user testing practices within their own workflows. And personally, it fundamentally changed how I approach feature development. Ultimately, I’m proud to do my part in strengthening Canonical’s culture of design thinking and user feedback across our engineering teams.

I no longer see my engineering work as simply delivering functionality. I see it as shaping experience. Once you begin observing how people actually use what you build, it becomes difficult to return to assumption-driven development.

The question isn’t whether you have time for research. It’s whether you’re comfortable building without it.