Several years ago, I was working on a product that required some attention from the software product teams. This happens to all software over time because a user’s needs change, features need to be added, and bugs happen (naturally). The undertaking was large enough, so our team agreed it would be ideal to talk with real users. We gathered participants and set dates to do either in-person interviews or phone interviews (days before Zoom’s popularity). A subject matter expert (SME) from our side of the house asked if they could “listen in”. It was not our common practice but we acquiesced.

The first interview went well, at least for the first 5 minutes. During the interview, our SME piped up with her thoughts on how the participant user should perform tasks to make their workflow easier. “Oh, no no, you should do it this way and tap that instead,” she mentioned. And on and on this went.

After our session ended, we sat down and had a conversation with our SME about how user interviews are not training sessions. We use them for the product teams to best understand how our product is actually being used. And we’d miss valuable research takeaways if the user thought they were being corrected rather than listened to.

Our best intentions get in the way. It’s hard to listen. When we listen with intent, we can better understand the problems or challenges that need to be overcome. That’s a great life lesson too. So I put together a list of common research mistakes I’ve seen (and made) during software projects and how to avoid them.

☹️ Mistake #1: Not doing any research at all

By far the most common mistake. There are several reasons why research doesn’t happen. It sounds overwhelming, incredibly expensive, and there’s no time to do research. The teams get pressed enough, deadlines go over, and competition delivers that feature to the market while you’re still working on yours. We don’t need research, our teams know what our people need.

This happens a lot when teams have been using software for a long time. We experience what’s called confirmation bias. Someone might say, “I don’t experience issues and no one else does either”. The folks in charge then incorrectly assume there are no issues. Or if issues come up they know exactly how to solve them. So, why research? I’m not picking on you for thinking that way. But here’s the concern, that rationale doesn’t consider day-to-day users. As an admin or power user, you know too much of the inner workings of the problems that exist (and the workarounds). What about onboarding new employees? Are you going to teach everyone the quirks of the system? That sounds expensive and time-consuming.

Avoiding Mistake #1

Research doesn’t have to be overwhelming. Ask yourself, “Who do I need to talk to in order to get feedback on how a feature is really performing?” Then be sure they aren’t staff, but real-world users. Get some feedback. And don’t forget to document it. Surveys are a wonderful place to start.

Research for large organizations can be really expensive. But if you’re a small company, it doesn’t have to be. Think of the cost of rework if a particular feature doesn’t turn out right. Research completed by DORA in 2017 estimates small to mid-sized companies are paying hundreds of thousands of dollars each year for “technical rework”. With improper direction, this is your fate.

You need to understand that investment in research can impact your ROI. It helps identify issues and affordable ways to solve them. You can also hire folks to help. Ehh-hem. We do research as well.

☹️ Mistake #2: Including the wrong people

Even the best intentions if they don’t serve the end goal can produce poor outcomes. This includes getting the wrong people involved. Or getting them involved at the wrong time.

Don’t recruit SMEs for research (at least not yet). When you recruit users who are also SMEs or people involved in building or delivering the software to other users, you’re inviting great feedback. But not the feedback you really want. They know too much and will have a negative impact on how your software is built. I encounter this a lot when we take on new projects that need some investment in design research. I ask for users and what we typically get are internal stakeholders and/or employees. They’re great people to talk with but not for software interviews.

Avoiding Mistake #2

Early on in your process identify what outcomes you want, fit the right research type (see below), then recruit the participant group. I tend to list out who I don’t want as well.

When I’m gathering participants, I look for non-experts. I want to speak with common users who encounter the good, the bad, and the ugly of the software. And then I give them permission to speak their mind. Finding these folks can be hard. Why? Because they tend to be the users who don’t say much. But when pressed they have a list (oh boy they have a list).

Then grab users on the other end of the spectrum. They might be brand new to the company or only use a small portion of the whole platform.

☹️ Mistake #3: Focusing on the wrong problems

I’ve heard it many times. “This one feature (fill in the blank) has been driving me nuts for years and we need to fix it”. And it might be something that should be fixed. But, this approach is like stepping into quicksand and smiling about it. Pragmatism slides out the window when you let your passion projects get in the way of a strategic roadmap for your software. It’s easy to fall into this because of your past success (or failures).

Tyranny of the urgent. These are the “fires” that exist day-to-day and require your attention now. Those sorts of problems crop up all the time and can leave you with a sense of accomplishment. However, in hindsight, your software hasn’t gotten more efficient, productive, or more feature rich.

Avoiding Mistake #3

Software problems typically fall into 3 categories: Feasibility, desirability, and viability.

What will likely have the most impact on most users (desirability)? Is this a feature that will make our business more efficient (viability)? Is this a good use of our technical resources (feasibility)?

An Eisenhower Matrix is a tool I leverage in research to help me identify the impact and related effort. With a sheet of paper and a pencil, you can quickly draw one. On your matrix, effort is vertical, and impact is horizontal. If the effort is high and the impact low, you should not focus on those challenges. If the effort is low and impact low, you shouldn’t tackle those either. Focus your time and energy on the “Goldilocks” zone—low effort, high impact.

This exercise removes urgency precedence as well. It allows you to reflect on what you need to accomplish to be competitive and flexible.

How to get started with research on your own software

The simplest form of research is casual user discussions—no pressure, just a chat. Schedule some time to sit with your users to learn about them and how they use the software. This will help you uncover a few of the problems that currently exist.

1. Ask questions about pain points they experience.
2. Listen to them (rather than tell them a better way).
3. Document your findings (Google Doc, Word, paper).
4. Work with your software development team to implement updates.
5. Test (is the problem solved?)

Next, you might want to consider hiring out your research. At Volano Software, we can perform several different kinds of research, which will greatly depend on your need.

Three 3 approaches of user research we leverage

1. Casual user discussions
   • Day in the life Leveraged for more general research or holistic understanding of the software users.
   • Task completion When a specific job to be done is a challenge and needs to be sorted.
   • Challenge & solutions Understanding what challenges the user faces and how they think it could be solved.

• Formal research
  • In-person or video interviews: Gather more detailed information about an entire workflow and the use cases.
  • Screen recording with empathy mapping: Used to document the interviews. Users are asked to share what they’re thinking about as they walk through the software.
  • Ethnographic research: Sit-in or live-in style research that seeks understanding beyond clicking software. But the challenges users face as they use the software.

• Multi-channel research and feedback
  • Surveys
  • User interviews
  • User testing
  • Mapping

Recap Summary

1. Don’t skip research. It has its own ROI and plays a role in successful projects small to large.
2. Involve the right people. Asking the right people makes a difference in the feedback you get.
3. Solve the right problems. Focus on documenting what to solve and the effort it will take by leveraging an Eisenhower Matrix.

At Volano, we have added many forms of research in lots of ways to build the best software products for our customers. The purpose might have a different bend for B2B than B2C; however, the outcomes are largely similar—happy users who get their work done faster with less frustration.

Interested in more content like this? Join our newsletter.