Several years ago, I was working on a product that required some attention from the software product teams. This happens to all software over time because a user’s needs change, features need to be added, and bugs happen (naturally). The undertaking was large enough, so our team agreed it would be ideal to talk with real users. We gathered participants and set dates to do either in-person interviews or phone interviews (days before Zoom’s popularity). A subject matter expert (SME) from our side of the house asked if they could “listen in”. It was not our common practice but we acquiesced.
The first interview went well, at least for the first 5 minutes. During the interview, our SME piped up with her thoughts on how the participant user should perform tasks to make their workflow easier. “Oh, no no, you should do it this way and tap that instead,” she mentioned. And on and on this went.
After our session ended, we sat down and had a conversation with our SME about how user interviews are not training sessions. We use them for the product teams to best understand how our product is actually being used. And we’d miss valuable research takeaways if the user thought they were being corrected rather than listened to.
Our best intentions get in the way. It’s hard to listen. When we listen with intent, we can better understand the problems or challenges that need to be overcome. That’s a great life lesson too. So I put together a list of common research mistakes I’ve seen (and made) during software projects and how to avoid them.
☹️ Mistake #1: Not doing any research at all
By far the most common mistake. There are several reasons why research doesn’t happen. It sounds overwhelming, incredibly expensive, and there’s no time to do research. The teams get pressed enough, deadlines go over, and competition delivers that feature to the market while you’re still working on yours. We don’t need research, our teams know what our people need.
This happens a lot when teams have been using software for a long time. We experience what’s called confirmation bias. Someone might say, “I don’t experience issues and no one else does either”. The folks in charge then incorrectly assume there are no issues. Or if issues come up they know exactly how to solve them. So, why research? I’m not picking on you for thinking that way. But here’s the concern, that rationale doesn’t consider day-to-day users. As an admin or power user, you know too much of the inner workings of the problems that exist (and the workarounds). What about onboarding new employees? Are you going to teach everyone the quirks of the system? That sounds expensive and time-consuming.
Avoiding Mistake #1
Research doesn’t have to be overwhelming. Ask yourself, “Who do I need to talk to in order to get feedback on how a feature is really performing?” Then be sure they aren’t staff, but real-world users. Get some feedback. And don’t forget to document it. Surveys are a wonderful place to start.
Research for large organizations can be really expensive. But if you’re a small company, it doesn’t have to be. Think of the cost of rework if a particular feature doesn’t turn out right. Research completed by DORA in 2017 estimates small to mid-sized companies are paying hundreds of thousands of dollars each year for “technical rework”. With improper direction, this is your fate.
You need to understand that investment in research can impact your ROI. It helps identify issues and affordable ways to solve them. You can also hire folks to help. Ehh-hem. We do research as well.
☹️ Mistake #2: Including the wrong people
Even the best intentions if they don’t serve the end goal can produce poor outcomes. This includes getting the wrong people involved. Or getting them involved at the wrong time.
Don’t recruit SMEs for research (at least not yet). When you recruit users who are also SMEs or people involved in building or delivering the software to other users, you’re inviting great feedback. But not the feedback you really want. They know too much and will have a negative impact on how your software is built. I encounter this a lot when we take on new projects that need some investment in design research. I ask for users and what we typically get are internal stakeholders and/or employees. They’re great people to talk with but not for software interviews.
Avoiding Mistake #2
Early on in your process identify what outcomes you want, fit the right research type (see below), then recruit the participant group. I tend to list out who I don’t want as well.
When I’m gathering participants, I look for non-experts. I want to speak with common users who encounter the good, the bad, and the ugly of the software. And then I give them permission to speak their mind. Finding these folks can be hard. Why? Because they tend to be the users who don’t say much. But when pressed they have a list (oh boy they have a list).
Then grab users on the other end of the spectrum. They might be brand new to the company or only use a small portion of the whole platform.
☹️ Mistake #3: Focusing on the wrong problems
I’ve heard it many times. “This one feature (fill in the blank) has been driving me nuts for years and we need to fix it”. And it might be something that should be fixed. But, this approach is like stepping into quicksand and smiling about it. Pragmatism slides out the window when you let your passion projects get in the way of a strategic roadmap for your software. It’s easy to fall into this because of your past success (or failures).
Tyranny of the urgent. These are the “fires” that exist day-to-day and require your attention now. Those sorts of problems crop up all the time and can leave you with a sense of accomplishment. However, in hindsight, your software hasn’t gotten more efficient, productive, or more feature rich.
Avoiding Mistake #3
Software problems typically fall into 3 categories: Feasibility, desirability, and viability.
What will likely have the most impact on most users (desirability)? Is this a feature that will make our business more efficient (viability)? Is this a good use of our technical resources (feasibility)?
An Eisenhower Matrix is a tool I leverage in research to help me identify the impact and related effort. With a sheet of paper and a pencil, you can quickly draw one. On your matrix, effort is vertical, and impact is horizontal. If the effort is high and the impact low, you should not focus on those challenges. If the effort is low and impact low, you shouldn’t tackle those either. Focus your time and energy on the “Goldilocks” zone—low effort, high impact.
This exercise removes urgency precedence as well. It allows you to reflect on what you need to accomplish to be competitive and flexible.
How to get started with research on your own software
The simplest form of research is casual user discussions—no pressure, just a chat. Schedule some time to sit with your users to learn about them and how they use the software. This will help you uncover a few of the problems that currently exist.
Next, you might want to consider hiring out your research. At Volano Software, we can perform several different kinds of research, which will greatly depend on your need.
Three 3 approaches of user research we leverage
At Volano, we have added many forms of research in lots of ways to build the best software products for our customers. The purpose might have a different bend for B2B than B2C; however, the outcomes are largely similar—happy users who get their work done faster with less frustration.
Interested in more content like this? Join our newsletter.
June 13, 2023
Data is like a vast set of building blocks, each has different shapes, sizes, and colors. Just like each brick has its unique utility, every piece of data carries a unique piece of information. As a business owner, how can you possibly start understanding what all the pieces of data from those fancy reports mean? […]
June 2, 2023
For small manufacturing companies with less than 100 employees and revenues of around $20–50 million, several key factors contribute to their success. Here are some important considerations: By focusing on these key factors, small manufacturing companies can enhance their competitiveness, achieve sustainable growth, and maintain profitability. It’s important to adapt these factors to the specific […]
May 30, 2023
There’s an ongoing debate: custom software versus off-the-shelf Software as a Service (SaaS). A few misconceptions tend to cloud everyone’s judgment and influence decisions in this area. It’s time to put these myths to rest and bring clarity to the conversation. Myth 1: Custom Software is Outdated Custom software is inherently outdated, which couldn’t be […]
March 15, 2023
Why continue to utilize a mess of spreadsheets to run your operations? We think there’s a better way. Here are the top 7 reasons you should switch to custom software.
February 3, 2023
Wait. What’s the problem again? Several years ago I was working in Healthcare for a tech startup. At the time, healthcare systems could not bill patients until a chart was signed off and locked by a provider (MD, PA, or NP). The provider had to step through every single screen and check a box regardless […]
September 14, 2022
Why build software of our own? Any good story of a project starts with a problem that needs to be addressed. We work with many clients in the manufacturing vertical and during our time spent with them, we noticed a challenge with synchronization with their dealers. Much of the dealers’ in-office time was spent hunting […]