Don’t Confuse Concept Testing With VoC

If we bring a prototype to a customer, this is “concept testing.” Something different from voice of the customer research.  But when definitions are not understood, these become conflated and we fool ourselves into believing that we’re more customer-centric than we really are.

This confusion results from how innovation and new product processes have evolved.

In an earlier era, we used to believe that the new product development cycle began with a new product idea. Early models of phased gate systems even began with a little light bulb, representing the idea, at the beginning of the process. And this older paradigm is also responsible for decades of failed new products. But that’s a story for another day. After all, we also once believed that the sun rotated around the Earth.

And if your company has baked this error into the product development process, it will not be prepared for battle with companies that employ more modern thinking and tools. In fact, it will be slower, less profitable, and have error-prone strategies.

This conventional wisdom has been left behind by the best performers.

The inputs to the innovation process, as Ulwick published nearly two decades ago, are customer needs, not ideas.  However, this was something that should have been understood all the while.  Certainly today, it should be obvious to anyone who’s taken a basic problem-solving course. After all, every problem-solving process worth its salt begins with problem definition, not idea brainstorming.

And so, when we refer to voice of the customer research, we are referring to something specific: a process to determine what problems that a new product should solve. With New Product Blueprinting, this occurs within Steps 2 (Discovery Interviews) and 3 (Preference Interviews).

Confusion about concept testing is especially common in B2B. Why?

Large B2B companies have structural advantages. It’s likely that they have good relationships with established customers. It’s also likely that they are deeply embedded within the value chain. This increases switching costs for customers to leave. Additionally, large B2B firms usually have additional barriers to competition such as IP, unique technologies and economies of scale.

However, despite these advantages, they also know that the competition is not resting. They know that they must continually innovate just to maintain their position. Therefore, B2B firms keep R&D scientists and engineers busy developing next-generation products – often ahead of the strategic marketing directives from product managers. 

And then, of course, they cannot wait to show these new cool things to customers. This is not necessarily bad. We should show prototypes to customers. But, as we do, we should not become disoriented as to where we are within the innovation process. And this location is concept testing, not voice of the customer research.

Bringing the prototype to the customer

When we bring a prototype to a customer, if we believe we’re executing a voice of the customer project, then we think we’re at the beginning of the NPD process. In which case, we are lost. And worse, we may consider ourselves free of biases. We may think that we’re just searching for honest customer feedback.

None of that is true. 

We’re not the at the beginning.  Just as if you use a map to navigate a course, and don’t know where you are… you’re lost. And if we’re concept testing when we think we’re doing VoC, then we’re unknowingly executing a risky innovation process.

Let’s imagine a scenario…

Let’s imagine a textile firm. Its primary customer base is the outdoor clothing industry, in particular, high-performance rain gear.

You’re the lead R&D engineer for your firm. You’ve been working on a new fabric that meets all previous standards for waterproofness, but with a new advantage: it’s three times more breathable than anything available. Your team began tinkering with this challenge a couple of years ago. Unfortunately, you were stuck in the technical aspect of the assumed problem:  increasing breathability without sacrificing waterproofness.

But then, your team found some unexpected luck. At a coatings conference, you met a Ph.D. student who had developed a new polyester fabric. Highly breathable and highly waterproof. Over lunch conversation, you realize that this fabric solved the problems that had remained so elusive in the lab.

Excited, you meet with your leadership, and are able to acquire the technology. You obtain the patents; all the IP. As your team tinkers with the new fabric, you find yourself giving presentations to higher and higher levels of the company. Your leaders likewise become more and more excited. And invested.

Finally, it’s time to get customer feedback. You target the best imaginable market for this is your core customer base – high-performance rain gear.

Time for the Customer Meeting

You set up a meeting with your biggest customer, Featherlite Clothing, to discuss. As your engineers meet with their engineers, everyone marvels at the breathability of this new fabric! You run some tests together. You return back to your company, ready to begin the commercialization of this new fabric.

As your team continues along the NPD process… a few weeks later, you get a call from your commercial team representative. She has some bad news. Featherlite is moving on to your biggest competitor, Nemesis Fabrics.

WHAT?!? You cannot believe it! You call your engineering contacts at Featherlite right away. Just a few short weeks ago, we all were together marveling at the breathability of the new fabric! What in the world has happened?

Your contact replies, “Yes, we were very impressed with the breathability. That would have been something really cool to have in our specs. But see… Nemesis Fabrics just came out with a new fabric that is so light, that we can make a fully functional rain jacket that weighs less than 125 grams. It’s been an arms race for years to break that weight barrier. This will be the biggest hit at the Outdoor Retailer show next year!”

Concept testing gone wrong: What happened?

You brought your prototype to the customer without understanding their priorities. And the focus of the conversation was your prototype. The conversation was biased around the best attribute of your fabric, breathability. Your customer and your technical folks marveled at that performance… when actually, the most important need they had was to minimize weight.

Having solved the wrong problem, your new product failed.

But if we have a prototype or even just an idea, what should we do?

First, we need to recognize where we are in the process. And it’s concept testing, not voice of the customer. Next, we take our product and work backward to the benefits with three questions:

1) What job-to-be-done will this help our customers to accomplish?

2) Against what desired outcomes does the product outperform the competition?

3) How certain are we that the customer values an improvement in this outcome?

If your team can easily and confidently answer these three questions, then, by all means, proceed! But not with concept testing quite yet. Before taking that step,  you should first test your product internally to see if it truly outperforms the competition. And if it does, then we can finally proceed with concept testing.

What if we’re unsure?

If we cannot answer all three questions with confidence, then we should not proceed with a concept test. Instead, we need to begin at the beginning, which is to execute a voice of the customer project. It begins with a qualitative phase to uncover all the customer outcomes. And continues with a quantitative phase to prioritize those outcomes.

Although this time, when doing quant work, we’re going to add a little twist. We’re going to include the outcome that our product addresses, whether or not it comes up during qualitative work. So if our new technology increases breathability while also preserving waterproofness, we’re going to include the following desired outcomes in our quant project:

• Maximize the amount of air that can pass from the inside to the outside of the fabric (breathability)
• Minimize the amount of moisture that can pass from the outside to the inside of the fabric (waterproofness)

And we will include these along with all the other outcomes that our customers suggest. Since our product addresses these two, where will customers rank these? How will they compare to the rest? If they rank them first and second, then we can feel great about the prospects for our technology. But what if instead they are ranked ninth and tenth (or worse)?

Well, if they are ranked ninth and tenth, we will not be upset too long because we’ll be more interested in what actually ranked number one. If it is “Minimize the weight of the fabric”, then we have found something highly valuable: an unmet market need. One that the customer will reward us for if solved. Armed with a new problem statement, we can turn our technical wizards loose. But this time, they will be looking for a solution to a problem that customers actually care about. A much better starting point than a concept testing exercise.

Concept Testing: what do we conclude?

We conclude that concept testing is a valid part of our development process. But one that is separate from voice of the customer research. Regarding this, we must be precise with our words – and not co-mingle the notions. If we do have an idea, we should proceed with caution – armed with self-awareness and good process.

Finally, we should remember that the surest path will always begin with customer’s desired outcomes, not a new product idea.