Validation: a wise investment, not an expensive gamble

Date
15 November 2022

An idea is often the first step towards a better future, and it’s no different in product development. A backlog full of crystal-clear priorities – epics, stories, bugs, and tickets – is a wealth of ideas for the future. Having the time and budget to actually work all these ideas out is something else altogether. How do you validate an idea without risking a serious chunk of your budget?

There are many situations where you would like to present an idea to your target group, just to find out whether it is financially feasible, but you can never be certain that an idea is worth the financial cost of testing.

A negative result after testing can be expensive. There are industries that operate at such a scale that a 2% drop can run to the millions. That’s why caution is advised.

header validation | iO

Caution: avoid unnecessary risk with carefully considered testing

Running a business is rarely risk-free. In fact, in different layers of the organisation it’s often impossible to validate ideas directly. Usually the decision is made, to just take the gamble and then see whether the idea and its development leads to the desired result: positive responses and a successful test. Like I said, anyone up for an expensive gamble?

Fortunately, digital product development offers ways to validate an idea without the risk of high development costs in the testing phase.

The numbers behind the testing process

Recently, Harvard Business School analysed 100,000 experiments on business websites. These experiments involved A/B tests in which an idea (a variant) in a 50%-50% distribution was compared to the current situation. Only 10% of these 100,000 experiments delivered a positive result. 60% returned an undecided result and 30% delivered in a negative result. You would want to reduce the risk if the chance is 3x greater that an idea produces a negative result wouldn’t you?

analyse-harvard | iO

The solution? Hypothetical validation

Put simply, it’s better to select a simple set-up to test your hypothesis at a limited level, rather than making significant investments that may ultimately do more harm than good. The success or failure of this approach is built on a solid plan of action. The plan is your starting point where you define and substantiating your idea:

  • What do we want to achieve with the idea?

  • How does the development of the idea fit in with the user’s needs?

  • What data do we need to validate the idea?

Further research and data collection is essential for validation. There’s a lot of different types of ways to research and identify sources of data:

  • Competitive analysis.

  • Feedback from users or customers.

  • Needs surveys such as interviews, surveys, or keyword research.

In many cases, the idea came from research. Even in situations like this it’s still worth drafting the best possible definition of the idea, and the problem that it’s trying to solve, or the answer you want to find.

risico

The 1:10:100 research rule

But conducting research costs money and time, doesn't it? Research costs are unfortunately elements that can take a hefty chunk out of your budget. However, the 1:10:100 rule demonstrates that every dollar you spend on research, becomes a 10 dollar spend on design and a 100 dollar spend to fix what went wrong. As a general rule, the closer to the source change happens, the cheaper it becomes.

1 10 100 rule

Ideations & prototypes

What do we really want to achieve? This is the starting point of the in the ideation phase. A lot can change in this phase, even the idea itself.

Within this phase, it’s important to take into account the technical possibilities needed to develop the idea. The real art of validating an idea without a budget, is to be creative with the available possibilities.

The development does not have to be perfect to get to prototype stage. Just good enough so that you can send it out into the world as a test.

For example: do you want to find out whether a webshop is a good step for your organisation? Then look for website elements that you can use to realistically imitate a webshop. You only need your development to be good enough to validate your idea.

Is your setup ready for the prototype phase? Make sure that your measures are accurate and reliable. Accurate, reliable data is essential for making good data-driven decisions.

swiss knife cartoon

Ready, test, go?

Are the previous phases complete? Then it is time to test, and to monitor how people react to the test. Do the results deviate greatly from expectations? If that’s the case, investigate what’s going wrong. Was it bad assumptions or have new, previously unknown factors suddenly revealed themselves?

During the test phase, you collect data about the idea itself, but also about the sentiment of the test. What do your users think about being in a test? After all, you’re dealing with your brand’s reputation, and you don’t want to throw that away just for the sake of a test.

Practical example: a webshop for charity

As a charity, every euro spent has to work hard and return double. This makes charitable organisations an excellent example of companies that can't afford to make expensive mistakes – not least for a webshop that we could test using hypothetical validation.

The organisation had a webshop, but one whose revenues did not match the costs of advertisements, procedures, packaging, and shipping. Was this because of the platform behind the webshop? Or were the products just not attractive enough?

We developed the hypothesis: what if, instead of standard products, we offered symbolic emergency aid packages? Products that are aligned with the purpose of the organisation. We wanted to know how many visitors would pay for these products. That’s why we didn’t just go ahead and change the webshop, we built an identical test page. We informed visitors that it was only a test, but we rewarded their interest and attention with a unique gift.

Of course, feedback from users was worth its weight in gold: that's why we provided sufficient opportunity to provide input with surveys, social advertising, thank you emails and a special web page. This meant we could monitor the responses and could intervene quickly in case of negative reactions.

Robert de Kok
About the author
Robert de Kok
CRO Consultant - iO

Eager to learn, highly disciplined and iO’s CRO team lead in the heart of Rotterdam - CRO consultant Robert de Kok is no stranger to efficient user research and data-driven optimisations. Together with his team, he works on digital innovation, customer journeys and conversion stats every day. Up next on the roadmap? Neuro usability testing – because why not?

Do you have ideas you want to validate?

Feel free to contact us to make an appointment. Our experts will be happy to listen to your story, your challenges, your wishes, and your vision. They’ll do the rest.

iOEindhoven