Testing Business Ideas summary

You might have a great idea for a new product, but how can you be sure that your brainwave will live up to its promise? Many entrepreneurs rush to take their product to market, figuring that fortune favors the bold. While this may be true, fortune also favors those who test, test, test. 

That’s where these blinks come in. You’ll learn how to use experimentation to transform your bright idea into a profitable business venture. You’ll discover how to save precious time and money by testing how feasible, desirable, and viable your plans are. From risk-reduction to data analysis, these blinks will show you how to make a success of your business from the word go. 

In these blinks, you’ll learn

• why you can’t trust what your customers tell you;
• how to paint your business canvas; and
• what not to do when running your experiments.

All great businesses start with a great idea. But what next? The answer is a loop – a design loop. The design loop is a way for you to shape and tweak your initial ideas, so that you can transform them into the best possible business model.

The design loop has two phases. The first phase is all about ideating. In this stage, you’ll come up with as many ideas as you can. But don't just go ahead with your earliest ideas. Instead, keep generating alternative concepts for how you can progress your venture.

In the second phase, you synthesize. This involves narrowing down your field of initial ideas – and deciding which of them are the most promising. 

Here’s the key message: You can use certain tools to make your ideas clearer and more tangible. 

One of these tools is the Business Model Canvas, a worksheet available for free on the authors’ website. 

The Business Model Canvas helps you define the risks and opportunities associated with your business ideas. It involves taking a broad look at many aspects of your potential new venture, asking yourself questions, and writing the answers down on your canvas. You might, for instance, ask yourself who your business is aimed at. Write down what sort of people you’re hoping to reach. How will you communicate with these people and reach them with your message? What sort of revenue can you expect to generate from your different customers? 

You’ll also ask yourself questions about the key resources and assets you’d need to have to get your business model up and running. What are the key tasks and activities you’d need to do to make your business model work? In other words, what would you actually be doing on a day-to-day basis? What kind of suppliers or other key partners would you need to form relationships with? The answers to all of these questions will go on your canvas. 

Another useful tool created by the authors is the Value Proposition Canvas, which can be used to collate your ideas.

This canvas clarifies your understanding of your customers’ lives – and how your offering can create value for them. It’s on this canvas that you include information about gain creators. This describes what your customers would gain from using your products or services. How would their lives tangibly improve? You’ll also list all the pain relievers associated with your offering. In other words, how can your business eliminate a pain or a fear that your customers are experiencing?

When you have a promising idea for a new venture, it’s tempting to rush ahead and implement it straight away. But slow down. Because at this stage of the game, all you really have are a bunch of assumptions – and assumptions can be wrong. 

It's crucial to test your assumptions before you start relying on them. Luckily, testing is a simple step-by-step process. 

The first thing you’re going to do is make a list of all the most crucial assumptions that underpin your great idea. Then, you need to turn these assumptions into hypotheses. These are statements that you assume to be true – but you need to test them in order to find out whether they actually are. 

The key message here is this: A well-formed business hypothesis has certain characteristics.

You can begin your hypothesis with the phrase We think that . . . . For instance, if your great idea is to start a children’s extracurricular science program, you could formulate a hypothesis that states, We think that young parents will pay for an extracurricular science program for their kids. Then you and your team can try to prove that this underlying assumption is true, and that young parents are indeed willing to pay for such a program. 

That’s the great thing about hypotheses – they’re testable. Based on the data you capture, it’s possible for your hypothesis to be proven either true or false. A testable hypothesis can be something like Young parents prefer carefully curated, educational science programs that are tailored to their children’s age group. In contrast, it can be much harder to capture evidence for an untestable hypothesis, which might be something like Young parents like arts and crafts programs. 

The best hypotheses are also precise, rather than vague. An example of a vague hypothesis would be Young parents will be willing to spend a lot on educational science programs, whereas a precise hypothesis would state that Young parents will be willing to spend $20 a month on educational science programs.

Finally, useful hypotheses are discrete, in that they only test one distinct thing at a time. So an indiscrete hypothesis would state We think we can buy and distribute science packs for a profit, whereas a more discrete formulation would break this hypothesis down into two different statements. One hypothesis would state We think we can buy science packs profitably and the other would state We think we can distribute science packs profitably. Afterward, two separate experiments could be run to test each hypothesis. 

Once you’ve decided on your hypothesis, you can test it by running some experiments. You might be worried that these experiments will involve additional investment, or even new hires on your part. But they don't have to. In fact, your early experiments should be run quickly and cheaply. This will enable you to learn fast and run even more experiments. And the more experiments you run, the more you’ll reduce your chances of spending time, money, and creative energy on things that won't work out. 

The key message here is: Learn to tell the difference between good data and better data. 

There are a number of factors that make up an effective experiment. First, every experiment starts off with a discrete, precise, and testable hypothesis. Next, you need to decide what you will actually do to test the hypothesis. This is where the actual experimentation comes in. 

All useful experiments also have a metrics component. This is the data that will be generated and measured when the experiment is run. Finally, each experiment comes with criteria. These describe how you define success, such as figuring out how your data should look to conclude that your hypothesis is correct.

If you run lots of experiments, you’ll find yourself generating a lot of data. This data forms your evidence base, and it's what you’ll use to determine whether your hypothesis is correct. But beware – not all evidence is created equal. When it comes to business experiments there is strong evidence and weak evidence. 

You can assess how strong a piece of evidence is by asking certain questions. 

First, ask whether the evidence is based on opinion or on fact. Facts are more valuable than opinions.

If you’re interviewing customers as part of an experiment, you can tell when they’re giving you opinions because they’ll talk about their beliefs. They might say I believe this is important or I like this. In contrast, when your customers are stating facts, they’ll talk about actual events. They might say something like I spent $20 dollars on a similar product last week. This is much stronger evidence. 

You’ll generate weak evidence if you only test potential customers under controlled, artificial conditions, such as in a focus group. The problem is that your customers know that you are watching or recording what they are doing and so they might not behave as they usually would. It's much better to run your experiments in real-world conditions, where people don't know they're being tested. 

There are many different experiments you can run, and most fall broadly into one of two categories: discovery experiments and validation experiments. In this blink we’ll look at discovery experiments. 

This type of test will help you examine the assumptions that underlie your business proposition – and determine whether said assumptions are right. Better yet, what you learn in these experiments will allow you to correct your course quickly if the evidence doesn't align with your assumptions.

Luckily, there are lots of useful discovery experiments to get started with. 

For example, you could use customer interviews to gain insights into the pains your customers have, what gains they hope to make, and how much they pay for your product or service. Of course, interviews won’t produce nearly as strong evidence as observing what your customers do in the real world, but they are still useful. 

Here’s the key message: Both weak and strong evidence is valuable in the discovery phase. 

Not only are interviews quite cheap and straightforward to set up, they also won't take much time to run. Ultimately, interviews will start to give you an idea of how well your value proposition matches up with your customers’ needs. They’ll also provide a rough idea of what your price point should be. 

To conduct a good customer interview, you should write a script and take notes during the sessions. You should also make notes on the body language of those you interview. Aim to interview around 15 to 20 people, and use the insights you glean to update your value proposition canvas. 

If you want stronger evidence of your customers’ desires, pains, and behaviors, then it's a good idea to carry out some web traffic analysis. 

This where you look at data, reporting, and analysis from websites to gauge patterns of your potential customers’ behavior. To run this experiment, consider what area of customer behavior you want to change. Do you want to boost your customer sign-ups, for instance, or increase the number of downloads? Or perhaps you’re looking for ways to increase online purchases? 

Once you know the area you want to concentrate on, look at the steps that lead up to a sign-up, download, or purchase. Collect data about customer behavior during those steps. At what point in the process are people dropping out – or changing their minds? What's the weakest point in the steps toward your desired outcome? You can use all of this information to make adjustments to your online presence.

Once you’ve run discovery experiments to test your initial ideas, you can move forward with your chosen concept. You might assume this is the end of the testing phase of your venture, but you still have some distance to go. However, now that you’ve decided on a direction, you can start the important phase of validation testing. 

This phase will help inform you whether the path you decided on was in fact the right one. Has it led to a promising concept that can meet your customers’ needs, or did you draw the wrong conclusions from the discovery phase? The only way you’ll find out is through continued testing. 

The key message here is this: Take the time to understand whether your direction is sound. 

One useful validation experiment is known as a single feature MVP, with MVP standing for minimum viable product. It works as follows: Imagine you wish to test your assumption about an important feature of your product. You want to learn whether this feature will really help your customers in the way you assume it will. To do so, you need to create the smallest possible version of this particular feature of your product, and then acquire customers to use it. You can then solicit feedback about how satisfied they were with it.

The good thing about MVP experiments is that they tend to be quite cheap to run. After all, you’re only producing a small and basic version of your product. The evidence you’ll gain from these tests, however, is very strong. Customers will actually have been using your product feature, rather than just thinking or talking about using it. Additionally, you will be charging customers for the use of your product, even if it is just a basic, one-feature version. This will lead you to gain valuable insights into their purchasing behavior. 

Of course, to make MVP tests a success, you’ll need to spend a fair amount of time setting up the feature, and testing it internally to ensure it all works properly. After all, if you’re charging customers to use it and taking their satisfaction as evidence, then you need to make it as good as it can be. 

Another useful experiment is crowdfunding. This type of experiment involves setting up an online presence and asking your target customer segment to provide start-up funding for your venture. This experiment will give you strong evidence about the desirability of your product or service. You can collect data on what sort of people are interested in your venture, and how much they are willing to pledge.

Sometimes the best-laid plans don’t work out. Failure is an inevitable occurrence on the path to success, but there are some common testing pitfalls that you can avoid falling into. 

The first trap is not spending enough time on testing. 

Teams often underestimate the time it will take to run multiple, high-quality experiments. Remember, when it comes to experimentation, you’ll only get out what you put in. To make sure you’re experimenting enough, consider setting aside a dedicated amount of time each week for your team to test, discover, and adapt. You can also make a plan of everything you want to learn within a given week, so that you have something tangible to work toward. 

The key message here is: Teams can fall into mindsets that hinder their testing.

Sometimes an unhelpful mindset can arise from being too careful. Your ideas are the lifeblood of your venture, but if you spend too much time on your concepts then you’re probably falling into the trap of analysis paralysis. 

Analysis paralysis happens when you can't move forward because you’re too preoccupied with choosing the right course of action. In this situation, the best thing to do is simply test your ideas and see what the evidence says. Don't get bogged down in long conversations that are based on opinions about what will or won’t work. Instead, base your decisions on data – what has or hasn't been shown to work. You should make a distinction between reversible and irreversible decisions. If you know you’ll be able to reverse a decision farther down the road, then don't agonize over it; if it's irreversible, then spend more time on it.

In a greater sense, you can avoid pitfalls by adopting this mantra: Strong opinions, weakly held. This phrase was coined by technology forecaster Paul Saffo. He advises business leaders to form strong opinions, but to keep an open mind about whether their hypothesis is correct. 

This is where rigorous testing comes in. Problems can arise when you only stick to the first part of Saffo’s formulation – in other words, when you set out to prove that you are right, rather than entertaining the possibility that you're wrong. 

Leaders can adhere to Saffo’s advice by creating an experimentation culture within their workplace. This type of culture is built by leading discussions with questions, rather than answers, and by paying attention to evidence, even when it doesn't tell you what you want to hear.