Basic Stats for Marketers: Variance
As someone who spends a lot of time dealing with maths (the joys of data vis development!), I spend a lot of time entrenched in statistics. Whilst that’s great fun, or so I like to think, I’m always aware that there’s a lot of people out there who were never really taught the why behind […]
The post Basic Stats for Marketers: Variance appeared first on Builtvisible – A Creative Digital Agency.
Basic Stats for Marketers: Variance
As someone who spends a lot of time dealing with maths (the joys of data vis development!), I spend a lot of time entrenched in statistics. Whilst that’s great fun, or so I like to think, I’m always aware that there’s a lot of people out there who were never really taught the why behind […]
The post Basic Stats for Marketers: Variance appeared first on Builtvisible – A Creative Digital Agency.
A Product-Based Approach to CRO
Posted by CraigBradford
Before joining the world of digital marketing, I was a product design engineer. Most of my clients at Distilled are now CRO projects and I’ve found my background to be surprisingly useful. There’s a lot of overlap between designing physical products and designing websites that convert well. I’d like to share some of the research methods that I use for CRO that I learned while designing physical products. I like to use a framework of Learn, Look, Ask, and Try.

I first came across this while at university. It’s the design research methods used by
IDEO. They released this as a pack of playing cards. In their own words:
“IDEO Method Cards show 51 of the methods we use to inspire great design and keep people at the center of our design process.”
There are 51 cards each with a research method that belongs to one of the above categories. The cards have since been made into an app. I find it useful when trying to come up with new ways to get customer insights. You can download the app
here.

Design methods in the learn section are about analyzing the information you’ve collected to identify patterns and insights. One you might not have heard of is error analysis.
Error analysis
In simple terms you can think of error analysis as going around your site and saying “what happens if I do that?” In product design you might hear this called failure mode effects analysis (FMEA).
“Failure modes” means the ways, or modes, in which something might fail. Failures are any errors or defects, especially ones that affect the customer.
”
Effects analysis” refers to studying the consequences of those failures.
While a lot of FMEA is overkill for designing a website (hopefully nobody is going to die if they click the wrong button) I think the principles can be used to help proactively find faults. Every website is different so you’ll need to think of your own scenarios but here are some to get you started:
- What if I use my email instead of username to login?
- What if I press the back button in the checkout funnel?
- What if I need a refund?
- What if I want to get the product delivered to my work address?
- What If I order the wrong product?
You can see that these potential errors can be a mix of usability and customer service. The point is to be proactive and anticipate what could go wrong. You can then fix true errors (things that are just broken) or put processes in place to ensure mistakes can be fixed easily when they do go wrong.
Extra tip
Google analytics has a great report that can help you find some of the most common errors or problems. Look at the reverse goal path report and pick a goal (for example people reaching the thank you page).
The report will then show you the most common routes that people take on their way to that page. The image below shows this report on one of my client’s sites.
I’ve had to blank a lot out for privacy reasons, but the point can still be seen: Out of the top 10 routes to the thank-you page,
- 4 included people visiting the terms and conditions page, and
- 2 included people visiting the FAQ page.
What are they looking for? If we can find out, we can make that information clearer and hopefully stop them having to go to those pages. Which brings me to my next section:

The “Ask” section is pretty simple; it’s about asking people to explicitly tell you what they do or do not like.
For the terms and conditions example above, the solution is an easy one—live surveys. I’m sure everyone has heard of
Quaraloo by now so I’m not going into detail on this. The solution is to ask people that leave the funnel via the T+C page what information they’re looking for using Quaraloo. Once we find out the reason, we can add that information to the pages leading up to the conversion and hopefully reduce anxiety and distractions leading up to the purchase.
The real research method I want to talk about for this section is extreme user interviews.
Extreme interviews
Any golfers reading? If so, you’ll recognise the image below as the famous Big Bertha.

Big Bertha is one of the bestselling drivers, so how did Callaway come up with the design? They did it by focusing on a particular demographic. While competitors were all focused on asking
golf players what they wanted, Callaway focused on a different set of users. They surveyed non-golfers. More specifically they wanted to know why people who loved sports, could afford to play and already belonged to country clubs chose not to play. In other words, why do people whom on paper should like and play golf choose not to?
By interviewing lots of people who fit that criterion, they were able to find the answer. People don’t want to feel embarrassed. If you’ve ever tried golf you’ll relate to how frustrating and embarrassing the first couple of years are, especially if you’re used to being good at other sports. This is where Callaway gained their insight and competitive advantage.
It turns out that consistently making contact with the face of a small driver is hard, really hard. If you fail, the ball can end up anywhere. Callaway decided to focus on the need of these people (people that should play but don’t) by designing a driver that had a massive club head and huge face. The result was it was much easier for beginners to hit the ball and avoid embarrassment of constantly losing the ball. To this day Big Bertha is one of the most successful drivers on the market.
So how do we use this for CRO? Extreme interviews? Instead of just surveying the people that do buy from you, or that are familiar with your brand, survey two groups of users.
- Experts: Repeat buyers or people that are familiar with your brand
- Novices: People that have never been to your website but that at least understand what your product or service does.
The hard thing about this is recruiting these people. You can’t just ask members of the public; you need to ask people that are on your site. A tool I like to use for this is
Ethn.io.
Ethn.io lets you recruit users for user testing and pre-qualify them. For example, in the above you can create a popup that looks something like this:

You can then ask users a qualifying question and group them appropriately.

Fly on the wall
Your customers are liars. Harsh, but true. Even in the extreme interviews technique you might not get the right insights that you’re looking for. If you ask people why they don’t convert, they might not be able to tell you. There are two reasons for this:
- The reasons are subconscious so they actually don’t know the answer.
- They don’t want to tell you for fear of embarrassment.
That’s why using the techniques in the look category is a great idea.
Here’s a scenario. Imagine for a second you gave someone this:

If you were to ask someone how they would open this package, you’d likely get a sensible answer like “I’d use scissors”, but as soon as you leave the room and they don’t think you’re looking, you’re likely to see something like the scene below:

If you’ve ever tried to open one of those blister packs, you’ll know how annoying they can be to get into. They’ve led to the
many injuries and the emergence of the term “wrap rage“.
Anyway, the point I’m trying to make is you can’t always trust your users to tell you all of the reasons they don’t do the things you want them to.
In the offline world, designers use a technique called “fly on the wall”. This involves watching people in the environment or using the product that you’re trying to design. Sometimes the subject is aware that you’re watching them (shadowing), other times (ideally) they’re not (fly on the wall). Watch this scene from Madmen and you’ll get the picture:
So how do we do this in the digital world? We can’t sit over everyone’s shoulders as they use your website, but there are some tools that come close. Both of which I’m sure you’ve heard of.
Shadowing: Usertesting.com
I like usertesting.com but my one complaint is the sample of users is unlikely to be representative of your customers. Even if you select the options like low level of internet experience—the fact they are on a UX testing website already puts them above the average internet user. They’re also not your customers. One way around this is to combine Ethnio with uerstesting.com. Recruit real users then set them tasks using usertesting.com or just watch them live using some kind of screen sharing tool.
Fly-on-the-wall: Clicktale
Clicktale might be above many peoples’ budget, but if you can afford it I’d recommend it. Clicktale records anonymous sessions on your website. Obviously it would take a long time to watch all sessions, but you can segment by things like location, browser, or even sessions that had errors. Using these tools you can dig into why certain segments of traffic may not be converting as well.
Another tool that I’ve heard of, but not personally tried, is
http://www.uxcam.com/ It’s like Clicktale but with a specialization in mobile usability testing. I spoke to the founder at one of our meetups about a month ago and was impressed by the features. It’s also still in beta so it’s free at the moment if you want to give it a try.

Empathy tools
As a product designer, your most powerful tool is empathy. If you can empathize with your customers and understand what they really want, you’ll create great products. The best way to empathize with your customers is to do what they do—try it.
In the offline world, product designers go to great lengths to understand users. For example, when designers are creating a product where the primary audience is the elderly, they might use empathy tools like those shown below:

The image above is a picture of me with coins strapped to the back of my knuckles. This can be used to simulate the limited dexterity that can come as a result of arthritis.

The designers in the image above strapped up their legs using a kind of split. Again, this one done to simulate reduced mobility when trying to climb stairs.
Finally and probably the one that could most easily be applied to the online world is a pair of empathy glasses. These can come in various types depending on the condition you want to simulate. If you want to get an idea of how various eye conditions can affect vision, look at the side-by-side comparison of healthy eyes to advanced stages of cataracts and glaucoma below. You can see the tools here:
Cataract simulator, Glaucoma simulator.
Cataract simulation

Glaucoma simulation

So if you’re not designing a website for the elderly, can you still use this technique? Absolutely. I like to use “scenario empathy.” This works by enforcing criteria on your users under test conditions, the most popular being a mix of:
- Time: You have six minutes to book a flight to London. GO!
- Money: Find the best product X for under $50.
- Product criteria: Find me a hotel that has a spa and is dog-friendly.
- Technology restrictions: You’re on a mobile with a slow edge connection.
You can then mix any of the above to create powerful scenario simulation. If you can make someone that’s short on time, has a low budget, really specific requirements and a slow connection happy, chances are the rest of your customers will also be happy.
That’s it.
In summary, there’s a lot of research techniques out there that can give you excellent insights about why your customers aren’t converting. Try them out; don’t just stick to the same techniques that you see on CRO blogs all the time.
For more ideas, take a look at the presentation I gave a few weeks ago at our
meetup. Also, for those in the UK, I’ll be presenting on a similar topic at Measurefest this week.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!
Google “Mapping” Real World With Incredible Precision For Self-Driving Cars
A fascinating article in the Atlantic appeared this weekend. It’s mostly about Google’s self-driving cars and how they operate technically. But it’s also about something much bigger: how Google is now effectively “crawling” the real world as it has crawled the web for…
Please visit Search Engine Land for the full article.
Coming Up: The Biggest Google Update of the Year?
Warning from Australia. Algoroo is showing 17th of May as a particularly turbulent day at 3.52 roos. So far there has been no official confirmation regarding the nature of the movement in search results but it’s worth noting that this is the highest roo value recorded this year and the second most turbulent day since the Penguin update in May 2013:
I have predicted some sort of an update due to unusual “quiet before the storm” SERP fluctuations, however this is not the first such prediction.…
The post Coming Up: The Biggest Google Update of the Year? appeared first on DEJAN SEO.
Doodle Equality: In 2014, Google Features Women In Special Logos Nearly Half The Time
If you haven’t noticed, the Google Doodle team — which creates those special Google logos — has been making up for lost time in 2014, adding significantly more women to the number of historic figures featured on Google’s various regional and global homepages. Now, nearly…
Please visit Search Engine Land for the full article.
Yahoo Likely To Slip Below 10 Percent Search Share Next Month
When comScore releases its “May 2014 U.S. Search Engine Rankings,” we’re likely to see Yahoo’s market share slip below 10 percent. Today (in April) it recorded a 10 percent share, down 0.1 percent from March. Microsoft’s share was up 0.1 percent and so was…
Please visit Search Engine Land for the full article.
SearchCap: Bing Snapshot Expands, How “Right To Be Forgotten” Works & More
Below is what happened in search today, as reported on Search Engine Land and from other places across the web. From Search Engine Land: Bing Expands Snapshot With New Food & Drug Entities Bing is getting smarter today with the addition of food- and drug-related entities to its library of…
Please visit Search Engine Land for the full article.
Bing Expands Snapshot With New Food & Drug Entities
Bing is getting smarter today with the addition of food- and drug-related entities to its library of facts. The new at-a-glance information for these searches will appear in the Snapshot area on the right side of Bing’s search results. In some cases, like the hamburger search shown below, the…
Please visit Search Engine Land for the full article.
Baidu Hires Andrew Ng, Google’s Lead “Brain”
Chinese search giant Baidu has stolen away the head of Google’s “deep learning” project, dubbed “Google Brain.” Andrew Ng is an artificial intelligence expert, Stanford professor and founder of online learning company Coursera. Google Brain is a machine learning…
Please visit Search Engine Land for the full article.
Google AdSense Magazine Ads
Google announced a new ad format for Google AdSense display ad units named magazine ads.
So instead of the display ad looking like a banner ad with animation or various popping colors, magazine ads are very minimalistic…
Google: If Content Isn’t Changed, Manual Actions Won’t Be Removed
An interesting Google Webmaster Help thread has a webmaster who has a manual action, he said he removed the content, and even left the page blank but the manual action won’t go away.
Google’s John Mueller explained why in an interesting way…
Google Updates Google Places Adds Manager Access & Social Features
Google’s Jade Wang announced on Google+ and Google Business Help forums that they have upgraded the Google Places bulk users to add social features and multi-admin functionality.
(1) Social Features: Each location in upgraded and verified accounts wil…
Video Recap of Weekly Search Buzz :: May 16, 2014
This week in search, Google says that there is no update going on even though a lot of things have been going on. Google is now forced to remove…
3 Key PPC Metrics Are Lying To You. Find Out How
When it comes to PPC, metrics are everything. We consume ourselves in pouring over numbers and analyzing our accounts down to the smallest detail. However, it turns out we are being misled by the data we rely on the most. We’re too quick to take our figures at face value without thinking…
Please visit Search Engine Land for the full article.
The Ultimate Entrepreneur’s SEO Checklist for 2014
How can entrepreneurs survive and swim the SEO sharks? Here’s some advice and top tips for business leaders looking for the ultimate entrepreneur SEO checklist for 2014, plus how to screen for social media fakes, phonies, and posers.
Top Five Local SEO Myths — And Truths
There seems to be no end to the range of declared “SEO truths” for local mom and pop shops. Search Engine Optimization strategies that work for the big brands differ from those that work for small, local players, so it’s no wonder that so many local marketers end up confused about…
Please visit Search Engine Land for the full article.
Search In Pics: YouTube Slide, Google Landscaping & SEO Gingerbread Hearts
In this week’s Search In Pictures, here are the latest images culled from the web, showing what people eat at the search engine companies, how they play, who they meet, where they speak, what toys they have, and more. Google Switzerland’s Honeycomb: Source: Google+ Google Dublin’s…
Please visit Search Engine Land for the full article.
Google Places Bulk Management Tool Integrates New Google+ Feature
Google announced new updates to its Google Places bulk location management tool this week, including a new Google+ social feature. From Googler Jade Wang’s announcement in a Google Product Forum: Each location in upgraded and verified accounts will have social features automatically enabled….
Please visit Search Engine Land for the full article.

