Bryan walked into my office the other day to point out an interesting item found while surfing: a left-sided add-to-cart button on a product detail page.
We chatted back and forth about the conversion issues involved with placing it there — and in fact, one of our Conversion Analysts, Peter, commented on this very topic in his latest post — but soon our conversation turned to something much more interesting than left-sided calls to action: the testing of left-sided calls to action.
«Do you think they tested it?» Bryan asked.
«Hmm, the Joker in me wants to say Yes, but I’m guessing the money bet is No,» I replied.
Now, that’s not because Crutchfield doesn’t test. In fact, I’ve no idea at all what sort of testing culture Crutchfield nurtures; I’m just saying that in our experience, only rarely does this sort of innovation ever come about from testing. Instead, it’s sadly de rigeur for it to arise from a designer wanting to try something «different», or an IT staff that doesn’t perceive one shopping cart as different from another, or maybe Matilda the Intern just forget an HTML tag. Anyway, the point is to go with the simplest explanation — which, in 2008, is that most companies still don’t test.
«I think you’re right,» Bryan continued, «cuz if they did test it, it probably wouldn’t do well.»
«Maybe some Clown in IT or Marketing just wanted to be ‘kewl’.»
Here’s what we’re talking about, as shown on Crutchfield.com:
Intuitively, I hope you’ll agree with us that right-sided feels like a better than even-money bet (though that in itself is a reason to do a test) — but what’s the point of leveraging your intuition to be «directionally correct» unless you eventually try to back it up with some evidence that you’re actually correct?
That started me down the road thinking about how to actually test this hypothesis.
(I can be wordy, so if you’ve lost the trail of thought, the question is, «Which converts better? Right- orLeft-sided Add-To-Carts?» and the hypothesis would be, «Right-sided Add-To-Carts convert better than Left-sided Add-To-Carts.»)
Here’s where it gets interesting: The supposition is that most Web surfers are so used to right-sided Add-To-Carts (and right-sided Calls-to-action, generally) that a left-sided one is bound to produce some cognitive dissonance. It might not be consciously noticed — less so on «narrower» sites and more so on wider ones — but the placement on the left will «feel» odd.
With that in mind, just how do you go about running a test you already know has a skew to it? How would you really determine whether the Clowns or the Jokers win The Great Add-To-Cart Positioning Debate of Aught-Eight?
Here’s what I would do: First off, start with the most obvious test, because we have to get a quick benchmark of just how far Clown is from Joker. Throw some percentage of traffic at the left-sided Add-To-Cart — enough for some statistical significance — and see just how well Right does vis-á-vis Left. (The fascinating thing about intuition is that a fair percentage of the time it’s fabulously, gloriously, achingly, wrong — and if this is one of those times, better to find out early and move on to the next good idea.)
Assuming we’ve shown some evidence of the skew in favor of right-sided shopping carts — otherwise, why continue reading this post? — how do we go about removing the skew that comes about from people being «trained» that right-sided is «normal» to answer the real question: If folks weren’t biased by convention, which side converts better?
To do that, what you’d really want is to look among your customers who’ve already successfully converted using one particular side and to present them with similarly-sided add-to-carts in the future (hmm, might have to set a cookie!), so you can gauge what the conversion rate is for people who’ve shown at least some indication that they can successfully convert.** The idea here is that, all else being equal — something the pre-existing bias hurts — the true question should be, «Do people actually have a preference for sidedness at all»?
By picking only from those who’ve successfully converted previously, you’re making a first attempt to say, «Hey, at least these folks don’t seem to be impeded by a systemic bias»; therefore, those who buy consistently using left-sided calls to action might then be expected to convert at approximately the same rate as those who buy consistently using right-sided calls to action.
«And surely,» you might argue, «those who show a preference for left-sided add-to-carts should convert better when consistently presented with left-sided add-to-carts than Right-Siders who are suddenly presented with a left-sided add-to-cart.»
See, you’ve switched the tables.
Get it? In short, you try to come up with series of tests — a Testing Campaign, if you will — which attempt to disprove the way your original hypothesis was leaning (we figured Right would do better, so let’s design tests that indicate when Right does poorer), and let us challenge any underlying bias (i.e., that Add-To-Cartss typically appear on the Right) that gives unfair advantage.
Well, those are my thoughts on the subject. What I hope you got out of that is that a «culture of testing» means thinking as deeply about the design of experiments as it does their performance.
I’d love to hear more about you. Are you a «Clown» or a «Joker»? Or are you just «Stuck in the Middle»? Would your brand loyalty or the customer’s familiarity with your site’s User Interface simply override any preference you have for being a Clown or a Joker?
– – – – – – –
**A few readers will feel reassured to know that, in actuality, you’d still send at least a few visitors who preferred one Side to see an opposite-Side call-to-action once in a while just to keep things honest; enough to get insight from the data, but not enough to cost the company too much from the loss from the expected conversion differential. I figured I’d say that as a footnote before some Sharp Tack out there writes in to scold me. 😉
Audio Bible
Ene 25, 2008 @ 15:13:42
I am going to make a guess here, the call to action button has been on the right side for so long, and I bet, few people will change it to the left to test it, that it will remain on the right side forever…..
I think we place things in certain locations because the customer expects it to be there. The call to action button is one of those elements.
Interesting idea, testing may tell thou……
Mark
Ene 25, 2008 @ 19:03:08
John,
newspapers, magazines etc have for decades charged a premium for ads (calls to action) that appear on the top right hand side and right facing pages. (Place one on the left side of a left page and you get a huge discount.)
They do this ‘cos they know that that is where people look.
Not sure when the research was done, probably middle of last century, or maybe they just figured it out over centuries of publishing. Be interesting to research that one and find out.
Point is, this whole right positioning thing has been around for centuries, a lot longer than web design.
Linda Bustos
Ene 26, 2008 @ 00:01:06
Mark makes a great point.
I was thinking you may need to test this with people who have never used the web before – and have not been conditioned to add to cart on the right. Now, that would make for an *interesting* and probably frustrating test!!
Ricardo
Ene 28, 2008 @ 11:30:58
You know how easy it would be for the e-tailer I work for to test add-to-cart button placement?
As easy as it would be for me to walk over to 34th St and topple the Empire State Building with my bare hands.
Dream on, guys.
John Quarto-vonTivadar
Ene 28, 2008 @ 11:59:29
Mark, Linda, Audio: thanks so much for your comments. Absolutely it is conventional for that AddToCart to be on the right. I suspect it goes hand in hand with reading from left to right (easy to find out how in right to left languages (Hebrew, Arabic) where do those language-specific sites tend to put AddToCart? And what about Japanese when read top to bottom? You’d think in all cases you’d put the AddToCart at the place in the prose where the persuasive thought comes to mind for the customer to say «I want».
John Quarto-vonTivadar
Ene 28, 2008 @ 12:32:23
Hi Ricardo,
It’s a sad state we’re in, isn’t it? It’s not that the left-side versus right-side AddToCart is not determinable — we could actually design experiments to measure it and a host of other important varaibles….but there’s no collective corporate Will to do so.
Zachary Abbell
Ene 28, 2008 @ 12:34:36
Ricardo,
It’s very easy for a 3rd party vendor to help you run such product page tests. My company runs countless such tests…You just need to be willing to look outside your current IT team!
Hope not so see you trying to push over buildings with your bare hands but please send pictures if/when you do.
Thanks.
Smufty
Ene 28, 2008 @ 14:34:19
I think the point Ricardo may be making was that to perform a test you need a reliable high volume of conversions, which excludes most etailers, plus the technical issues of product-based websites which may be doing all sorts of wizardry in the background.
Once you get into the «You previously bought that, so you may like this» sort of stuff, plus tracking such data, auto-generated offers, seasonal stuff and all that jazz – moving the button from one side of the screen to the other is not always as simple as it sounds!
Sometimes there’s a neat and elegant way of doing it, and sometimes it would be easiest to run the entire system twice rather than mess around trying to be smart. Most off the shelf CMS shopping systems were designed before such testing became popular so it’s not a no-brainer by any means.
It strikes me as one of those things some bright young spark comes up with, the IT claim they can do it rather than look silly and then it all goes horribly wrong when some customers get billed twice, some lose their orders and by the way, here’s an email saying our logo is half off the screen and it looks crap in Firefox.
Of course that was the bad old day, WE are so damn smart we never have ANY bugs or screwups and everything will work perfectly what with CSS and being net 2.0 an’ all. It’s just that old-school management have heard all that before and know how quickly «There’s a problem…» features in conversations.
John Quarto-vonTivadar
Ene 28, 2008 @ 15:09:02
hmm, I *hope* that’s not the point Ricardo is making, because you don’t in fact need a huge volume of conversions to do this sort of test. You’ll do your first test (as originally described) simply to distinguish whether left or right does better generically and at that point it doesn’t take a large sample to distinguish a non-random distribution of results (i.e., something other than 50/50) due to something other than random chance. You don’t need high conversion rates per se, you simply need sufficient traffic to each element in the test for the results to be statistically significant, regardless of what they do once they see the AddToCart.
Then, following part 2 of the test, you’d only be looking at on-going conversion of those who did convert earlier, i.e., you’re repeat customers. Although that, presumably, is where the left-side AddToCart customers will need to build up some traffic for significance, you’re also drawing that traffic from a sub-group of your customers who’ve by definition have already shown a propensity to buy your products (repeat customer typically have a higher average conversion rate than first-time customers — atleast for businesses who have legitimately good products that satisfy customer needs). So you’re starting from a higher conversion rate population anyway, which will help tremendously with the return traffic you’ll need for significance
In other words, if your site’s conversion rate is so low that you can’t get enough data to make a start at this sort of experiment then this isn’t the sort of experiment you should be considering anyway — you should be doing much simpler optimizations that increase overall conversion (the so-called «low hanging fruit») before tackling what may in fact be a persuasional challenge for your company’s business topology.
John Quarto-vonTivadar
Ene 29, 2008 @ 17:27:06
Hmm, Cart Blog, please do not put words in my mouth. I did not say I wasn’t a fan of Crutchfield, nor that Crutchfield doesn’t test — in fact, I bent over backward to mention that we had no special insight into Crutchfield’s approach to testing one way or the other.
What I did assert outright was that testing is infrequently done at companies, even in 2008. And so, in the absence of any evidence to the contrary, unorthodox UI use should best be assumed as non-tested.
Nevertheless, the point of the article — perhaps missed in your rush to link to us — is not what Crutchfield is doing in the specific, but how companies in general can begin to think about testing techniques for unorthodox elements when they find themselves in a testing culture where such experiments can occur.
LynnS
Ene 30, 2008 @ 14:42:51
I do find it interesting that the left nav is replaced by the buy option on the product page – a position where people are used to looking for further navigation options … Has anyone ever tested what happens at that point in the buying cycle when someone is presented with something new to choose from?
MichelleT
Ene 31, 2008 @ 14:02:36
Web usability experts (Jakob Nielsen) have done studies that show web viewing patterns following an «F» pattern where people tend to look across the top of a site, and down the left hand side. Based on that logic, the Crutchfield layout is actually right in line with such eye-pattern studies. It would be nice to test that theory – but as many have already pointed out, that seems very difficult for the average site.
Mark G
Ene 31, 2008 @ 18:05:00
Michelle,
but there’s also plenty of other research which proves completely the opposite. What to do?
As the crew at Future Now keep pointing out, conversion rates for most sites are till stuck at 2%.
We need to be careful about accepting anything as definitive when it comes to web design. With conversion rates that low, everything is up for reinvention.
The F pattern merely shows that that’s how most web designers layout their pages, which forces people to look in that pattern.
There’s another great post on this site. (the bunnyfoot sunsilk ad test) that shows that just by changing the direction of a model’s eyes, you can drastically change where people look.
Now, that’s got me thinking. The effectiveness of Calls to Action are obviously also dependent on what’s around them. So how do we go about getting some definitive tests?
As I said in my earlier reply, publishing and advertising in the west already charge a premium for right placed content.
It would be great to find out their research. Anybody know anyone in advertising who can get hold of this research. I know Ogilvy did a whole lot in the 60’s and 70’s but I lent my book of his to somebody and I never saw it again.
PS John, this is obviously a great post and hot topic. Maybe theirs another article or chapter in an upcoming book on this?
John Quarto-vonTivadar
Ene 31, 2008 @ 18:18:12
Hi Mark, you’ve brought up a great point. I think it’s worth repeating the other side of the same coin — the «F» pattern reported by Neilsen et al, is simply a reported observation of how visitors eye-scanned a page. This does not guarantee it’s a preference for how they gather such information and if designers know about the «F» pattern and repeat it, they run the real risk of a «lemmings effect» wherein they are all doing it — and visitors are all scanning that way — for no reason other than everyone else also doing it. Hardly a way to stand out in a crowd, but nevertheless safe. Which came first, the designer or the visitor?
It makes you wonder, are visitors just trained that way now for web surfing? And if so, how strong is the habit? I’ve got a pretty strong habit of good penmanship (Thank You, Sister Mary Agnes! Your wooden ruler inflicted quite a bit of unintended «cursive», if you’ll pardon the pun). That penmanship would be hard to break now, for sure. But I’ve also got a more conventional habit of putting my fork on the left when sitting to dinner — a habit I could easily by-pass with just a little effort. So is the «F» effect a deep or a shallow scanning habit? Does it come from the actual preferences of the visitors, or does the common elements of design during the web’s first decade simply imbue a certain level of «F» proclivity?
And you’re correct, Bryan and I are in fact in the midst of a new book, precisely what we were working on when the Left-Side Add-To-Cart discussion first occured. I’ll leave details for a later post once we have a better sense from the major publisher as to release date and the like.
Steen Rasmussen
Feb 04, 2008 @ 17:17:36
Some interesting aspect and points to this discussion but at the same time it seem we try to remove the discussion of left and right from the actual design of the page.
There might be some universal truths about left and right, but at the end of the day the main issue is connected to the individual design – and the way the design drives our eyes.
When looking at the page live on the site (crutchfield.com/S-fahcfnLTVOg/App/Product/Item/Main.aspx?I=472TOUCH16&cc=01&wm=cl), the lack of content to the right of the product is affecting your behavior.
It forces your attention in one of two other directions – either to the left to the price and the call to action or down to… absolutely nothing.
The most prominent problem with this design might well be the lack of a secondary call to action at the bottom of the page to collect the «lost» users. Having just one “call to action” on this page seems something of a gambit.
When all is said and done I do agree with everyone that challenging a convention is a challenge you want to measure the result of, not gamble you business on.
carpet protection
Feb 23, 2008 @ 15:31:07
Hmmm…very interesting. Has anyone tried to contact their web team to see if they actually did test this? Anyone willing to test it themselves?
florida internet marketing
Dic 14, 2008 @ 14:49:45
Not to be controversial, but I’ve tried the exact design from crutchfield before. My boss practically made me clone their website, and we didn’t see any improvement over our previous design. We did however see an improvement when we copied adirondackchairs.com That’s a site that’s part of the netshops network. It’s really usable, but it’s not that exciting. It tested really well against other designs we tried, and we tried to test out things that we didn’t like, but they always lost against the adirondack design.
John Quarto-vonTivadar
Dic 14, 2008 @ 14:58:51
Thanks, Florida. Certainly nothing controversial there — I think you’re echoing the theme of this blog post — that the left side add-to-cart doesn’t seem to improve conversion at all, almost as expected.
I wouldn’t mind hearing more details about how you did the Crutchfield test — are you able to share them?
canadian bodybuilding
Abr 07, 2009 @ 11:03:07
very interesting