Interesting results, so I tried some experiments with the color of my own sign up buttons.

On the Collectorz.com website the main call-to-action buttons (Try and Buy) are a dark shade of blue (what we call “collectorz blue”). This results in a nice and quiet look. But if I can get more sign ups by introducing a noisy Sign Up button, I won’t complain.

The Red Button

I tried a red one first. It sure stands out
The results after about 400 sign ups:
The conversion rate dropped by 9%.
Not quite what I had expected. Maybe red is too much of a danger color? Are people scared to click a big red button? Or maybe it’s too red and will a softer shade of red do better?

The Green Button

Next attempt: a green sign up button. IMO it looks a bit nicer than the red one. But that’s not what counts, is it? Will it get more people to sign up for the trial? Ehm… no…
The results (400 sign ups counted):
13% less sign ups.
That’s even worse than the red one.

Back to Blue it is

Well, that didn’t work. So for now I am back to the blue buttons. I still think that making the free trial option stand out more is a good idea. But apparently not like this.
Maybe I should have tried Dan McGrady’s 3rd tip first: “Change the button text from “Signup for Free” to “Get Started Now”. Dan says:

“Get Started Now” is an easier sounding commitment than signing up. Signing up also has connotations with paying.

Make sense, so that’s what I am testing at the moment.

The last time I did this, the main thing that struck me was all the Try/Buy “noise” on the right side of the screen. There’s a Sign Up for the Free Trial box, a Get it Free (TrialPay) option and five (!) different buying options (Standard, Pro, two “Pro + scanner” options, plus a “custom order” option). I realized that I had been replicating most of my shop page right there on the landing page.

So I started experimenting with a different approach, aimed at getting more people to sign up for the free trial. Then just let the trial edition and the autoresponder sequence do the conversion to sales. I didn’t want to remove all buying options though, because we see a lot of customers buying without trying. But it would have to be reduced to just one button, simply taking the user to our recently re-designed shop page.

The original design ( = Version A )

This is the A-version of my A/B split test, the original look of the product home page. The left side of the page offering lots of product information and screenshots (even separate screens for the Windows and Mac editions), the right side having all the try and buy options.

This layout has worked well for us for a long time, but for some first time visitors it may be a little overwhelming. Let’s see if we can make it a little easier on the eye.

The first attempt ( = Version B1)

Here’s my first “less is more” redesign. To reduce the noise on the right, I completely removed all the trying and buying boxes from the panel on the right, replacing them with 3 recent testimonials.
I rewrote the top part of the left side, aiming to give a full overview of the software with just 1 headline, 2 pictures and 3 bullets. Followed by two large boxes, one for the free trial, one for buying. Below these two call-to-action boxes I only placed a demo video and some more testimonials (not shown in the screen shot).

The results
Immediately after starting this test, the sign up conversion rate shot up. That was hopeful. But sales dropped.
Here’s the outcome of the A/B test after 7 days:

• Sign Ups: +12.3%
• Sales: -9.3%
• Average First Purchase: +8.6%
• Total Profits: -2.8%

After 7 days, sales in the B category had gotten stronger again, but were still 9.3 % lower than the A category sales. But interestingly, the average value of the purchases were 8.6% higher in B. So in the end, the drop in profits for version B wasn’t too bad: down by 2.8%.

What’s happening here? First, the removal of main call-to-action (buy!) from the top right seems to have resulted in the decrease of immediate purchases. The extra sign ups (and thus extra people on my autoresponder sequence) seem to compensate for this decrease later on. Maybe the autoresponder, and its Buying Guide on day 2, is also helping to increase the value of the first purchase.

The second attempt ( = Version B2)

I like the increase in sign up rate and the high value purchases of the B1 design, so it would be great if we could keep those and generate more sales at the same time.
That’s what I tried to accomplish in the in B2 version shown below, by adding two call-to-action boxes on the top right again, in the same style as the big ones on the left.

The results>

I tested this B2 version against the original A version, still my control. I let the test run for 18 days, so that the trial version and trial autoresponder would have the time to do their jobs.

• Sign Ups: +8.8%
• Sales: +5.9%
• Average First Purchase: 0%
• Total Profits: +5.9%

All numbers are green, so that’s great
But if we look more closely, we find two interesting numbers, especially if we compare these results with those of the previous test:
First, the increase in sign up rate is lower for B2. Which is strange because there are two sign up boxes now, one on the left and one on the right. Did version B1 have more focus on the two big boxes on the left, just because the panel on the right was more noise-free?
Also, we lost the increase in purchase values that B1 gave us. However, looking at the details I found that the average purchase was $50.30 for both B1 and B2. For some strange reason the value for A had gone down to $46.30 during the first test.

What’s next

I am preparing a 3rd test now. Version B2 will be my new control ( = my new version A). For the new challenger I am reintroducing the big screenshots of my iPhone apps (because these has gotten lost in the redesign).

From the shop, via the upsell interstitial page and the name/address page, to the payment details page and finally the order confirmation page. I have written some new back-end PHP classes and am now rewriting all website pages based on those classes, making the pages more consistent, and more importantly, easier to maintain. Easy maintenance is essential, because I am tweaking these pages quite often.

I have already posted about my upsell interstitial page a while ago, which was the first page I based on the new back-end classes. This long Easter weekend I have completely rewritten the first page of the check-out, the shop page.

The Collectorz.com shop

This page has always been a challenge, because it easily becomes pretty complex. We have seven downloadable products, all with a Standard and a Pro edition. For 3 products we also have Mac editions. Plus we sell subscriptions for our 3 Connect web-apps.
Then there’s the Priority Support and CD-ROM delivery options. And of course we offer two types of barcode scanners (a third one will be added soon).

In fact, this page is so complex that for our recommended Buying Choices on our product pages, we skip this shop page, taking visitors directly to our upsell page. But a full shop page that shows everything we sell is a necessary evil, so I keep trying to make it as simple as possible.

What’s New?

Of course the rewrite was a perfect opportunity to implement further improvements. Let me give you an overview of the changes I made compared to the old page. The best way to see how it works is to play with the live Collectorz.com shop, but here’s a screenshot for easy reference:

The changes:

• Cleaner design, using white-space for separation, not borders.
• Both Windows and Mac programs are now visible. The old page showed just Mac OR Windows editions based on the browser agent, but that caused confusion.
• Better handling of empty carts. Now shows “- your cart is empty – ” and disables the Continue to Checkout button.
• Easier switching between Standard and Pro editions. Previously one had to remove the Pro edition from the cart, then add the Standard edition.
• It’s now easier to remove items from the cart (people missed the little red X-buttons in the cart). You can now remove on the left side too.
• Added our iPhone apps, even though they cannot be purchased through our shop, just linking them to the iTunes App Store.
• Prominent random (but recent) testimonial.
• Added images of most important payment methods.
• Prominent 90-day money back guarantee
• Several question-mark icons that open small explanation popups (created with Thickbox). The old page had some “more info” links too, but these linked to other pages on the website, taking people away from the Shop page.
  The new page has popups for:
  • Standard vs Pro
  • Priority Support
  • Barcode scanner comparison
  • Shipping and handling (about delivery times)
  • Coupon Code (with the hint: “Don’t have a coupon code? Google is your friend!” )

What’s next?

Currently, the Shop and the subsequent Up-sell pages are consistent in look and feel. Next up is a rewrite of the other pages, the name/address page, the payment details page and the order confirmation (or “thank you” page). I am planning to make the look of all these pages fully consistent, then add some kind of “Step 2 of 5″ indicator at the top, showing the buyer where he is in the check-out process.
The payment details page is going to require some creative programming (read “hacking”), because that step does not happen within our site, but on the RBS server.

Oh, before I forget, all feedback on the Shop is very very welcome.
Just post below. Thanks!

So in August 2008 I ran a simple A/B split test, testing just that:

• Group A: my regular product pages, no TrialPay option
• Group B: An extra “Get it Free” button, taking visitors to the TrialPay offer page. Just for our Standard edition (regular price $29.95). The button was a regular HTML button, just like our other “Add to Cart” buttons. Like so ->

The results were surprising: After tracking 240 sales (in 10 days), the profits in the B group were 30% higher. The extra TrialPay sales (only 13) didn’t account for that increase, so having the TrialPay option was generating more regular sales too.

My guess is is that the presence of the “Get it free” button had lowered our bounce rate and got first-time visitors to stick around just that little bit longer. Then they eventually ended up buying in the normal fashion (maybe because they want the PRO edition or something).

New website design, new TrialPay box

Then in May 2009, while toying and struggling with a new website design, I introduced a new TrialPay box. With a nice TrialPay logo, a short explanation and a big “Get it Free with TrialPay” button. Pretty cool looking box.
And without testing, I just added it to the new product pages. I know, completely unlike me as I normally test everything. My only defence is that I was testing other important changes at the time. Still stupid, I confess

A couple of weeks ago, I noticed that my TrialPay profits were slowly decreasing in the past couple of months. Also, we had been getting some confused customer emails, who had clicked on to the TrialPay website, looking to download our trial edition.
“trial pay, get it free” <-> “free trial edition” … I can see the confusion, especially if the two options are placed close to each other.

Anyway, reason enough to look into the presentation of the TrialPay offer again.
Of course, with another A/B split test:

Split testing TrialPay presentation

Here’s the two designs I tested against each other:

• Version A: a very prominent TrialPay box, right below the Free Trial box and above the Buying Options. The first buying box offers both the Standard and the PRO edition, as side by side choices.
• Version B: here I went back to the old box, offering TrialPay very subtly, as a free “buying option” inside the Standard edition buy box. The PRO edition has been given its own box.

The results

After four weeks and a total of 1290 sales, version B is resulting in 4% more customers and 4% more profits. The number of trial sign-ups was 2% higher too. The average first purchase was exactly the same in A and B.

The difference is not large, but after tracking this many sales, we can safely conclude that version B is doing better. On top of that, during the entire test, version B has been ahead. After two weeks of testing (enough data to make a decision), B was ahead by 10%. So I am quite confident in choosing for design B.

Also, the test shows that the way you present the TrialPay offer on your landing page can have a considerable impact on performance. In this case, it looks like bigger wasn’t better. And placing it right besides the “free trial” option could be confusing, maybe because of the “trial pay” name?

Other ways to use TrialPay

The above only discusses the usage of TrialPay on your product page, or landing page. But of course there are various other ways to use TrialPay.
At the moment we are also presenting users with the TrialPay option in our “trial expired” screen and on our upgrade pages (where people pay for the upgrade to a new version). Both with limited success. I am getting some “sales” that way though.

I am still planning to experiment with other placements, like:

• Download page: offering the “free trial” and a “get the full version for free” side by side. TrialPay tells me that this has been quite successful for several of their vendors.
• Exit popup: showing the TrialPay option in a popup when people leave your site without buying
• After uninstall: a bit desperate maybe, but who knows?
• Pirates: Can you detect pirated copies or usage? Maybe you can convince some pirates to get a full version for free?

Other ideas are welcome. Post below if you are using TrialPay in a creative way.

In terms of numbers of sales, it’s doing about the same as the old design. The profits for category B are slightly higher (about 3%), mainly because the average first purchase is still higher for the new design.

All in all, the new product pages are not a great success. But at least they are now performing well enough to switch to the new pages and start optimizing their details. I already have a couple of ideas I want to try:

• Headlines: Emotional / Fluffy / Benefit oriented vs Practical / Feature oriented
• Pricing: .95 vs rounded pricing
• Upsells: Move stuff like CD Delivery, Priority Support etc… to an interstitial upsell page, as opposed to including them in my “Recommended Buying Choices”

Today, I have started the “Fluffy vs Practical” headline test.

Version A:

Starting right now, you will enjoy a perfectly organized DVD collection…

Movie Collector Instantly Catalogs Your DVD & Blu-Ray discs.

You’ll Save Time and Money Today.

Finally Have Total Control Over Your Movie Collection.

Version B:

Automatic DVD Database Software for Windows, Mac OS X & iPhone

Movie Collector Automatically Catalogs Your DVD Collection.

Just Enter Movie Titles or Scan DVD Barcodes.

Instantly Download All Movie Details and Cover Images.

I am interested to see how this one works out. I A/B tested similar messages in my Adwords ads and the practical ones worked better there, that is, they resulted in a higher CTR.

I love A/B testing
Remind me to do another post about how I do these tests exactly.

• Downloads: down 4.22%
• Sign ups: up 1.50%
• Sales: down 19,11%
• Profits: down 11,52%
• Average First Purchase Value: up 10%

Something’s not right in the new design, but it’s hard to determine what it is.
Yes, I know, that is because we changed too many things at once.

My guess is the problem is in the different set of buying options that is offered in design B (we lost the TrialPay option, the delivery on CD option for the PRO edition, the “Customize Your Order” button, etc…).

So to have a test between the two designs that is more fair, I have re-introduced these buying options in design B. Oh, and I also brought back the cool looking Vista Media Center screenshots for Movie Collector, somehow these had gotten lost too. Ouch…

This is the challenger for the upcoming seven days. Will this one finally beat my control?

• We decreased the height of the photo of the bald guy in the top left
• Now we could move the headline over to the left
• Which created some space top right, just enough for one testimonial (or at least part of it)

We also centered the headline above the first two pictures, which seems to make it stand out a bit more.

Here’s the result:

These are the results after 7 days (A is the old design, B is the new design):

• Downloads: About the same
• Sign ups: 20 percent more for design B
• Sales: 19 percent less for design B
• Profits: 10 percent less for design B
• Average First Purchase Value: 10 percent higher for B

The bad…

The number of sales are down. Not good. Maybe moving the download/buy panel lower on the page wasn’t such a good idea after all.
Profits are down too, but interestingly enough not by as much as the number of sales. Interesting.

The good…

The Average First Purchase Value is up. Which explains the relatively low drop in profits.
Not sure why though, maybe because the new buy boxes explain the difference between the bundles and the scanners better, so that more people opt for a more expensive package?

The number of sign ups for the free trial are up too. That’s great.
Why? Maybe because the new look and the handsome bald guy top-left lowered the bounce rate? (I see no direct evidence of that in my website stats though).
Or because of the clear bold red “Click here for FREE trial” link under the main screenshot?

It is worrying that the increased number of sign-ups does not seem to translate into sales. At least, not within 7 days.
(On average, our customers buy within two days after their visit, though I am seeing slower conversions lately. Maybe the financial crisis is causing this?)

So… what to do?

Overall, it seems that the lower number of sales is killing the results of the B version now. So let’s try to get those up by making the buy boxes more “in-your-face”.

Today I started a new A/B test where B has the main “call for action” boxes on the top right again ( A is still the old design). The rest of the new look is kept as is, including the design of the download/buy boxes themselves.
I am sad that the testimonials at the top right had to go. I liked those there…

Anyway, here’s the new B design. Waddaya think, will this one beat the old layout?

But this time I am implementing a more radical change: a completely new layout. In the next couple of weeks I will be testing the new layout vs the old layout using an A/B split test. That is, fifty percent of new visitors will be seeing the old layout, the other fifty percent will get the new layout. Then we’ll be measuring sign-ups, downloads, sales and profits in both groups.

Version A: the “old” design

Here’s a screenshot of the old Movie Collector product page:

Version B: the new design

The goals I had for the new layout and design:

• Get rid of the noisy and crowded download/buy panel on the top-right.
• More focus on the result of cataloging your collection, as opposed to the process itself. In other words: promote benefits, not features.
• Replace the box-shot in the top left with something more interesting, preferably a photo of a person.
• Add more recency / up-to-dateness.
• Make better use of user testimonials.

After a couple of days, this is what we arrived at:

And the winner is ….

Today I have started the A/B test. I am planning to let it run for 14 days. Unless the results after one week are dramatic (one way or the other).

What do you think? Which one will win? And by how much?