File this post under: Technical Nerd stuff for the .001% of you out there who might be interested.
For those of you who may not know it, Buzzillions makes its money in part from our users clicking through to retailer websites from various sections of our product pages. Users arrive on Buzzillions, select products, read reviews and continue on to one of our retailers to purchase. We want to satisfy the user by providing useful buying options when they are ready to buy and satisfy our merchants who are always desiring more of our above-average converting traffic. As part of an effort to maximize these clicks and conversion rate, we embarked upon Multivariate Testing using Google’s Website Optimizer, a framework that has many attractive features – chief among them being “its free”. Varying the content and measuring the success or failure of a tweak or series of tweaks to our user interface in attracting clicks is the name of the game.
Traditionally (in the web world that means in the last year or so), Website Optimizer is used to inject test content into a page dynamically via javascript. The content is served up from Google’s side after we set up our tests and tell it what to inject into our page. Could be a different-looking button or it could be an entire chunk of a page. This is tricky when you want to test a section of a page that is dynamic and has many repeating areas. From what I’ve seen most developers are working around it by serving up all the variations in their HTML and then using either css or javascript on the Website Optimizer side to either show or hide the content variations.
Problem is, page performance (size, speed) for users and keyword density for Google take a negative hit when you add the variations to the code. Doesn’t amount to much if the variations are small (button appearance, small text changes) BUT the toll could be very negative if your variations bloat your HTML by, say 25-50% like ours were. So we needed to come up with another way of doing things, especially since we had a lot of variations to test and very little time to test them in (sound familiar, software developers?).
After banging our heads against the wall for a few days, we came up with something that fit the bill (note that we did create an internal server side testing framework but it wasn’t multivariate and didn’t have statistically sophisticated reporting). Since our Where To Buy section is the one that is the most heavily tested and also the heaviest in terms of page weight and number of variations BUT is below the fold, we decided that we should try to AJAX the page section onto the page dynamically after the page loaded. The benefit being that it would have no negative impact on our site performance in terms of page size, speed or keyword density — all metrics that matter to our users and Google, especially during the busy holiday season for us. To accomplish this, we would need to create a backend web service and hook it up to several javascript components – the YUI Javascript Library which we use to create and retrieve our AJAX call, our own javascript for manipulating the AJAX’d data once returned, and the Trimpath Library for some front-end templating to make it all look pretty and seamless.
This combination of technologies allowed us to use a free and sophisticated tool such as Google Website Optimizer, retain the dynamic complexity of our page delivery all the while preserving page speed, size and keyword density. The team consisted of one backend engineer to create the web service, a designer to work up the variations, a business owner to shepherd the project and winnow the variations to a manageable size and a front end engineer to glue it all together and set up the Google Website Optimizer test itself.
Leading up to the 2009 Holiday season, we tested over 200 versions of our product page to maximize buying options for our users and clicks to our merchants. We were able to create a variation (variation #86 to be specific) that showed a DOUBLING of the click through rate to our merchants (without any sacrifice to the conversion rate). The click through rate growth was actually 60% once we installed variation #86 permanently before the Holidays started (we assume this discrepancy is due to the statistical wobble inherent in the multivariate reporting). Still, a 60% increase in click through rate was great news for our users (who found more buying options) and great news for our merchants (who received more traffic at the same above average conversion rate).
Here are some more details about how to integrate the above technologies into your site:
Since Google’s Website Optimizer code is required at the very top of the HTML to work effectively and because (for performance reasons) we load all of our javascript at the very end of the page just like any other website that is aware of performance’s impact on the user, we had to come up with a way of making Website Optimizer aware of what our javascript components were doing. To do this, we came up with the idea of having Website Optimizer poll for our components. The code that went inside of Website Optimizer was super-simple:
<script>
function poll() {
if (typeof BZGWOREADY==”undefined” || !BZGWOREADY) {
setTimeout(‘poll()’, 1000);
} else {
bZ.wtb.gwo.fetchData(‘variation1′, ‘variation3′, ‘variation9′);
}
}
poll();
<script>
This simple javascript function basically tells Website Optimizer to look for a javascript object called “BZGWOREADY” that lives at the very end of the javascript on our side of the fence; if it doesnt find it, it waits for one second and then looks for it again. Pretty basic actually.
Then, at the very end of the page after all of our javascript has loaded which deals with the AJAX call, the Trimpath templating and all the other javascripted parts of our test content, we instantiate the object that Website Optimizer is looking for simply like this:
var BZGWOREADY = {};
Instantiating the BZGWOREADY object like this tells the Website Optimizer code that we are ready to make our AJAX call and get the ball rolling – fetching and formatting the data and telling our javascript which variations to use (in the example above, the 3 parameters passed to the function bZ.wtb.gwo.fetchData which lives inside of the javascript on “our” side tells our code which variations to use).
Using this technique can be time-consuming in terms of development, but if you are concerned with page bloat and keyword density, you might want to check it out. Might be worth it for you. Have fun.
Jim is out and about again, this time representing Buzzillions at the 