While there’s some consensus that conversion rates should be measured by unique visitors, in some industries it can be naive to ignore a conversion rate calculated based on visits – especially under (very rare) specific circumstances. Tools like Google Website Optimizer rely solely on unique visitors, meanwhile Google Analytics relies upon visits to calculate conversion rates.

Let’s see why this is a problem…

First, an example

Let’s say you’ve built a fantastic site about health insurance and you’re testing three different variations. Google Website Optimizer (using unique visitors to calculate its conversion rate) tells you there are no winners.

In your own investigation however, Google Analytics’ goal conversion rates shows one variation performing dramatically better. You might end up with the following scenarios:

Assuming one variation increases the number of visits that each visitor makes to the site and another variation decreases the number of visits to the site, this could result in a higher or lower CPA (if visitors return to the site through a costly medium such as PPC). But you wouldn’t see this in Google Website Optimizer! Check out the difference in CPA that could be achieved here (yes it’s drastic, but it proves a point):

While GWO shows no clear winner calculating conversion rate on unique visits, an conversion rate calculated using visits (or clicks) might show a different winner in terms of CPA

Worse yet… You might place yourself in the position where Google Website Optimizer tells you one thing and Google Analytics tells you another thing:

In the example above, whilst “Visits CR Low” looks nice in GWO, you’d be better off choosing the losing variation.

Assumptions in the cases above

The examples above make a number of assumptions:

1. Subsequent visits return to the site through branded search (not resuming their session after having their cookies expire)
2. The number of return visits increases significantly
3. CPCs are unforgivably high
4. You’re heavily CPA driven

Likelihood of encountering this?

This is a very rare circumstance. In fact, I’ve encountered only one or two sites where this is of SOME concern. If you’re in doubt, you’re most likely going to want to measure your website’s conversion rate in terms of unique visitors.

How can you measure/keep-an-eye-on this?

First you’ll need to start tracking your Google Website Optimizer tests in Google Analytics and second…

Well to tell you the truth, I don’t know how to measure this very accurately myself.

Any ideas?