A/B Testing Post from 37Signals

I found this post from 37 Signals to be an interesting viewpoint on testing.  It is about the process, not about the outcome.  

 

I only judge a test based on whether we designed and administered it properly.

As an industry, we don’t yet have a complete analytical model of how people make decisions, so we can’t know in advance what variations will work. This means that there’s no shame in running variations that don’t improve conversion. We also lack any real ability to understand why a variation may have succeeded, so I don’t care much whether or not we understood the results at a deeper level.

The only thing we can fully control is how we set up the experiment.....

 

See more at...  http://37signals.com/svn/posts/3169-ab-testing-its-not-about-the-results-and-its-definitely-not-about-the-why