I've been reading "Blink" by Malcom Galdwell and it has some relevant
ideas to the audio testing debate.

Malcom gives numerous examples in which food testing, such as blind
"central location" sip testing of sodas, doesn't translate perceptions
during normal, home use. I'm sure Harry knows a lot about this.

To be sure, some of this happens because during "normal" use one knows
the identity of the product and has associations to packing, brand,
etc. But even the experience of a food changes depending on whether we
eat a little or a lot of it. What tastes very sweet can become cloying
with time. What tastes dry can become engaging.

It is obvious that the subjective impression will differ depending on
context. That's what "Blink" points out over and over. What we need to
ask is: are the consicous impressions different because they process
different patterns in the item, or are they different because they
involve different conscious processing?

In other words, does product A taste different in long-term use because
(1) we are just changing our impression, or because (2) we are picking
up new patterns in the item? Everything I know about perception
suggests that (2) is possible.

Now audio: let's say A and B, two audio devices, produce different
sound. We put them into a quick switch comparison. Now, maybe you'll
hear a difference, and maybe not. Let's say you don't. There are two
cases: (1) perhaps A and B are only slightly different, below the
resolution of the ear. Or perhaps (2) A and B differ in *a pattern that
isn't used in forming the conscious impression*. The ear can hear it;
it just doesn't reach consciousness.

Since long-term listening results in a different conscious impression,
it seems likely (we certainly can't rule it out on the available
evidence) that these impressions are built on different patterns in the
sound. Cases (1) and (2) can't be distinguished by a quick-switch test.

Mike