John Iverson, one of the meany flawed thinkers at SP wrote the following:

Advocates of blind listening tests who say that ABX or similar tests prove
anything about audio components or can be useful in Stereophile product
reviews have it upside down and backward. They're living in negativeland.
The fact that any two of us in the same room taking the same test end up
scoring different results is all you need to know about the nature of blind
audio testing.

http://www.stereophile.com/asweseeit/805awsi/

It tells you something alright, but not what Mr. Iverson would have us
believe.

It tells us that when relying on only one's ears, people's guesses about
which device they are listening to are different. They have a 50/50 chance
of guessing correct every time they choose. That one may guess correctly at
one time and then get the next 5 in row wrong, while another person gusesses
two in row correctly, while getting the next 8 in row wrong, doesn't mean
the test procedure is flawed, only that they guessed. People being
different, they guess differently.



Iverson also said:

"Human beings are not pieces of objective test equipment. Some listeners
consistently score above average, others bounce around the middle, and still
others never get it right. My own blind-test results are not consistent. And
all the while, the differences that do or do not exist among products remain
fixed-and likely undiscovered. "

Where are the ones who score above average? Why aren't they laying claim to
some of the prize money available to thme if they can hear what nobody else
seems to have been able to hear?



He further states:

"Also consider that someone with an axe to grind can always use a room full
of cloth ears or a poorly designed test to "prove" that no meaningful
differences exist between two components."

Aside from this obvious slap in the face at Mike Fremer, what evidence does
he have that anybody doing DBT's are using cloth ears, such is certaqinly
not the case at Harman's facilities.

More nuttiness:

" Another series of tests, with a single properly trained listener who
consistently beats the odds, can prove the opposite"

But where is that person, doesn't he know he could become a millionaire?



Also:

"A blind test could be developed in which a known distortion is added to one
of two components; then we can see who can hear it. We could then repeat the
test with less and less distortion, until no one scores."

IIRC this has been done or something similar. Participants were allowed to
do sighted and blind listening to components, while THD was added to the
signal. The sighted listeners could not detect it until it became very
high. When the same THD was added to the blind lisgtening, it was detected
much earlier

As usual the opponents of blind comparisons are the ones who get the facts
and their interpretations wrong.



That wouldn't be so bad if they didn't also actively campaign for snake oil.