First I can say that I don't have much of a bias for or against any
type of speaker wire (or speaker cable system with terminations) or
any low voltage interconnect cable except I'd like the former to
deliver as much current as possible without changing impedence and
the latter to be super well shielded and matched to the components
being connected in terms the capacitative and inductive
characteristics the component's designers considered appropriate when
the preamp, tuner or other was designed and when it's specifications
were determined. I have been given dozens of different kinds of
speaker wire and cable systems to use in my CES booth displays and to
be used when I did speaker selling demos in stores. 20 years of
research testing loudspeakers for seller and buyers showed me that
the distortions in most speakers are so huge that it becomes nearly
impossible to hear any unique characteristics of speaker wire except
its ability to deliver current to cone speakers and voltage to
electrostatics. The very few loudspeakers that might be able to
demonstrate whether there was any audible improvement using one kind
or brand of wire (or even a cable system including unique
terminations), like the Quad 63 have other electrical components like
transformers in their systems making it seems unlikely that subtle
differences in wire can be significant enough to hear.

I believe this to be fact because of testing that I've done when
designing loudspeaker systems for audio companies, testing done as
the loudspeaker system development Imagineer for Disney during the
building of EPCOT and Tokyo Disneyland, and testing I've done for the
AES paper I delivered March 4, 1982 in Montreaux and later that April
to the Los Angeles chapter that showed that voice coils in
loudspeakers undergo so many changes of "sound reproduction
capability", that hearing the difference created by different wires
of the same guage and current carrying capacity would be like finding
the molecules of a specific raindrop after it had fallen into the
ocean. When I gave my paper on loudspeaker compression to the AES
chapter in Los Angeles in April 1982 a few of the engineers in
attendance calculated that the just the factors of non-constant
dynamic impedence and compression changes in loudspeakers that my
testing showed, would likely show up as fast variations in
temperature on the voice coil in excess of 100 degrees centigrade.
Almost immediately the chief engineer of Cetec-Gauss stood up to
mention that while he didn't think that it had really been important
to mention his company's testing in this arena up to that point in
time, he could verfiy that fast changes in voice coil temperatures of
well in excess of 100 degrees centigrade had been measured in many of
the tests Cetec-Gauss had done on their own products and those of
their competitors. These kinds of massive forces obliterate "subtle"
differences.

The louspeakers being discussed at that moment were large voice coil
types with designs that made cooling one of the most important
considerations (although I've tested hundreds of different component
loudspeakers for professional and home hifi use). The loudspeakers
used in home hifi systems do not cool nearly as well as pro
loudspeakers, and the negative effects on sonic characteristics due
to temperature change are much greater than those demonstrated by
professional loudspeakers. And that is just one of the factors
involved with loudspeaker compression. These changes in sonic
characteristics make it nearly impossible to do any real research on
speaker wires that could be relevant to audiophile listening because
the test to make such comparison "fair" would be literally impossible
to design. By the time listeners could focus on the sound playback of
of one of the test wire/products the second product in the test would
already be unfairly tested because the test loudspeaker system
(acoustic microscope equivalent) will not likely "sound" the same as
it did 30 seconds ago. This means that the test passage would need to
be made longer and restarted after a specific cooling off time and by
then the human acoustic memory is gone. This could be why so many
anecdotal testimonials involve hearing "things" after the time was
taken to disconnect one set of speaker wires and connect a second
set. The speakers probably cooled down and sounded better after the
"wire changing period". Maybe comparitive testing of large voltage
wire should be done using thin film or electrostatic headphones
because these devices are actually better at exposing tiny
differences in audio characteristics. As an aside I might suggest
that the way some speaker cable systems are produced they seem to
want to be part of the crossover for the louspeaker or maybe adjust
the frequency response. I've always wanted speaker wire to be able to
deliver gobs of current to some of the loudspeakers I've designed or
listened to, except with electrostatics when I want the speaker wire
to conduct the correct voltage from the amp to the speaker's input
irrespective of the changes in the input impedence of the speaker.

If there really was a wire that was substantially better sonically
then "super consumers" like Disney would do the testing (at whatever
cost) so that they could deliver the AES papers that would make
greater prestige for Disney. Disney has bought numerous "home hifi"
products and tested them (and sometimes even used them) because they
have an incredible investment in not only having the best of
"whatever" in their facilities, but in knowing more than any other
audio consumer in the world. Matsu****a may be an incredible audio
manufacturer and their leaf tweeter with extended response to 80kHz
is one of the few devices that can be used to test whether human
hearing really processes 20+kHz information when our brain decides on
how much "acoustic reality" is being reproduced. But even Matsu****a
has recognized that no company is as concerned with knowing what is
what in audio as much as Disney. And cost is no object for Disney, so
whether speaker wire was $.50 a foot or $5.00 a foot wouldn't matter
to a Disney engineer because (through AES) Disney promotes their use
of the audio equipment and materials proven to be sonically superior
in their theme parks. The test labs at WED Imagineering are better
outfitted by Bruel and Kjaer than almost any audio manufacturer in
the world. As "super-consumers", Disney is unrivalled and there isn't
really much difference in speaker wire sonically or there would be
Disney engineers giving papers on their "wire" findings at AES
conventions. As it is, Disney has strict rules about having enough
copper to carry the current or voltage to the speakers designed into
any facility or show in their theme parks, but even in the most
critical applications there is no special type of wire specified.

It's also a sure bet that the manufacturers of wire or wiring
harnesses for speakers don't really want there to ever be any one
special kind of wire or wiring system that proves to be superior to
others. In the audiophile marketplace, confusion about which wire is
best increases sales. Otherwise it wouldn't be possible to sell the
same consumers 3 or 4 different sets of wire or interconnect cable at
enormous cost. And the different types of manufacturers all have
their own reasons for not wanting any one type of wire to be
determined "best". These reasons vary even if sometimes when
confronted by an insecure retailer, a manufacturer may SAY their wire
is superior or the most cost effective sonically, but like I say,
"Confusion increases sales" so doing a real comparison test is the
opposite of what these manufacturers want. Higher sales keep
companies alive and that is their prime directive, so confusion in
the marketplace best fulfills that directive. Look at all the
possibilities.

The wire seller who truly believes his product is superior won't want
to bother wasting time, money and effort to prove something he
already believes. The wire seller who knows his product is NOT
superior won't want to be shown up. The wire seller who doesn't care
if his product is superior won't want to bother with something as
unimportant as testing, especially because of the costs involved and
the possibility he MAY be shown up. So it is against the interests of
wire sellers to participate in any tests which would clearly
determine which speaker wire was superior or at least which was best
for which speaker or for the money. Even if one crusading wire maker
were to support a true test of wire it would only allow every other
manufacturer to look at the results and then make a copy of the
product. And that doesn't consider the great unknown that could be
very deadly to any and all of the wire companies. It can happen in
the most bizarre ways but the results can be quite staggering.

In 1978 I was the professional products marketing manager at ESS
(Electrostatic Sound Systems) at a time when they sold professional
versions of the Heil tweeter, some professional amplifiers and had
the rights to import some European professional products. The
company took on a new ad agency for consumer products and this agency
did a month of field research in 20 retail outlets to see what they
had to work with for ESS' new ad/marketing campaign. The ad agency
found that in head to head sales demos using ESS speakers against any
other speaker brand of comparable price, store salespeople were able
to sell the sound superiority of ESS loudspeakers against any other
brand. With the various retailers around the country carrying various
mixes of loudspeaker lines, ESS seemed to be able to sell their
products against any other brand based on sound quality during a
demo. Store salesmen usually take the path of least resistance and
they would have sold more ESS loudspeakers except for one thing.
Sometimes when store salespeople suggested to potential customers
that they listen to speakers, the customers didn't want to listen to
ESS loudspeakers. Buyers would come in predisposed to listen to a JBL
speaker vs maybe an AR speaker, or a Bose speaker vs an Infinity
speaker. If the salepeople suggested listening to a JBL speaker vs an
ESS speaker, the potential customer would often change the second
contender to one of the other brands. ESS was confident they could
sell their product against any other brand but they needed to give
the market a reason to even listen to them, and their technology
didn't seem to be motivating consumers enough.

So at the big meetings with the ad agency about what sort of ad
campaign would convince consumers to at least listen to ESS
loudspeakers, it was determined that just using repetative ads
wouldn't do it. ESS had already had a high frequency of ads. Since
consumers said they gave JBL a listen due to their belief that pros
used JBL so perhaps JBL made the consumer product they wanted.
Consumers also said that as the inventor of the acoustic suspension
or direct reflecting or "whatever" loudspeaker, other companies had
some credibility with consumers. So the ad agency proposed that ESS
go out and do a sextuple blind listening test nationwide with
thousands of consumers under tightly controlled conditions, to get
documented evidence that ESS loudspeakers sounded better in various
price ranges compared to 9 other brands of loudspeakers. This was to
be the "credibility hook" ESS supposedly needed to have consumers
give them a listen.

The Physics, Psychology/psychoacoustics, Audiology and Music
departments at 4 major universities in California, Washington state,
Wisconsin and Georgia would check the test controls and an outside
accounting firm was brought in to tabulate and document the results.
Of course ESS wanted to keep as much of the information gained as
possible to themselves for future product development. But the gamble
was that if their products were sonically superior, they could create
an ad campaign that would convince more consumers to give their
speakers a listen in retail stores. They gave me the job of actually
running the test and managing all the different groups involved. I
was chosen because the test was supposed to use the highest quality
source material of classical, jazz, rock and pop music and some
natural sounds. This meant getting access to original master tapes
used to cut master pressing disks. I had the expertise needed to work
with the studio people to make the part of the test program material
using these tapes, I could manage a touring "road show" and I knew
how to use quality test equipment to replicate the test conditions in
each location.

The musical passages had to be long enough to have some kind of
repetative or sustained characeristics that would allow listeners to
hear differences in loudspeakers. The program was 40 minutes long and
covered every kind of musical material (although there was quite a
bit of well recorded vocals in nearly half of the recordings). The
speakers were all set up behind acoustically transparent screens that
were visually opaque. About half of the listeners were in the near
field. Almost every seat in the listening area was in a "sweet" spot.
The colleges did allot to promote the participation of their
students, staff and anyone who was interested, in newspapers and
local radio. There were four different programs and there were four
different price levels of speakers, although the programs were
rotated between the different price levels. Consumers could come back
to take the "test" up to three times if they so desired. Prizes were
given out via drawings and the various departments at the
universities were allowed to share some of the data collected at
their facility. The playback system was never driven to clipping, the
level of the balancing pre-amps was set for each speaker based on
their average output using pink noise and flat, A, B and C weighted
measurements both in the near and far fields. The locations of the
speakers on stands was varied and with 9 speakers of which only 6
could be tested at a time even the comparison match-ups were
constantly varied. There were tens of thousands of comparisons made
by thousands of listeners. Quite a few audio business theories and
guesses were tested in these circumstances.

The program material did make a difference in the test results. Two
different loudspeakers that were compared with different program
material might show different audience preferences depending on
whether the program material was pop, jazz or classical, so it seemed
that even the best loudspeakers were not always "best" for all kinds
of music. Of course there were some good loudspeakers that always
seemed to do well in head to head comparisons using any program
material and others that were just bad and were disliked by most
listeners no matter what the program material. Sometimes the room's
acoustics influenced the test results so that if the heat in the room
built up and the window's opening positions were changed, the test
results could change. ESS was able to show in random blind testing
that their products were sonically superior "enough" (statistically
valid results 5-8x greater than the margin of error) so that the test
results were about the equivalent of the in-store research done by
the ad agency. An interesting device was the "Comparison
Identifier", a large box in the front of the room which displayed a
number for "one" speaker in comparison vs the number for the "other"
speaker being tested. The box would produce a number between 1 and 9
but the speakers could be labeled with any number at any time even
during one listening test session. The test was actually quite well
designed so that no speaker had an advantage. A few of the really
poor sounding speakers were weeded out after the first hundred or so
testing/listening sessions and some cross price-level comparisons
could then be made. The marketing campaign (ESS Wins on Campus) was
introduced before the tour of test locations was completed and so in
Wisconsin during Homecoming week and at Georgia Tech the crowds of
listeners wanting to "take the test" were enormous.

The information not used for marketing purposes involved how 2
non-ESS branded loudspeakers did compared to each other. But one very
startling fact came to light and it forced the testing to stop and
the ad campaign to change. The most prefered loudspeaker for all
kinds of music was the same at every college. It was the same for any
test and any type of program material. Unfortunately this obviously
"most preferred" loudspeaker was not the most expensive ESS
loudspeaker. It was one of the least expensive ESS loudspeakers. The
design criteria for this model of loudspeaker was actually quite
elegant and as a model it shows how a device could be purposely made
so it might "beat a test". By the time I had watched listeners in
thousands of tests I could have designed a loudspeaker that would
have always won the test by a statistically significant margin. ESS
wasn't happy that their most expensive loudspeaker was only the most
preferred by listeners when compared to other company's expensive
loudspeakers. It seemed that when people made choices without visual
cues as to which speaker "should" be best, their ears guided them in
the direction of the most accurate sound, and that wasn't always the
most expensive or the biggest loudspeaker.

In addition this "up-the-line superiority" wasn't the same for all
companies. Whether by design or accident, some companies produced
speakers that sounded better as the price increased, but other
companies had results like ESS where the tests determined that some
of their less expensive loudspeakers were actually their most
accurate. It turned out that the corporate philosophy of ESS that was
dictated in part by the requirement for all mainline models to use
the Heil tweeter, worked against them in some ways.

There was also the fact that if the test could be duplicated because
it was controlled, then any company could eventually figure out the 4
or 5 keys factors in the listener's sonic decision and design a
loudspeaker that would "beat the test". This was an obvious outcome
because there really could only be one set of major sonic priorities
that people use to decide which loudspeaker sounds more accurate than
another. Perhaps the priorities would be slightly different so my 1,
2, 3, 4,and 5 sonic priorities are 2, 5, 1, 6 and 3 for you, but
generally there are only so many criteria that determine "realistic
sound". Our brains and ears have evolved over a few millions of years
making life and death decisions based on what our ancestors heard. So
our hearing developed in a certain way prioritizing certain acoustic
criteria. What was also obvious was that no one speaker company in
1978 made speaker systems that incorporated the most possible of
these criteria in their designs. There were also limitations imposed
by the fact that all the tested loudspeakers were bookshelf models
because that is what dominated the market at the time. Strangely
though, larger floor standing cabinets have many many sonic factors
going against them and so it is less expensive to make a more
realistic sounding speaker if it is a small or bookshelf model.

There is also the "monkey-wrench" factor that dictates that people
will be more disposed to spend more money if the loudpseaker they are
auditioning looks bigger and more imposing because that loudspeaker
seem visually to be worth more money and our visuals are telling our
brain that the larger loudspeaker should sound better than smaller
loudspeakers. In point of fact, the larger loudspeaker will likely
sound less accurate than if the same money was put into a smaller
loudspeaker. But the fact that this listening/test research could
have been used against its sponsor and the loudspeaker industry as a
whole points up how a simple sextuple blind test could become so
dangerous for the status quo in loudspeakers in general.

This should all give people pause when discussing comparison testing.
Is it possible to actually design a test so that the second item
being listened to, is not immediately disadvantaged (or advantaged)
by it's position in the test? Is there any reason for the
manufacturers to support such a test or even to acknowledge the
results as being valid for reasons of their own? Is it possible in
certain circumstances to even hear sonic differences without
resorting to a basic change in venue (like the need to use
headphones)? In fact, the loudspeaker and the source material are
acknowledged to be the links of the chain that have the most
problems, so if there is no effort to actually define how to reduce
the magnitude of distortons in these areas, how can any other portion
of the system even be tested? As I said in "the emperor's clothes"
thread, there have been less than perhaps 10 loudspeaker systems ever
made that can realistically reproduce the voice, the piano and
natural sounds "accurately". If there has been so little effort to
make realistic sounding loudspeakers and program material, what does
it matter if the other parts of systems are .1% improved or not?

Finally I have another anecdote that actually applies to me and what
I want out of a listening test for my audiophile system. In the 80s
when the compact disc was getting a firm foothold in audio, AES was
concerned enough about what the 18kHz brick wall filter would sound
like to try to develop an international listening test to see what
kind of sonic impact these filters could have. They developed a test
using playback systems that had lots of response output up to 50KHz,
the way the Panasonic leaf tweeter could provide and then using
playback material recorded with a sampling rate way past 80KHz they
tested AES members to see how easily they could detect the insertion
of filters of various types at various frequencies while music or
real life sounds were being played.

The idea was to push a button when you heard a filter inserted and
release it when the filter was removed. At 3KHz everything sounded
like it was playing over a telephone when the filter was inserted so
that was easy. At 10KHz almost everybody noticed filter insertion. At
15 and 18KHz there were still many many engineers who noticed the
filters. But at 23KHz I for one have only been able detect a filter
about 20% of the time and at 25KHz I didn't hear any differences. But
there was one young European recording engineer who kept getting it
right all the time well past 20KHz. He was able to focus in on the
hiss from the condenser microphone and pre-amp, plus the noise from
the mixing console and the dither in the recording while excluding
the other sounds in other spectra and he really could hear when these
very high frequency filters were inserted and removed. That makes a
total of one guy. I am absolutely certain that I would not want that
one guy to ever make a decision (for me) in a test about any audio
component item because I cannot hear what he hears so I don't want
the decision about what is "best" to be made on the basis of any
crieria except the ones I can hear.

I want a test to determine anything in audio to be so repeatable and
available that I can take the test myself because I don't want to
base what will be best for me on what someone else can hear or
someone else's taste in music. This is one of the things I learned by
running a test taken by thousands of people. We all don't hear the
same. We all don't make judgements on what sounds best using the same
program material. Our sonic priorities for phase/square wave
response, frequency response, dynamic reproduction capability, the
intrusion of spurious cabinet noises and constant directivity might
be similar but what is vastly different is what we DON'T hear. If I
can't hear something then it isn't worth me paying for it in a
product. I am a fussy listener compared to most and I am a trained
audio engineer who can focus on individual specta and instruments
while excluding others. But even so I don't want to take someone
else's word for what's best even if they are similarly trained ,
because someone else will either have better or worse hearing than
me, and I'll be the one spending the money on the equipment. Also
whatever test will be developed to make the decisions about what is
best, I want to be able to take it. But in case I didn't mention it,
I'd rather not pay for the test itself because I know from
experience, just how incredibly expensive this will all be. Paying
for credible comparative listening tests is the part I haven't quite
worked out yet. By the way, I've spoken to Ed Meitner about sonics
many times in the past and I'm sure that he could easily produce the
source player, turn-down/turn-up switches (we wouldn't want to make
full power switches between products would we?) and level balancing
pre-amp circuits needed for the kinds of test people here discuss,
but, of course there would be a price to pay and I wouldn't want to
be the one paying that price. Watchking

Listening isn't a competative sport, but buying equipment is.

We don't get enough sand in our glass

watch king wrote:
The loudspeakers
used in home hifi systems do not cool nearly as well as pro
loudspeakers, and the negative effects on sonic characteristics due
to temperature change are much greater than those demonstrated by
professional loudspeakers. And that is just one of the factors
involved with loudspeaker compression. These changes in sonic
characteristics make it nearly impossible to do any real research on
speaker wires that could be relevant to audiophile listening because
the test to make such comparison "fair" would be literally impossible
to design. By the time listeners could focus on the sound playback of
of one of the test wire/products the second product in the test would
already be unfairly tested because the test loudspeaker system
(acoustic microscope equivalent) will not likely "sound" the same as
it did 30 seconds ago. This means that the test passage would need to
be made longer and restarted after a specific cooling off time and by
then the human acoustic memory is gone. This could be why so many
anecdotal testimonials involve hearing "things" after the time was
taken to disconnect one set of speaker wires and connect a second
set. The speakers probably cooled down and sounded better after the
"wire changing period".

So quick A/B switching and using short snippets of sound are the most
effective for discrimination. I also found pink noise to be very
revealing for detecting level and frequency response differences.

Thank you for a very intersting and thought-provoking post. I completely agree
that "testing" is not so simple as some would have us believe and that
experience and 'listening biases' play a large part.

One comment below:

"watch king" wrote:
If there really was a wire that was substantially better sonically
then "super consumers" like Disney would do the testing (at whatever
cost) so that they could deliver the AES papers that would make
greater prestige for Disney.

A couple of years ago, MIT was commissioned to completely rewire the
recording/performance facility at George Lucas' Skywalker Ranch in No. CA with
their cables.

Listening isn't a competative sport, but buying equipment is.

Right on.

Regards,
Mike

I'm not sure this will work because Google groups seems to be an
unreliable post portal for this .rec group but here goes.

Are we assuming this is a double blind test with an indicator display
and that you are testing something other than loudspeakers? CD
players, tuners and interconnect cables are the easiest to test. Phono
cartridges and loudspeakers are difficult and headphones are nearly
impossible. Power amps and preamps are in the middle difficulty-wise.

Whenever possible it is best to give the test listeners a sense of the
music when music is the source. So for an easy to test item there
would be a reasonable period of musical lead-in and then a countdown
as the music came up to test level. Then during the sustained passage
(some operatic overtures are good for this and some symphonic passages
as well, and of course much of the quartet music produced is great for
this, as well as some repetative piano music), the music would be
brought to "test" level after which a number of 8-10 second
comparisons could easily be made. Usually it is best to have 3 or 4
direct head to head comparisons with one passage because that allows
the listener to be absolutely sure they can hear clearly which test
item is better than the other. (Of course Unsure should always be a
choice, but if Unsure is the most common response then there would
likely be no difference between test product X vs test product Y).
With test program other than music like pure spoken voice or natural
sounds, the listeners need to know what the material really sounds
like or the test isn't really valid. This would also be the case with
material like single classical guitar (eg. Segovia plays Bach) or
single flute, or single "a capella" voice. This is also especially
important for any pipe organ music. Musical memory is "helpful" here.
Go listen to an organ concert, record it binaurally and then use
headphones and when you test the CD player or interconnects, your
musical memory will help you judge.

Of course speakers shouldn't be tested this way. Speakers should be
played both together, loudish to warm up, with the test listeners "not
listening" hands over ears will help and then the warm speakers can be
compared. Alternatively a different speaker not part of that
particular head to head comparison can be played with the musical lead
in and then take a 1 second break and start the comparisons at full
test level between speaker X and speaker Y. Or alternatively with
speakers (only), a "test" can be run at full loudness but the results
not counted just to give the listeners the sense of what's coming
(more like a countdown 8 switch 7 switch 6 switch etc), then after the
first chorus and a return to the main, a real comparison test can be
run for useful results (eg. 4-5, 4-5, 5-4, 5-4 END, with 4 and 5
randomly chosen numbers for the two tested speakers for this one
portion of the test). Follow that with another comparison and another
until the good parts of that song are used up.

But for CD players and interconnect cables the testing can be very
straightforward. The "moderator" cannot alas partake of the test for
the sake of non-biased presentation. All lead-ins, song intros and
explanations have to be prerecorded and "played" to the listening
testers. Once the test starts it must finish or all results are
unusable. There are many many "control restrictions" needed and
requiring pre-test documentation of procedures. The musical program
material should be rotated throughout the program during different
tests to reduce the biases that "program material position" in the
test program can create. As often as possible the order which any item
is tested first should change. For high power switching of items like
amps, speaker cable and speakers make the loudness turndown steps
between test items pretty short on the order of .1 seconds from full
loudness to 0, then switch, then turn up in .1 seconds. By putting
time code onto a CD and having the switches time code driven this can
be accomplished. We used telephone touchtone signals to activate the
numerical display box. The switch shutdown/turnup can be programmed
right onto the CD material although duplicate disks would need to be
synchronized somehow if 2 CD players were being compared.

The most listening testers can seem to hold their sonic concentration
is betwen 20 minutes and 40 minutes. It is an intense experience. On
the other hand testers don't seem to be able to fully concentrate
until about 2 comparisons into the test or about 1-2 minutes. 20
minutes gives you barely enough time for one throwaway opener and then
6 bits of test material and 40 minutes can allow for 12 or so tests
passages but people start getting headaches and listening fatigue. If
need be, run the test a number of times with different program
material and with intervals of 30-75 minutes between tests. Don't
drink too many liquids before a test session. Getting up for the
bathroom ends any test with "No valid results". In other words no
distractions should be tolerated (no cellphones, no doorbells, no
chatting or physical communication between test listeners, sadly-no
crying babies and especially no "Just listen to this" kind of cueing.)
It's either done professionally or it's useless.

This is not to say that perhaps the character of test items A & B will
not be immediately noticable after 15 minutes of testing. They may
well be different enough to be immediately recognizable, but keep
concentration so as to provide results which can be used to determine
which item is more accurate or "better". When using one of the very
rare "transparent" test listening speakers to test other items,
between one and three chairs is about all a Quad ESL 63 or Martin
Logan CL-3 can accomodate in the sweet listening spot. Only a very
tiny (point souce) loudspeaker can produce the kind of superior
quality and wide soundstage with pinpoint imaging needed to make tests
with perhaps as many as a dozen possible test seats. Very small
loudspeakers with high power handling, very low spurious noise
generated by the cabinet, constant directivity, a single driver for
the voice band, reasonable bandwidth and phase alignment capability,
limit the number of louspeakers that can be used to perhaps 2 or 3
models that have ever been made in the history of audio. Big boxes
will not work for this kind of testing because front row seats will
hear something dramatically different from middle and back seats.
Remove any chairs not full of test listeners. Use preprinted pages
with only 2 columns of numbers on them to allow the two test item
numbers to be circled or a box to be checked. Don't be surprised if
the choice changes with program materials.

Listening tests may be exciting but they may not be fun. No matter how
people have travelled and might be leaving or how tight their
schedules are, if some component used but not being tested develops a
buzz or glitch or if the test aparatus malfunctions don't use any of
the results. Use the prerecorded "moderator" intros to cue listeners
as to what they might listen for, (eg. "the following quartet is
composed of flute, cello, violin and trumpet", or "on this recording
the piano is the only acoustic instrument and it is mic'd with 2
overstring and one soundboard mix microphone", or "the test comparison
will be done during the middle of the 3 minute drum solo") because if
there are anomolies to be heard let the testers know when to
concentrate the most closely. Watchking

listening isn't a competative sport, buying equipment is.

We don't get enough sand in our glass.

chung wrote in message news:4uVPb.124026$8H.329218@attbi_s03...
watch king wrote:
The loudspeakers
used in home hifi systems do not cool nearly as well as pro
loudspeakers, and the negative effects on sonic characteristics due
to temperature change are much greater than those demonstrated by
professional loudspeakers. And that is just one of the factors
involved with loudspeaker compression. These changes in sonic
characteristics make it nearly impossible to do any real research on
speaker wires that could be relevant to audiophile listening because
the test to make such comparison "fair" would be literally impossible
to design. By the time listeners could focus on the sound playback of
of one of the test wire/products the second product in the test would
already be unfairly tested because the test loudspeaker system
(acoustic microscope equivalent) will not likely "sound" the same as
it did 30 seconds ago. This means that the test passage would need to
be made longer and restarted after a specific cooling off time and by
then the human acoustic memory is gone. This could be why so many
anecdotal testimonials involve hearing "things" after the time was
taken to disconnect one set of speaker wires and connect a second
set. The speakers probably cooled down and sounded better after the
"wire changing period".

So quick A/B switching and using short snippets of sound are the most
effective for discrimination. I also found pink noise to be very
revealing for detecting level and frequency response differences.

(Mkuller) wrote:

Thank you for a very intersting and thought-provoking post.

Yes, I agree. Thank you.

I completely
agree
that "testing" is not so simple as some would have us believe and that
experience and 'listening biases' play a large part.

The latter for sure. That's exactly why we need bias controls. Level matched
and double blind is the best.

One comment below:

"watch king" wrote:
If there really was a wire that was substantially better sonically
then "super consumers" like Disney would do the testing (at whatever
cost) so that they could deliver the AES papers that would make
greater prestige for Disney.

A couple of years ago, MIT was commissioned to completely rewire the
recording/performance facility at George Lucas' Skywalker Ranch in No. CA
with
their cables.

Was MIT "commisioned" or did they volunteer? The reason I ask is that when I've
asked recording industry personnel about the brand-names that have appeared in
liner notes they have always told me that cable companies approached them about
cabling needs and would "offer" to supply cabling at no cost in return for the
brand name appearing in liner notes or other studio literature.


Listening isn't a competative sport, but buying equipment is.

Right on.

Regards,
Mike

Neither is a competitive "sport" for those who are interested in maximizing
sonic quality through-put and not ego-satisfaction or ego-insecurity
minimization.

(Watch King) wrote:

snip

But for CD players and interconnect cables the testing can be very
straightforward. The "moderator" cannot alas partake of the test for
the sake of non-biased presentation. All lead-ins, song intros and
explanations have to be prerecorded and "played" to the listening
testers. Once the test starts it must finish or all results are
unusable. There are many many "control restrictions" needed and
requiring pre-test documentation of procedures. The musical program
material should be rotated throughout the program during different
tests to reduce the biases that "program material position" in the
test program can create. As often as possible the order which any item
is tested first should change. For high power switching of items like
amps, speaker cable and speakers make the loudness turndown steps
between test items pretty short on the order of .1 seconds from full
loudness to 0, then switch, then turn up in .1 seconds. By putting
time code onto a CD and having the switches time code driven this can
be accomplished. We used telephone touchtone signals to activate the
numerical display box. The switch shutdown/turnup can be programmed
right onto the CD material although duplicate disks would need to be
synchronized somehow if 2 CD players were being compared.

The most listening testers can seem to hold their sonic concentration
is betwen 20 minutes and 40 minutes. It is an intense experience. On
the other hand testers don't seem to be able to fully concentrate
until about 2 comparisons into the test or about 1-2 minutes. 20
minutes gives you barely enough time for one throwaway opener and then
6 bits of test material and 40 minutes can allow for 12 or so tests
passages but people start getting headaches and listening fatigue. If
need be, run the test a number of times with different program
material and with intervals of 30-75 minutes between tests. Don't
drink too many liquids before a test session. Getting up for the
bathroom ends any test with "No valid results". In other words no
distractions should be tolerated (no cellphones, no doorbells, no
chatting or physical communication between test listeners, sadly-no
crying babies and especially no "Just listen to this" kind of cueing.)
It's either done professionally or it's useless.


Once again you have brought up many important variables that may affect the
outcome and validity of any open-ended audio component comparison DBT using
music, particularly the amateur DIY variety that are strongly advocated by some
posters here.

What is your experience in using 'highly experienced listeners" (reviewers,
disc masterers, etc.) versus 'average listeners' for blind tests to determine
whether there is an audible difference between components?

One variable you did not mention is 'control of the switch'. John Atkinson of
Stereophile has said he personally has a lot of problems with any blind test
where he can't control the switch. What are your thoughts on that issue?
Regards,
Mike

Mkuller wrote:
(Watch King) wrote:

snip

But for CD players and interconnect cables the testing can be very
straightforward. The "moderator" cannot alas partake of the test for
the sake of non-biased presentation. All lead-ins, song intros and
explanations have to be prerecorded and "played" to the listening
testers. Once the test starts it must finish or all results are
unusable. There are many many "control restrictions" needed and
requiring pre-test documentation of procedures. The musical program
material should be rotated throughout the program during different
tests to reduce the biases that "program material position" in the
test program can create. As often as possible the order which any item
is tested first should change. For high power switching of items like
amps, speaker cable and speakers make the loudness turndown steps
between test items pretty short on the order of .1 seconds from full
loudness to 0, then switch, then turn up in .1 seconds. By putting
time code onto a CD and having the switches time code driven this can
be accomplished. We used telephone touchtone signals to activate the
numerical display box. The switch shutdown/turnup can be programmed
right onto the CD material although duplicate disks would need to be
synchronized somehow if 2 CD players were being compared.

The most listening testers can seem to hold their sonic concentration
is betwen 20 minutes and 40 minutes. It is an intense experience. On
the other hand testers don't seem to be able to fully concentrate
until about 2 comparisons into the test or about 1-2 minutes. 20
minutes gives you barely enough time for one throwaway opener and then
6 bits of test material and 40 minutes can allow for 12 or so tests
passages but people start getting headaches and listening fatigue. If
need be, run the test a number of times with different program
material and with intervals of 30-75 minutes between tests. Don't
drink too many liquids before a test session. Getting up for the
bathroom ends any test with "No valid results". In other words no
distractions should be tolerated (no cellphones, no doorbells, no
chatting or physical communication between test listeners, sadly-no
crying babies and especially no "Just listen to this" kind of cueing.)
It's either done professionally or it's useless.


Once again you have brought up many important variables that may affect the
outcome and validity of any open-ended audio component comparison DBT using
music, particularly the amateur DIY variety that are strongly advocated by some
posters here.

Any of the provisos he's cited would *also* apply to sighted comparison,
of course...but they certainly don't seem to be applied in the sighted
comparisons I read about every month.

But then again, nothing he's written even remotely supports the idea that
*sighted*, 'open ended' comparison, using music (and please, feel free
to add whatever new conditions you can conjure up),
advocated and practiced by the most audiophiles, including the two
main audiophile magazines, is a good way to test for difference at all.

And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.
In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,
as all researchers in the field of perception acknowledge.

DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.

--

-S.

"They've got God on their side. All we've got is science and reason."
-- Dawn Hulsey, Talent Director

"Mkuller" wrote in message
*snip* quoted text
What is your experience in using 'highly experienced listeners"
(reviewers,
disc masterers, etc.) versus 'average listeners' for blind tests to
determine
whether there is an audible difference between components?

Mike,

You have repeatedly brought up the notion that DBTs, particularly of the ABX
variety, only have validity in 'trained' listeners and are useless to the
untrained. So I ask, if we were to set up a double blind cable
discrimination test and prior to running the test, had each testor engage in
some ABX training and we subsequently charted their sensitivity to known
types of distortions, would you conclude that the ensuing cable test would
be valid even if all testors failed to discriminate between the cables?

One variable you did not mention is 'control of the switch'. John
Atkinson of
Stereophile has said he personally has a lot of problems with any blind
test
where he can't control the switch. What are your thoughts on that issue?
Regards,
Mike

And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.
In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,
as all researchers in the field of perception acknowledge.

DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.


Neither one is perfect as it stands now. You happen to prefer your imperfectly
applied DBT which obscures differences over my method which doesn't provide
"adequate controls" for bias.

Otherwise, provide me an example of a 'perfect' DBT with a sensitivity which
has been verified to be, say around 0.2db - two times the difference Pinkerton
is demanding for his $4.5K Cable Challenge. (Our money is safe.)
Regards,
Mike

Bruce Abrams wrote:
Mike,

You have repeatedly brought up the notion that DBTs, particularly of the ABX
variety, only have validity in 'trained' listeners and are useless to the
untrained.

My point is that to be able to identify audible differences in a blind test,
the listeners need to be experienced enough to be able to 1.)recognize them and
2.)categorize them for memory. When I was first starting out in High End
audio, I recall comparing two preamps in an audio store with a friend. I could
tell they sounded a little different, but I was unable to tell exactly what was
different or decide which one I thought sounded better for my frined to
purchase. Today, after 20 or so years experience comparing differences in
audio components I would be much more likely to be able identify and qualify
the differences.

In research, subjects get training in learning to hear the specific artifact
they are supposed to identify. Since the differences with audio equipment can
be so many different things (timbre, dynamics, soundstaging, bass, etc.) how do
you train a subject adequetly?

So I ask, if we were to set up a double blind cable
discrimination test and prior to running the test, had each testor engage in
some ABX training and we subsequently charted their sensitivity to known
types of distortions, would you conclude that the ensuing cable test would
be valid even if all testors failed to discriminate between the cables?


No, subject training is only one of the many necessary variables. What about
demonstrating the actual sensitivity of the test source material (i.e.0.5dB or
5.0dB) before applying it to a situation where the amount of audible difference
is unknown. If the audible difference is 2dB and the test is only sensitive to
5dB, then you will get null results, as most do.
Regards,
Mike

Any of the provisos he's cited would *also* apply to sighted comparison,
of course...but they certainly don't seem to be applied in the sighted
comparisons I read about every month.

If you don't like them why are you reading them?


But then again, nothing he's written even remotely supports the idea that
*sighted*, 'open ended' comparison, using music (and please, feel free
to add whatever new conditions you can conjure up),
advocated and practiced by the most audiophiles, including the two
main audiophile magazines, is a good way to test for difference at all.


Equipment reviews are not tests for differences per se. They are subjective
reviews of equipment used by the reviewer in the likely manner that the
consumer would use the product.


And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.

Varification is not an issue in subjective review for the most part. Using the
product as the consumer would use it is a reasonable way to evaluate equipment
if the consumer who reads the magazine evaluates equipment the same way. If you
read the reviews and don't like the fact that they are not scientifically
reliable, I suggest you read the disclaimer that suggests consumers shouldn't
rely on reviews alone and should audition equipment for themselves before
making any purchases.

In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,

Yes they are. As is the case for any subjective review. Stereophile is not
trying to be a scientific journal. Most journals that do subjective reviews of
hardware in any number of fields are every bit as unscientific.


DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.


I wouldn't expect such absolute claims from a scientist.

S888Wheel wrote:
Any of the provisos he's cited would *also* apply to sighted comparison,
of course...but they certainly don't seem to be applied in the sighted
comparisons I read about every month.

If you don't like them why are you reading them?

Component reviews are typically a mix of fact -- how many
inputs/outputs does the device have, what formats does it play,
what materials and parts were used in its construction, how is the
remote laid out , how does it differ from previous or competing
models in terms of features , etc -- and possibly spurious
subjective impression about the sound . Guess which part I
find most valuable.


But then again, nothing he's written even remotely supports the idea that
*sighted*, 'open ended' comparison, using music (and please, feel free
to add whatever new conditions you can conjure up),
advocated and practiced by the most audiophiles, including the two
main audiophile magazines, is a good way to test for difference at all.


Equipment reviews are not tests for differences per se. They are subjective
reviews of equipment used by the reviewer in the likely manner that the
consumer would use the product.

Which does not make the conclusions drawn any more tenable...all it means
is that both reviewer and consumer tend to evaluate the sound of a
component in a manner that is unlikely to be particularly
reliable or accurate. But it does make them feel good.

And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.

Varification is not an issue in subjective review for the most part.


Indeed. But these 'subjective reviews' you talk about do *not* confine
themselves purely to statemenst such as, "I liked this one", "I didn't like
that one"; "This one made me feel good", or the like. They instead often
*do* involve comparisons (often to the reviewer's 'reference system')
and they *do* make *specific claims* about one 'throwing a better soundstage'
or 'sounding less harsh' or 'clarifying the bass'. Under thy typical
review conditions there is a distinct possibility that these
impressions are NOT the result of
actual sonic differences, but are *entirely* psychological in origin -- a
likelihood that science tells us cannot be confidently dismissed without more
controlled comparison. Reviewers pretend this isn't the case.



Using the
product as the consumer would use it is a reasonable way to evaluate equipment
if the consumer who reads the magazine evaluates equipment the same way.

Yes, the community of belief is a strong component of the hobby. But is the belief
warranted by the facts? At what point does the hobby become more
'faith based' than fact-based?


If you
read the reviews and don't like the fact that they are not scientifically
reliable, I suggest you read the disclaimer that suggests consumers shouldn't
rely on reviews alone and should audition equipment for themselves before
making any purchases.

Do these disclaimers -- where are they printed , btw, in the current issues
of the usual audio journals? -- further recommend that the personal auditions be
carried out in a scientifically reliable manner? Repeating the error of the
reviewer yourself isn't going to get you closer to reliable.

In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,

Yes they are.
As is the case for any subjective review. Stereophile is not
trying to be a scientific journal. Most journals that do subjective reviews of
hardware in any number of fields are every bit as unscientific.

I'll never understand why you think telling me that *lots* of journals
have poor standards of objective proof for their claims, is any sort of
argument *for* sighted comparison.

What sorts of products are you thinking of?

DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.


I wouldn't expect such absolute claims from a scientist.

All it means is that you can 'refine' a sighted method forever, and the results will
always have a huge, intrinsic question mark over them. By contrast DBT can shrink the
size of that question mark to the point of *scientific* certainty...which is the
strongest kind of certainty we can achieve about the natural world.



--

-S.

"They've got God on their side. All we've got is science and reason."
-- Dawn Hulsey, Talent Director

Component reviews are typically a mix of fact -- how many
inputs/outputs does the device have, what formats does it play,
what materials and parts were used in its construction, how is the
remote laid out , how does it differ from previous or competing
models in terms of features , etc -- and possibly spurious
subjective impression about the sound . Guess which part I
find most valuable.

Certainly the same part I do.


Which does not make the conclusions drawn any more tenable...all it means
is that both reviewer and consumer tend to evaluate the sound of a
component in a manner that is unlikely to be particularly
reliable or accurate. But it does make them feel good.


Which is the bottom line in any hobby.


Indeed. But these 'subjective reviews' you talk about do *not* confine
themselves purely to statemenst such as, "I liked this one", "I didn't like
that one"; "This one made me feel good", or the like. They instead often
*do* involve comparisons (often to the reviewer's 'reference system')
and they *do* make *specific claims* about one 'throwing a better soundstage'

or 'sounding less harsh' or 'clarifying the bass'. Under thy typical
review conditions there is a distinct possibility that these
impressions are NOT the result of
actual sonic differences, but are *entirely* psychological in origin -- a
likelihood that science tells us cannot be confidently dismissed without more
controlled comparison.

All true and all a possible reaction that any given consumer may have to such a
new piece of equipment. Just because the reviewer is more detailed in
describing their impressions of a given piece of equipment doesn't make their
impression any less real or any less like the sort of impressions that any
consumer may also have.

Reviewers pretend this isn't the case.

The disclaimer says enough IMO. It is clear that the editor of Stereophile does
not hold the subjective opinions of the reviewers as definitive. I haven't seen
any reviewer specifically claim that thier subjective impressions are
universally infalable.


Yes, the community of belief is a strong component of the hobby. But is the
belief
warranted by the facts? At what point does the hobby become more
'faith based' than fact-based?

At the point where people draw their conclusions based on what they are told
rather than what they experience. You cannot blame a magazine that tells it's
readers to not use subjective reviews as definitive claims of performance and
to audition equipment for themselves before purchasing equipment for some
audiophiles acting on faith.


Do these disclaimers -- where are they printed , btw, in the current issues
of the usual audio journals? -- further recommend that the personal auditions
be
carried out in a scientifically reliable manner? Repeating the error of the
reviewer yourself isn't going to get you closer to reliable.


They do not suggest how people audition equipment for themslves. Just that they
do so. I believe these disclaimers can be found in the introduction to every
recomended components list. It is your opinion that a reviewer or a consumer is
in error if they do not use scientifically valid methods for testing thier
subjective impressions. But then *all* of the advocates of DBTs on RAO clearly
chose the one component that they believe matters most in the same unreliable
manner. The bottom line how a hobbyist feels about his or her participation in
the hobby.


I'll never understand why you think telling me that *lots* of journals
have poor standards of objective proof for their claims, is any sort of
argument *for* sighted comparison.

I agree that it is unlikely you will ever understand. I suggest you give
careful thought as to why skeptics spend their time on bigfoot UFOs and
holistic medicine and not hobbies like audio and photography.


What sorts of products are you thinking of?

Toys for grownups.


All it means is that you can 'refine' a sighted method forever, and the
results will
always have a huge, intrinsic question mark over them. By contrast DBT can
shrink the
size of that question mark to the point of *scientific* certainty...which is
the
strongest kind of certainty we can achieve about the natural world.


OK, if this is what you meant by 'perfectable' than I am OK with your claim. I
think perfectable per se goes beyond scientific certainty.

Steven Sullivan wrote:
S888Wheel wrote:
Any of the provisos he's cited would *also* apply to sighted comparison,
of course...but they certainly don't seem to be applied in the sighted
comparisons I read about every month.

If you don't like them why are you reading them?

Component reviews are typically a mix of fact -- how many
inputs/outputs does the device have, what formats does it play,
what materials and parts were used in its construction, how is the
remote laid out , how does it differ from previous or competing
models in terms of features , etc -- and possibly spurious
subjective impression about the sound . Guess which part I
find most valuable.

I also read them for the pure entertainement value. I find amusing how
many different ways someone can describe the "sound" of a cable.

"watch king" wrote:

.....snip to content......

By the way let me restate that no one should be trying to concentrate
on listening tests for more than an hour at a time. Remember the ear
canal closes down under periods of stress of many kinds, changing the
sensitivity, frequency response and comb filter responces of the
ears. Test for an hour max and then rest for an hour is a good rule
of thumb. Also don't run 3 minutes of testing and then figure you've
finished. Even the most professionally trained listeners usually need
a time to "zero their sonic focus" so the results during the fisrt
few minutes of testing are usually suspect though not totally
worthless.

Because of the human sensory system is highly adpatable sensitvity is generally
highest at first exposure or at change of state. It is true that some effects
may require training (some people have be 'taught' to hear a phantom image for
example) but once learned acoustical effects are most strongly apparent at
change of state.

Listening fatique is also an interesting factor. But I think it's useful to
differentiate it from listening 'stress.' Fatique may occur with long testing
periods because of repetitive tasks but is exaggerated by the difficulty of the
task. IME most listening fatique and listening stress occurs when subjects are
attempting to "hear" an inaudible effect.

IME, with multiple subjects, boredom, and loss of attention, is a larger factor
with highly sensitive listeners.

There is also much misunderstanding about the role of stress in sensory
perception. A himan wil be at his most sensitive to any physical stimuli when
under the greates stress imaginable .... a case of life or death.

I once had a Proponent argue that a bias-controlled listening test introduced
sensitivity reducing stress akin to pointing a gun at the subject. My response
was that if the effect were truly audible a gun-to-the-head would be the most
sensitivity enhancing device possible.
The stress factor would only be debilitating only IF the stimulus were NOT
audible.

S888Wheel wrote:
Which does not make the conclusions drawn any more tenable...all it means
is that both reviewer and consumer tend to evaluate the sound of a
component in a manner that is unlikely to be particularly
reliable or accurate. But it does make them feel good.


Which is the bottom line in any hobby.


In butterfly collecting, one wants to know the
*actual* facts of the creature's taxonomy, life cycle,
geographical distribution, etc....not a collection of
comforting communal beliefs. One may *want* to believe
that a Luna moth is a rare species, when one has one in
a collection -- they are quite pretty, after all --
or that it's life cycle is peculiar, but in fact it's not
and it isn't. Lepidopterists accept these facts with
no complaint or cavilling about the small-mindedness
of entomologists. They don't indulge in special pleading
that 'maybe science just doesn't know' or 'science
can't measure everything'.

In most hobbies, claims of difference are based on
incontrovertibly visual evidence.


Indeed. But these 'subjective reviews' you talk about do *not* confine
themselves purely to statemenst such as, "I liked this one", "I didn't like
that one"; "This one made me feel good", or the like. They instead often
*do* involve comparisons (often to the reviewer's 'reference system')
and they *do* make *specific claims* about one 'throwing a better soundstage'

or 'sounding less harsh' or 'clarifying the bass'. Under thy typical
review conditions there is a distinct possibility that these
impressions are NOT the result of
actual sonic differences, but are *entirely* psychological in origin -- a
likelihood that science tells us cannot be confidently dismissed without more
controlled comparison.

All true and all a possible reaction that any given consumer may have to such a
new piece of equipment. Just because the reviewer is more detailed in
describing their impressions of a given piece of equipment doesn't make their
impression any less real or any less like the sort of impressions that any
consumer may also have.

And it doesn't make these 'impressions' any closer to *true*. In the
case where in fact there is *NO DIFFERENCE*, isn't it absurd to talk of the
impressions being 'real'? They are only 'real' in the trivial sense that
a false statement remains 'really' a statement.

Your line of reasoning ends up having to assert that the audiophile hobby
isn't particularly concerned with what's true.

Reviewers pretend this isn't the case.

The disclaimer says enough IMO. It is clear that the editor of Stereophile does
not hold the subjective opinions of the reviewers as definitive. I haven't seen
any reviewer specifically claim that thier subjective impressions are
universally infalable.

What disclaimer? Have you seen any reviewer claim that their impressions
could in fact be entirely spurious? Have you seen any admit that,
absent controls for bias, level matching, etc, the likelihood that impressions
about cable or amp or CD 'sound' are *entirely* spurious, cannot be dismissed?

Yes, the community of belief is a strong component of the hobby. But is the
belief
warranted by the facts? At what point does the hobby become more
'faith based' than fact-based?

At the point where people draw their conclusions based on what they are told
rather than what they experience. You cannot blame a magazine that tells it's
readers to not use subjective reviews as definitive claims of performance and
to audition equipment for themselves before purchasing equipment for some
audiophiles acting on faith.

But 'audition for yourselves' is not a sufficiently detailed piece of
advice, Scott, as regards *verifying audible difference*,
any more than 'trust your ears' is. It's little more
than slogan or mantra, unless accompanied by information about factors
that can produce false impressions during those auditions.

Do these disclaimers -- where are they printed , btw, in the current issues
of the usual audio journals? -- further recommend that the personal auditions
be
carried out in a scientifically reliable manner? Repeating the error of the
reviewer yourself isn't going to get you closer to reliable.


They do not suggest how people audition equipment for themslves. Just that they
do so.

Exactly. Which makes these disclaimers no more useful as truth-finding
advice about the 'sound' , than the reviews themselves. The same
essential flaws in the methodology apply.

I believe these disclaimers can be found in the introduction to every
recomended components list. It is your opinion that a reviewer or a consumer is
in error if they do not use scientifically valid methods for testing thier
subjective impressions. But then *all* of the advocates of DBTs on RAO clearly
chose the one component that they believe matters most in the same unreliable
manner. The bottom line how a hobbyist feels about his or her participation in
the hobby.

For the hundredth time...

People choose components for lots of reasons, not all of which involve the
sound: looks, budget, features. Visual differences are easy to verify.
Sonic ones often aren't. But journals don't confine themselves only
to reviewing the looks and features , do they?

DBT advocates differe from others only in their willingnes to
*admit* that without DBT, their impression of the 'sound' of
cables and amps stands a strong chance of being entirely wrong.
Or else they differ in their willingness to actually perform such
comparisons.

It's also rather schizophrenic of journals to display test measurement
results, but then perform their reviews sighted. Why make a nod
to some science, but ignore other science? Clearly the journals want
to present *some* sort of semblance of technical accuracy.



I'll never understand why you think telling me that *lots* of journals
have poor standards of objective proof for their claims, is any sort of
argument *for* sighted comparison.

I agree that it is unlikely you will ever understand. I suggest you give
careful thought as to why skeptics spend their time on bigfoot UFOs and
holistic medicine and not hobbies like audio and photography.

The answer is that far more of the public at large cares about Bigfoot
and UFOs and holistic medicine, than about the folklore of audiophiles.
And what controversies in photography are analogous to 'cable sound' or
'green pens', Scott? Are there treatments or devices that claim to
induce *visible* differences in photographic results, that require
a blind comparison for verification?


What sorts of products are you thinking of?

Toys for grownups.

And because they're 'toys', it's acceptable for the hobby to be pervaded by
child-like, magical thinking about how those toys work? How odd.
Usually children grow out of such beliefs about their toys.






--

-S.

"They've got God on their side. All we've got is science and reason."
-- Dawn Hulsey, Talent Director

"Mkuller" wrote in message
news:5GgQb.105187$Rc4.724489@attbi_s54...
And that's because -- and this is the crucial thing -- it
can't *ever* be a good method, for verifying subtle differnces.
In other words, in contrast to scientific methods,
the method advocated by
the main 'voices' of audiophilila, and people like yourself, is
*fundamentally and essentially flawed*,
as all researchers in the field of perception acknowledge.

DBT for audible difference is 'perfectable' -- sighted listening simply
*isn't*.


Neither one is perfect as it stands now.

That's like saying neither Barry Bonds nor Randy Johnson are "perfect"
hitters
because neither bats 1.000.

- Gary Rosen

Steven Sullivan wrote:
In most hobbies, claims of difference are based on
incontrovertibly visual evidence.


That's simply not true. One close to High End audio is sports cars. Everyone
has their own opinion which they prefer and a lot of it is based on 'the feel
of driving it'. I used to own a '95 Acura NSX T-top. Road and Track compared
it to the '95 Porsche Targa and '95 Ferrari F-355 convertible. 0-60mph and
skid pad specs were almost identical for the three cars. Objectivists might
say the three cars had no significant differences. Any subjectivist that drove
all three would be able to tell you the differences and which they preferred.

snip
Your line of reasoning ends up having to assert that the audiophile hobby
isn't particularly concerned with what's true.


*Truth* has many dimensions. If you chose to look at truth through your own
filter, you may miss what's true to others...
Regards,
Mike

Mkuller wrote:
Steven Sullivan wrote:
In most hobbies, claims of difference are based on
incontrovertibly visual evidence.


That's simply not true. One close to High End audio is sports cars. Everyone
has their own opinion which they prefer and a lot of it is based on 'the feel
of driving it'. I used to own a '95 Acura NSX T-top. Road and Track compared
it to the '95 Porsche Targa and '95 Ferrari F-355 convertible. 0-60mph and
skid pad specs were almost identical for the three cars. Objectivists might
say the three cars had no significant differences. Any subjectivist that drove
all three would be able to tell you the differences and which they preferred.

I'm not an auto aficionado. Are 0-60 and skid pad spec considered sufficient to
determine if two cars are *indistinguishable*? Is 'almost identical' equivalent to
*imperceptibly different* in such cases? On what bases are these beliefs founded?
If they are based on data that's as strong as those from psychoacoustics, then
the rational thing to do would be to test the subjectivists claims in some
sort of controlled protocol, where obvious differnces -- such as visual ones --
can have no effect.


snip
Your line of reasoning ends up having to assert that the audiophile hobby
isn't particularly concerned with what's true.


*Truth* has many dimensions. If you chose to look at truth through your own
filter, you may miss what's true to others...


One may believe that the earth is flat, but looking at the world through
*that* filter doesn't make the belief any more true.

We're not discussing philosophy or aesthetics or religion, Mr. Kuller. We're
not talking about the philosophy or semantics of the word 'true' -- or
at least, I refuse to go down that pointless road from this point on.

Real devices in real systems that induce real sound waves in real air
and generate real stimulus from real hair cells in real ears that result
in real brain activity that we experience as 'hearing'. That's
what we're talking about. It's an unfortunate fact that our judgement
about what we hear -- about what *really* was contained in those sound waves
-- is demosntrably fallible. Out belief does not necessarily map to
what happened. We can even believe two things are different when
they are demonstrably the same (e.g., the same component presented twice
when we are told it was switched). Our perceptual systems can
be fooled. Deny any of that, and you are denying *reality*.







--

-S.

"They've got God on their side. All we've got is science and reason."
-- Dawn Hulsey, Talent Director

Steven said

Which does not make the conclusions drawn any more tenable...all it means
is that both reviewer and consumer tend to evaluate the sound of a
component in a manner that is unlikely to be particularly
reliable or accurate. But it does make them feel good.



I said


Which is the bottom line in any hobby.


Steven said


In butterfly collecting, one wants to know the
*actual* facts of the creature's taxonomy, life cycle,
geographical distribution, etc....not a collection of
comforting communal beliefs.

How on earth do you know what all butterfly collectors want?

Steven said

One may *want* to believe
that a Luna moth is a rare species, when one has one in
a collection -- they are quite pretty, after all --
or that it's life cycle is peculiar, but in fact it's not
and it isn't. Lepidopterists accept these facts with
no complaint or cavilling about the small-mindedness
of entomologists. They don't indulge in special pleading
that 'maybe science just doesn't know' or 'science
can't measure everything'.

What is your point? Are you saying that audiophiles have unrealistic ideas
about the rarity of their music or equipment? I would like to point out however
that science does take an interest in the biology of butterflies. It has become
quite evident that no such interest exists for audiophilia.

Steven said


In most hobbies, claims of difference are based on
incontrovertibly visual evidence.

OSAF.

Steven said


Indeed. But these 'subjective reviews' you talk about do *not* confine
themselves purely to statemenst such as, "I liked this one", "I didn't
like
that one"; "This one made me feel good", or the like. They instead often
*do* involve comparisons (often to the reviewer's 'reference system')
and they *do* make *specific claims* about one 'throwing a better
soundstage'

or 'sounding less harsh' or 'clarifying the bass'. Under thy typical
review conditions there is a distinct possibility that these
impressions are NOT the result of
actual sonic differences, but are *entirely* psychological in origin -- a

likelihood that science tells us cannot be confidently dismissed without
more
controlled comparison.

I said


All true and all a possible reaction that any given consumer may have to
such a
new piece of equipment. Just because the reviewer is more detailed in
describing their impressions of a given piece of equipment doesn't make
their
impression any less real or any less like the sort of impressions that any
consumer may also have.

Steven said


And it doesn't make these 'impressions' any closer to *true*.

It does for the person having those impressions.

Steven said

In the
case where in fact there is *NO DIFFERENCE*, isn't it absurd to talk of the
impressions being 'real'?

No. Not if you understand what an impression is.

Steven said

They are only 'real' in the trivial sense that
a false statement remains 'really' a statement.


No. They are real in that the person who has a given impression indeed does
have that impression.

Steven said


Your line of reasoning ends up having to assert that the audiophile hobby
isn't particularly concerned with what's true.

Audiophilia is very much about personal aesthetic experiences for many
audiophiles. Why this is so difficult for some to understand seems to be a
reflection of those people's limmited scope of understanding aesthetic
experiences.

Steven said


Reviewers pretend this isn't the case.


I said


The disclaimer says enough IMO. It is clear that the editor of Stereophile
does
not hold the subjective opinions of the reviewers as definitive. I haven't
seen
any reviewer specifically claim that thier subjective impressions are
universally infalable.

Steven said


What disclaimer?

I already answered this.

Steven said

Have you seen any reviewer claim that their impressions
could in fact be entirely spurious?

Not entirely. But it would be pretty absurd to think their impressions are
"entirely spurious" IMO.

Steven said

Have you seen any admit that,
absent controls for bias, level matching, etc, the likelihood that
impressions
about cable or amp or CD 'sound' are *entirely* spurious, cannot be
dismissed?

I have seen no laundry lists of all the possible caveats that any review may be
subject to. I think the general disclaimer suggesting that people listen and
decide for themselves covers it. But I already told you this.

Steven said


Yes, the community of belief is a strong component of the hobby. But is
the
belief
warranted by the facts? At what point does the hobby become more
'faith based' than fact-based?

I said


At the point where people draw their conclusions based on what they are
told
rather than what they experience. You cannot blame a magazine that tells
it's
readers to not use subjective reviews as definitive claims of performance
and
to audition equipment for themselves before purchasing equipment for some
audiophiles acting on faith.

Steven said


But 'audition for yourselves' is not a sufficiently detailed piece of
advice, Scott, as regards *verifying audible difference*,
any more than 'trust your ears' is.

Why does the advice need to be detailed? Are you suggesting that reviewers are
obligated to give detailed instructions on how to audition equipment?

Steven said

It's little more
than slogan or mantra, unless accompanied by information about factors
that can produce false impressions during those auditions.


I disagree. It is IMO a simple disclaimer that points out the reader may not
agree with the reviewer's findings.

Steven said


Do these disclaimers -- where are they printed , btw, in the current
issues
of the usual audio journals? -- further recommend that the personal
auditions
be
carried out in a scientifically reliable manner? Repeating the error of
the
reviewer yourself isn't going to get you closer to reliable.


I said


They do not suggest how people audition equipment for themslves. Just that
they
do so.

Steven said


Exactly. Which makes these disclaimers no more useful as truth-finding
advice about the 'sound' , than the reviews themselves. The same
essential flaws in the methodology apply.

Balony. They let the readers know that the reviews are not to be taken as
dogma. That is very useful information for the highly impressionable readers
you seem to be so concerned about.

I said


I believe these disclaimers can be found in the introduction to every
recomended components list. It is your opinion that a reviewer or a
consumer is
in error if they do not use scientifically valid methods for testing thier
subjective impressions. But then *all* of the advocates of DBTs on RAO
clearly
chose the one component that they believe matters most in the same
unreliable
manner. The bottom line how a hobbyist feels about his or her participation
in
the hobby.

Steven said


For the hundredth time...

People choose components for lots of reasons, not all of which involve the
sound: looks, budget, features. Visual differences are easy to verify.
Sonic ones often aren't. But journals don't confine themselves only
to reviewing the looks and features , do they?

No. If you have a problem with it don't support any of the journals that offer
subjective reviews of equipment. That would be all of them by the way.

Steven said


DBT advocates differe from others only in their willingnes to
*admit* that without DBT, their impression of the 'sound' of
cables and amps stands a strong chance of being entirely wrong.
Or else they differ in their willingness to actually perform such
comparisons.

This is just plain funny given the fact that those same advocates seem to be
quite cetain thier impressions are quite right. Maybe you could cite a DBT
advocate who believes thier "impressions" of the sound of any cable or amp
"stands a strong chance of being entirely wrong." I will say though, plenty of
people I know who don't rely on DBTs are quite aware that their impressions may
be in some part wrong and is not entirely reliable. I include myslef in this
group.

Steven said


It's also rather schizophrenic of journals to display test measurement
results, but then perform their reviews sighted.

Since they all do this and you seem to think it is crazy. Maybe you should
start a journal that does all it's subjective listening double blind.

Steven said

Why make a nod
to some science, but ignore other science?

Reporting bench tests is no more a nod to science than reading a clock or a
thermometer.

Steven said

Clearly the journals want
to present *some* sort of semblance of technical accuracy.


Yeah. So?

Steven said



I'll never understand why you think telling me that *lots* of journals
have poor standards of objective proof for their claims, is any sort of
argument *for* sighted comparison.

I said


I agree that it is unlikely you will ever understand. I suggest you give
careful thought as to why skeptics spend their time on bigfoot UFOs and
holistic medicine and not hobbies like audio and photography.


Steven said


The answer is that far more of the public at large cares about Bigfoot
and UFOs and holistic medicine, than about the folklore of audiophiles.
And what controversies in photography are analogous to 'cable sound' or
'green pens', Scott?

Lense quality is a hotly debated subjective issue.

Steven said

Are there treatments or devices that claim to
induce *visible* differences in photographic results, that require
a blind comparison for verification?


Interesting question. It seems the lines drawn in photography never beg for
blind verification. I suspect a blind verification would remove the possibility
of sighted bias.

Steven said



What sorts of products are you thinking of?

I said


Toys for grownups.

Steven said


And because they're 'toys', it's acceptable for the hobby to be pervaded by
child-like, magical thinking about how those toys work? How odd.
Usually children grow out of such beliefs about their toys.

Usually truly objective people don't burn so many straw men.

"watch king" wrote in message ...


Maybe comparitive testing of large voltage
wire should be done using thin film or electrostatic headphones
because these devices are actually better at exposing tiny
differences in audio characteristics.

Guess how I do my listening tests?

"Mkuller" wrote in message
...
Steven Sullivan wrote:
In most hobbies, claims of difference are based on
incontrovertibly visual evidence.


That's simply not true. One close to High End audio is sports cars.
Everyone
has their own opinion which they prefer and a lot of it is based on 'the
feel
of driving it'. I used to own a '95 Acura NSX T-top. Road and Track
compared
it to the '95 Porsche Targa and '95 Ferrari F-355 convertible. 0-60mph
and
skid pad specs were almost identical for the three cars. Objectivists
might
say the three cars had no significant differences. Any subjectivist that
drove
all three would be able to tell you the differences and which they
preferred.

No objectivist would say the cars had "no significant differences" based on
two narrow measurements. But a subjectivist might say one car "felt"
faster when the difference was really the firmness of the seat padding.

- Gary Rosen

On 26 Jan 2004 18:53:04 GMT, (Mkuller) wrote:

Steven Sullivan wrote:
In most hobbies, claims of difference are based on
incontrovertibly visual evidence.


That's simply not true. One close to High End audio is sports cars. Everyone
has their own opinion which they prefer and a lot of it is based on 'the feel
of driving it'. I used to own a '95 Acura NSX T-top. Road and Track compared
it to the '95 Porsche Targa and '95 Ferrari F-355 convertible. 0-60mph and
skid pad specs were almost identical for the three cars. Objectivists might
say the three cars had no significant differences. Any subjectivist that drove
all three would be able to tell you the differences and which they preferred.

Actually, objectivists would point to readily measurable differences
in spring/damper rates, roll centres, steering rack gearing, engine
gearing, and exhaust note tuning, to explain how these cars feel on
public roads as opposed to test tracks. High performance cars don't
just *happen*, y'know, they are very carefully *engineered* to feel
the way they do. How do you think that BMW manage to make both
razor-sharp rorty-sounding Z4s and big wallowy 7-Series barges?

As with SNR and THD, 0-60 and lateral G are pretty useless measures
for high performance products.

snip
Your line of reasoning ends up having to assert that the audiophile hobby
isn't particularly concerned with what's true.

*Truth* has many dimensions. If you chose to look at truth through your own
filter, you may miss what's true to others...

I'm sure you'd like to think that truth is somehow able to mould
itself to your personal prejudices, but that isn't the case, the
non-existence of 'cable sound' being but one example.
--

Stewart Pinkerton | Music is Art - Audio is Engineering

First there seems (from the emails I've received directly) to be some
misunderstanding about my statements concerning blind testing. I
think blind testing is the best way to make determinations about
whether one audio product is superior in the circumstances where
products either measure identically or if their various distortions
and specifications differe in ways that would not immediately
demonstrate the superiority of one item or items. The only
considerations are that the tests be designed to be equal for each
item tested, that the listeners would not know except from listening
which item is being used at any moment, that the test be repeatable
and that the order of the items being tested should change at random
moments so that (for example) if the first set of tests has one item
going first 1st, 3rd, 6th and 8th, then during the next round with
the same program material (and everything else) the test have that
same item going 2nd, 4th, 5th and 7th.

Now a comment on the idea of the "trained listener" that is being
bandied about. What does this mean? In my experience professional
recording engineers are trained listeners. They really aren't any
better at listening testing than any other listeners and in some
circumstances they can have an incredible bias that makes them
useless as test listeners. In the circumstance that a recording
engineer has played a piece of music he is intimately familiar with,
through his own studio monitors, his preferences afterwards if that
recording is ever played is worthless. I've been to the former JVC
Cutting Studio many times. The place was full of well trained
professional recording engineers and they listened to hundreds of
different master recordings (both digital and analog) through their
own mastering monitors. They would be poor listening test subjects.
It would be no different for engineers who listened mostly on UREI
Time Aligns(Altec 604s), or any number of other JBL, Tannoy, Fostex,
Yamaha, Westlake or other studio monitors. These listeners just have
such huge built in listening biases that it would be difficult for
them to be objective about the "total quality" of one audio product
VS another in a blind listening test unless their loudspeakers and
facilites were used. These loudspeakers and facilites might never be
able to demonstrate a variety of audio characteristics.

I have worked with dozens of other professionally trained listeners
and engineers who also had enormous biases about what constitutes
natural sound. Even Disney Imagineers may be highly biased, even
though their goal is supposed to be designing sound systems (or
purchasing premade systems) based on the best sonic quality or most
natural sound (along with reliability, sufficient output to do the
job, and other non-sonic quality considerations). And all Imagineers
are professionally trained listeners. It's true that some Imagineers
also reseachr the best of the esoteric audio world but not many.
Still I'd say that Disney Imagineers are generally more interested in
every and any device that could enhance sonic quality. But Imagineers
would not be much better as test listeners than any novice listener
that really had a broad interest in music, spoken word, natural
sounds or cinema.

So does the reference to "trained listeners" imply that there is some
other expertise these listeners need to have that would make them
better or different from the normal recording engineer's skill at
listening to specific spectra or perhaps goes beyond the trained
engineer's long years of experiencing the sound of real instruments
and voices? If so what is this quality that makes these "trained
listeners" better than professional audio engineers? I've never seen
it (or heard it). Even when I was selling esoteric audio at retail I
never met a "golden ear" who had anything over the trained
professional recording engineers I met later in my career.

Even the most highly "respected" golden earred listeners of
audiophile equipment usually only seemed to have their own specific
biases that was supported by their own little group of followers.
These "audiophiles" could usually argue well, and were superb at
justifying why the faults of the products they labeled superior were
less important than the faults of the products they labeled inferior.
Generally I met almost no audiophiles whose sonic judgement I would
trust more than many trained professional recording engineers. The
best of each group could be possibly very focused listeners, very
fussy listeners or there are even some I like to label "savant"
listeners. Some have big egos and some don't. But there is not one
trained listener I've met who would be the "best" listening test
subject. Some trained listeners can make faster determinations about
why one product sounds better than another compared to most people,
but the decisions they make are usually very much parallel to those
made by untrained listeners as long as the test is properly designed
to be fair and meaningful.

I've often discussed audio in the halls of CES with amp designers of
products like Krell, B&K, Bryston, Carver, Threshold, Conrad Johnson,
Quad and Lazarus and not only have these people been willing to lend
their amps to me for use in my displays, I've had many other
designers come into my displays to listen and discuss audio. Amp
designers listen allot because they want to be sure their products
don't have "bad reactions" to all the different loudspeakers on the
market. These are electronics engineers with thousands of hours of
focused listening. Perhaps they are self-trained but they are often
very skilled at hearing tiny differences between products. Again I've
noticed that even this group of trained listeners might hear quality
differences faster, but they don't hear them any better. So I don't
understand this emphasis on trained listeners being some kind of
especially capable listener who can hear things other listeners
can't. In fact it usually works the other way around .

Trained listeners who don't get a constant diet of nearfield
experience with live music often tend to be more "in the box" as
opposed to being open-minded. Sometimes it is the sad result of ego
forcing them to always choose a product whose "sound" is one they
recognize or have publicly stated is superior. If not for gigantic
egos unwilling to be openminded, there would likely only be 25 or 30
loudspeaker makers instead of hundreds. Also if it were a certainty
that any particular listener was better than others, the best
financed companies would hire them to compared that company's
products to everything else on the market. But this isn't the case no
matter how many "golden earred" listeners claim they hear better than
anyone else.

In fact normal listeners are just as good at determining which audio
product sounds better than others (if there are any differences to
hear). The 21 year old European novice recording engineer who
demonstrated this when he was able to "hear" 98% of the time when a
23khz filter was inserted into the AES program material was more like
an untrained listener. No other trained listeners could do it more
than 40% of the time. Remember this was a filter-in/filter-out test
with button pushed & released to match when the change in sonic
character occurred. Even more important to remember was this was a
test to determine whether people could hear a "difference" sonically
when there was a definite electrical difference. Maybe the engineer
"liked" the filtered program better than the unfiltered program, even
though the unfiltered material would measure better for
phase/squarewave response, noise, and the total amount of distortion
in the "audible" program (Thus explaining Quad fans).

ANY listener who is careful, focused and has some sonic reference
points remembered from hearing live voices and instruments can be a
good listening test subject. A good listening test particiapant is
something completely different from knowing allot about audio and
music in general. This is what seems to confuse most people and it is
something people with allot of general audio and music knowledge want
to deny because they want to believe they are special golden earred
listeners because they know allot about audio or music. A good test
listener has to be able to hear differences in sound only if they are
there and they have to be able to make judgements about which item
sounds better than the other if there really is a difference. It's
that simple.

Human beings are a group of interactive sensors and measuring
instruments, tied into a large central processor that not only makes
quality judgements based on direct measurement but also based on each
individual's history, temprement and education. Normally if there is
a difference in sonic quality it can be measured or quantified.
Sometimes measuring product A's 9 non-linearities vs product B's 9
different non-linearities makes it difficult to determine which
item's overall sonic quality is superior to the other's. Properly
arranged listening tests just narrow the focus of these quality
judgements so they can be more easily identified.

If the purpose of an audio playback system is to fool the ear/brain
combo into believing the listener has been transported to some space
where a live performance or sound exists, then there can be tests to
see how close or far away that goal is by testing one product in the
chain at a time. Let's be careful that we hear real quality
differences. Sometimes there can be two distoritons present, one
irritating and one masking. If a product removes the masking
distorion then perhaps the irritating distorion becomes more audible.
This is a real possibility. But usually differences are singular, and
other times listening tests point out which of two different
distortions is the most irritating. But training beyond an hour or
two to warm up test listeners doen't really make a difference in
whether anyone can hear sonic quality differences. Healthy ears,
focused minds, and a personal liking for the sounds they are hearing
usually makes the best listeners for tests. People who always have to
be right, those who have been too acoustically imprinted by previous
listening of certain program material or those who have a large
amount of image and ego invested in the tests' outcomes usually make
the worst listeneing subjects.

Now for some of my opinons about consumer magazines that have product
"tests". The electronic tests may be helpful or fun information
especially when features are described. The desire to please
potential advertisers make most magazines focus on saying something
nice about products sonically. The fact that most magazines will not
point out real sonic problems is a diservice to readers. No magazine
I've ever read does real blind listening tests to even choose their
best and most recommended products. About the best I can say is that
if a reader finds after comparing a few dozen product auditions that
any reviewer has done, with their own auditions of audio products and
then finds themselves always in agreement with that particular
reviewer, then reading that reviewer's articles about products could
save time. Otherwise I find most magazines are either driven by
advertising and thus could care less about consumers, or they are
driven by the need to suck up to or be courted by manufacturers to
get equipment and favors or special treatment at shows and thus could
care less about consumers.

A few of these product "reviewers" can be funny or edgy writers, and
reading their reviewes can be interesting. This doesn't mean they can
hear anything of any special importance that others can't hear but
maybe their writing is readable. But beware of the priorities of most
reviewers. I visited many magazine editors and test "facilities" in
both the audiophile and mid-fi markets. I've often asked about this
or that product to get opinions when I saw the product was at the
magazine for a review. I was often astounded to find that products
which sounded great and were very reliable could be panned while
other products that were less reliable or sounded worse would get
glowing reviews. When asking for specifics, I was told that the knob
on one device wasn't what the reviewer thought was right, or the
connector on another device was not up to snuff. This attitude had a
disasterous effect in some cases.

In the 60s a barrier strip was the normal way speakers were connected
to receivers and integrated amps. A spade lug or better yet a
circular lug could usually be mounted on a speaker wire and a very
solid connection could be made. Magazines later touted the use of
push connectors because consumers rated them more convenient. What a
terrible move. Push contact connectors had a much smaller conductive
surface and barrier strip connectors could have much more secure
connections. The switch to push/insert connectors probably hurt the
sound quality of systems as much as almost any other change in audio
up to the switching power supply fiasco. On the audiophile end, few
reviewers for high end publications have much knowledge of room
acoutics and acoustic treatments. Some of the listening rooms I saw
were abominable. I understand why there seemed to be no relationship
between the reviews I read in audiophile magazines and my actual
auditions of the same products. This kind of inconsistancy makes it
difficult or impossible for companies making really good products to
get consistantly good reviews and develop audience confidence (which
was why McIntosh originally decided not offer their products for
review in most cases when they were ascending the audio heap).

So while I am certain from experience that double (or triple or
quadruple etc.) blind listening tests can best determine #1) if there
are differences between products and #2) whether differences make one
product better than another, I don't think these tests should utilize
any particular level of trained listener, as long as the listeners
are interested, open-minded and careful. Comparing audio to cars or
politics seems a bit far afield, but a good comparison might be to
compare audio to 3D video imaging. It is still difficult to measure
3D animations to know which animation might look most realistic and
current animations fall short of reality in multiple ways.

But we still test computers, monitors and storage devices to be used
in 3D animation production or viewing. Images are getting better but
not all the program material is of the highest quality either
visually or thematically. Equipment is also getting better.
Eventually it may be possible to make 3D animation which can fool the
eye/brain combo the way we try to fool the ear/brain combo with
audio. Our eyes' ability to see immediately whether something is
animation or real on a 2 D screen is much like determining "reality"
with audio program and playback material. Once in a great while we
will see a piece of animation that looks authentically "real', and
I've had the chance to hear some materials that were able to create
"ear/brain fooling" sonic quality. It doesn't happen often and seems
to be more rare now in audio than it was in the 70s and 80s, while
animation is still moving forward towards "reality". Perhaps this is
because in 1973 dollars there is less money being spent on audio now
than then. Perhaps audio will become more important again when 3D
animation and video get so good that it is always easy to fool the
eye/brain combo. We can always hope. Watchking

We don't get enough sand in our glass

"watch king" wrote in message ...


Now a comment on the idea of the "trained listener" that is being
bandied about. What does this mean? In my experience professional
recording engineers are trained listeners.

I'll go with that. Of course, as a professional recording engineer I
probably have a vested interest in agreeing with that! Plus, as an
undergrad 26 years ago I took two semesters of Listening & Analysis,
so perhaps I'm even more of a "trained listener" than most? Hey, I got
an A...

They really aren't any
better at listening testing than any other listeners and in some
circumstances they can have an incredible bias that makes them
useless as test listeners. In the circumstance that a recording
engineer has played a piece of music he is intimately familiar with,
through his own studio monitors, his preferences afterwards if that
recording is ever played is worthless. [snip]
They would be poor listening test subjects.
It would be no different for engineers who listened mostly on UREI
Time Aligns(Altec 604s), or any number of other JBL, Tannoy, Fostex,
Yamaha, Westlake or other studio monitors. These listeners just have
such huge built in listening biases that it would be difficult for
them to be objective about the "total quality" of one audio product
VS another in a blind listening test unless their loudspeakers and
facilites were used. These loudspeakers and facilites might never be
able to demonstrate a variety of audio characteristics.


Sorry, I must disagree with you here. In the first place, our job as
professional recording engineers starts with the capturing of a
musical event. If we're not intimately familiar with what the actual
live performance of that event sounds like...if we haven't gotten our
lazy asses out of the control booth and stood out there in the studio
or concert hall with the musicians and heard the actual sound coming
out of their instruments...then we're not really "professionals".
Fortunately, many of us are, and do. Our listening biases are based on
an intimate familiarity with live musical performance; *then* and only
then do we train ourselves how that sound translates through the
monitor speakers of choice.

Moreover, it strikes me as pointless to subjectively test audio
components for "total quality" in a wide open scenario such as you
imply, where one's familarity with some other audio playback system
would be considered the benchmark against which any variables would be
compared. What sort of information (other than very gross
generalizations) would you glean from *any* test which simply asks
"Which sounds better?" If you're going to leave the testing that wide
open at the very least A) subjects should be required to elaborate on
WHY one component sounds "better" (an explanation which, btw, I
believe professional recording engineers are usually well qualified to
offer), and B) subjects should be comparing to the benchmark of live
music.

It will take me a while to get through the rest of your entire post,
but this early paragraph caught my eye (ear?) & I wanted to respond to
it while the notion was fresh.