[Leica] performance analysis (was MTF)

[imxputs <imxputs@knoware.nl>]

Alan wrote
        >it does seem weird to me that people would start up today, on a 
competitive
        >market, in serious testing (renting or purchasing of benchmarking 
hardware,
        >human resources needed to intervene at the various stages of the 
workflow,
        >logistics to be managed, etc) if they should not do it in a way that 
might help >them gain readers and
        marketshare...

        Well the most elaborate testing equipment lab is the owe used by 
PopPhoto. Still most Luggers would not
        even care to consider the results and Eric wrote recently that the 
PopPhoto results are questionable. Testing
        in my opinion has absolutely nothing to do with gaining readers and/or 
marketshare. Testing should be done
        to reveal the strong and weak points of a product. Reporting on these 
characteristics has to done in such a
        way that prospective users can form an inteligent opinion about this 
product and know how it will perform
        when used in their personal work and environment. That is why I refuse 
to use simple numbers or
        starratings and/or easy to remember stickers like 'super' or 'awesome' 
or 'bad'.
        One of the reasons of the continious debate about relative performance 
is just this: if a mgazine gives two
        lenses both 4 stars or both an A-, what in heavens sake do we know 
then. That is will be good performers
        relative to what is available on the market. But does it tell you 
which of both lenses is good for you. That is
        also why I always refer to companion lenses or predecessors to give 
some reference. A case in hand is the
        repeated question whch lens is better: Summilux-R 1,4/80 or Summilux-M 
1,4/75 or Elmarit 2.8/90. They
        all get 4 or 5 stars. Still no one knows how to evaluate this. My 
reports clearly state in normal photographic
        parlance what to expect from a lens and how it compares in detail with 
companion lenses. I could easliy
        say: get the MTF graph and look for yourself. That will not work.

        NBWatson referred to MTF testing as a kind of dynanometer that 
impresses car buyers in a non relevant
        way. Well as far as I know a dynanometer tests the kilowatt power of 
an engine, which in todays traffic is
        indeed not very relevant. If you were tuning an engine for maximum 
performance it is indispensable. An
        MTF graph for a designer is not the only tool. They use many other 
tools to evaluate a lens (spot diagrams
        for instance). The MTF graph is just a convenient shorthand 
description of a lens performance IF measured
        and interpreted competently. To denigrate MTF graphs as non-relevant 
for photographers is a statement I
        cannot support. It is the most important information you can get about 
a lens.
        AND it correlates very well with photographically relevant quality 
criteria. It is well known that you can
        prove everything with statistics and that statistics can lie. So it is 
with an MTF: it can prove hatever you
        wish and it also can lie about the real performance. I try to be as 
honest as I can and not to lie and yes I
        study MTF graphs in order to give them a meaningfull interpretation 
for readers of my reports. Without the
        bravery of Zeiss (yes Marc, you are right about Zeiss, they invented 
the SLR, they improved on the Leica
        III and they invented lens analysis as is stands now)we were much more 
ignorant about the real image
        quality of lenses. But is the credit of Leica to advance the state of 
the art two steps ahead, thanks to
        rigorous MTF checking. And I assume Mr Watson can appreciate that too.


        Erwin