LENS TESTS - DO THEY REALLY TELL US WHAT WE WANT TO KNOW?

 

A large number of test reports on cameras and lenses can be found in many camera magazines and on the internet, and it seems that in many cases these are more confusing than helpful.  Owners of Leica equipment often wonder why their supposedly superior equipment in some cases does not perform to their high expectations.  The reasons are diverse and difficult to explain.

As we all know, a modern photographic lens is an immensely complicated instrument, with a great number of performance characteristics.  For years, photographers have tried to find a valid method to determine the overall performance of a lens.  Unfortunately, it is almost as difficult to measure true lens performance as it is to design the lens in the first place.  The emphasis here is placed on “true” performance.

Many readers of magazines and the internet are misled to consider resolution as a valid measure in determining the quality of a lens.  The obvious argument is that a lens must be good if it is capable of high resolution.  Unfortunately, resolution is not a very good measure, especially if it is taken alone and at face value.  This is the very reason why some test reports also publish contrast figures.  The contrast level of a lens is at least as important for overall performance as the resolution.  As a matter of fact, in the past Leica has at times sacrificed overall resolution of lenses in order to gain a higher contrast level.  Contrast is a very difficult to explain criterion.  In simple terms, it can be referred to as the ability of a lens to reproduce high resolution at a recognizable level.  This is not the same as increasing the contrast of a photograph.  Lack of contrast of a lens cannot be corrected by any printing technique or digital manipulation.

Virtually all resolution tests are done with high contrast (black and white) targets which make it very easy to differentiate between minute details.  The results are expressed by lines per millimeter, l/mm, where each black or white line is counted or by line pairs per millimeter, lp/mm, where one black and one white line are counted as one line pair. Actual picture taking is different, however, because a lens is rarely subjected to the ideal conditions of a test target.  Instead the lens is confronted with multicolored scenes or subjects, all of which need to be distinguished.  Regardless of photographing in color or black and white, a photographic lens must (in most cases) reproduce a colored subject or scene.  A lesser lens might very well reproduce two almost identical shades of the same color as one color or the same shade of gray, and thus not be able to resolve this slight difference at all, while a better lens will be capable of making that distinction.  On the other hand, in a common lens test, when confronted with high contrast test targets, the lesser lens will inevitably show a higher resolution than it is capable of under everyday conditions.

 

This is a perfect example showing the importance of contrast over resolution.  When viewed from a normal distance of approximately two feet, the bottom picture will immediately look better.  Upon closer inspection one will notice, however, that the actual resolution of the top picture is substantially better.  This can be seen especially by the small test targets in the center.  The actual resolution of the bottom picture is only 50% of that on the top.  Yet the substantially higher contrast of the bottom picture immediately gives the impression of a better image, even higher resolution.  As I pointed out, a pure black and white target will show relatively high resolution, even with a low contrast lens.  Had the center target been separated only by different shades of gray and white, rather than black and white, the differences would not have been visible, in spite of the relatively high resolution as displayed by the top image.  This is a rather extreme example.  Differences from one lens to another rarely are as great as in this example.

 

The gradual disappearance of detail with low contrast subject matter

For that and a number of other reasons, lens manufacturers use a computer read out, the modulation transfer function (MTF), to better determine lens performance.  Rather than giving resolution figures, the MTF is based on a fixed resolution value, giving the contrast level of the lens over the entire image area at the various f/stops.

MTF functions also rely on a test target.  Like in the example above, the target shows vertical and horizontal lines.  This is done because most lenses can distinguish fine detail better in one direction than the opposite.  This is the reason why MTF functions always show two curves, one for the sagittal test patterns and another for the tangential or meridional one.  Either one shows the lens performance starting at the center of the image and going out to the edge of the image.

 

This is the MTF function of the Leica 50mm f/2 Apo-Summicron-M at f/2, f/2.8 and f/5.6

The four curve pairs show the level of contrast at 10, 20, 40 and 80 lines per millimeter resolution.

The curve at f/5.6 is an optics designer's dream, virtually flat across the top of the chart,

a performance level unrivaled by other manufacturers.

A competitor's MTF function for a similar lens.

Please note: The function is only for 10 and 30 Lines/mm.

Higher resolution functions would be noticeably worse.

Most published MTF curves are based on a rather low resolution of 10 and 30 lines per millimeter at the most.  The same “standard” is used by most magazines and digitally published tests for their resolution figures.  It is obvious that 30 l/mm is a criterion that can be met by virtually any lens short of the bottom of a pop bottle.  So it is no wonder that the majority of the lenses tested do achieve relatively good figures; especially in view of the fact that a 100% contrast rendering is unattainable.  But even with these relatively low criteria, the much more even performance level of the Leica lenses is still revealed.  Even at maximum aperture they usually perform to levels that many other lenses do not achieve unless stopped down, if at all.  If the contrast figures were based on a resolution of 60 l/mm or even higher, it might become apparent that there are substantially greater differences among various brands of lenses than we are usually led to believe.  I have often maintained that photographic magazines and websites which get the majority of their revenue from advertising, have to conduct themselves in a manner that is not detrimental to their advertisers.  Although these test are generally very good and unbiased, they are also conducted and presented via criteria which work to the advantage of the publication's subsidizers.

I think it is (or should be) obvious that optical companies other than Leica are all very much capable of making first rate equipment.  But their merchandise is also in most cases heavily mass produced.  The bench-made process, as used by Leica, gives them a good competitive edge.  But the superiority of Leica lenses especially shows itself in extreme situations.  This includes lens performance at maximum apertures and performance under adverse conditions.  Unfortunately, published tests do not always consider this and subsequently Leica lenses often seem to be less of an advantage than generally expected.

One might also notice in the written evaluations of Leica lenses that the writers often seem to be a bit strained to avoid the overuse of superlatives.  But we have seen comments like “...the lens against which all others have to be measured (400mm Telyt)...one of the best lenses ever tested (50mm Summicron)...the closest to our test standards ever (40mm Focotar).”

Measuring the surface of an aspherical lens element at Leica

In recent years aspherical lens elements have been used more and more in obtain better optical correction of lenses.  Leica has come a long way since they introduced the first commercially produced lens with aspherical elements back in 1966.  That was the original Noctiluc 50mm f/1.2.  Aspherical elements could only be manufactured with careful hand grinding and/or hand finishing, a very expensive preposition.

Today lens manufacturers use three different approaches.  The least expensive one is to make a standard lens element with spherical surfaces and then apply a thin, asperical plastic layer on top of it, usually made of a high quality acrylic.  This method was first developed by Zeiss.  They ultimately discarded this process because it could not match their high quality standards.  The problem was, and still is, that even the best acrylics contain rather huge molecules.  Light, when transmitting through such materials, will literally scatter and thus adversely influence lens performance.  The result is that only relatively inexpensive lenses are made with this process.

A noticeably better performance can be achieved by precision molding glass elements.  Here the prefinished glass elements are reheated to the point where they become pliable and are then precision molded into their final shape.  However not all photographic glasses lend itself to this process.  Especially glasses with a high refractive index and a low level of dispersion cannot be used for this process.  Subsequently such glasses, which are generally used to increase optical performance, must be replaced with glasses of less desirable properties.

All Leica lens elements are individually ground by precision grinding machines

The best and unfortunately most expensive method to make aspherical lens element is to grind them into their required shape.  Leitz has developed this process to a level unavailable to other manufacturers.  The result is that Leica today is the foremost manufacturer of high end aspherical lens elements.  The complexity of this process is shown by the fact that Leica tried to have some other lens manufacturers make some of their aspherical lens elements for them, to increase production.  The process requires such high precision that Leica often can only produce as few as 10 or less such elements in one day.  This did not work out because the lens manufacturers they contacted either were unable to make the lens elements with the required precision or they were unable to make them within the cost parameters required by Leica.

The grinding compounds for many lens elements is of an exotic nature that requires constant agitation to avoid deterioration  

So far we have talked much about optical performance, the sole criterion used in most lens tests.  The mechanical quality of a lens, however, is just as important.  Poor mechanical design and quality will eventually degrade even the best lens to mediocre performance.  Automobile magazines routinely run 50,000 or even 100,000 mile performance tests on automobiles and I have often hoped that something like this would be done with cameras and lenses as well.  The performance of a lens, when brand new, is one thing, especially if it is delivered for tests by the manufacturer after careful tweaking and adjusting to assure the best possible performance.  But what about its capabilities after it has gone through several months of use, after it has been knocked around in gadget bags, in cars and airplanes for an extended period of time?  Heavy use and the rigors of professional use take a great toll and only the best will remain on top.  This is where mechanical quality translates into optical performance and shortcuts in quality will show lower grade results immediately.

Price is the final criterion I should mention.  Nobody in the optical industry today is able to perform any miracles.  Although mass production does have a beneficial influence on the price of lenses and cameras, this advantage quickly fades when we deal with lenses of an exotic nature that are sold in a much smaller quantity and cannot be mass produced.  Suddenly we find lenses from competing companies to be equally expensive than their Leica counterparts.  Extreme wide angle lenses and long telephoto lenses, especially those that have proven to be close in performance to their Leica counterparts, usually sell for not much less than their equivalent of Leica lenses.  This is the case with most of the major lens manufacturers.  Yet when dealing with absolutes in performance, none of them have anything to offer that can compete on all levels with the newly designed Leica lenses featuring apochromatic design and aspherical lens elements.  The heart of any camera system is its line of lenses, and this is still one of the main reasons for choosing a Leica.

 For other articles on this blog scroll down in the column to the right to BLOG ARCHIVE

To comment or to read comments please scroll past the ads below.

All ads present items of interest to Leica owners.

___________________________________________________________________________

                                  www.classicconnection.com                                     

 www.leicastoremiami.com  

      www.eddycam.com        

      

Inventive Camera Bags for All of Your Gear

   http://www.tamarkin.com/

Buy vintage Leica cameras from 

America's premier Leica specialist 

www.tamarkin.com

                          

           http://www.tamarkinauctions.com/               http://www.tamarkin.com/leicagallery/upcoming-show

GIFT IDEAS FOR THE LEICA ENTHUSIAST

Click on image to enlarge

Order: info@gmpphoto.com

Please make payment via PayPal to GMP Photography

Click on image to enlarge

Order: info@gmpphoto.com

Please make payment via PayPal to GMP Photography

Click on image to enlarge

Order: info@gmpphoto.com

Please make payment via PayPal to GMP Photography