LensRentals is a big rental house for video and photography gear and Roger Cicala and gang have a blog where they talk about lens testing and whatnot…
Today, at the end of the story about testing Sigma’s 14-24mm f/2.8 Art Series Lens, I read something astounding. Something that everyone is finding astounding.
During testing of Sigma’s lenses apparently some abnormalities in the results showed up, Roger Cicala talked to Sigma about it, and they asked him to ship the lenses back to them.
Addendum: Sigma’s Response
Our protocol is always the same: the intake people inspect and test lenses on test charts, fail anything that’s obviously bad and pass the rest on for testing. I test the first ten lenses they send me to write the blog post.This time a weird thing happened. On testing day 1, I tested the first eight copies of the Sigma. At 24mm five of them were identical and excellent. Three were not OK and was a big gap between the two groups. This is NOT normal. Generally, we see a cluster of lenses that are good, a lens that’s not quite that good, another not quite as good as that second lens, etc. In other words, the variance is always a steady trend away from best.
This is not something I’ve ever seen before in testing hundreds of lenses. I have a good working relationship with Sigma, so I emailed them as a courtesy, as I do with any manufacturer that corresponds with me. (They generally ask me to retest and want to argue about why my results aren’t valid. I generally retest and find my results are valid.)
Sigma didn’t do this. Instead, they asked for the serial numbers of the lenses. A few hours later they told me they had pulled the tests of those lenses at the end of the assembly, that the three lenses in question had tested much better than this at the factory, and asked me to send them to Japan for repeat testing.
Most of you don’t realize the enormous shock this caused me. I’ve been testing lenses for a decade and working with manufacturers for nearly that long. Leica has this kind of testing for every lens. Zeiss has it for certain lenses in their lineup. Nobody else does. Leica lenses. Some Zeiss lenses. Sigma Art lenses. Yes, I know all about other claims of ‘we test every lens’; that means it was mounted to a camera to program the AF firmware. Nobody else (with the possible exception of Canon with some of their newest lenses, and I’m not certain of that) is testing every copy optically and keeping the results on file.
Ten days later (shipping to Japan takes a while), Sigma emailed to tell me the lenses I sent back tested much worse than their original end-of assembly tests. Their new tests agreed with my findings and were, in their words ‘not acceptable for this lens.’ I’ve heard ‘this lens is within spec’ from manufacturers on lenses I knew were bad so often, for so long, that I reread the email just to be sure. To paraphrase a bit: “We have retested the lenses and agree with your findings. They aren’t good now. They were quite good when they left here. We’re investigating to find out what happened.”
Then they had their engineers disassemble the lenses and found some minor damage to rollers consistent with an impact that had caused the problem. I’ll be blunt, usually, I’d expect a ‘so you guys must have dropped them’ next, because these came from 3 different suppliers. And that is certainly possible. The lenses passed through several hands here on their way to me, and while very rare, things get dropped (and that people dropping don’t always admit they’ve dropped).
Instead, Sigma said they would track the shipment and see if the lenses were in the same crate or pallet during transit. Then they told me something that made me nearly weep with gratitude, showing that in a few companies engineering, not marketing, still runs the show:
Our engineers have also taken a number of new lenses from the assembly line, dropped them in various ways, and then disassembling them to look for damage. We did this until we had reproduced this type of damage. We will now see if there are changes we can make in the lenses or packaging to prevent this from happening in the future.
This is, in my opinion, what superb engineering and quality assurance are all about; look for any weaknesses and strengthen them. I’ve been working in this industry for many years, and this is the attitude I wish every company had, but few do.
Also, to be as certain as I can that the tests I’ve presented here are representative, I’ve tested a total of 25 copies now; the three damaged lenses were the only ones with any issues.
Read the entire article here: https://www.lensrentals.com/blog/2018/07/mtf-tests-for-the-sigma-14-24mm-f2-8-art-series-lens/
That is just crazy.
Let’s be honest, lens manufacturers make products that we love. But no manufacturing method is perfect and the reality is that if you’re one of the unlucky people to get a damaged or bad copy of a lens… wow are you screwed.
Customer service is hellish, they ALWAYS directly claim or imply it is your fault, and don’t actually seem to care about their products once they leave the factory… because if they genuinely cared they would be doing exactly what Sigma just did day in and day out.
It’s sad that people taking responsibility for their products, actively working to make things better, is so unusual in our world that it’s shocking.