Of all the tools in the Lean Six Sigma toolbox, Gage R&R is one of the least sexy or at least it’s one of the more complicated sounding procedures. Just take the name; does this concept promote “Rest & Relaxation” for…gages?
And what’s with the odd spelling of “gauge”? Was someone trying to reduce the letter count for Twitter and decided to drop the “u”?
At some point a kind-hearted quality soul renamed it “Measurement Systems Analysis” to make it less geeky, but I think I nodded off while I was typing that phrase. It’s like they just strung three quality words together and called it a day. So whether it sounds formidably technical or confusingly bland, the tool does not command the attention it deserves.
So whether it sounds formidably technical or confusingly bland, the tool does not command the attention it deserves.
What if we could tie this method to American Football? That’s immediately going to be interesting to, well, mostly American men. And what if we could tie the concept to Tom Brady, the preternaturally handsome quarterback for the New England Patriots?
Well, now that might be of interest to women. Or at least it’s interesting to one woman – his wife the supermodel Gisele Bundchen.
And, finally, what if we drew the inevitable connection to the infamous “Deflategate” – the controversy that, albeit delayed, ultimately led to Tom Brady’s 4-game suspension from the NFL in 2016?
Now we’ve got this little-loved Lean Six Sigma technique tied to a modern football scandal that just won’t die.
Now we’ve got this little-loved Lean Six Sigma technique tied to a modern football scandal that just won’t die. It’s been a few years since the fateful game but it’s still relevant enough that I remember one of the presidential hopefuls declared to a roaring Massachusetts crowd, “Tom Brady was framed!” True or not, it got the crowd’s attention, and regardless of your stance on Tom’s innocence or guilt – now you’re dying to know the connection to Gage R&R, right?
Let’s break down exactly how this story relates to Gage R&R.
In order to understand the situation, I interviewed Molly Goodwin, a member of the USA Football Board of Directors and an owner of the Boston Renegades, a full-contact professional women’s football team.
Molly knows a lot more than me about football (which is not hard) and she tried to break down the issues for me. For those of you who somehow managed to miss the story, here’s the gist: The NFL alleged that the footballs used in the AFC Championship game between the Patriots and the Indianapolis Colts had been tampered with by the Patriots. In particular, the charge was that Tom Brady was somehow involved in having the balls under inflated so they were easier for him to throw. I’m probably doing the controversy a disservice, but you get the idea.
When I asked Molly about Deflategate she said, “There’s a report there were two or more air pressure gauges involved and they got different psi (pounds per square inch) measurements at different times during the game.” This is when my ears picked up. “Gauges?” or should I say, “Gages?” Do tell!
the idea is to ensure that when using a gauge, you are measuring things properly by testing for both “repeatability” and “reproducibility” of the measurements.
For those of you unfamiliar with Gage R&R, the idea is to ensure that when using a gauge, you are measuring things properly by testing for both “repeatability” and “reproducibility” of the measurements. Okay, now you’re nodding off so let me clarify.
What’s a Gage/Gauge?
A gage could be something with that name in the title such as a pressure gauge that measures pounds of pressure per square inch or it could be a thermometer that gauges whether or not you’ve got a fever. It could also be:
- a scale
- an odometer
- an altimeter
- a speedometer
…you get the picture?
Now let’s address repeatability. Let’s say my butcher uses a scale to determine that my ribeye steak is 16 ounces (1 pound). If I ask him to weigh it again and, once again, the scale reads exactly 16 ounces, then, based solely on my steak, his measurement system is repeatable.
Repeatability means that the same person gets the same result when measuring the same unit twice.
When people conduct actual Gage R&R tests, they use at least 10 units, and I have no objection to having 10 steaks but I’m just trying to clarify the topic. Repeatability means that the same person gets the same result when measuring the same unit twice.
Now let’s take the concept of reproducibility. Maybe when I asked him to reweigh the ribeye, my butcher had already moved on to constructing an 8-foot sandwich for a Super Bowl party.
Say he asks his daughter Sally to step in and reweigh the steak for me. If Sally puts my ribeye back on the scale and it reads 16 ounces again, then this measurement system is reproducible.
Again, this is just to clarify that reproducibility means that two different people get the same result when measuring the same unit. Of course it’s the day before the Super Bowl and these people are busy so let’s get back to Brady.
When I asked Molly to please elaborate on Deflategate, she explained that the balls were tested by the head referee before the game, and then tested again at half-time by alternate game officials, and the two groups got different results. Whether they used the same gauges or not, they lacked reproducibility!
So if we go back to the Deli – that would be akin to Sally determining that my steak was 14.5 ounces which is a bit shy of a pound. If I’m being charged for a pound, I might get concerned that I’m paying for more steak then I’m getting. Even if she used the same scale, what’s important is that she gets the right measurement.
The scales have to be properly calibrated.
“Calibration” is a word you hear a lot with Gage R&R. Since there is measurement equipment involved with continuous data, each gauge has to be true to a standard.
We’ll use the gauges in your car to illustrate this concept. You probably don’t doubt the accuracy of your odometer, and you might like to claim your speedometer is off if you get stopped for speeding, but you know it’s not. The one indicator that never seems properly calibrated is the gas gauge. The needle doesn’t seem to move for days after you fill up the tank, but as soon as it’s half-full it seems the gas level plummets and before you know it it’s on empty. Those early indications of an everlasting full tank might lull you into a false sense of security.
For the record, that is my official excuse for the time I ran out of gas – it was a calibration issue.
Another thing Molly mentioned is that temperature can have an influence on pressure. There’s a positive correlation between the temperature and pressure of a gas. But NFL guidelines don’t specify the temperature at which the measurements should be made. On January 18th, the day of the game, the outside temperature in Foxboro, Massachusetts was anywhere from 15 to 28 degrees Fahrenheit.
And what about games in San Diego or sunny Florida?
And what about games in San Diego or sunny Florida? Would all those balls test at a higher psi? In Lean Six Sigma, if a factor like temperature could change the measurement, then that would have to be part of the Operational Definition that details how to measure the unit.
When it come to sports, competition and the NFL, there’s a not a lot of grey area in the passions of fans and detractors. Brady has become a lightning rod so I think it’s wise for me to leave football in the hands of professionals. But I do know something about Gage R&R so I’ve listed some guidelines to help the NFL avoid this type of scandal in the future.
Guidelines for Using Gage R&R in the NFL
- The Gage:
- Make sure all the pressure gauges are properly calibrated
- The Operators:
- Choose people who are familiar with the process like the officiating crew
- Make sure they’re trained on how to use the pressure gauges
- Make sure they get instructions on how, when, where and at what temperature to measure the units (footballs)
- Parts or Units:
- Make sure there are at least 10 units (footballs) to test
- Randomize the units so the operators measure them “blind”
- Trials or Replicates:
- Have each operator test the pressure of each unit (football) twice
- Make sure the units are measured with the same ambient temperature (run all the tests in the same officials’ locker room)
- Measurements or Response:
- Record the psi measured for each unit (football), each time by each operator
- Enter the measurements into a statistical package (Minitab, SigmaXL etc.)
- Run Gage R&R to assess if the level of variation is acceptable (<= 10%)
Okay NFL – I hope you’re copying all this down. I’m here for you. I’m offering my services as a Lean Six Sigma Sideline Statistician. Please forward tickets to me for all key games – my husband and Molly would like to assist so maybe just send 4 tickets to cover all bases.
With Deflategate fading into the past, it could be time to look at some other measurements regarding the Super Bowl – ones that don’t involve PSI. Sports are great for statistics: Passing Yards, % Completed Throws, # Interceptions, Rushing Yards, etc. Those are all much easier to monitor and compare. For measuring yards you’ve got the Chain Crew and the signal poles. For measuring completions you’ve got the naked eye (and sometimes instant replay). So much easier than pressure gauges and thermometers. But this Sunday, we have two teams focused on just one statistic – Points won. Good luck!
So, who are you rooting for this Sunday? Let us know in the comments below!