The Hidden Cost of Manual Compliance Testing - And the 60% Time Reduction We Found

Three engineers, a stamp, and a quarter-million-dollar P&L. Until 2023, the group spent about 40% of its calendar on actual regulatory work, interpreting standards, talking to certification bodies, deciding whether a new product feature triggers new requirements. The other 60% was lab work. Settling test conditions, recording numbers, transcribing them into spreadsheets, generating PDF evidence packages, signing them, and mailing them. \ I worked with Karen’s group for nine months in 2023 to automate the lab work. The headline number, the one that goes in pitches and press releases, is that we cut compliance testing time by 60%. That number is real. What I want to write about is the part of compliance work that I didn’t see, didn’t expect, and that the 60% number doesn’t capture. The Visible Cost The first thing you notice when you sit in a compliance lab is that everybody is waiting. Power-draw measurements per IEC 62301 require 30-minute settling periods, then sometimes hour-long stable measurement windows. The engineer is staring at a meter that’s reading 0.42, 0.42, 0.43, 0.42, 0.42, with a clipboard. The compliance test takes maybe 90 minutes of measurement. The compliance engineer is in the room for 90 minutes. They are doing nothing for 75 of them. \ We had four product families, each tested in four regions (US, EU, Canada, India), each tested at two voltages, each requiring a minimum: standby, network-attached standby, idle, and on-mode measurements. That’s a lot of clipboards. \ The visible cost is hours. Karen’s team logged about 220 hours per certification cycle in 2021 and 2022. Two of her engineers were burning 50% of their calendar on this. The work didn’t need three people; it needed three people because it took that long manually. \ When we automated, the 220 hours dropped to 78. That’s the 60% reduction. It’s the number that gets used. The Invisible Cost What I didn’t see in 2021 was that the manual testing wasn’t just slow. It was creating second-order costs that nobody was attributing to compliance. \ Engineering bandwidth held hostage. Whenever a firmware change touched the power-management code, somebody had to assess “does this affect compliance?” and the answer required actually running the tests, which took a week of lab time. So firmware engineers held off on power-management changes. They batched them. They avoided them. The system optimization work was being deferred not because nobody had any ideas, but because compliance verification was so painful, the team didn’t want to incur it. \ When automated compliance landed, this changed almost immediately. Power-management changes started shipping faster because the verification cycle was hours, not weeks. Mike on firmware told me, casually, in a hallway about three months in, that he had “a backlog of about fourteen power optimizations” he’d been sitting on because he didn’t want to drag the compliance team in. He was shipping them now, one a week, and the standby-power numbers on our products were dropping with each release. We hadn’t planned this benefit. It just showed up. \ Late discovery of regressions. A compliance test that runs once per certification cycle (i.e., once every 6-12 months) catches regressions only at the end of the cycle. We had three separate incidents in 2021-2022 where a firmware regression had introduced a power-draw increase, the team didn’t know, and we had to roll back several months of work to ship a compliant product. Each rollback was about 2-3 person-weeks of lost work. \ When automated compliance ran in CI, regressions surfaced within a day. The lost-work figure dropped to roughly zero. We attributed maybe 6 person-weeks per year to “compliance regression rework” before automation. After: less than one. That’s another invisible benefit, and depending on how you account for it, it’s worth more than the direct lab-hour savings. \ The recruitment cost. Two engineers had told Karen, on separate occasions, that they were considering leaving because the lab work was repetitive and they wanted to do “real” engineering. We didn’t lose either of them, but the threat was real, and the conversations were real. \ After automation, both engineers shifted to higher-leverage work, interpreting new EU regulations, building relationships with certifying bodies, and doing failure-mode analysis for products that were on the boundary of compliance. They both told me, separately, that they were happier. One has been with the team for eight years; she’s now the lead. The other was considering leaving and is still with us. \ The vendor relationship. Our outside testing labs charge by the hour. When we sent a product for certification, they would re-run a portion of the tests we’d run internally. Our manual data, in the form of hand-typed spreadsheets, was treated by the lab as untrustworthy; they’d double-check almost everything, costing extra hours we paid for. \ After automation, our data was machine-generated, time-stamped, and in their preferred formats. They started accepting our results as the primary input and only running confirmation samples. Our lab fees dropped by about 40%. This was a number we never would have known to estimate beforehand. The 60% number, in context. The 60% reduction is a clean comparison: hours per certification cycle, before and after. It’s the only metric that’s easy to communicate. But the hours-saved figure leaves out everything I just described. \ If I add up the second-order effects: • Firmware optimization velocity: hard to put a single number on, but at least 14 optimizations shipped that wouldn’t have. Some of those translated into MEPS rating improvements that affected sales. • Regression rework prevented: ~5-6 person-weeks per year recovered. At engineering-loaded cost, that’s something in the $30-50K range. • Lab fees reduced: about $48K per year in saved external testing. • Engineer retention: not quantifiable, but neither engineer left, and replacement cost would have been substantial. \ The direct hours saved (220 → 78 = 142 hours × 4 cycles per year × loaded engineer cost) is roughly $80K/year. The second-order effects add up to at least $100K more. The full cost of the manual was double the visible number. Why this kind of cost is invisible. I think there’s a structural reason these costs are invisible until automation removes them. Manual compliance testing creates costs that show up in other teams’ budgets and other teams’ velocity. The compliance team sees the hours. The firmware team sees the deferred optimizations. The product team sees the rollbacks. Finance sees the lab fees. HR sees the retention conversations. No one team aggregates the full picture. \ When you propose automation, the business case is built on the visible cost, the hours. That’s typically enough to justify the work, so nobody goes looking for the rest. The hidden cost reveals itself only afterward, when the constraints disappear. \ This is, I think, a general pattern. Any sufficiently slow manual process creates a constellation of costs that get budgeted across other teams. The cost of “we can’t do X because Y is too slow” is an unspoken expectation that becomes part of how the company operates. Removing the slow process doesn’t just save time on the slow process; it removes the constraint that other teams were quietly paying. What I’d tell people who are considering automation. Two things, mostly aimed at engineers proposing this work to leadership. \ The hours-saved business case will be enough to get approved. Use it. But you’ll see other benefits later, and you should track them so the next person doesn’t have to relitigate the case from scratch. We track three numbers monthly now: lab hours, firmware power optimizations shipped, and external lab fees. The trend lines all moved in the right direction at roughly the same time. \ The thing that surprised me most was how much the team’s morale changed. Manual compliance testing isn’t physically hard. It’s mind-numbing. The engineers who do it know they’re doing rote work, and they know an automation could be replaced by a machine. They feel that. \ After automation, the team’s regular monthly retro went from “we’re tired” to “we found this weird thing in the EU 2024 amendment, what do you all think?” That conversation is the one Karen wanted to be having all along. The automation didn’t just save time; it gave her team back the work they’d signed up for. The 60% claim, again. If anyone asks: yes, 60% is real. Karen will tell you the same. We measured it the same way each time, the methodology is documented internally, and I’m confident in the number. \ But if I were writing the press release again, I’d say something different. I’d say: we automated compliance testing and discovered that the manual process had been silently constraining a half-dozen other workflows. The hours we saved were the smallest part of the win. The biggest part was that, after automation, we found out what our team could do when they weren’t watching a meter read 0.42, 0.42, 0.43. \ That’s not as snappy. The 60% number is going to keep getting used. Just remember that 60% is the floor, not the ceiling.