The Security Bug That Almost Shipped

Most engineering war stories I’ve heard in my career fall into a couple of recognizable shapes. There’s the outage story: something broke at 3 a.m., the team got paged, the on-call engineer figured it out in some dramatic way, the system was restored, lessons were learned. There’s the heroic-feature story: a team shipped something difficult on an aggressive timeline, somebody pulled a long week, the launch went well. The story I want to tell isn’t either of those. It’s the story of a vulnerability that almost shipped to production, was caught in the last days of a release, and quietly got fixed. There was no outage. There were no users affected. There was no public postmortem because the issue never became public. These stories almost never get written down, which is a shame, because the catch-late stories teach things the ship-late stories don’t. The almost-shipped vulnerabilities are the ones that, in aggregate, determine whether a security program is doing its job. Each individual one isn’t dramatic. The pattern of catching them is what matters. I’m going to walk through one of these stories, anonymized but with the engineering shape preserved, and try to draw out what I think is generalizable. The shape of the bug The bug, in the abstract: SQL injection in a search field. A legacy product in our portfolio assembled its search query by string concatenation — user types into the box, the input goes into the SQL string, the string goes to the database. The oldest vulnerability class in the book. The kind of thing every security training opens with. The reason it survived is the reason most legacy vulnerabilities survive: the search worked. It had worked for years. Nobody was in that code, because nobody needed to be. The release we were preparing was centered on a new product, and all the security attention — the scans, the reviews, the threat modeling — was pointed at the new product. The legacy product was in the release as an integration, and integrations inherit an assumption: it’s been running in production for years, so it must be fine. It had been built, shipped, and quietly running long before the current process existed. It almost shipped again. How it was caught This is the part that’s most worth examining, because the catch wasn’t the formal review process — it was a game. Just before the release, the office ran an internal competition: find security vulnerabilities, any product, any severity. Not a scheduled pen test, not a compliance gate. A competition, with the permission structure that implies — anyone could attack anything, including things they didn’t own and would never normally touch. A developer went after the legacy product. Not because it was assigned; because it was there, and because everyone else was crowded around the new product. They put a quote character in the search field, watched the error come back, and had a working injection shortly after. Every formal control had technically done its job. The new product had been scanned and reviewed thoroughly — and was clean. The legacy product hadn’t been scanned at all in that cycle, because it wasn’t the thing being released; it was just along for the ride. The unit tests on the legacy search tested that search returned results, which it did, faithfully, for any input including hostile ones. The catch didn’t come from a tool or a checklist. It came from creating a window where a developer had permission and incentive to point adversarial attention at something outside their lane. What the formal process missed I want to dwell on this, because the miss here is a different species from the usual one. SQL injection by concatenation is precisely what SAST tools are good at — it’s a syntactic pattern, and any modern scanner flags it reliably. The tooling didn’t fail to detect the bug. The tooling was never pointed at the code. The release process scoped security work to what was changing. New product: full scans, code review, the works. Legacy integration: assumed safe by longevity. That assumption is seductive because it’s usually cheap and usually right — until the release makes the legacy product reachable in new ways, at which point “it’s been fine for years” is a statement about the old exposure, not the new one. Every layer of the formal process was working as designed. The design just had a boundary, and the bug lived on the far side of it. Scope is the blind spot that doesn’t look like one: everything inside the scope is covered thoroughly, which produces exactly the same green dashboards as everything being covered. What the catch implied The first reaction after the catch — once the immediate fix was in — was to pull the thread. If the legacy search had this, what else did the legacy products have? Nobody had been looking at them; one finding said nothing about whether it was the only one. So the same in-depth security scanning that the new product had received was run across the legacy products too. That work is why the release slipped — not the fix itself, which was small, but the honest acknowledgment that we couldn’t ship until we knew whether the finding was an outlier or a sample. Pushing a release date to scan code that “hasn’t changed in years” is a hard sell; the working exploit from the competition is what made it sellable. The structural change came after: the process was updated so legacy and integrated products fall inside the security scope of a release, not outside it. The assumption that age implies safety got replaced with the assumption that unscanned means unknown. What I think the lesson is The lesson I keep coming back to from catches like this isn’t about the specific bug. SQL injection is a category that’s well-documented; OWASP describes it; every security training mentions it. The lesson is about the catch. The catch happened because somebody had permission to look at something that wasn’t theirs. Not assigned to that product. Not part of the formal scope for that release. Just pointed at the area by a game that said: attack anything. My honest read on why the competition worked where the process hadn’t: it wasn’t that the competition found what tools couldn’t. It’s that the competition had no scope. The process pointed everyone at the new product because that’s what was changing; the competition said “anything,” and a developer took that literally. Fresh eyes mattered, the game incentive mattered, but the decisive ingredient was permission — permission to attack, and permission to wander outside your lane without justifying it. Teams where that wandering is normal catch these things; teams where the scope document is a fence don’t. That kind of attention is hard to formalize — but not impossible, which is what the competition demonstrated. You can’t put “have engineers look at code that’s not theirs” on a process diagram, but you can create structured windows where that behavior is not just allowed but rewarded. The competition was exactly that: a mechanism for manufacturing out-of-lane adversarial attention on demand. The version that didn’t get caught For balance, I’ll mention the corollary. Not every almost-shipped bug gets caught. I’ve seen production incidents where, in retrospect, the bug was clearly there for months before it got noticed. The damage wasn’t always severe — sometimes the conditions for exploitation didn’t actually arise during the window — but the bug shipped, ran in production, and was only discovered later. The catch I’ve described is the visible part of the iceberg. The bugs that ship without being caught are the part underneath. We don’t know how many there are. We can’t measure what we don’t see. But the math suggests that for every catch like this one, there’s some number of similar bugs that didn’t get caught, and that ran in production for some period before either being discovered or being patched as a side effect of unrelated work. This is the uncomfortable part of security engineering. The wins are silent. The losses are sometimes silent too. The metrics that we report — open findings, mean time to fix, coverage — don’t fully capture either. What I take away The version of this story I’d tell new engineers: The formal process doesn’t cover everything. It covers its scope, thoroughly — and its scope has edges. The dashboards look identical whether everything is covered or only everything inside the boundary is. The thing that finds bugs outside the scope is human attention with permission to wander. Engineers who feel free to poke at products they don’t own, who can ask “should I look at this?” without needing a ticket to justify it. The teams that build that kind of attention into their culture ship fewer vulnerabilities. The teams where everyone stays in their lane are efficient for delivery and blind at the edges. Catches are easier to celebrate than near-misses. The almost-shipped bug doesn’t get a public postmortem; it’s just a small fix in a release branch and a slipped date nobody outside the team remembers. But the pattern of catches is what determines whether the security program is doing its job. The cost of building catch culture isn’t free. It requires engineers to spend time on things they weren’t directly assigned. The output looks like wasted effort if you measure narrowly. The teams that measure it broadly — by the absence of post-release vulnerabilities, by the rate of internal findings versus external findings — see the value. What the catch changed, concretely: the competition became a regular fixture — but moved well away from the release date. Running it in the last days of a release means every finding is a schedule decision, and that pressure pushes toward triaging findings by their impact on the date rather than their impact on users. Run early, the same competition produces the same findings with time to fix them properly. The mechanism was right; the timing was the bug. If I could go back, that’s the version I’d tell myself: manufacture adversarial attention deliberately, aim it at whatever the process considers out of scope, and do it early enough that the answer can’t threaten the ship date. The bug almost shipped. It didn’t. The reasons it didn’t were partly luck, partly process, partly culture. The lessons I’ve kept from the catch are about the third one. The first two are easier to influence. The third one is the one that pays back over time. The story isn’t dramatic. The fix was small. The catch came from a game. But the difference between the version where the bug shipped and the version where it didn’t was a developer with permission to attack a product that wasn’t theirs, in code the process had decided didn’t need looking at. That’s the system. The catches happen because humans get to pay attention to things they weren’t asked to. Build the kind of team where that’s normal — and build the windows where it’s rewarded — and the bugs don’t ship. Build the kind where everyone stays in their lane, and they do.