Agile vs. Waterfall: From the front lines of technology transformation
Washington hired the talent to fix government technology. It neglected to rewire procurement.
America's federal government spends more than $100bn a year on information technology. By the Government Accountability Office's count, about 80% of it goes on running and maintaining what agencies already have. The rest is meant to buy something new.(1) Too often it buys busy work that doesn’t result in real change.
Crystal Philcox has watched this from several seats. She was chief of staff at the Internal Revenue Service when, by her count, the agency faced 32 congressional hearings in a single year and its commissioner, John Koskinen, faced an impeachment resolution.(2) She then ran operations at the Technology Transformation Service (TTS), a unit of the General Services Administration (GSA) stocked with recruits from the technology industry and led by Rob Cook, a veteran of Pixar. She now works at the Digital Services Coalition, a trade group for firms that build software in small, testable pieces.
Her diagnosis is that talent is not the problem. TTS had plenty. Her job there, she says, was to be the "government sherpa": telling clever newcomers what regulations meant, who to talk to about them, and how to understand dependencies within them. These are all real issues, but the the deeper obstacle, according to Crystal, is how projects are approved and paid for. Years ago, contractors sold agencies a "system development lifecycle", a fixed sequence of milestones. Milestone one is planning. Milestone two is more planning. Milestone three is a set of reports showing the planning was done. Working software arrives at the end, if at all, by which time the requirements have changed.
The alternative is easy to state. Milestone one: get the data right. Milestone two: ship something people can use. Milestone three: ship the next piece. Pockets of government already work this way. Spreading it means changing governance boards, investment reviews, budget cycles and the procurement templates that encode the old sequence. Those are dull documents, which is one reason reformers neglect them.
History adds friction. GSA was created under Harry Truman; the software industry as Silicon Valley knows it is perhaps 30 years old. Startups build on empty land. Agencies build on decades of law, contracts and systems that still pay pensions and process tax returns. The brief, noisy tenure of the Department of Government Efficiency (DOGE), which promised both speed and technology, has left career officials wary of anyone coming to sell either.
Ms Philcox's fix is unglamorous, yet effective: write iterative delivery into the paperwork that governs money. As a middle manager running waterfall projects, she and her colleagues used to daydream about being allowed simply to talk to developers. At TTS the daydream came true. Her task now is to make it routine.
-
GAO-23-106821: Agencies Need to Continue Addressing Critical Legacy Systems (more than $100bn a year on IT, about 80% on operations and maintenance)
HCD Measurement Guides
Your project hit milestone three on schedule. The report is approved and the status slide is green. The problem is that nobody can say for sure whether a single person's day got easier or more complicated. You know you’ve done the work; you don’t know the impact of it.
The HCD Measurement Guides show you how to tie a program's big-picture goal to indicators that matter, and down to data you can actually track. That means that a milestone means movement people can see and feel the effects of. Written by Ana Monroe, author of the federal HCD Guide Series commissioned by VA, OPM and GSA.
$25. Get the Guides
From the Masthead
One of the things I like about wicked problems (1) is that they are dependable. They dependably resist both definition and solution. Whether you take a technocratic approach or try to hack your way through, the chances are you will not solve them completely.
Why try, in that case? To public servants like Crystal, the answer is obvious: they can't not try. The potential benefits to the public are too big, and the maleffects of throwing up one's hands are contrary to their nature. The real question is not whether to start, but where.
Crystal's answer is specific. Milestone one is getting the data right. Milestone two is something people can use. Milestone three is iterating on milestone two. That sounds like a simple change in process from waterfall to agile, but it is a decision, and like any decision it effects how the solution impacts the problem. Where you start sets what you notice and what you build next. Start with planning documents and you will end up measuring planning documents. Start with working software and you will measure that software.
So the first thing to know about any wicked problem you take on is why you are starting where you are starting. The honest answer, for most of us, is that we start where our tools are. A full-stack engineer starts with the system. The lone expert deep inside an agency starts with the process she knows. I start with design thinking and historical research; my colleagues at Ishmael start with data analytics and project management.
That is a fine place to start. Nobody gets to approach a wicked problem with every tool. But a toolset is also a direction. It decides which parts of the problem look solvable and which drop out of view. An engineer will find the technical debt. A researcher will find the history. Neither will find what their tools can't see unless they ask what that is. LLMs are a case in point: they appear to have deep toolsets to work from, but the actual depth and rigor of their analysis and recommendations is anyone's guess.
That is the measurement I'd ask of you. Before the first step, write down why you chose it, what your tools can do and what they can't. Then check, at every cycle, whether the direction you're heading is the one the problem needs or just the one your tools lead to.
Wicked problems won't be solved absolutely. Modernizing a lumbering, old organization like a US federal agency will never be finished. But we can solve parts of it, and those solves count, as long as we know why we started where we did.
What we’re into this week
Ana
Talking to Crystal made me remember how hard it is to reconcile talent developed outside of the public sector vertical with talent developed inside of it. But our conversation reminded me of a Partnership for Public Service report called The Benefits, Challenges and Potential of Federal Public-Private Talent Exchanges about this very issue. It only talks about US methods for integrating parallel talent, but it’s a great read for anyone seeking to understand more about how to increase cross pollination in talent pools.
Aaron
That’s a good one to surface, Ana. We had really different paths to service, but developed a workable solution for tracking DX [eds. note: digital experience] because our skillsets complemented each other; they didn’t match. Which is the opposite of what’s happening right now, where software seems to be calling all the shots.
Scott
This is some real inside baseball talk, yall. Because I’m the one member of this team who has never been a career fed, I’m staying out of this one.
Ana
You were once a contractor, though.
Scott
A contractor-to-a-contractor. I didn’t like it.
Aaron
Probably a good instinct; the government does seem concerned about contractors.
Scott
😮💨I had not seen that.
Credits
Guest: Crystal Philcox | Host: Ana Monroe | words: A. Adams