In short
Odd Lots Podcast Episode Summary
Episode Title
This Is What Happens When Governments Build Software
Episode Description The podcast explores the challenges faced by the public sector in developing and maintaining software systems. While frustrations about the government's ability to build infrastructure are well known, the episode highlights the often-overlooked difficulties experienced in software development, illustrated by past failures like the Obamacare website and pandemic-driven unemployment insurance systems. Guests Jennifer Pahlka and Dave Guarino discuss their experiences and insights related to public sector software challenges.
Key Guests
- Jennifer Pahlka: Founder and former executive director of Code for America, author of *Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better*.
- Dave Guarino: Formerly worked at the Department of Labor on upgrading the unemployment insurance system and has extensive experience with public sector software systems.
---
Key Themes and Discussions
- Challenges in Government Software Development
- Government software systems often face significant issues due to outdated technology and processes.
- Notable failures include the rollout of healthcare.gov and state unemployment insurance systems during the COVID-19 pandemic.
- The pandemic emphasized the fragility of these systems, as states struggled to handle an unprecedented surge in claims.
- Systemic Issues in Public Sector Software
- Waterfall Model: The traditional approach to government projects often relies on lengthy requirements gathering and rigid compliance standards, leading to inflexible solutions.
- Overlapping Policies: The complexity of policy and bureaucratic processes often hinders technological improvements and creates inefficiencies.
- Accountability and Capacity Challenges
- There is a pervasive culture in government where technology is viewed as secondary to policy-making, leading to a lack of empowerment for tech teams.
- Many government projects are hampered by rigid structures that discourage innovation and responsiveness to user needs.
- Hiring processes are lengthy, making it difficult to attract and retain skilled tech professionals.
- Learning from Failure
- During the discussion, examples of specific failures in IT systems were highlighted, including the Veterans Affairs healthcare application that failed to work outside the office due to outdated specifications.
- The importance of iterative learning in software development was emphasized, contrasting with the static nature of traditional government IT projects.
- Potential for Change
- Despite the challenges, there are signs of progress, including recent successful projects like COVIDtest.gov, which demonstrated that rapid, effective government software can be achieved.
- Encouraging a culture of continuous improvement and user-centered design in government software is essential for meaningful change.
---
Key Takeaways
- Incremental Improvement: Emphasizing small, continuous improvements in government software can lead to better outcomes than attempting to overhaul entire systems at once.
- Internal Capacity: Building internal capacity and capability within government agencies is crucial for effective software development and maintenance.
- Policy Reformation: Simpler and clearer policy frameworks are needed to allow for agile responses in software development, moving away from overly complex and bureaucratic requirements.
- Recognizing User Experience: Government software should prioritize user experience and be tested against real-world usage to identify and resolve issues proactively.
- Cross-Disciplinary Dialogue: Acknowledging the interplay between policy, technology, and user needs is essential for developing effective public sector software solutions.
---
Conclusion The episode provides in-depth insights into the complexities of software development within the public sector, emphasizing the importance of adopting modern practices, fostering internal capabilities, and simplifying policies to enhance user experiences and government efficiency. Jennifer Pahlka and Dave Guarino's experiences serve as a hopeful reminder that change is possible, even in entrenched systems.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Transcript
Automatic transcript. May contain errors.0:00You're being sold an AI future where you're obsolete or irrelevant. That vision is wrong. At Palantir, they're building AI that helps workers and unlocks their full potential. American workers are our nation's greatest strength. AI shouldn't eliminate them. It should elevate them. Palantir is here to tell their stories. From factories to hospitals, AI is freeing people from drudgery, letting them do what humans do best. Create. Solve. Build. Palantir, making Americans irreplaceable.
0:38So you're telling me that the AI that's meant to make everyone's job easier to manage just adds more to manage on top of the thousands of apps the IT department already manages. Funny how that works. Any business can add AI. IBM helps you scale and manage AI to change how you do business. Let's create smarter business, IBM.
1:15Hello, and welcome to another episode of the Odd Lots podcast. I'm Jill Weisenthal. And I'm Tracy Alloway. Tracy, you know what I thought was a really interesting thing in a recent debt ceiling episode that we did was about all the coding it would take if the Treasury were to introduce a new kind of a bill. Yes. That was interesting to me too, because I think there's a perception out there that if Treasury wakes up tomorrow and decides to sell a new type of bond, they could just do it. But actually, there are all these back-end changes that would need to be made. And as you and I know from multiple conversations at this point, it feels like government technology can be a little bit clunky sometimes.
1:57Is that a fair way of putting it? It is. I would say, you know, in defense, I mean, get into this. Yes. You know, corporate technology can be clunky. But that point reminded me that like we've done software is like a fascinating thing. And we talk a lot on the show, I think, about like the sort of built economy, like the Chips Act and battery investment and semiconductor and all these things are like, can the U.S. build things again? Like as a government, state capacity, these ideas. But in the 21st century, also like building software and tech has to be part of that question. Right. So it's one thing to announce that you're going to do this new thing.
2:34But almost everything that we do nowadays comes with some sort of software requirement. Like I'm sure there is a system for processing applications for chips funding and things like that. And someone has to actually build that. So there's two questions contained in the can America build things question. It's like, can it build the actual things? And then can it build the software systems that would allow it to build the things? And, you know, in the in the pandemic period, we got sort of smacked in the face, I would say, by this realization that the software component is not trivial at all. And most notably, it was like every state had some sort of problem with their unemployment insurance system.
3:13And there was just such a surge in claims, obviously, in those initial, you know, really for that first year, just on a scale that no one had ever seen. And it reminded people like, oh, yeah, 50 different state systems, much with like tech that was probably built by some contractor. And I don't know, the 80s or 90s and the people who knew how to run it have left. And, you know, it was a I get eventually I guess it got smoothed out. But it sort of exposed like, OK, why did that happen? And what else is lurking out there that we don't really know how it works until it's broken? I'm sure this is going to end up being a classic episode where we mentioned COBOL quite a few times.
3:48It's inevitable, right? It's coming. You can feel it. One day we're going to do an actual just like what is COBOL episode where we just focus on that directly. But yes, we're going to talk about software and we're going to talk about it from the public sector perspective. What does it mean when the government tries to fix something? What happens when the government tries to build something? What happens when the government tries to buy something? some of these themes. We talked about them from a private sector perspective with Patrick McKenzie earlier in the year, but more to do on this topic.
4:17Yeah, I'm excited. I have a lot. This is going to be a really good chance to actually ask the decisions that are going into some of these things being built. I'm still floored every time I log into the Treasury Direct website that you You have to click the little buttons, like the actual letters and numbers to type your password. It won't let you just type your password. How did that come to be? I have questions. That is a great question. So we have, I think, literally the two perfect guests to discuss this. Two guests with extensive experience answering and working on exactly these problems. We're going to be speaking with Jennifer Palka.
4:58She is the founder of Code for America. She helped found the U.S. Digital Service. She co-chaired the California Task Force on fixing its unemployment insurance system during the pandemic. And she is the author of a new book called Recoding America. And we're going to be speaking with Dave Guarino, who's a number of things. Currently an independent consultant researcher focused on this stuff. Founding engineer at Code for America helped create GetCal Fresh, which was like allowed Californians to easily access Snap. has recently worked on the unemployment insurance modernization at the Department of Labor, also worked in California.
5:38So two people with extensive experience on exactly these things. So Jennifer and David, thank you so much for coming on OddLots. Great. Thanks for having us. Absolutely. Yeah, great to be here. Now I want to ask, Tracy, can I steal your first question? Yeah, do it. How does it happen that a government website doesn't let you type in something and you have to like click buttons like that, like resemble a mini keyboard on a website. What happened? You see that. Do you have like in your mind, you're like, oh, I know exactly the meeting that caused this to happen. I think Dave has been in those meetings and has come out like with his hair on fire fuming.
6:12So I will let him answer that. I mean, not. Oh, well, I don't have. Yeah, I wasn't in that specific meeting. Maybe the best answer I can give is and there's deeper issues behind it, is that the way government tends to build technology. The reason for that is no one wrote down specifically as a requirement upfront that people should be able to use the keypad and the numbers on their keyboard to enter the digits. And so someone's like, well, I've met the requirements. Like the requirements are that someone can do this. They can enter it. And the way they do that is they click in this somewhat, I suppose, insane way.
6:51This is going to be a conversation where we're like banging our head on the desk a bunch, I feel like, as we listen to these answers. Anyway. Yeah, get ready. Okay, well, let me ask maybe a big picture question to start. But what is the process by which government software is developed? Because, you know, that answer just then, it makes it sound like someone gets a mandate. We need a website, for instance, through which people can buy government bonds, and then someone else goes off and I guess they execute it along the lines of the mandate that's been set. I mean, I'll jump in on that one. I mean, first of all, it is changing.
7:25So just before we get everyone sort of banging their heads on the table, there's a lot of movement in good directions. But what has been the process is exactly that. There's sort of this mandate. It often comes from legislation or regulation that very frequently now says there will be a website. They'll even put the URL in the legislation. And then it kicks off what's a really long process that does exactly what Dave was saying. It is first centered around requirements gathering. And that process of just requirements gathering can easily take 10 years. Now, when it's something like healthcare.gov, you know, and they have a launch date that's three years out, they don't take 10 years, but they still gather sort of every imaginable requirement and they kind of throw it in one giant bucket.
8:11And that's how the RFP is created, the request for proposal. That's what vendors bid on. And it's sort of like an undifferentiated, unprioritized mess of everything everyone could think of, plus things that have sort of pulled over from previous RFPs, plus a bunch of compliance stuff around security, compliance stuff around, well, various other things Dave can maybe chime in. But what that isn't is product management, right? There's no one saying, here's the important things this should do. And as Dave said, there's never a requirement for it to just work. So then the vendor, I mean, I'll skip several steps in the middle where, you know, it takes a long time to bid it out.
8:51And then there's like a protest because one vendor thinks the other vendor shouldn't have got it. And that delays it a couple more years. But then they go into development. And their job, as Dave said, is just like check all these boxes. And it's thousands. I mean, when we were at California, state of California, working on the unemployment insurance, they were actually about to put out a business system modernization RFP. They were actually, I think, about to award it to a vendor. And I think it had 6 ,700 requirements. We can check that. But that's really a normal number of requirements. And so they do all these things where you can check a box, but what they don't do is actually check that it works.
9:30So, I mean, maybe I'll just tell a quick story from the book. Sure, please. You know, there was famously this application for veterans' health care benefits at the VA that didn't work outside the building. And they couldn't see it. So basically somewhere in the specs it had that this form needed to work on a very specific and outdated combination of Internet Explorer and Adobe Reader. And the reason no one knew that it didn't work inside the building is that's how all the computers in there were set up. But if you were outside the building and had any other possible combination of those two pieces of software, it literally wouldn't load.
10:10And so they had very, very few people applying for these benefits online. And, you know, veterans were really, really, really frustrated. And it took a team going out there recording a veteran who had tried to do this dozens and dozens of times, bringing that video back and showing it to the deputy secretary for them to be able to say, oh, OK, actually, there is something to be fixed here. OK, you can go ahead and make a new form. But up until then, they said, sorry, it's fine. We're looking at the requirements. The requirements have been met. There's technically nothing wrong. That's amazing.
10:44It's also kind of crazy to think that people would have been looking at that and just been like, oh, well, I guess demand for veterans benefits is lower than we thought it would be. But actually, it was a tech issue. I think what they said was demand for them doing it online is low. These veterans must not have access, which is absolutely not true. Or they can't figure out the computer when, in fact, it was other people who couldn't figure out the software. They would tell them and they told this guy, Dominic, the veteran that they interviewed, they kept telling him it's user error. There's something wrong with you.
11:11Can I ask a question about that bidding process? When I hear government goes out to bidders for a website, I imagine these like big sort of faceless offices in Roslyn, Virginia, where, you know, people get like chopped salad for lunch and stuff like that. And there's like three or four of them and they sort of like rotate who wins the bid and stuff like that. How competitive is it? Would a typical RFP auction or bidding process be? And how hard would it be for, say, a more agile, innovative firm or someone wanting to do it differently to break into the pursuit of that contract? That's changing a lot.
11:50So now you've got a set of vendors. I think there's about 35 of them in this thing called the Digital Services Alliance, which are smaller companies that work in a more sort of agile, user-centered way. And they're just a lot smaller than the Beltway Bandit, so they're still a tiny portion of the market, but they're really viable businesses, and people in government can contract out to them. In fact, I think there's a mechanism by which you can put an RFP out just to those companies. But what has happened in the past is, and Dave can pile onto this, if you were bidding out, say, benefit system in a state, pick your benefit, Usually they would require in the RFP that the bidder had to have done a similar system in a different state in the same category, which means that right there it limited to like three vendors.
12:42And so you were never going to get any disruptors because they were boxed out in the very first step. Just to maybe jump in. That's exactly right. And if you I mean, to some extent, it's rational. If you are thinking about this as someone who's not a technologist, you run an agency, and you're thinking, we need to do this very large project. It's going to take a very long time. We really need it to go right. It is hard to think about, well, why wouldn't I look for a vendor who's done this 5, 10, 15 times? The problem is, as Jen said, well, I think there's two problems in that. One is you've now very, very drastically narrowed your vendor pool.
13:24So there's a lot less competition. And two, you also may have too big of a project. Like really, what level of distraction are you trying to work towards? And is it the case that really you need someone or a vendor who's worked on this specific type of system in another entity that works exactly the same way? And all of those details, the exact same program, the exact same structure of agencies. Probably not, because probably it's about more generically software. But if you really focus only on the track record there, you kind of get stuck, as Jen was saying, in this trap where there's only a small number of vendors who can meet those criteria, even if you think they're rational.
14:05And it's hard because the other thing I would say is as much as there are new vendors, and maybe Jenny can speak more to this, but it's also the case that if you have a new vendor that wants to work in a more agile way, wants to work in a more user-centered way, but they are bidding on an RFP that is structured in a very waterfall, top-down, meet all the requirements, and that's what success looks like, it's also not going to be successful. because you're not giving them any space to say, oh, well, I know your requirement says a way to enter phone numbers using the mouse. But when we tested it with people, like people turns out really don't like using the mouse to click numbers to enter digits.
14:48They prefer to use the keyboard. Could we change that requirement? And honestly, I mean, that's an extreme example, but a lot of, and there's a lot of what Jen's book is about and what resonated with me in it is that in doing these things and building software, you learn those details that you never could have gotten right up front. And so if you make success defined by implementing everything as we understand it before we ever get started, you almost ensure that you're not going to get what you really wanted because you're not allowing any information after that cutoff point. Right when you might be getting to the point where in doing it, you're going to learn a lot.
15:42Silicon Valley is selling you a future where you're obsolete, or worse, identical. At Palantir, they're witnessing something different and revolutionary, From re-industrializing the nation's defense base to shipyard workers building faster and frontline workers boosting productivity, AI is transforming work across the nation. AI is not replacing American workers or flattening them into conformity. It's unleashing what makes each one irreplaceable. Their judgment, their craft, their creativity. When American workers become more powerfully themselves, they own the future. Palantir, making Americans irreplaceable.
16:50Discover more at brightviewseniorliving.com. Equal housing opportunity. I definitely want to ask you more about that top-down waterfall nature of how a lot of these software projects get commissioned. But before I do, I'm just curious about the actual vendors because I'm sort of imagining these like big faceless offices. But what types of vendors are we talking about? And I'm trying to think how to phrase this kind of diplomatically. But I can imagine a company that is putting itself out bidding for government contracts on software development. Like maybe they're not as experimental or cutting edge as a software company that's, you know, somewhere in San Francisco and it's building like new exciting products for the corporate world.
17:40Although we've also done an episode on how bad internal corporate tech can be. But it seems like it might be a little bit more, I hesitate to say boring, but sort of basic, playing it safe. Well, you know, one thing that I will say, like, I hear both incredible frustration and even anger from folks that have to hire these companies. They're just like, there's no alternatives. This has gone badly. It will go badly. They kind of know that. Like, I had this woman in a major state, I won't say which, who, but I said, you know, your project's going to fail. She said, do you think we don't know that?
18:18The last seven projects have failed. So there's this, like, huge frustration. But there's also this real feeling that the company in San Francisco with its 40 people who make consumer software, like, great, people love using that software, but they don't understand us, right? They're not going to actually understand the constraints of government, so we have to go with these big companies. And they're not wrong in the sense that the constraints of working with government are real. There's a huge compliance burden. And that 40-person company in San Francisco mostly does not want these jobs. Like there is so much that goes into getting the bid.
19:02You know, it's sort of famously said that these companies know how to get the work, not how to deliver on the work. That's not probably what a 40-person company wants to do. But, you know, it's true that you have to deliver on stuff that is mind-bogglingly complex. Like when we're working in unemployment insurance again during the pandemic, my colleague was talking with the claims processors like week over week. and we're trying to like dissect it and figure out what's going wrong and like clear this backlog. And one of these guys keeps saying, well, I'm not quite sure about that answer. I'm the new guy.
19:43I'm the new guy. And she finally says, how long have you been here? And he says, I've been here 17 years. The guys who really know how this works have been here 25 years or more. So think about like, you know, going from doing some simple, cool, you know, tech app, you know, easy consumer app to trying to build or fix or improve upon a system that is so complex that it takes 25 years to learn how to process a claim. Yeah. That's, that's sort of, I think what needs to be on the table as part of this agenda is not just like, can the tech be better, but can we go back and simplify the accumulated like 90 years of policy and process that's making that so hard to make.
20:27Well, why don't we sort of back up then? And I, again, I'll kind of steal Tracy's question because I had the same thought. What is it about government buying that creates these massive waterfalled RFPs in which you can't, you know, in which A, like create these constraints that the software developer would not necessarily have with a private buyer, and B, puts government agencies in a position where they will say to you, of course, we know it's going to fail, which again, presumably, that actually probably does happen in the private sector too, but what are the, to some extent, but what are the dynamics that make these things so hard to change and so hard to come up with a new system?
21:11Well, this is a good Dave question, but I'm going to start. I really spent a lot of time reflecting on this. I've had sort of three years since I stepped down from Code for America where I just sat in a room and interviewed people and thought about it and thought about my own experiences. And I think that there's a deep-seated culture in government where the policy people are the important people. They do the important stuff. And technology, digital, is just part of implementation, which is not just the bottom of a software development waterfall. It's the bottom of a big, rigid hierarchy in which information and power and insights only flows from the top to the bottom.
21:52And so it's problematic in part because the people who are doing the tech are really just sort of downstream of everything else. And the power and ability and willingness to step up and say, hey, like, we probably shouldn't do those 6 ,700 requirements. We should probably focus on these 200, get that out the door, and then, you know, add edge cases as later. Like that's just not there's no permission really to say that. And until we if you compare that with, say, I will call it metaphysical Silicon Valley, because I don't mean actually Silicon Valley, like the people who I write code started the companies.
22:31They're in power. They're at the top. Like compliance is below them in sort of the D.C. government hierarchy. Compliance is like way above, right? Like policy is way above. And there's not this sort of build, measure, learn cycle that Dave was referring to where, you know, people are learning from each other as they're doing the work. And the technology, certainly the policy has an impact on the technology, but the building of the technology needs to have an impact on the policy. Like that fundamental culture is something that needs to be looked at and called out. And where it's not happening, where there's actual conversation, dialogue beats directives, right?
23:18Where that's happening, we should be lifting that up and saying this is possible within government because that's like the foundation, I think, of getting away from these mega projects that fail. but Dave you you are you're going to disagree with me on this and I'm going to love it I think that's I actually do think that's right I think a big part of it flows downstream from budgeting meaning a lot of this the way that and again everything that I think I just like there's some degree of generalization here and there's always outliers and there's always exceptions and of course there's also a lot of people trying to change these things that said maybe the the dominant mode is we're going to budget for a large project.
23:58We get one-time funding to spend to do that project. So we got to get everything right. We're only going to get this chance once every so often. We're going to replace everything. We're going to have this new system. It's going to be great. And the way that this paradigm breaks down is in what's called a design development phase, where you build the system, you design what it should do, you think about that. Again, you're kind of making all these decisions up front, which I think is very counter to a lot of the, if you want to call it the Silicon Valley approach, but then you're building the system and then you enter what's called maintenance and operation.
24:32And it's supposed to be very, very little. You're not going to change much. It's just kind of keeping the system going. And the problem is modern software, you kind of learn the most the second you've actually deployed to real users. That is the point when you are getting people saying, oh, this doesn't work or you're getting an error or you're getting, hey, I tried to put in this address, but I have a funky address and it won't let me do that. So all of a sudden you have all this information. And unfortunately, in sort of what gets called a big bang launch project, where you just put it up and you're done and you're not going to touch it again.
25:05You now almost immediately on day one can see potentially all of these issues. And a lot of people prepare for this in these kinds of projects. They're like the first month is going to be really hard. You all of a sudden know all the things that were assumptions you made that you got wrong. But the model that you had that you were given is, well, we're going to do this one time. Now, contrast that with actually going live is the first step. And you're going to have funding that's continuous and kind of flat over 10, over 20, over 30 years. And you're going to have those people on staff. And you're going to continuously change the system as you learn more about how users interact with it.
25:41As you learn more about like what things you didn't get right up front. Did you you made it possible to upload a certain type of document you you accept PDFs and TIFF images T I F F is a real example. But turns out you don't support JPEGs or the I think iPhone proprietary default image format. So now all your mobile users are saying, hey, we can't upload or a bunch of them are saying I can't upload documents. It says I can't submit this file type right now a lot of, you know, well intentioned folks in government get to that situation and then they're stuck because they don't have the mandate or the funding to say, now we're going to change that.
26:18They're waiting for the next big project to replace the system and now support those new things. Whereas instead, if you have a model where you're saying, we expect to continuously change this, it's a very, very different thing. And that's a little bit, that is more how Silicon Valley operates. I mean, this may be a sort of hyperbole, but Google didn't build search and then lay off 90 % of their staff and say, we're done. This is great. You know, like, look, we got the search thing. And I think that difference flowing downstream from how tech is budgeted for where you're budgeting for a one time project.
Read the full transcript
26:50And this comes from legislatures, you know, from Congress, a one time project versus we want to have ongoing funding, because this is a living, breathing system is a huge paradigm shift and has an uncomfortable aspect, which is it may seem like it's costing more money because it's potentially dollars ongoing because it's staff, not one time by. But I think a lot of the dysfunction that we see probably flows down from that root cause. Okay. So here's a big question. If we can identify the cause of a lot of this dysfunction, and if we can trace it back to this waterfall structure, the sort of top-down mandates, or maybe like theorists versus practitioners.
27:31So you have a politician who has a big idea and they want that big bang reveal of the software. They're going to expand benefits and it's going to come with a really cool website and everyone's going to be able to access them easily. And yet it leads to these problems that we've been discussing. Why do we keep doing it? That's a hard question. Well, what is the system that is like keeping this way of doing things in place versus maybe, you know, after a few decades of developing these types of systems, why isn't someone going, well, actually, the way we've been doing things hasn't really been working?
28:08Well, I think a lot of people have been saying that. And that's why I say things are changing, like and I will offer as evidence, COVID test.gov. Do you remember that one? Beginning of 2021, the president announced that there would be this site where you could request tests. and I think he gave a date that was, I want to say, six weeks out. It launched a date early. It launched in multiple languages. It took 11 seconds for me to use. Other people say eight seconds, 15 seconds, right? And your test showed up a couple days later. Like that sounds like it could have been really like, okay, so it's much simpler than say signing up for healthcare.gov or whatever, but like they could have made it way more complex.
28:51They could have said, like, let's gather every possible requirement and try to fulfill every possible requirement. But you had internal capacity. Like to Dave's point, there are people in-house who were saying, hey, we have been doing it a way that hasn't been working. Let's try this way. And it's great. Like it serves as a fantastic example, I think, within government and outside government, right? So part of it is how people in government think, what they believe. And part of it is what the public thinks, right? The public couldn't imagine something that easy, but we have to start imagining it and holding our government accountable to doing that.
29:30So, I mean, I will say it is changing, but the belief that we aren't good at it, that we couldn't ever do, like a covidtest.gov, drives politicians, pendants, vendors to say government's bad at this and so we should outsource everything. Now, of course, we're going to have vendors and we should have vendors, but there has to be enough internal capacity and competency to outsource well and to make those decisions like, hey, let's have this thing only ask name, address, and not ask for their health insurance numbers and not do different numbers of tests per household. Like, that's an internal capacity question that is really important.
30:18And we don't have enough internal capacity for these kinds of decisions in government because we believe we're not good at it. And we believe the only people who are good at it are these outside companies. Plenty of evidence that that's totally wrong.
31:10We'll see you next time. September 31st, 2025, and save. Discover more at brightviewseniorliving.com. Equal housing opportunity. Hey, Ryan Reynolds here for Mint Mobile. You know, one of the perks about having four kids that you know about is actually getting a direct line to the big man up north. And this year, he wants you to know the best gift that you can give someone is the gift of Mint Mobile's unlimited wireless for$15 a month. Now, you don't even need to wrap it. Give it a try at mintmobile.com slash switch.
31:50I want to go back to internal capacity in a minute. But since you mentioned the COVID test website, it's sort of a good chance to pivot a little bit here. You know, COVID specifically seemed to open people's minds a little bit about how fast we can work and how much money we can spend. And obviously, a historically fast pace of vaccine development that we've never seen prior to that. And so it sort of had this brief moment where it sort of opened people's minds. I'm not sure if it's closing again. But, you know, since both of you worked on the California unemployment insurance system and generally like I think almost every state on some level is seen by some debacle, like what was, I guess, the biggest eye opening thing that going into that environment taught you or about the system?
32:35And then what was like, you know, what's the key sort of takeaway of like, OK, here's the thing that we can learn from this that then whether it's UI in other states or more broadly, OK, this is something that you learned that then can like have extensible lessons. So I think you're right. COVID had sort of twin impacts on our thinking. One was, holy cow, we can actually do stuff. And the other one was, holy cow, we're really screwed. Yeah, it's a good way to put it. And there's evidence of both, which is really fair. I mean, I think for me, I'd been doing this for 10 years when I got pulled into the unemployment insurance.
33:17I call it a rescue. We were clearing the backlog. And I felt like nothing here can scare me. I have seen the VA. I've seen the operating of some pretty janky government agencies. But I actually really did learn a lot and was kind of shocked at how the unemployment insurance system worked. And the metaphor that came to mind for me was like, everyone kept saying, well, it's a system. Obviously, you can just fix it. And it isn't a system. And I think that's true of other things we think of as systems too. Like it is archaeological layers of technology that sort of map to archaeological layers of policy that didn't ever get designed.
34:05It just accrued. So nobody ever goes back and says, okay, how is this going to work with this? There's such little appetite for anything that's like backward looking that would actually update and make these layers work together well, that it just isn't what people think it is. When you go look at it, you're just like, whoa, of course you can't do these things. Like this thing isn't connected in any meaningful way to this thing. And like this was built in a certain era for a certain purpose. And we just glommed stuff onto it when we needed to say add internet access, like let people apply online.
34:43I mean, the biggest thing was like, I think we haven't grappled with the fact that when these systems were quote unquote designed, I actually would say when they were started because they really have never been designed in a lot of them. And there's huge advantage to actually doing some design. But you went into an office and showed who you were. Like you identified, you validated your identity by going in. And then we moved online and we never figured out how to validate people's identity. And there were a number of problems in California with unemployment insurance, the fact that it took 25 years to learn how to do it meant that any claim that couldn't be processed automatically was a huge, huge bottleneck.
35:28And you couldn't add claims processors to do that. You would have to go back in time 25 years, start new claims processors, and have them ready to come online during a downturn. That's never going to scale. But the other big problem was that they only went through the automatic sort of pipeline if they felt like they could verify your identity. But they weren't really doing any meaningful identity verification. And we, unlike most other industrialized countries, don't have a national system for that. Like, we don't have a national technology platform for that. But also, like, again, back to this all derives from, like, the legal and policy framework.
36:12We don't actually have a reasonable policy framework for identifying people and knowing who they are when we start to do a transaction. And that is causing problems everywhere. And there is interesting conversations about how to fix it. There's some stuff out there. But it's really contested. And it's not just a problem for the pandemic. It's going to continue to be a problem. Yeah, I mean, I'll just add, and my vantage point was somewhat different, right? I was coming at it from a different angle. I think the major thing that I took away, especially with so much focus on technology and so much focus on COBOL mainframes, and that was, you know, that was the dialogue.
36:56We need to do a COBOL, like, drinking game or something. Take a shot. We need to time how long it took us to get to that word. I think it's been about 30. Oh, I can't believe, you know, and the best part is, I mean, my next point was that's misplaced causation. And yet I am the one who brought up COBOL. Yeah, I mean, I think that is a, you know, for so much focus on, oh, these tech systems are falling over, they're not good. I think there's the number one lesson I learned was that, and I think this is true, but we don't tend to think about it, Which is the technology systems are part of a larger socio-technical system.
37:33What I mean by that is, as Jen mentioned, you have staff who can do stuff manually and you have a system that might be able to do stuff in an automated way. One of the problems with programs that are so complex is if you have a massive spike of volume and your technology system isn't set up to just cover all those cases in an automated way, you try to throw bodies at it and you try to hire people. But if it takes a year and a half to train someone and like train someone to a relatively basic level with some of the complexity of these programs, you're not able to hire fast enough. And unfortunately, you know, this is why.
38:11Can I just step in? We were hiring really fast. We hired 5 ,000 people in California to help process these claims. But because they couldn't do anything, they were taking up the time of the experienced claims processors. And we calculated that every single new hire the state made slowed processing down. Oh, man. And one of the big things we did was actually just get them to reassign staff away. I mean, some of it was just reassign staff to other things. Like nobody was opening the mail, like, which was pretty critical. And yet opening the mail is something you can do if you just joined and have no background.
38:51But like, yeah, it was it was pretty frightening. So sorry to interrupt, Dave. I just like we were hiring. It's just that the hiring was having a perverse effect. Well, and that's I guess that's my point is that's exactly right. And I guess the number one lesson I learned was if you look at the history of a program like unemployment insurance, but specifically UI, you have this thing where once every 10 years or so, there's a major recession. Or in this case, there's a pandemic, which was like a 10x recession, right? Because it was overnight as opposed to gradual. It was overnight. So much of the economy, so many of the working force, labor force were just out of work immediately.
39:29So all those claims came in simultaneously. But we have this cycle of every 10 years or so we have a recession or I mean, I don't know. I'm not an economist. I couldn't tell you exactly when it is. But it happens in a recurring way. And then everyone gets frustrated with how difficult it is to get unemployment insurance because they're overloaded volume. And then the claims volume goes down, recession ends. And then there's a lot there honestly isn't a lot of focus on, okay, we're gonna sort of put more money into the UI system so that we're ready for next time. but unfortunately you can't have resilient systems if you optimize for pure pure pure efficiency like in the down years in the years when you have very very low volume if you are just going to cut staff down to very very bare bones skeleton crew it comes back to this point of not only are you not able to do preparation if you don't have the funding to have staff you can't even make the changes that would be good and you can't when you get the money in a new recession hire the people because you can't train them.
40:28And so we kind of get stuck in the cycle. So that was the number one from a systems modeling perspective lesson I took away from it. I got to add, though, you know, we did invest several billion across states during the Great Recession or after the Great Recession. So I think about half of the states technically modernized then. They did take advantage of pretty significant funds. None of those states did any better than average in this downturn. So I agree with Dave, and I would argue that taking a different approach to quote unquote modernization, to policy and process simplification, to having goal-driven modernization, all of those things is equally important as investing during the downturn.
41:17Investing the way we have been doing it is not working. There are so many different threads to pick out of there. But just on that last point, you know, when you were talking about how a lot of these government systems were not designed to be that way, I kept thinking of a lot of the software used by the banks where, you know, you think you think of like JP Morgan software system. It wasn't designed to be JP Morgan software system. It's the result of many, many acquisitions of other banks. And, you know, every time they take over this business, it gets sort of duct taped on to the rest of the system.
41:52But I'm curious, is there a moment that you have seen in your careers at which someone says this system is just like irredeemable? We're going to have to start from scratch. And like, what is the trigger point for making that decision versus reaching for the band-aids and the duct tape and just trying to make it work? I mean, and I forget if someone else has mentioned this. I listened to the Patrick McKenzie episode. Maybe he mentioned this. But in general, a good rule of thumb is never rewrite a system from scratch that's working. I mean, other people might disagree with that. But the reason I don't have a good answer for sort of a moment.
42:34Lots of people have tried to do that thing where they just rebuild it from scratch. And again, it tends to go poorly because, and this is the specific thing, the old system encodes so much tacit knowledge and so many of the many, many, many little details. And some of those details might no longer be relevant. but many of them might encode some really hard learned truth about the world because at the end of the day the software is just modeling the real world it's just saying okay well we have customers they have addresses okay so maybe you have two address fields in your old system and you think oh well let's just streamline that down to one but then you launch your new system has one address and everybody calls you up like what are you doing like we the reason we have two addresses is because our billing department is over here and the customer facing address is different.
43:23So I think the throw it away and start from scratch is an impulse everybody has, but it has a lot of risk as well. And I think the better approach tends to be, can you get to a place where you can start to incrementally change your systems as you go in small ways that are safe, where it doesn't take it, you know, It's not like we have to test for six weeks any change before making it live. We can ship weekly. We can ship daily. We can ship multiple times a day. That tends to yield better things. There are plenty of projects out there that we're going to build this from scratch, from the bottom up.
44:08Most of the ones I'm aware of don't have great launch stories. Maybe Jen disagrees. That's my... No, they're still in that mode of like, get it all in once and then we're done and we have sort of maintenance. And so they're pretty bad. But I will say two things. One, on a practical level, just because I was talking with a colleague on the way over here about this. There are great examples now of what Dave's talking about, like not waiting for this big bang project, but saying we are just going to be making this a little bit better every day. And one of them is New Jersey's Department of Labor.
44:43Their unemployment insurance response during the pandemic was quite good, but they've also learned from that and said, we're not going to wait for some big system to come that's going to fix all of our problems. We have a team now that can just incrementally fix things as we go. And eventually that will involve some bigger investments as they learn X, Y, and Z, but they're just not waiting. And it's just a great, great example. But I will say, Tracy, like I think about what you're asking about all the time. I don't think this will ever happen. But if you do get a chance to sort of zero base it and start over, you have to zero base the policy.
45:28Like you can't – could you do a new unemployment insurance system in California that encodes the 25 years of knowledge that those claims processors have? Well, I mean maybe AI will help you now. I don't want to talk about AI. But like that's a problem. Like I say in the book, there's this great team that's working on this Medicare problem. And it's like there's nine different definitions of even like what a medical group is. Like even the first question doctors are supposed to answer is like wildly complex. And the policy, the delivery team is sort of fighting with the policy team. And she's like, I get that it's complex.
46:08It has to make sense to a person. And I like fight, fight, fight, not over the tech, but out of like over collapsing these nine definitions into two. They want to get to one. They can't. At least they get to two. And like this idea that it has to make sense to a person, we've like lost track of that. I mean, of course you're frustrated when you apply for unemployment insurance. Like, it's wildly complex in its policy and the process that occurred from those policies. Like, if you were going to start over, you can't start over with the tech. You would have to say, okay, let's figure out, let's, like, look at this 90 years of accumulated policy and say, what was this actually trying to do?
46:50Can we kind of, you know, either go back and just, like, rationalize all the changes, right? It's not like, like I say, there's not one binder of regulations that covers UI. There's 90 years of memos that changed the previous memo, that changed the previous thing, that changed the previous thing. And you could at least go back and like condense that and like figure out all of the conflicts in it. But it would make actually, I think, a lot more sense. And I will be clear, this is never going to happen. if you just said, let's design the policy for an unemployment insurance system that makes sense in 2023.
47:28And then you could build tech for that. There's so many different threads and places we could go. There's such a fascinating conversation. I just have one more question. And it sort of relates to something, Jennifer, that you said earlier about like, in government, the tech people are like, so secondary, likely to like the policy walks and people designing these things and that that's a really big difference between government, public and private sector. And, you know, this internal capacity, how much does that hinder? And I think this is another thing that Patrick McKenzie brought up, but like hiring.
48:00And so, and the challenge of like, you know, public sector pay scales are not private sector pay scales. I have to imagine for a lot of these things he talked about a little bit, you know, and so, you know, what you've worked on some of these like sort of volunteer things that sort of bridge the private and public sector. But can you talk a little bit about that constraint of like, even if like just the challenge of in those in those roles, getting like experienced, talented people? I'm glad you brought that up because it's a little bit of a soapbox I like to get on because we have, you know, like politicians, like the leaders who are so frustrated with delivery in technology and like they're generally they're well-intentioned, but the sort of things they push on are kind of unhelpful.
48:44You know, they kind of increase the risk aversion of the bureaucracy by yelling them about things and not recognizing the constraints on the bureaucracy. I call this the accountability trap. But there are things they actually could do. And fixing the civil service and civil service hiring rules is really, really important. I do know what Patrick McKenzie said on your show about pay scales is not totally, I don't totally agree with that. The biggest problem is not the pay, it's the time to hire. So if it takes, as is really common right now in federal government, nine months to make an offer, of course you lose everybody.
49:25You've got actually, you know, when we and I started this work, there was not like a line of people out the door with great tech and design skills wanting to work from government. Now there actually is, and we can't get them in the door. I mean, I know people who do wait nine months for that offer because they really want to help. And the pay is fine. Like, it's not fantastic. I think he was comparing to like Google, whatever. But like, you know, compared to a startup where, yes, you have the hope of getting an exit, but you're probably not going to make like$500 ,000, right, as a programmer, you're going to make, what, half of that?
50:02I mean, we don't have to get into like exact numbers, but like the pay is in the right ballpark for like mid-level folks. It's not going to pay at the executive level. But as he mentioned, like at the executive level, if you've been at Google for 20 years, pay is not your biggest concern. I do want to push back a little bit on the notion that that is who is in government. There are some of those people, but there are far more people who just have good tech and design skills who just want to make a difference more than anything else. And we should be working on all of the constraints that are keeping that hiring process at nine months more than we're like calling up the people who, you know, manage the Department of Labor in a state and yelling at them because they had these failures.
50:48Like fix the environment in which they're working, give them that capacity, then you can hold them accountable. But they can't hire people right now. you also you need to be able to hire people and you need them to be able to use their judgment to make higher order decisions than just do we enter the phone number with the keyboard or by clicking on a virtual pad of numbers like it is also the case that some of these systems are the way they are because it's just jen this might be your phrase uh it's like policy vomit it's just the rules put onto a screen, just like all the rules in statute put onto a screen.
51:30And so I think the thing I'd like to add is the whole point of product management is to arbitrate the trade-offs between user experience, compliance goals, technical cost of change, and technical trade-offs. And if you don't give those people, if you hire them, but they don't have the ability to say, well, maybe we need to simplify this to make it make sense to a person. If instead, the only response they get is, well, we need to put on the page exactly what's in statute, then or whatever other compliance goals there are like, oh, we need to strictly follow this process. It's procedurally what it is.
52:13If those are counter to the outcome, you need a change in government where outcomes start to be outcomes in this area around technology start to sort of have a higher importance than the strict process and the strict procedures. And I do think that that's a necessary corollary to bringing smart, capable people in. Because if you bring in a superb product manager and they have no ability to sort of bring these trade-offs up and make those calls on the ground, you're going to get the same status quo, I think. So I can't resist asking this question. I take all the points about like, here's what we need to do in order for this to change.
52:56But can I just ask, because both of you have that practical experience of doing this, what was the craziest thing that you saw during your respective tenors? Like, what was something that just really surprised you or shocked you? I'm back to the banging our head on the table portion of this conversation. I mean, I guess the thing that comes to mind, just because it really made an impression on me, and it's a negative story that I want to sort of balance out with something positive, but I was working in the White House and had decided to do what we convinced the Veterans Administration to do what we call discovery sprint on the Veterans Benefit Management System.
53:42and it was going really slowly. There was like huge latency. And I got these two folks in to help out for just a couple of weeks and just do an analysis of what was wrong before they go like, you know, trying to figure, you can't solve the problem until you know what it is. So like the first thing was we show up and the guy's like, oh, I'm glad they sent the White House to verify that everything's fine now. And it turns out everything was fine because the leader had defined latency as under two minutes. Okay. So you've just defined the problem away. So people who are trying to process benefit applications, like if they clicked and it took one minute and 59 seconds for the application to respond, it was fine.
54:26No problem here. But then I kept talking to the guy and I kept asking him questions about the system. Like, well, why is it designed this way? Why was it designed that way? And he said, I've spent my career teaching people not to have an opinion on business requirements. The IT people should have no opinions there. And I said, why? He said, well, if they ask us to build a concrete boat, we'll build a concrete boat because then it's not our fault. And I was just like, I felt like I'd been gut punched. Like it was so hard to hear that. Like there were, I think at the time, 18 veterans a day committing suicide, most of whom couldn't get their health care benefits.
55:09And I just couldn't understand how someone could take that approach. But honestly, you know, in the 10 years since then, I've seen a lot of reasons why he felt like he could say that. And, you know, I don't want to blame him. I want to blame the system. But yeah, if you tell us to build a concrete boat, we'll build a concrete boat. Like how, what way is that to run a country? I have a much more technical problem. And please yell at me if this is too technically in the weeds. So at one point, we in my old work on Snap, we observed a system, one of the one of the official systems that was user facing.
55:46So external users, like folks trying to apply for food stamps to use it. And we noticed one day that it wasn't loading, right? And it wasn't loading in the following way. You would go to the website, it would have buttons and everything loaded. But then when you try to enter click on anything, it wouldn't work. And it wouldn't work, weirdly, for about a minute and a half. Like, then all of a sudden, you click on things. And we're like, what is going on here? This is bizarre. And we looked around, and so we did some debugging on the client side, meaning like we opened up the web browser inspector, and we said, okay, what's going on?
56:23And what we found was this website was loading, I forget exactly the name, but it was like translations.js or translations.xml. And it was taking a minute and a half. Everything else was loading fine, but this was taking a minute and a half. And counter to best practice, it was, you know, blocking any interaction with the page until it was fully, fully loaded. And it was loading very, very slowly. And we looked at it, and it was this giant file. I was like, I don't know, it was like 50 megabytes. It was a giant file. Like, what is going on here? looked into it and it was a translation of every page on the entire system into like eight languages and we call it the thing and that was like okay this is not great and there's ways to fix that like i said you can let people interact with the page until before that's fully loaded and it's fine also you definitely shouldn't put all the translations for every page in your website on eight pages into a single file that you send to the client.
57:21Like that's just bad development practice. But the thing that maddened me is we being good hearted people who are not out to, you know, just sort of lob bombs and talk trash. We contacted the entity who ran it and we said, hey, this is happening. We're seeing it consistently. This is the issue that's going on. And they said, no, no, no, no, that's not happening. It's totally fine for us. And the reason is because we diagnosed it down to one of two reasons. One was they'd already loaded the file once, so it was cached. And so it was on their computer. So every time they went to it, it was fine.
57:56And like, oh, yeah, it works for us. The other was, and this was our better guess, is their server that was serving this was on their network and actually sending it much more quickly. And they were just like, well, it works on our network. It works totally fine. And I don't know exactly what it was, but I think that is the reason that's so maddening. And it gets back to a little bit like Jen's point about accountability is the truth is we can't fix problems that we don't see or don't recognize. And I do think that a big part of all of this is we have some expectations that websites should work, that technology should work.
58:33We also need to give more. We have to set an expectation, I think, government-wide about what stuff working looks like. Like Jen gave an implicit one of a normal human should be able to understand this. That should be tested on every system. In the e-commerce world, like what's the number one metric you care about? It's from start to finish, how many people start a transaction and actually end up purchasing that conversion rate. That, I mean, it would be a really useful feedback loop to measure the success rate of users trying to do a transaction across every transaction in government, right? Like people applying for food stamps, people applying for whatever.
59:07If you saw massive drop-off or massive disparities across systems, at least it would tell you, oh, there's something we need to fix here. So I think there's another layer of all this. And that's my depressing story because eventually it got fixed. And I think probably they figured out we were right like a week later. but it took a week and it was a very, I think that's why in my mind, there's also, if we can get to better accountability and give people better on the government side, better visibility into these issues with possible routes to solving them, they want to solve them. But right now we don't have a system that necessarily looks for those issues.
59:46And I think that would be a better form of accountability we have relative to say, did you follow every IT checklist bullet point in policy, right? Like, if we're getting bad outcomes while following that, then maybe something else needs to change. And maybe there's a different form of accountability necessary there. We should give a quick shout out to the President Biden's customer experience executive order that is trying to get agencies moving in that direction. There is so much to talk about and so many different subversions of this conversation we have. But that was an amazing conversation.
1:00:22Jennifer Palka, Dave Gorino, thank you both so much for coming on OddLots, sharing your time and insight and direct experience in all this stuff. It was a delight. Thanks so much. Thank you.
1:00:45Tracy, that was an extremely illuminating conversation I found. I mean, it's like easy enough to say like, oh, government is bureaucratic and they don't pay enough and that's why they can't build software. But there was like so much richer and more complex than I appreciated. Right. Right. The emphasis on incentive was really interesting and sort of fits into a lot of the discussions we have here. And also Jen's point about a lot of this emanates from the policy side. Yes. Right. And, you know, it's people trying to tick various boxes that have been set in stone by policy. I got to say one thing coming out of that, though, I feel a newfound appreciation for being a journalist.
1:01:22And actually, you know, when you and I produce an episode or write an article, we sort of send it out in the world and then it's done and we can move on to the next thing. We don't have to go back and refine it in response to customer feedback and various testing and things like that. If we were if we were Wikipedia editors, we would have to be continually editing them for years. I thought that was really great. And again, Jen's point that you just brought up was like this sort of like reflection. like the real solution like has to emanate from the policy level i was actually getting drinks recently with a friend of mine who's a software engineer and he said there's like this phenomenon he said it's called conway's law like every organization he said like ships its org structure and this then it makes like so much sense like the software is complicated because u.s politics is like complicated 90 years of policy changes and democrats and republicans switching who's in power and et cetera.
1:02:16It's like, no wonder the ultimate product of that thing is going to not look like Google.com. Well, also the story of the guy who was like, it takes 25 years to really know how to file a claim. You can imagine if you spend 25 years developing expertise in this one very specific skill set, you're not really incentivized to start changing that anytime soon. And so the system just kind of perpetuates itself. There's so many different directions we could go with that conversation. So many different ways the system is self-perpetuating. I learned a lot from that. Yep. Shall we leave it there? Let's leave it there.
1:02:53This has been another episode of the Odd Lots podcast. I'm Tracy Alloway. You can follow me on Twitter at Tracy Alloway. And I'm Joe Weisenthal. You can follow me on Twitter at The Stalwart. Follow our guests on Twitter, Jennifer Palka. She's at PalkaDot, and she is the author of the new book, Recoding America. definitely check it out. Follow Dave Guarino. He's at Dave underscore Guarino. Follow our producers, Carmen Rodriguez at Carmen Armin and Dashiell Bennett at Dashbot. And check out all of the Bloomberg podcasts under the handle at podcasts. And for more OddLots content, go to bloomberg.com slash OddLots, where we post transcripts.
1:03:32Tracy and I have a blog and a newsletter that comes out every Friday. And you can talk about all of these things 24-7 with fellow listeners on the OddLots Discord. It's really fun. I hang out there a lot. Go to discord.gg slash oddlots. It's a real blast and a fun place to hang out on the internet. Thanks for listening.
1:04:27We'll see you next time.
1:04:39Meals, safety and security, transportation, resort-style amenities, and high-quality care. Take financial possession of your apartment by December 31, 2025 and save. Discover more at brightviewseniorliving.com. Equal housing opportunity. Amazon Five Star Theater presents Real Customer Reviews, performed by Eva Longoria. Tonight's review, sports briefs. Oh, boy. Where do I even start with these performance mesh boxer briefs? These boxer briefs are like a magician's trick. You know the one where you go, where did that rabbit come from? So if you're looking for underwear that not only performs well, but also gives your package the attention it deserves, then look no further.
1:05:29Five stars, Nicolicious. Shop the perfect gift this holiday season on Amazon.
From the publisher
There's a lot of frustration about the government's ability to build things in the US. Subways. Bridges. High-speed rail. Electricity transmission. But there's another crucial area where the public sector often struggles, and that is software. We saw it with the infamous rollout of Obamacare. We see it in the UX of the Treasury Direct website. And we saw it in the way state unemployment insurance systems broke during the pandemic. So why is it so hard for the public sector to build and maintain software? On this episode we speak with Jennifer Pahlka, the founder and former executive director of Code for America and author of the new book Recoding America: Why Government Is Failing in the Digital Age and How We Can Do Better, as well as Dave Guarino, who recently left the Department of Labor after working on upgrading the unemployment insurance system. Both have a long history of working on public sector software systems and they explain why the problem is so tricky.
See omnystudio.com/listener for privacy information.
