Data and curiosity: a conversation with DJ Patil

Welcome! I’m Tuhin Srivastava and today we’re sitting down with DJ Patil. DJ was appointed by President Obama as the first U.S. Chief Data Scientist, where he worked on using data to responsibly benefit the American people. His work includes establishing healthcare programs like the Precision Medicine Initiative and new criminal justice reforms like the Data-Driven Justice and Police Data Initiatives. 

DJ has also served as the CTO for Devoted Health and as Chief Scientist and Chief Security Officer at LinkedIn.

In this interview, you’ll hear how DJ applies his personal philosophy on investing in people and going above and beyond to find the most impactful opportunities to apply data-driven solutions.

Super excited to introduce this fun and heartwarming interview to you! Here’s my conversation with DJ Patil:

‍Note: Transcription has been slightly modified/reformatted to deliver the highest-quality reading experience.

Welcome DJ! I know this is open-ended, but can you tell us more about how you got here? 

DJ: I think the way we fall into the roles in our lives is really a function of a couple of things. One is curiosity, it's where we find passion. And the second is who you're fortunate enough to have around you or the choices you make of who you're willing to put around yourself. And I think I've had the very good fortune early on in my life, especially in high school, where I didn't always make the best choices on that by some of the people I hung out with, some of my own type of life choices where I wasn't a good student. And I had somehow convinced myself I was really bad at math. And I wasn't going to be ever good at math. But luckily, I had a really great junior college and that forced me into this world of taking elementary math classes because my girlfriend was taking them, so of course I had to take them.

And what it turned out was my prior assumptions about myself were wrong. That I could be good at math. I could actually fall in love with math, weirdly enough as that sounds. And from that, it kind of carried me forward into this world of what you can do with that. There's a quote I keep on my desk, it's actually from one of my PhD advisors Jim Yorke who's the guy who coined the term chaos theory: "A degree in math is a license to study the universe." And for me, the extension of that is data also. What I was really excited about was this idea that, well, what if we use data to understand the world around us? Which for many people, they’re like, duh, isn't that how you're supposed to do science? And it is, but what's novel is the volume and the amounts of data that we can process.

And people have been trying this for years. The Mayans had been doing this forever ago, right? But what we're able now to do is bring a different degree of computation. That's been able to unlock so many different things. And for me, what I've been most lucky about is that I've been able to wander from different project areas to different disciplines that have a thematic sense of data, design, technology, all underpinning it. And because data is this universal license to explore the universe, I feel I've been in a very privileged position to be able to explore a lot of very different domains.  

Would you say that, through junior college and being reintroduced to math in that way, and being given that license to fail, that's really what allowed you to flourish and get more excited about math?  

DJ: I think there's something special about our society, especially here in the United States, very Silicon Valley-esque—and I use Silicon Valley here as a marker for all these technology ecosystems—is your permission to fail. It's okay to fail. And what I didn't realize until early on was like, hey, everyone fails at something. And what I was lucky enough to do was fail very early and was able to realize like, wait a second, you could do this. I think one of the most important things for me about getting a PhD in math was you have to learn how to rely on yourself at the deepest, darkest moments.

And you're a founder. This is like, this is a quintessential moment of a founder, there's no one else except for you to rely on yourself. It's this aspect when you get the freedom to design a new company, but it's also the freedom when you get access to a database and you get to ask any question you want. The number of times a data scientist or a person with a clever idea has changed the outcome of a company is not small, it's a lot of people.

Yeah. As a founder, the buck stops—really does stop—at you. And I think your analogy of that with being a mathematician is fantastic. But I think the upside of being a founder is that people are also willing to stop thinking about everything that can go wrong for a minute to think about what can happen if something succeeds. 

DJ: You nailed it. And part of this that’s important to also highlight is that these journeys can be incredibly taxing. You know, they can be very lonely, they can be mentally exhausting and there's a lot of impact that we're seeing in the world from mental health issues. How do you counter that? It's by having good people that you can rely on. Trusted people. It's not like I just was a good mathematician and that just happened. That's far, far from it. There were people like Takashi Nishikawa, who was kind of my partner in crime for so much of my research. There's Ishfan Sunyo who was next door to me in my office. All my advisors. There was Suzanne Cindy who I bounced ideas off of, there was like a team. There was a team of people that I could say: I don't know how to do this. What do you think? And they might show me. 

There's a saying I like to often say to others and hope that it helps, but as much as I have to say it to myself, many times over: data science is a team sport. And I think when we treat it, as a company, in life, as a team sport, then it's not only more fun, but we tend to be able to have more success. 

In just the past two years, you've helped with the COVID-19 response, you've been the CTO of Devoted Health, been the fellow at the Belfer Center, you're an angel investor. And I’m sure that’s only a small part of all the things you do. How do you decide where and what to spend your time on?

DJ: It's a really good question. So first, you know, people think like I have a team that I work with that's like team DJ or something. Unfortunately, it's just me. And Tuhin you know it is only me because I'm really bad at email. What I would say is I operate across a few different models. 

The first is, I really learned in life early on that you should always have a portfolio of things you're somewhat bouncing between. Doesn't mean like you're so scattershot, but there's a few different things that you're focused on because something inevitably will stall on one and you kind of run up against a wall, but then you kind of rotate to that next thing. And then you kind of maybe rotate back, you kind of go back and forth and something breaks through. There's also a set of things that are all consuming. So, during the COVID response, there was nothing else in life. That was it, and it was heads-down 18+ hours a day. Now, once you kind of back off of that, you can create a little bit of room to work on other things or let other things kind of percolate. Same with helping run the Biden-Harris transition, there was no room for anything else. But then if you get the spaces where other things can happen and you can kind of go, well, where can I add value through my time or energy? 

And the analogy I literally use is momentum. And momentum is mass times velocity. And so mass is either people or dollars. And if you don't start with anything, it's going to take some work. You got to figure out how to get that. Velocity is a vector. So there's a magnitude, which is how fast something is moving and the direction it's going. And so you can change the direction. But if it's moving really fast, if you change direction a little bit, it's going to take a while for it to change.And so you can use this analogy to ask yourself: of the things you're working on, what has momentum? And some things are just not right to get traction and other things absolutely are. There's a saying that is often attributed in policy circles: "never let a good crisis go to waste."

So there's a ton we want to do on policing and policing reform, community policing, mental health issues in the space of, of using data and technology. We didn't have the window to do so until, unfortunately, there was a number of these murders, police shootings, offenses that then showed predominantly people of color, especially black men who were murdered. And that then suddenly gave us a window to say, could we do something different? Could we show something different? And that movement gave us the ability to bring in people. Bring in dollars. So now we had mass and it allowed us to start with something that had very little velocity, but find other people who had ideas and say, hey, we're not about reinventing the space. Is there a way we could come together? Could we make this a team sport? Could we align our vectors so that we get one super scalar? And that's really the approach I'm kind of looking at across different things. Can I add value? And if there's something where I can't add value, my, my kind of usual thing is like, I don't think I can add value here. Time to move on. 

So having a lot of irons in the fire creates this building ground, and when the opportunity is right, you can jump onto whichever one you need to?

DJ: Yeah, I believe it was Pasteur who said "opportunity favors the prepared mind." And there's a version of that. Now you can easily be a mile wide and an inch deep, and it can cut the other way. What I have found is that if you are always going a little bit beyond, you may not have the bandwidth to do all these other things, but if you have the ability to do one other job.

Can I give you a very concrete example? When I was at the White House, and this is sort of a very high-flying example of this, you'll see in a second. But it sort of shows that this can happen at the highest levels of government or power. So it was very explicit that President Obama said that national security should not be part of my portfolio. And the reason for that is that it had been one of my big focus areas previously. And he wanted people like me who had technical skills to focus on domestic issues, predominantly healthcare. Like how do we develop tailored genetic treatments?

So it was off my list. But I happened to run into a friend who was a VC and her husband had just been appointed as Secretary of Defense. And you know, I didn't really know her husband, but knew of him. And I had some ideas and he's like, yeah, well, why don't we see if we can do something together? But it's very rare that somebody from the White House just flies on traveling with the Secretary of Defense. It's not the way it's supposed to work. And Secretary Carter did not have anyone from the National Security Council travel with him. He had his core team. 

The first time he was going out to Silicon Valley, I wrote up this quick thing saying, hey, here's how you might want to think about engaging the community. It's the first time in 30 years, a Secretary of Defense has actually traveled out to Silicon Valley, which is hard to believe, but here's some ideas. So they said, hey, could you just fly out with us so that we have some concrete time to work through these issues? So I'm on the jet, and it's like all fancy, you've got logos on the plane. You have a flag. It’s pretty cool. And then they've got like, his Chief of Staff and there are actually desks on the plane where they're running everything. Then there's the next row of seats, which is kind of the next level of staffers. And then there's press. So depending on where you are close up to the Secretary is how important you are. I'm not very important. I'm back near the press. 

So I just walked up to the Chief of Staff and the comms team. I'm like, hey, just letting you know. I'm here on the jet. If there's something I can help with totally here to help. If not, that's also okay. So they gave me the speech. They’re like, read this. So I go back and I rewrite everything and you know, the next thing I know we're just working together. We're doing stuff together.

We go, it turns out the trip is a success. We go home. I write up a memo for the Secretary of Defense at like 3:00 AM. I'm already on no sleep. I've already got my day job at the White House. And I write this extra thing up about how we might be able to engage in national security in a different way, which ultimately became the Defense Digital Service. So now I had a side hustle, which is supporting the Secretary of Defense. When the President has said, you shouldn't really do that. So the Secretary of Defense, Ash Carter, goes to President Obama and says, hey, I need DJ on these specific issues. And he says yes. 

The reason I'm going into this detail is even at the White House, I kind of had a side hustle to support an agency. If I can do that, there's nothing that prevents somebody else from doing it. The question is how do you approach doing it. And by me, just kind of going up and saying, like, can I add value? If I can't add value that's cool. But I'm going to take a little bit of my sleep, my time, to try to add value and solve a problem for you. That creates collaboration. 

It’s really about going one step above and beyond. And if you hadn’t written that memo, this probably wouldn't have happened, right?

DJ: How many times in life have we thought, I should have, could have, right? But if we keep that one extra step, and I tell people this in companies all the time, go look at everyone else's goals. Choose one person where you can help solve their goal, their problem. You will learn more about the company, more about them. You'll develop a lifelong friend, a companion, and it'll be more fun. 

And they'll come to you next time as well right? Spending those few extra hours to get something started. That's really the activation energy a lot of the time.

DJ: The thing that I think we often fail to do, and especially as data scientists, we're guilty of this, is we look at problems as data, and we forget who's behind the data. I remember, you know, when I was first going into country, one of the most important things was to find people who had done this before. And there were all these amazing people who basically taught me what I needed to do. I met like, you know, these people who had defected, who I got connected to. Now, what is the difference between this and doing customer discovery? Nothing. No difference. What prevents us from just saying, hey, I wonder why this is an issue of importance to somebody? We rarely have the courage to go ask and say tell me about it. 

I wanted to touch on your time as the Chief Data Scientist. It seems super ambitious and forward thinking for the U.S. government to invest in that early, back in 2014 right?

DJ: That sounds about right. 

Only ~5% of tech companies probably had data science functions that were mature at that time. Why was this position created, and what were your key learnings and challenges?

DJ: Well, I think the real credit of this lies with President Obama. I mean, this is his vision. And I think what started this is during his campaign, you got to see how really grassroots efforts were utilizing technology and data to create a movement. You know, they weren't this sort of big behemoth vehicle. They had this very loosely coupled federation of grassroots organizers and community organizers that were able to use data and technology in novel ways. And I think he's got to see that, wow, something is changing. This is simultaneously, you know, companies like LinkedIn and Facebook, are starting to use data in novel ways and Google, and so we were like, hey, the beginning of data science is really taking hold at the beginning of the Obama administration. 

In that, he decides to create a Chief Technology Officer for the United States, which is kind of surprising that there isn't one. Yeah. There's a Science Advisor. And you realize that science and technology are now such large drivers of our economy and life. It’s actually a job that's too big for one person. They need to be two different roles. And so that first CTO is Aneesh Chopra and, Aneesh and I both moved to DC early in our careers at the same time so we knew each other. And the second CTO was Todd Park and Todd also was a very strong data evangelist. And so Aneesh and Todd had both carried the baton of open data, primarily through healthcare, but so many other ways. And why isn't all the data that is federal by default open to the public? We paid for it. It's our taxpayer money. We support this. Why do we have to buy our own data back? 

So President Obama kind of looked at those pieces and said yeah, we should codify this. When the position of CTO was then handed to Megan Smith, and government is very much a baton that you hand to the next person, she picked up the baton to run with it. And her focus by the mandate of the President was like, look, we've done a lot on data. We need to focus on education, broadband, all these disparities, there's so much to focus on. And the question was, what do we do with all this data stuff? And it was like, hey, isn't this time to graduate it? And so that graduation became, who's gonna look after this forever going forward, and that should be the U.S. Chief Data Scientist. 

And the role there, that's very specific, the reason it's Data Scientist and not Data Officer is the mission statement: to responsibly unleash the power of data to benefit all Americans. The words that were very carefully chosen by the President and his team were “responsibly,” just because we can, doesn't mean we should, and “benefit all Americans.” And you could easily extend this to the whole world, right? There's no reason that you can't extend it. That role and that charter then says, how are you going to do this? Well, one, what are we going to focus on? And so the framework was: impacts more than 50% of the population, $1 trillion of U.S. spend, or a population that has no recourse.

What falls into that? Healthcare, criminal justice reform, national security, but there's so many other things that people forget about. The fact that a person who's trans or non-binary and walks through a scanner at the airport and the scanner doesn't know what to do, because it has only been configured to say male, female, and look for issues. That is embarrassing given that we are the country that has flown a probe or landed on every single planet in the solar system. We are the only country that has ever done that. And yet we can't get a TSA scanner to recognize how to think about a person in a dignified, respectable way? That is a failure of using technology both responsibly as well as to benefit people.

So those are the types of problems that the U.S. Chief Data Scientist should tackle. And we created everything from the precision medicine initiative to data-driven justice, to all these other initiatives, to show how that happens as well as create data scientists and data officers across the federal organization to create a constellation of people who are going to ensure that the mission is, is held true for the American people. 

It sounds very functional. Oftentimes when people talk about the government, they wouldn’t think that. 

DJ: Yeah to our credit, I think one thing we have to recognize is this movement wasn't only being done by the United States. You know, there's a team in the United Kingdom who'd been really focusing on this at the state and local level, Code for America had been doing this. So there's been a collective movement. And I'll say one of the most powerful things that I learned as Chief Data Scientist: you can call anybody you want, and people will have ideas for you. I used to go to roundtables and be like, I don't know. The federal government isn't great at innovation. The federal government is great at what we call scout and scale. You look for ideas and you say, oh, I wonder if that idea with this idea, maybe that'll work across the country or the world. And so you can do that.

Before your role at the White House, you were the Chief Scientist and Chief Security Officer at LinkedIn. What did you learn there about how data teams can be more impactful within their own organizations?

DJ: Oh, there's so many thoughts that come to mind. Maybe the first is it kind of starts with trying to understand who your customers are in the company and who the other people are. People were always surprised by how I was spending my time, trying to learn about a day in the life of everybody else in the company, because it was just kind of like: what sucks? And then my job was to say, let's make it suck a little less. 

I remember I was out at our call center in Oklahoma and I was just like, wait, what? You're dealing with this spam? I was like, we can just reconfigure your Barracuda system. That’s not even supposed to be a problem, but no one had ever asked them. So now suddenly I had a different relationship. And it may seem trivial, but when I needed something, they were going to come to bat or we would build out this technology layer and we're like, look, it might be able to help you guys on this problem and help you over here on this problem. And they'd be like, we're in. You know why you're in, because this is a bet on you. 

Now, the other part that is equally important in this is that we had an amazing team. You had Jonathan Goldman who created "People You May Know," you had Steve Stegman who did "Who viewed my profile?" You had Monica Regati who built out the job talent matching algorithm. I remember Monica, Jay Kreps, the founder of Confluent and creator of Kafka, and a bunch of us went to Hadoop World. And we're like, oh my gosh, we know nothing. We're like, oh, this is so embarrassing, we’re like the idiots here. And I remember getting the team together afterwards and being like next year, we're the presenters. Tell me what it's going to take to get us here. We're going to get this done. And so the team would come back with ideas, and my job was to figure out how to get the door open. If that meant figuring out how to convince the executive team, the storytelling, it's no different than, Tuhin, what you've had to do with starting something. You have to find a way to get it done and you have to tell a story and then you have to deliver. And so we had a saying: we never miss our deadlines or deliverables. We never miss it. We say we are going to do something. We do it. And so we used to joke. We were like, literally the analytics team, The A-Team as in that TV show or depending on how old you are the movie. And it was like, nah, if you could find us, you can hire us. We will solve your problem. And you develop that mantra and that approach. And then you're able, with people across the organization, to craft a vision. 

And by the way, it's not always nice. There is knockdown, drag out political fights, all these issues, all the horse trading, and in that what’re we looking for? Never let a crisis go to waste. Show you can add value, take a pain away from a person, add the value on top of it. And then before you know it, you start to develop that momentum, the momentum starts, and then people are like, it's easy to double down on momentum. It's hard to double down on nothing. And why did we spend much time in those days of LinkedIn on storytelling? It's to allow other people out there to take our stories and go to executives or other people who may be naysayers and say, see, this worked. We could do this too. And to know that this broader community is a team sport and that if we all kind of move together, you can do it. If somebody is struggling to get buy-in at a company, tell them there was a time where there was no Chief Data Scientist, and someone had to have the courage to make that impact and the transformation that that has led to around the world. If that can happen at the largest, most bureaucratic institution in the world, it sure as heck can happen at a startup or some other Fortune 500 company. 

You're remarkably consistent across your experiences about what you preach: listen to people, invest in relationships, and go above and beyond. 

DJ: It's pretty simple. And the other part, which is important in this, is to have fun doing it. If you've got the right people around you. It's a joy. It's fun. If it's not, and it's, it sucks and somebody's not treating you with the respect you deserve. They put you in a box that is not too limiting, or they don't view your skills because they can't have an inclusive mindset. You need to move on. You need to figure out how to get yourself into a place where you deserve that because no one should ever be in a situation where that's happened. You should always be in a home or a place where people can be the best version of yourself. And I say that from a position of incredible privilege, I recognize that. But I also want people to try to know that they aren't alone and that if you'd make it a team sport, others will help you get into the right place. 

It goes back to what you said earlier, if you don't think you can have an impact, it’s okay to cut your losses. If you don't make the progress you want, document your findings and have it ready for the next time around.

DJ: Totally. People think I haven't had any failures. That's far from it. I've had so many colossal catastrophic failures and that's okay. But what picked me up at the end of the day? Like you said, put it in a box, move on. Take those lessons learned, apply them later, and sometimes, you know what, some of the stuff is baggage that isn't helpful, especially the emotional part. And you have to find a way to let go of it and move on. 

Well I think that's a really good point to end on. Thank you so much for the time.

DJ: My pleasure.

‍