John List

Guy Kawasaki:
Hi, it's Guy Kawasaki. This is the Remarkable People podcast.
Our remarkable guest for this episode is John A. List.
He is the Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago. He was also a professor at the University of Central Florida, University of Arizona, and University of Maryland prior to joining the University of Chicago.
John received his Bachelor's in Economics at the University of Wisconsin-Stevens Point, and his PhD in Economics at the University of Wyoming.
He has served as senior economist on the President's Council of Economic Advisers from 2002 to 2004.
And he's not just an accomplished educator, he's also been the chief economist of Uber and of Lyft.
John's research includes 200 peer-reviewed journal articles and several books, including the 2013 bestseller, The Why Axis.
His latest book is called The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale.
This episode is about experimentation, field testing and optimal decision making. He explains what academic studies really mean and how to apply science to the art of scaling a product, service, or idea.
Entrepreneurs and marketers need to pay special attention to this episode.
I start off with a question about Vince Lombardi because John is a huge Green Bay Packers fan. Vince Lombardi was the Packers legendary coach.
Two of his famous quotes, first, “Winning isn't everything; it's the only thing.”
Second, “Winners never quit and quitters never win.”
I'm Guy Kawasaki. This is Remarkable People. And now, here's the remarkable John A. List.
So, John, I have a theory that for a startup, there are two fundamental roles. One is an engineer who makes it and the other is the behavioral economist who sells it. That's the only two things you really need.
Steve Jobs was a behavioral economist and Steve Wozniak was the engineer. That's all you really need. Everything else is bullshit. MBAs, VPs of strategic development, all that is bullshit. Somebody make it, somebody sell it. That's all you need.
John List:
As an economist, I like it when people have parsimonious and accurate theories. And that feels like a parsimonious and an accurate theory.
Guy Kawasaki:
High phrase, indeed. High phrase, indeed.
I have a lot of questions for you because I'm really into this. I'm a huge Bob Cialdini, Katy Milkman, Angela Duckworth fan, much less Freakonomics and all that kind of stuff.
And you're right down that sweet spot. What do you think Vince Lombardi would say to you at this point in your career?
John List:
Vince would probably say, "John, you are the biggest failure in the world."
Guy Kawasaki:
At golf.
John List:
At life because, "John, you have quit. And, John, you have quit too much, and quitters never win." That's what Vince would tell me, I fear, and that breaks my heart because Vince is a hero of mine, is a Wisconsin native.
Guy Kawasaki:
You're at the University of Chicago, Economics Department. You're the department head. How was that a failure?
John List:
I had a dream. And the only reason why I went to college was to become a golf professional. If it was not for golf, I would be a truck driver right now.
My grandfather was a truck driver. My dad was a truck driver. My brother is a truck driver. I went to the University of Wisconsin at Stevens Point on a partial golf scholarship. And the only reason why I went to college is to become a golf pro.
Very early on, I realized this isn't going to work.
I quit. I quit on that dream. I didn't quit the team, so Vince would be happy about that. But nevertheless, I quit on that dream and I said, "I'm going to have a new dream and that's to be an econ professor."
Guy Kawasaki:
Nothing wrong with that.
John List:
I'll take it. I'll take it.
Guy Kawasaki:
Bob Cialdini tells a similar story that Bob Cialdini in college apparently was a very good baseball player. And one day, a scout pulled him aside and told him, "Bob, you're a really good college player, but you're not going to make it to the pros so you got to look into something else." And the rest is history with Bob Cialdini.
John List:
And that's a pretty good history too, isn't it?
Guy Kawasaki:
Yeah. Let's say that you, Phil Zimbardo and Stanley Milgram go into a bar. And what do you say to them about the prison experiment and the obedience experiment?
Just in case you're not familiar with the work of Phil Zimbardo and Stanley Milgram, let me explain.
Phil Zimbardo, a past guest on this podcast, did the famous Stanford Prison Experiment. He randomly assigned students the roles of prisoners or guards in a fake prison in the basement of the Stanford psychology department.
However, the experiment had to be cut short because the students accepted the roles of prisoners and guards too well. Things got out of control.
Stanley Milgram did a study of people accepting authority. In this experiment, people were given permission to shock a Confederate when the Confederate made mistakes. A high percentage of the people continue to give shock even though the Confederate screamed out in pain.
The implication is that people will listen to authority figures and do things they probably normally wouldn't do. The issue you have to wonder about for both Zimbardo and Milgram is whether people would behave this way in non-experimental surroundings.
John List:
Wow, wow. So, I don't think that all three of us will come out of the bar alive because we will either drink way too much because we're having so much fun, or we're going to come to fisticuffs because we'll probably begin with the ethics of those experiments and we'll talk about exactly what we've learned.
We will talk about how we could scale those results or generalize those results to other populations. But that would be a conversation that I would give a lot of money to have.
Guy Kawasaki:
I can't help you with the Stanley Milgram part but I can help you with the Phil Zimbardo part.
John List:
I would love that.
Guy Kawasaki:
Let's say that Phil says, "We can't depend on undergraduate college student trying to do something over Christmas or Easter vacation for credit."
How would you do an in vivo Stanford prison experiment? Abu Ghraib? What do you do? How would you even construct that experiment?
John List:
No, I think it is true. I think the ingenious part about that work is he did it as natural as he possibly could. He did cross a few lines that people might not like. But if you need to produce scientific evidence on this particular topic, the guy was ingenious in the setup and how he went about his science.
So, you really can't complain about that. Now, you can have other complaints about the ethics of the science or whether we should be doing that kind of work. But conditional on doing it, the work is ingenious.
Guy Kawasaki:
I got a lot of suppositions and stories here. As you can tell, I did a lot of research for this. This is not something that somebody handed me or Wikipedia profile.
John List:
You have done a lot of work. You have not taken this job lightly.
Guy Kawasaki:
Not at all. Vince Lombardi would be proud of me.
Let's say that now you run a not-for-profit and you hear Bob Cialdini tell the story that if students on the campus of the University of Chicago are trying to raise money for, let's say, student organization.
And Bob Cialdini says, if the student starts off the conversation by saying, “Like you, I am a University of Chicago student” versus just diving straight into the spiel, you will raise, I don't know, 25 percent more money.
So, now, you're the person who runs a not-for-profit. What's your immediate reaction? Do you change everybody's pitch or do you do an experiment? What do you take as a practical step when you hear something like that?
John List:
No, absolutely. That's a great question. The one fact of life that I have learned about through my travels, when I work with for-profits or nonprofits or I work in the government, nearly every time if somebody reads an academic paper or hears from an authoritative figure, they take that as the gospel. They say, “It's peer-reviewed so it must be the truth.”
When in reality, if it's a surprising insight, it's likely not true.
It's likely just what economists or social scientists call a false positive or in the book, I say the data are lying.
So, in this particular case, I would say, "Bob, that's a really interesting insight and I can understand the theory behind it."
What I want to do is I want to do a small-scale pilot and I want to make sure that it works for my organization. And after I find out it works for my organization, I then want to find out who does it work for.
Because in many cases, our messages or our nudges or our experiments work for some people but not everyone. And in a lot of cases, we roll out programs and ideas ubiquitously and they only work for a small corner of the world.
So, early on, when you have great ideas, you do not want to be in the business of an efficacy test.
And by efficacy, I mean, let's give the idea its best shot. That's like standard business because people like to show big results, et cetera.
I want you to take the opposite view. I want you to kick every tire and figure out how can I make my idea fail. And if you can't do that, we're in business. That's something that can truly scale.
Guy Kawasaki:
So, we have been talking about the scenarios with social psychologists. And after doing the research about you, I had one fundamental question, which is what is the difference between behavioral economics and social psychology?
John List:
Yeah, so it's a great question. Behavioral economics, the watershed moment was really Kahneman and Tversky. It does go back.
Gary Becker did some behavioral economics, one of my late colleagues, Herbert Simon, Adam Smith. So, there were, I believe, bonafide behavioral economists before Danny Kahneman and Amos Tversky. But it's really the watershed moment.
Late seventies, they have a great paper on loss aversion.
Now, because of that starting point bias, most of behavioral economics has been along the lines of using psychological foundations and adding those to the economic model. That's social psychology.
Now, behavioral economics is broader than that. Behavioral economics is not only using insights from social psychology, but it's also using insights from sociology, from computer science, from divinity, et cetera, et cetera.
So, I want you to think about behavioral economics as using another field to augment the traditional approach in economics, even though most of our celebrated examples are through psychological insights. That's only a subset of behavioral economics.
And I predict in the next several decades that there will be competitors to the crown of psychology and the competitors that I can see will come through computer science.
They will come through sociology, when you think about group theory, network theory, my own work on spill overs and how spill overs work in a market. The theories in sociology can really lend deep insights to those kinds of questions.
Guy Kawasaki:
Just for the skeptic listening, how much of what we think we know is based on undergraduates trying to get credits for being subjects of experiments for professors?
John List:
Now, you're trying to get me in trouble. So, when I came on the scene in the early nineties, I was coming into academia as a graduate student who had been raised in the barons of baseball card markets.
So, what I mean by that is you have these big conventions where people entered the convention to buy, sell and trade baseball cards and football cards and basketball cards. So, I was a dealer and I was doing experiments in the eighties and early nineties in exploring how does economic theory do in these markets.
Okay. So, now, as a graduate student at the University of Wyoming, I start to read all of these lab experiments. I'm like, "Wow. Are they saying that from sophomores in college, that result will generalize to important markets?"
Now, I believe you can learn a lot from lab experiments with students. I do some of them myself. But I believe to tie the entire story together, you need to use field experiments to generate insights, not only to verify the lab, but importantly to figure out what are the mediators at work and what are the moderators at work.
And basically, what I'm saying here is, if I go to the real world, and that's been my career since the early nineties, to use the world as my lab. I suspect that every one of your listeners has been an experimental subject in one of my experiments.
So, I want to stop here and thank all of them for allowing me to learn about the real world. No, it's not creepy in that I can't attach your name to your data. But what I can do is I can tell you, your listeners, when I raise the price at Lyft... I'm the chief economist at Lyft.
If I raise the price by 10 percent on your Lyft trip, I have a pretty good idea how you're going to behave.
Now, those types of insights can only be truly gotten in the field. So, I think it's important to recognize that the lab and field are complements and they're complements in the discovery process.
And we will never really learn exactly what's happening in markets and in environments that we really care about unless we have both kinds of data.
Guy Kawasaki:
So, now, let's suppose that I'm a CMO and I read about this study conducted by Stanford at Draeger's Market in Menlo Park, California. So, the gist of that study is when you put twenty-six kinds of jelly out on the table, people buy less jelly than if you put five kinds of jelly. So, now, I don't know, I'm CMO of Apple.
I say, "Maybe we shouldn't have sixteen different colors of iPhone and sixteen configurations. Maybe we should just have five."
Is that a logical conclusion or you say to yourself, that's jelly? That's not phones. That's not cars. That's jelly.
John List:
Exactly. It might be because the consumers are different than students or the jelly consumers. It might be that the products are different than some important way.
So, here is what I would tell you. I would tell you that this is an interesting result and it's something that we should at least consider when we roll out our product variety because it could be true that our own consumers have choice overload.
And if we end up giving them a really, really rich choice set, we could undo all the good stuff that we were trying to do by creating that great choice sentiment. There are good reasons why they have a deep choice set because people are different.
People have different preferences. The phone that I want might be very different than the phone that somebody else wants.
But it is true that there's a limit to that, and the limit is because of our human cognition that says, if I just get too many stimuli thrown at me, I might just throw my hands up and go to somebody else where they only have three choices, which is clearly inferior but it's not making me confused.
So, to me, those types of results are signals to organizations. That's something we should look into, and it won't be very hard to look into it.
The thing that I believe most organizations typically fail at is that they don't understand the opportunity cost of time.
And what I mean by that is, every moment that goes by, that you're not doing the best you can, that's a moment that you've lost.
And this is a customer group that you may have lost or a profit that you may have lost or charitable dollars that you may have lost because you haven't tried to innovate new things based on science.
A lot of times people say, "Well, that's just the science of undergrads. That doesn't work for us." I think about it differently. I would say, "That's a potential idea. Let's put it in our idea hopper and that's something we're going to test to see if it does work."
Guy Kawasaki:
What if somebody pushes back on you and says, "John, you brought up the topic of opportunity cost." If I'm the CMO of Apple, and you're telling me I need to do in vivo experiments where I offer twenty-five colors of iPhone and five colors of iPhone, that will take six to nine months and by then, I missed the Christmas season.
That's the opportunity cost.
John List:
Yeah. If you do it that way, I'm thinking you can have the results in three days with the resources that a CMO has. Look at Lyft, didn't we have a new idea? I can know within twenty-four or forty-eight hours whether this idea is spinning and working in some markets.
And then, when it starts to work for a lot of people, within a week, I'm spinning it out all across America.
So, I don't know what Apple's problem is, but they need to get kicking with it. It's too long.
Guy Kawasaki:
With Lyft, you tell some programmer, just change the app slightly or change the price.
But with Apple, you've got to get the magenta and the blueberry and the cherry and tangerine color phone.
John List:
No, but they already have it. They already have it pretty much. It's easy like with VR to create it and with technology. These things are not that hard. It's easy to create these opportunities.
And it's not that you have to gin up a new facility in China or Topeka or whatever. It's not that at all. We have technology to make these choice sets very possible, I believe.
Guy Kawasaki:
To continue to pound this dead horse into the sand here, so I guess with the CMO of Apple, what you could do is you'll say, "We're going to take some of these stores and we're going to take certain colors of iPhone out of the display. And we'll see if they do better than other stores with more iPhones."
John List:
If you want to do it in the physical sense, as long as you have the array, you can always take off items from the array.
But where you went was, we want to make the array even bigger. And I'm saying if you want to make the array bigger, we can do that with technology.
But you're right, if you want to do it physically, you just take things away.
Guy Kawasaki:
Now, with your perspective on all this, what do you think is more dangerous, changing too fast or changing too slow?
John List:
This is almost like asking me, "John, of your eight favorite kids, which is your favorite?" And in that case, I would say all. In your particular example, I would say both. But in terms of scaling, we typically make errors in the move fast and break things mentality.
And the reason why I say that is because the mentality is throw a bunch of spaghetti against the wall. If it sticks, we're in business. If it doesn't, we move on to something else after we run out of money.
You have to understand that for an organization, if you think about a business or a nonprofit or a governmental organization, a lot of times when you start down a path, all of our biases start to kick in, things like sunk-cost fallacy.
We spent so much time on this already. We have to keep going. Confirmation bias, all the signals that you get that it's bad, you put those on the sidelines and say, "Actually, my idea is good."
Now, it's also hard for many organizations to pivot. And what I mean by that, it's hard to make change because our general mentality is the status quo is best.
That's the way our minds tend to think. So, the way I look at it, if you make a six-month mistake, it's probably in the end, an eighteen to twenty-four-month mistake because you're afraid to let it go and you don't pivot fast enough.
So, that's why I would argue that making the initial mistake in many cases is worse because we keep doubling and tripling down because we don't understand the optimal quitting rule and that there's a true opportunity cost.
Guy Kawasaki:
We're going to do some case studies here.
So, what do you think was going through Reed Hastings' mind when Netflix had its near-death experience? Why did they do what they did?
John List:
I hope what was going through in his mind is that, "Oh, my gosh, I should have set up a really small-scale set of experiments to figure out that this is not the right move to make." They got really lucky and they made it.
But they had to have a set of fortunate, let's say, situations happen, for them to work their way out of that near-death spiral. And it would have been very easy to avoid what had happened there with a little bit of experimentation.
Guy Kawasaki:
What would they have to do?
John List:
This is in The Why Axis now, right?
Guy Kawasaki:
Yes, yes.
John List:
We wrote about this in 2013. That would have been easy to test whether you should be rolling out that nationwide. It would have been easy to test that in really small-scale markets to see how people would respond. Back then, there was an uproar.
When you look at the stock chart and what exactly happened, that would have been easy to avoid that. There's no reason to roll that out nationwide.
Guy Kawasaki:
You could just spend thirty seconds to explain what they did because people may not be old enough to remember that exactly. I'm talking about the time in The Why Axis. Wasn't it the time where they jacked up all their prices and enforced membership?
John List:
Yes. You could have looked at varying price and looked at memberships whether they need to be obligatory or mandatory. You could do both of those things.
We do pricing experiments all the time. There are membership experiments going on all the time. Before rolling those out and going to a near-death spiral, you could have tested those in a small-scale setting to figure out they're not going to work and that would have saved a lot of money and a lot of time. That's exactly what I'm saying.
Guy Kawasaki:
But what I was really trying to ask with this question is, what would have gone through Reed Hastings' mind and the board and his exec team to make such a dangerous decision with very little data?
John List:
Let's face it, most decisions are made without data and without evidence. Most decisions are made with gut feeling or with some evidence that doesn't make sense like bring in a focus group or something and bully them into telling us what we want to hear.
If you gave me a nickel for every major decision that was made around America, in government and in private firms, I would be as wealthy as Elon Musk right now. This is just the culture of decision making.
A fundamental difference in a few decades will be that those kinds of firms and those kinds of decisions will be made extinct. And they will be made extinct because the data and the technology and the ability to get evidence will just be a lot easier to obtain the relevant evidence that you need.
In the past few decades, this information revolution has changed the entire landscape. But I think you're going to find that you will not have an excuse not to have a data-backed decision.
Where back in the day, let's face it, there just was not the data available, or it was costly and it took a lot of time to find the relevant data. That was the culture mindset that they were probably in.
And they all sat around a table and said, "Isn't this a great idea?" And they all said, "Yeah, it's a great idea. It works for me." Now, a lot of times what works for me doesn't always work for everyone else.
Guy Kawasaki:
I would say that you are a lot more optimistic about the near future than I am. And so, case in point, do you think that Facebook tested the name change to Meta?
John List:
I don't but I also think in that case, it was hard for them because they also wanted a splash. And they also wanted a big surprise moment. And in the end, those types of things, I understand branding is important.
But Facebook was at a fork in the road. They needed a change. And it was pretty clear they need a brand and an image change and there, it makes sense for some decisions. I don't work with Facebook, but my guess is they probably did focus group it and it came out to be something that people liked.
Guy Kawasaki:
Shit.
John List:
For sure they did. Guy, you know they focus group.
Guy Kawasaki:
They probably paid McKinsey ten million bucks for that.
John List:
To focus group it, right? This is good.
You have asked a lot of questions that I have not thought about for a long time. You're very different than your questions in that you are asking very fundamental and deep questions about the social sciences, which tells me that your mind works in a different way than most people's mind.
Guy Kawasaki:
Is that a compliment or an insult?
John List:
I make it as a compliment. So, if you took it as a compliment, you interpreted it correctly.
Guy Kawasaki:
Next question. So, what does your framework make of a Steve Jobs or Elon Musk? Because in my opinion, they at least superficially appear to violate everything you stand for.
John List:
Yeah, I don't think so actually. When I look at the truly and deeply successfully scaled businesses, you find features of my five vital signs in everyone.
So, let's take an example in the vaccination world and let's talk about Jonas Salk. And I'll come back and we'll talk about the businesses in a moment here.
So, Jonas Salk has an idea about a polio vaccination. So, what he does is he tries it on his own kids and it works. So, then he tries it on a bigger group of kids and it works.
And he replicates it on a much bigger group and it works. So, he checks off, this is not a false positive. Then he also finds that it works for every kid.
So, now, the population of people that it works for is ubiquitous. The polio vaccination works, whether you're black or white, low or high SES, et cetera, et cetera.
Now, comes the stand at the O.K. Corral because in my book, the third hurdle is called Properties of The Situation.
So, you say, "How is Salk going to get that vaccination in people's arms?" That's really what we face right now with the COVID vaccinations is the delivery mechanism.
How do we get people to take the medication? In the case of polio vaccinations, this is the epiphany. Because what happens here is, and all of your listeners will know this, when you have a baby, what happens is the baby comes out. You're super excited in the delivery room, and they give your baby some vaccinations.
They then give you a schedule, you need to come back in six months for the checkup. You get more vaccinations. You come back in a year, you get more vaccinations.
So, what's brilliant about the polio vaccination is that it leverages the health care system because every adult wants to do by their baby. So, they will bring their baby in for the doctor checkup. And guess what? You get the medicine.
Okay. Vital sign number four in my book says are their spillovers in general market effects? Yes, that's where the polio vaccination strikes again. Once you get the vaccination, you can't give it to everyone.
And then five, which is super important, is the supply side. That's the fifth vital sign in that's do we have economies of scale in our product? And what that means is, as I expand my customer base, does it get more costly to provide the goods or less costly to provide the goods?
Now, in the vaccination case, it gets less costly. Why? Because most of the expenditures in medicines are for R&D.
So, to create the pill or to create the vaccination, it's pretty cheap after you have the compounds. Now, it's also cheap if people bring their babies to you, which happens in the vaccination case because of leveraging of the health care system.
Okay. So, that's now a perfect idea that checks all five of my boxes.
Now, let's go to Musk. When you look at everything that Musk is about, whether it's recycling batteries, whether it's sending spaceships up beyond the earth, he takes advantage every time of economies of scale. That's big for him.
It's also big for people like Bezos. The way that they've gained their competitive advantage is they're able to leverage economies of scale. Same thing at Uber. I used to be the chief economist at Uber. Same thing happened there.
At Lyft, same thing happens. At Walmart, same thing happens. So, they also have the other characteristics, too. When you think about Amazon, nearly everybody likes Amazon products.
So, it's not a false positive that Amazon is good. In some cases, there are certain vital signs that play a bigger role than in other cases. But every idea will have a strong component of these features, at least every idea that I've seen that's worked and I've explored that hasn't worked.
These five vital signs that I talked about in the book, they're always key players. One of them in some cases might be more important than in others, but there will always be a mixture of them in every narrative that we tell about a business idea or a policy idea.
Guy Kawasaki:
But if I may push back a little.
John List:
Absolutely, you should push back a lot.
Guy Kawasaki:
Let's just say I know from firsthand experience, and I don't know Elon Musk, but I would assume this is the case. It's not like Steve Jobs did a lot of experiments to figure out what to do in vivo. He just went for it.
So, is he just a unicorn or did he just get lucky? Isn't he contrary to your experiment, and find out what's really happening before you jump in with all two feet?
John List:
That's a great way to think about it. Let's think about it in a slightly different way. If I go into a bar and I observe an expert billiards player, what they will show me is that they have figured out the physics of being an expert billiards player.
So, if I look at what they're doing, it will conform to a physics model that says this is how you should play billiards.
So, you shouldn't be tricked by thinking, does a billiards player know the physics? No, of course, they don't. Of course, they don't.
But they were lucky enough to practice enough or they just fell upon it by sheer luck to have the physics right and then they become expert billiards players. There are millions and millions of people who have tried to push their ideas, and they failed.
Now, just because one person does it right, who didn't have my recipe beforehand, doesn't mean they can't just be lucky and have the five features without knowing them.
Just like the billiards player doesn't know the physics of playing billiards but they got it right. But there are a million other billiards players who get it wrong. That doesn't mean that my model is incorrect.
In fact, my model is based off of looking at billiards players who are good, and looking at billiards players who are bad. So, my whole book is about observing both people who got it right, Jobs, Musk, et cetera, et cetera, and people who got it wrong.
So, this isn't an innovation of my mind. I'm observing real data and then bringing it back and putting it in a way that I hope is digestible. But that's one of my data points in my science.
Guy Kawasaki:
Let's talk about voltage.
So, my first question when I encountered your Voltage book is that is a voltage drop like Geoffrey Moore's Crossing the Chasm? Is a voltage drop a chasm?
John List:
Yeah, that's a good way to think about it. The way I think about voltage drops is that in the hard sciences, we have physical laws. We have quantitative laws that we have used experimentation in theories to discover.
In the social sciences where I live, we have laws but they're not as cool as the hard science laws. Our laws, the best we can do would be like the law of demand.
You can say, "What's the law of demand?" Well, the law of demand is obvious. When price goes up, the quantity demanded goes down.
It basically means, people buy less when the price goes up. You can say, "What kind of law is that?" But our laws tend to be moderated by variables.
So, anytime you're dealing with a human, you have these situational features that affect how strong is the law or how weak is the law. When I first started doing research on the voltage effect some years ago, I began to wonder, "Is this like a social science law? Is this something that will happen every time because of the way that the knowledge creation market is set up?"
And what I found is that the voltage effect really is like the law of demand. And it's very clear that nearly every time it will happen unless we do something about it.
So, you're right, there's a chasm. And unless we create some kind of bridge, or some kind of alternative route, we're going to fall prey, or we're going to fall into the trap of a voltage drop.
And I think the feature of my book is I'm trying to bring forward science and bring science into this very, very important question about scaling ideas.
So, that I feel is my contribution, it's I'm bringing science to an area that everyone agrees is super important.
Probably one of the most important questions that we face in modern society is “Why do we have decades of research on poverty alleviation? Why do we have decades of research on public education? And we can't solve those puzzles?
A big contributing factor is because we haven't figured out how to scale the idea. And we haven't figured out that from the very beginning of the research agenda, we need to have scaling in mind.
So, to answer your question, absolutely, it's a chasm, but it's something that I hope now we understand what are the elements we need to put in place to avoid it.
Guy Kawasaki:
I'm going to offer you something, maybe you thought of it but just in case. One author and a wannabe behavioral economist to a real behavioral economist.
I'm going to offer you something. Are you ready?
John List:
I'm ready. Give it to me.
Guy Kawasaki:
So, your five vital signs about false positive, overestimating market share, dependence on unscalable items, unintended results, supply side, you could make the case that they are essentially the GFCIs of entrepreneurship.
Hopefully, you know what a GFCI is.
In case you're not familiar with the term GFCI, it stands for ‘ground-fault circuit interrupter’.
You may have experienced it in your house. When lights don't go on or electrical outlets don't work, a GFCI has detected a leak in the system.
That is, electricity is going out typically because of water.
Now, you may be wondering what's a circuit breaker then. Circuit breakers have a very different function. Circuit breakers ensure that an abnormally high amount of electricity is not going through the circuit.
So, circuit breaker makes sure you don't have too much electricity going in. And GFCI shut off electricity when electricity is leaking. This may be too much information about electricity, but for a book called The Voltage Effect, GFCI makes sense.
When there is a leak in the business, the system should shut down until you fix the leak.
John List:
I like the way that you're marketing the result because I think you're right. That's a good way to put it. No, I think you're exactly right. And I've never thought of it that way.
The way I've been thinking about it is, in the world, we put up too many resistors and then we're not allowing the voltage from the very beginning to happen on the transmission line the way it should. But you're right, it's a GFCI, a 100 percent.
Guy Kawasaki:
So, false positive means voltage has leaked, you better check why the GFCI went off.
So, speaking of false positives, half of entrepreneurship is ignoring the fact that something could be a false positive and going forward anyway, but I digress.
John List:
No, you're right. You're right. When I look at my data, a big part of it is there was never voltage to begin with but people ran with it for a crazy reasons.
But you're right, a lot of this problem is there was never voltage to begin with.
Guy Kawasaki:
But does that mean you hate the concept of fake it till you make it?
John List:
If the concept fake it till you make it is meant to be a duper, until you make it, then I'm against it because there are altogether too many dupers out there. And let me tell you a story.
So, a good friend of mine is a venture capitalist, VC guy. He has hired three people from the FBI to figure out and do background checks on people who tell him their ideas because of the duper problem.
So, the fake it till you make it might be great for you as an individual, but it's not very good for investors in the capital market who actually think you're telling the truth.
And they're spinning up millions and millions of dollars in your ideas that you've been faking.
So, once you think about the externalities of your behavior or how it affects others, I can't be on board with the fake it till you make it idea.
And in chapter one, I talked about dupers. I talked about Elizabeth Holmes with Theranos. In fact, I talked about my friend in that first chapter. If it takes people to hire ex-FBI agents to try to stop this from happening, we're not in a good part of the world if we're telling people fake it till you make it.
Guy Kawasaki:
What's the difference between a duper and a dreamer? In that, a dreamer is not necessarily criminally lying about something but is saying that such and such could work. Give me money to prove it. That's not duping.
John List:
That's not a problem but that's not a duper and that's not faking it either.
So, when I hear fake it till you make it, it feels to me it's a duper faking data. And you keep faking data and finally, you might end up with an idea that works. That's not a dreamer.
Look, I was a dreamer when I wanted to make the PGA Tour when I was in high school. That's not duping. I'm saying, "Look, this is my dream. Here's how good I am. You should invest in me."
Here would be the duping. If I would go to people and say, "You know what, I'm shooting a sixty-five every round on these hard courses," and then giving them the scorecards and the scorecards would be made up. That's a duper.
But going to them and saying, "I'm a good golfer, let's go out and golf together," this is my dream. That's a dreamer. I'm all for dreamers. Where I have a problem with a dreamer is that sometimes they dream too long.
And they dream too long because they see the headlines that say, every time we watch the Olympics, in fact, when you air this episode, we will probably be right in the middle of the Winter Olympics.
And there will be feel good stories about a woman who has been at it for thirty years. And now, she finally made her dream and she got seventh place, et cetera, et cetera. Those headlines are sending, on the one hand, a great story that Angela Duckworth, I'm a huge fan of Angela and she's a good friend of mine, grit is important.
Don't get me wrong, but misguided grit is just misguided. And all I'm saying is that if you're a dreamer and you have grit, let's focus on the right ideas and the right ways to use your time.
Because I've never seen the headline that said, John List just spent his whole life trying to be a golf professional, and he just died and he never accomplished anything.
There are millions of headlines like that that should be written. They're not.
You know why they're not? Because they're not feel good stories.
But because they're not written, people believe the norm is that I shouldn't quit something and I should have grit into this idea. And in the end, we don't quit enough and we dream for too long. All of your listeners should take one piece.
If they take nothing else from me, if you have been grueling over something that you're thinking about quitting, whether it's a job or relationship, whatever, a project, and you've been grueling over it for six months, you probably should have quit already. That's what the data would suggest.
And just read chapter nine in Voltage Effect and you'll learn why. There's science behind that.
Guy Kawasaki:
Six months? Six months, that's all?
John List:
Six months is way too long. If you're grueling over it, that means for the year before, you were just contemplating it. But six months of grueling, that probably means you've been thinking about quitting for two and a half years. Probably too long.
Guy Kawasaki:
Wait. So, I've been podcasting for two years. And I know it's not exactly like I'm Joe Rogan or Terry Gross.
So, should I quit podcasting?
John List:
No, because you're successful.
Guy Kawasaki:
Not compared to Joe Rogan or Terry Gross.
John List:
Yeah, that's another thing that we should be careful about. And I want to make sure that people understand what I'm saying in the book.
My dad, my grandpa, and my brother, when they were truckers... my brother is a trucker right now. My grandpa unfortunately is gone. My dad is retired. But in their business, it's one man, one truck, one good life. The secret sauce to their business is their charisma but they realize that they can't expand or scale.
Their business is what it is, and they're happy with that. All I'm saying is that before you invest a lot of money or time into an idea, understand how big the tent can be.
The tent doesn't need to be huge to invest in it. If the tent is one man, one truck, one good life and you're happy, you go for it. It doesn't need to be “let's take over the world to be a good idea”.
You don't need to be number one in the podcast business to be a great idea that changes the world.
I don't want my book to be viewed as I'm only going to go after ideas that everyone in the world is going to purchase. It's not what I'm talking about. I'm talking about understand your market.
And if that market is big enough to make you happy, great, go for it. But if it's not, pivot to a slightly different idea or pivot to something that does meet your threshold for your time.
What I see happening is that people are overestimating the ability of their idea to change the world.
And then, they're investing in it accordingly with this false impression or this false dream that will just never come true.
And then, what you have is wasted years and wasted money for something that we could have figured it out much, much earlier.
That's my message.
Guy Kawasaki:
I'm going to tell you an Apple story. Well, I'll tell you the story.
The story is back when the Macintosh division was first coming out with Macintosh, eight-three, eight-four. Inevitably, every press person would ask, “What large companies are going to use Macintosh?” because they needed external confirmation that Macintosh was a good operating system.
And so, our story, our response to that question was always along the lines of, well, Peat Marwick, for example.
Peat Marwick is buying Macintoshes so that their onsite auditors can take a computer and do spreadsheets at the client. So, Peat Marwick is an example of a large well-known organization that's going to use Macintosh.
So, I'll just tell you that behind the scenes, Peat Marwick was the only company that was going to do that.
So, now, in Guy’s' perspective, we were faking it until we made it. We told people, there's Peat Marwick, for example, as if there were many other examples but it was literally only Peat Marwick.
So, was that wrong?
John List:
I don't think it was wrong in that if you didn't know that nobody else would follow Peat, which you didn't know, you had no idea. It's easy to say it now, thirty-seven years later, you can look back and say, "Maybe we should have known it was going to only be him who would do it."
But at the time, I am sure you are saying, "This dude is a visionary. And if Peat is going to do it, we think we can take over the world."
That's not faking it at all. That's taking a piece of market information and trying to use it in a way that you think will be helpful and that you can change the world with.
Look, Monday morning quarterbacking, when I see... my team is Packers, of course. I could look back on Monday morning and say, "I can't believe they ran that tight end out. They should have been running the fly there. They were dumb."
It's really easy ex-post to justify the way we were thinking. But ex-ante, it feels to me like you had some insights. You saw visionary was going to use it. You said, "Maybe there are million other Peats who are going to come forward and they're going to do it as well. We're going to blow up the world."
I don't call that a faking it at all.
Guy Kawasaki:
Okay. One more example of Peat Marwick being applied to your five vital signs.
So, with Peat Marwick, in order for Peat Marwick to use Macintosh for their auditors, we have to do a lot of stuff for them.
Help them write custom software, do training for them, all this kind of stuff, which frankly, could not scale. If every sale of a Macintosh took what we did for Peat Marwick, we would have sold a thousand Macintosh. It just could not happen.
So, one of your five vital signs is, “Are we depending on unscalable elements?” and in that case, we were. So, didn't we just violate the vital sign?
John List:
No, I think that's right. If the world would have stayed static, you're right. But sometimes the world changes and there are new markets that come up. And innovation can help you overcome that non-negotiable.
So, if your product is really good, maybe there's a third party that will come in and do a little bit of consulting to make it easier. So, then you're not providing it at a high cost, but you're creating another market that makes your good look more viable.
But if the world would have stayed static and you would have had to expend all of those resources to make this thing go larger, then absolutely, that's going to be a constraint on the amount of scalability of your idea.
But don't underestimate new markets and new innovations happening not only from yourself, but from third parties that can help attenuate these non-negotiables. Sometimes they don't.
Let me give you an example when they won't.
So, I started in early childhood program here in Chicago with Roland Fryer and Steve Levitt called the Chicago Heights Early Childhood Center.
And what we figured out is that we were really good at teaching three, four and five-year-olds when we had really good teachers. But we only had to hire thirty teachers. If we wanted to scale that thing in Chicago, we might have to hire forty or fifty or 60,000 good teachers.
It's one thing to hire a high quality teacher when you only have to hire thirty of them. It's an altogether different thing when you have to hire 30,000 of them.
So, that's going to be a constraint that in the very beginning, we should have tested with a bunch of really bad teachers, and said, "Do we have a good program with bad teachers?"
If so, that's awesome because that's going to be something that's scalable because we're going to be able to hire bad teachers in mass.
So, that's all I'm saying is that from the beginning of the research, kick it and figure out what are the constraints at scale, and what are the warts that we're going to have to withstand at scale? And if the idea still works with those warts, then you have a scalable idea on that dimension.
So, in a way, this is a revolution in that we typically talk about evidence-based policy. What I'm talking about here is reversing that and thinking along the lines of policy-based evidence. Look at the constraints at scale and bring it back to the petri dish.
Guy Kawasaki:
Wow. I mean, that really is a very interesting concept. Wow. I have to wrap my mind around that. It might take me the weekend to figure it out.
Now, we're coming up on an hour, but I got about four more questions that I'm just dying to ask you.
John List:
Sure. As long as you have time, let's keep going.
Guy Kawasaki:
As you can see, I'm really into this stuff.
John List:
No, I love it.
Guy Kawasaki:
Before I forget, I got to get one question off my checklist here, which is, let us talk about extrinsic versus intrinsic motivation for kids to study, because you fly in the face of a lot of people's thinking here.
So, what's the Gospel according to John about extrinsic motivation?
John List:
So, let's first of all start with my critics. So, I've been working in the world of public education for close to twenty years. And my laboratory in the world of public education has tended to be areas where there are deep structural problems in the communities.
And the educational, let's say, facilities, from K-Twelve are largely failing. And what I'm talking about here are your Chicago's, your LA, your Denver, your Atlanta, your New York City, your DC.
These are large urban settings that have fundamental problems in education.
So, John List goes into those settings and he explores ways in which he can induce both parents and kids to try harder. Because guess what, there's a fundamental problem.
In my high schools in Chicago Heights, maybe a thousand kids enter high school every year. It's a fundamental problem if fewer than 500 of them make it out with a high school degree.
It's a fundamental problem when a three, four, and five-year-old, the parents don't have time to give them the educational resources and inputs to help their cognitive and non-cognitive advances, or human capital acquisition.
So, this is a world that I enter. What I have found is that I can use extrinsic incentives to get people to try harder, whether it's teachers, parents, or the kids themselves. It works.
I use behavioral tricks like loss aversion. I use extrinsic incentives like giving people cash payments if they work harder, whether they're adolescents or young. It works.
Now, my critics come along and say, "You're bribing kids." They say, "It's not appropriate to bribe kids to do something they should be doing anyway."
And then, the next critic comes along and says, "You know what, John, when you use extrinsic incentives, you are crowding out their intrinsic incentives to do it."
I look at them, you know what I say, Guy? I say, "It's hard to crowd out a kid that has zero intrinsic incentives because you can't go lower than zero." When there's nothing to crowd out for some of these kids, it doesn't make sense to abide by conventional wisdom.
If it's not there, it's not there. We can use programs to try to give them intrinsic motivation. I do that, too. I have a lot of programs that are trying to push executive function skill development, with both the parents and the children.
But if it's just not there at baseline, what should you do? Say “Just use your intrinsic motives” or should you understand that there's a constraint on intrinsic motives?
So, I better do everything I can and use extrinsic motives to get better outcomes. Because once the children and the parents realize how to do things and how if I do work harder, I do get praise in school and I get things. This is self-reinforcing now.
Because once you have skills, you have to understand that skills beget skills, and it's a lot easier to learn after you have a better base.
So, if my critics can show me how to give kids intrinsic motivation or their parents, when it's not there, I want to do that, too.
But when it's not there, you have to resort to things like what we do with smoking. How do we get people to stop smoking? We do a mixture.
We do advertisements to say it's bad for your health, but we also have higher tax rates. Those higher taxes are pecuniary ways to get people to stop smoking.
But we also try to use knowledge, nudges, et cetera, and show them really terrible things, emphysema, or cancer, or really bad outcomes when you smoke.
In the end, I think it should be a mixture of extrinsic and intrinsic motives, and that's what I do as a social scientist.
Guy Kawasaki:
So, cutting to the chase, if I say to my son, "If you get at least a 3.5 GPA, I will buy you a car."
Is that extrinsic? Well, that's obviously extrinsic. But is that a bad idea?
John List:
It all depends on a few features. So, let me start with number one.
Do you think that your son has enough intrinsic motivation to start whereby if you come in with that, if in five years, if that extrinsic reward isn't there, is your son then not going to perform because you gave him the car five years before?
If so, then you should be very careful because life doesn't always present us with extrinsic motives.
So, if your son or child has intrinsic motivation to begin with, you should always be careful because I do believe that you could crowd these things out, and you could not only crowd them out during the task, but in the future.
It's potentially that it's like Pavlov's dog.
If you condition them to only doing things for an outside reward and then they realize they don't have it and if they don't give effort in that case, you've just undone all the good stuff that you were trying to do.
So, to me, a lot of it is what is the baseline level of intrinsic effort that your son will put forward? And do you think the extrinsic reward will not only crowd out the intrinsic reward during the task but also after? And then, that's a calculus that you need to go through.
But if you're going to do it, I'm going to urge you to do something slightly different. I'm going to have you give your son the car to use, and then you're going to tell your son, "I'm going to take away that car if you don't achieve."
That's going to be a much higher reward. That's called the clawback and that's going to give you much higher output if you want to go that route.
Okay, Guy?
Guy Kawasaki:
Okay. Of course, you understand I'm asking for a friend. This is not my family.
John List:
Absolutely, just a friend.
Guy Kawasaki:
Okay. Yeah, I know somebody who we touched on this earlier and I chose not to dive into it because we could have really gone down a rat hole, but we talked about vaccination and how it's succeeded.
So, what's your analysis of vaccination today?
John List:
So, I do talk a bit about how we should think about scaling around COVID. And in the book, what I argue is there's a lot of blame to be placed all around. The president made a fair number of mistakes.
But I think there were other-
Guy Kawasaki:
Which president?
John List:
... political mistakes made. President Trump. I think the current president makes some mistakes now too, but these aren't easy questions. I'm not trying to be unfair to people.
I was part of some of the early ways to think about, first of all, the innovation, and what should be the incentives that the government gives Pfizer, and Moderna, and Johnson to actually innovate. And then, after the innovation, how should we think about the rollout?
How should we think about incentives around the rollout? And also, during the process with wearing masks and how we should think about public safety precautions, there were a lot of good things that were done.
There were a lot of mistakes that were made as well. I think the minute we politicized vaccinations, it became a much deeper problem than if we would have just said this is a public health issue and it's an individual choice.
And we need to figure out how can we leverage the health care system to get as many people vaccinated as possible. That's been our biggest failure is the rollout in terms of making the vaccinations both available and in high demand.
And we failed on the high demand side the minute we politicized it because in this polar world, we then get vaxxers and anti-vaxxers. And then, right away, it becomes a question of “Whose side are you?” on rather than “Let's do this for public health.”
So, right away, we're off to the wrong start. Now, let's just pretend that everyone said they wanted to get vaccinated. There still needs to be a way that we roll out the vaccinations in the public health system so it is, let's say, costless as possible.
Because you can't make getting the medicine in people's arms. The minute you start to add a friction, or a cost to it, that's when you start to lose people.
My work in medication adherence, it's really hard to get people to take their meds, even when they know they should. There's a problem I've worked on with Humana and with some drug companies. People have the medication sitting at their bedside and they still don't take it because they need to be reminded. Then, we do reminder text.
Now, the nice thing about the vaccination is you don't need to do it every day. You just need to get people to do it once and then maybe you have to do it in six or nine months again.
So, it's an easier problem, but it's a harder problem in that the needle isn't sitting by their bedside.
And when they put their phone up to their head, they don't get the vaccination. So, we still need to get them to go to the Walgreens, or the CVS, or the Walmart or the doctor to do it.
And that's a part where I don't think we've quite figured out the suite of behavioral interventions and the suite of economic incentives that are at the sweet spot of getting everyone to do it in a cost effective way. And I think scientists were working through that.
Some states are doing high value and lottery incentives. So, this is where we have failed in that when you look at these nationwide policies that have worked in the past, so take yourself back to the New Deal. Most of the ideas of the New Deal actually came from Governor Al Smith in New York.
Roosevelt took a lot of Al Smith's ideas. And that's a great part about a federation is you have a bunch of states trying different things.
And then, when a state finds itself upon something really successful, you can ramp that up to the national level. That's what Justice Brandeis talked about when he said this is laboratories of democracy. We should allow all these laboratories to give it a try.
And then, what works, we take up to the national level or other states. Use what works.
I don't think our federal government has incented states in an optimal way to try innovative things to get the medicine in people's arms. That's where one global area where I think we failed.
Guy Kawasaki:
I want to end with one last question and this is because it is so important to expand people's minds about the simplicity of what many people have concluded, which is you have a very different take on things like racism, and gender pay gap, that it's not sexism and animosity and hatred and racism as we traditionally define it, but it's more of a different kind of discrimination.
So, can you explain that?
John List:
So, when you think about a complex problem like “Why are people treated differently based on race?” or “Why are women treated differently than men in labor markets?” and in particular, “Why do women earn less money than men?” These are complex problems that are multi-dimensional.
But one set of features that people oftentimes don't think about are that on the one hand, preferences and constraints.
So, the differences in preferences between men and women and the differences in constraints between men and women, in many cases are important variables that cause the gender pay gap.
Let me give you an example.
So, some colleagues and I did a large scale piece of empirical research with Uber drivers. And what we find is that men earn about 7 percent more per hour than female Uber drivers. And a lot of times, people would say that's because of discrimination.
So, here, the discrimination could be because of customer discrimination or it could be because Uber dispatch was discriminating against women. We learned that neither of those kinds of discrimination exist in our data, that it wasn't customer discrimination.
In fact, if there was anything, the customer discrimination went the other way because people tip female drivers more than they tip male drivers. And we also found that there wasn't dispatched discrimination.
So then, you can say there's no discrimination but there's still a 7 percent pay gap.
Now, you could say maybe it's because of bargaining. A lot of times, women don't bargain as much as men. In some settings, that's true but at Uber, there's no bargaining. Men and women are paid the exact same rate card.
You're paid by your time that you have in terms of driving a person from point A to point B, and the distance. So, it's time and distance. Men and women are paid the same.
Okay. So, we take off that explanation.
So, you can say, "My gosh, what is happening here?" And the reason why I say it's preferences and constraints is because there are three factors that determine this gender pay gap.
One factor is where and when people drive.
So, in a lot of cases, women don't drive during the morning rush hour. They don't drive during the morning rush hour in many cases because they have to take care of the kids and so that's a constraint in their life.
They also don't drive at bar time.
So, at bar time, it's a very lucrative time at one to three in the morning. Women don't like to drive there because it's more dangerous for women to drive at night. So, that's also a constraint.
Now, what we also find is that the features of how often they drive matters.
So, if you gain a lot of experience driving, there's a learning effect to driving. So, when you drive more hours or take more trips, you end up learning where and when to drive. So, there's a learning effect.
So, these two things are 50 percent of the gender pay gap. The other 50 percent is how fast men and women drive. So, men drive faster than women.
And because they drive faster, they're able to take more trips per hour. That's the other 50 percent of the pay gap and that's a preference. Because when you look in general at men and women and how they drive, even when they're not driving for Uber, men drive faster than women.
So, it's not being an Uber driver that's causing it. It's just a preference for taking on risk. So, in this case, I want the readership and the listeners to know that there's no animus here, there's no discrimination. It's your preferences and constraints that are leading to this gender pay gap.
And these are features that occur in every labor market. So, we need to start understanding how important are those features, preferences and constraints, to the gender pay gap.
Guy Kawasaki:
Can you apply your thinking also to racism?
John List:
I believe so. I believe racism is one part animus.
In terms of racism, let's be clear that there are racists, and there are bigots, and that should not be undervalued, but it's not the whole story.
Part of the story is that in some cases, people are acting based on preferences and constraints and profit motives. If you need to understand both, you need to understand what would be called taste-based discrimination or bigotry.
That's important.
But you also need to understand preferences and constraints and profit motives.
You won't understand the complete reasons why we have discrimination, whether it's for gender, age, race, sexual orientation until we understand all of the motivations that are going on in these markets.
Guy Kawasaki:
There you have it, John A. List. What a diverse and fascinating interview.
We covered the Green Bay Packers and Vince Lombardi, Phil Zimbardo's Prison Experiment, flavors of jelly, GFCI versus circuit breakers, the gender pay gap, and last but not least, how to help your friends motivate their kids to study, all of this in one mere episode of Remarkable People.
I'm Guy Kawasaki.
My thanks to Jeff Sieh, Peg Fitzpatrick, and Shannon Hernandez, the GFCIs of the Remarkable People podcast.
And thanks to the rest of the team, Alexis Nishimura, Luis Magana, and the drop-in queen, Madisun Nuismer.
This is the team that makes Remarkable People remarkable. Mahalo and Aloha.