Author of “Ultralearning: Master Hard Skills, Outsmart the Competition, and Accelerate Your Career,” Entrepreneur, and Programmer

1Huddle Podcast Episode #88

On this Bring It In podcast episode, 1Huddle’s CEO and Founder Sam Caucci sat down with Scott Young, author of the book Ultralearning: Master Hard Skills, Outsmart the Competition, and Accelerate Your Career. He is also a programmer and entrepreneur, known for The MIT Challenge and The Year Without English.

On this episode of Bring It In season three, Scott sat down with Sam and discussed desirable difficulties, retrieval practices, and effective feedback.

Audio available on Apple Podcasts, Spotify, and Google Podcasts. 

TOP 3 HIGHLIGHTS

Below are some of the insights Dr. Lynch shared during our chat, edited for length and clarity. You can find more Bring It In podcast episodes here.

• “Increasingly the difficulty of the training has a more beneficial effect on long-term learning.”
• “I hope education and training have more options.”
• “Your benefit in terms of your ability to remember something (or recall) is inversely related to how difficult it was to retrieve.”

FULL TRANSCRIPTION

Sam: So Scott, Ultra Learning, why’d you write it? 

Scott: Well, I wrote it because, not only myself, but I’ve met all these people that have had these cool experiences, learning difficult subjects, difficult skills in somewhat unconventional ways, often not going to school, not doing the normal thing that you do to learn it.

And I wanted to share those stories because I think we live in a time where we know that these training and education experiences are getting harder to, more expensive, and people want other alternatives for getting good at things. 

Sam: When I was reading through the book, the premise of these sort of nine universal principles that you talk about, how did you come across those?

Scott: Well, I mean, again, they’re sort of drawn from this sort of pastiche of different stories and personal experiences. So some of these just seem to be true to me, and then others are based directly on cognitive science research. So I had a chapter on retrieval and even that word is straight up plucked from cognitive science.

And it was something that, when I encountered the research, I was like, why don’t students know about this? Why don’t they know? Because they’re spending a lot of time studying in a way that we know to be inefficient. And so I think there’s a sort of a nice mixture of the practical hands-on derived from doing it, and also, there’s a lot of cognitive science out there that tells us how we learn things. And so I think both need to be appreciated. 

Sam: It’s hard to explain things like retrieval to people, or the concept that, when I found myself talking about some of the concepts, you’ll get somebody who says, yeah, but where does the learning happen? If I’m tested on something or I’m forced to recall something, they’ll say things like, don’t I have to learn it first before I’m challenged by a topic? How do you make it easy to communicate?

Scott: Well, I think you’re right. I definitely think retrieval, if there’s nothing to retrieve, doesn’t always work. There is some sort of interesting kind of speculative evidence of having some initial failures. So you try some problem that you don’t actually have the tools to solve. Maybe it helps with some processes of learning later. That’s a little bit of a kind of an on-the-edge, controversial point, but what’s not controversial is the idea that you read something once, but as we all know, reading something once does not make you an expert in it. What would you do to build that knowledge so that you would be able to answer more difficult questions about it later? And this is what students do when they’re studying. They’ve sat through the class. They’ve hopefully read maybe some of the reading assignments, but that doesn’t mean they’re gonna get an A on the test. What should they do with that extra time? 

And what the retrieval practice research shows is that after you’ve had some exposure to the material, it’s far better to spend the time with the book closed, trying to recall the material, and especially if you can then later go back and check whether you got it right, that’s gonna be much more efficient than just reading it over and over and over again. So, I really like Robert Bjork’s Theory of Retrieval and Storage Strength here, but basically just argues that your benefit in terms of your ability to remember something or recall something is sort of inversely related to how difficult it was to retrieve.

So if you have something that’s like, you’ve just seen it and then you practice retrieving it, you’ll be successful at it, but there won’t be a corresponding increase in that retrieval strength later. So that sounds really technical, but I think the basic idea here is that you wanna be engaged in some kind of practice.

You wanna be engaged in a practice where you’re getting feedback. And hopefully also, as I said in the beginning, that you know what you’re doing, so you’re not just guessing, but that’s very important for studying all sorts of things, especially things like book topics and things from lectures and stuff like that, where very often the default training strategy is just to just kind of skim it over again.

Sam: You know, a lot of the folks listening are sitting in environments where they have to onboard workers quickly. It could be a restaurant, could be a retailer, it could be a hotel. And given this moment where the race back to work is real, even the moment that workers are shifting and switching jobs, that saving time and that onboarding motion is important. What specific recommendations would you have to someone that’s a manager or responsible for onboarding? Maybe a few strategies they can implement tomorrow that would accelerate and make their onboarding exercise more effective? 

Scott: Yeah. I mean, well, we just talked about one right here, which is that if you’re gonna give people material that you want them to understand in a very basic way, this could be like standard operating procedures or it could be things that are more sophisticated than that, if you’re having accountants or programmers, maybe there’s some design ethos that you want them to follow. Then it’s pretty clear that what we want to do is to have some kind of testing of the people we want to not just give the material, just read over this book and then go apply it, but we should try to have little tests to see whether or not they’ve learned it.

And I think the way that we often think about tests is as a way of measuring what has been learned. But this research just clearly shows that they’re very much a way of encouraging learning as well. And so I think, probably, one of the things that I would shift to, if I were designing these kinds of systems, is ones where you present the material in small chunks, and then you provide kind of repeated testing of that material throughout this sort of training period. And that not only lets you know what people are remembering and recalling from the situation, but it actually strengthens their memories. So they’re more likely to remember it and use it later.

Sam: You use the term ‘desirable difficulties in the book. What is that? 

Scott: So desirable difficulties again is Robert Bjork’s concept and he points to a few areas where there can be, what’s called a disconnect between learning and performance. So when you train people to do something, what you’re measuring is their performance on the tasks. So you get them to do something and then you see they have some performance curve, whereas learning is supposed to be the kind of invisible latent, like how much better have they gotten in general? And that’s not what you’re measuring, you’re measuring their performance. 

And what they’ve noticed is that sometimes there can be a bit of a paradox that things that make performance worse, meaning that you seem to learn worse in the short term under which the training takes place actually results in much better performance, either at a longer time horizon or for a larger variety of tasks and things like this.

So he lists a few of them. One of them is the spacing effect, which is basically that if you spread out the intervals that you make someone practice something, they will remember it much longer than if you clump them together, but if you clump them together, then immediately after, which is presumably when you’re testing them, they’ll do really well. It’s just, they forget it really quickly. 

Similarly, there’s one about interleaving or contextual interference, which is the idea that, let’s say there’s 10 different things that you wanna get someone to learn one way would be teach them task, a, teach them task B, teach them task C, just down the line until they’ve gotten it. And then you move on to the next one. Whereas interleaving would be, you have to do a little bit to make sure that they can at least get it, but then when you’re actually testing it, you do A, B, C, C, A B, and you kind of mix them up and maybe even in a random order. So they can’t predict what they’re gonna be tested on.

Again, performance is gonna be low immediately after this thing or lower than it would be if you do AAA, BBB, CCC as these testing paradigms are. They’re gonna be able to remember it longer, and they’re also gonna be able to transfer it better to actually use it in the real situation. And part of this is because real situations don’t come masked out as tasks AAA, BBB, CCC. They are a big jumble of things, and if you can’t discriminate which thing you need to do, you’re not gonna be very effective at it. So there’s a few others that he talks about as well. I know there’s some research on levels of feedback, having an influence is a desirable difficulty, there’s also some stuff on, some of these other little things that you can adjust.

I don’t wanna say all difficulties are desirable. You can definitely make training a lot harder for no benefit. For instance, by not providing any instructions or any guidance for how to do something, then it’s a lot of trial and error, but it just shows that, in some circumstances increasing the difficulty of the program, which drops a performance, has a much longer and more beneficial effect on your long term learning.

Sam: Yeah. It’s like when I started in sales and I had that sales manager that I hope I never see again do the, ‘can you sell me this pen’ exercise. It’s been like made famous and it was just really hard just for the sake of being hard. 

Scott: Well, I think you bring up a good point. I think that, especially when we have people in a training context and work often, one of the problems is that we don’t explain nearly enough. So there’s little gaps in how you perform this task that we’re expecting employees, new people to figure out. And if they have to figure it out, they just have to try things at random until they get something that works, which can waste a lot of time and it can be dangerous if the things they’re trying out are bad for the company.

And so I definitely think that the more you can provide structure and guidelines so that someone would know that this is the right way to do this particular type of thing, or these are good ways to do this kind of thing. The more they can hone in on that, the better they’re gonna be. So definitely that’s an undesirable difficulty. Kind of like, ‘all right, I’m not gonna give you any help. Prove your worth to me.’ I mean, maybe if you’re trying to filter candidates on raw intelligence, that’s one thing to do, but if you’re actually trying to train people to do well, then it definitely helps to provide more structure and guidance.

Sam: Do you have a perspective on the right amount of positive recognition or positive reinforcement? Cuz you’re talking about a lot of failure and going through that process. What’s the right amount as a manager, or how important is that in this process? 

Scott: Yeah. So it’s interesting. The research on feedback, one of the studies that I reviewed in Ultra Learning, found it actually in a nontrivial amount, I think it was like in 37% of the studies, giving people feedback actually had a negative effect. So we tend to think of feedback as good, but it can backfire, and there’s different ways it can backfire. One of the main ways it can backfire is what you’re talking about. You give someone negative feedback, they take it personally, and then they don’t wanna work as hard or they shut down on the task. And I think this can be particularly true in workplace settings. I talked to one of the authors of that paper and he was saying that you have to be very careful, when they study feedback in the workplace, that often even employees who are seeking feedback are not really seeking feedback, they’re seeking validation.

They want someone to give ’em a gold star and say they’re doing a good job. And so if they actually ask for feedback and you give some criticism, that wasn’t what they wanted and they kind of shrink away. And so all the research I’ve read on feedback says that it works much better the more it can be corrective and applied to a particular task. So it’s not an evaluation of, this is what your performance is like, ‘but when you do this, maybe try doing it this way, this kind of thing.’ And then that way they can apply it. It’s helpful, as opposed to, ‘you know what, you’re not very good at this kind of work,’ and that’s it. 

Sam: What have you seen regarding the timing of feedback? I’ve heard some folks say that immediate feedback is helpful, but in some cases maybe delayed it would be more productive, I guess. How do you know when the right moment to give feedback is?

Scott: So again, I think the research tends to be more in favor of more immediate feedback, just because there’s just a general timing issue. If you wait two weeks, whatever decision you made that resulted in the feedback is going be faded from memory, it’s gonna be harder to control that. There are some studies that show that delayed feedback has some beneficial effects, but I usually think of those in terms of a spacing effect. So they’re probably because by spreading something out, as opposed to giving immediate feedback, you get multiple exposures to the idea spread out in time. so I think for most practical situations, immediate feedback is probably gonna be better. And I think that’s especially true if you’re dealing with some kind of control type task where you’re not quite sure how to do this and you’re making it and you’re making all these decisions. You wanna get feedback quite soon. 

Now how much feedback you need to give and how immediate it is is a little bit like there’s some experiments where, if you’re doing some kind of online tutoring system, getting people to not correct their errors instantly helps them spot when they’re making mistakes and self correct, so there can be some benefit there, but I think in a practical workplace context, the problem is overwhelmingly that feedback is either non-existent, super delayed out in time, or directed at the individual and not at the specifics of their actual behavior performance. 

Sam: Interesting. You had a lot of stories that you shared, and I’m sure you have a lot more you are saving for the next book or weren’t able to fit in. What was your favorite story in the book? 

Scott: Well, I think the favorite one that I had was because I kind of got to witness it first hand was Tristan de Montabello. And he was someone who took on a little project to get better at public speaking. And it was very much something that I was on the ground floor of. He’s like, ‘I kind of wanna do this, I know you’re writing this book about learning. Maybe I could be like a little guinea pig and you could kind of coach me through this.’ And I was like, oh, okay. And he just had such a phenomenal result. He went from having very little public speaking experience, like maybe giving a handful of speeches in his life, to being in the top 10 for Toastmasters world competition, which I think has something like 30,000 people or something to compete elimination style. So that means that he made it through all the rounds to get to the top 10 on the stage there.

And I mean, obviously his success is quite dramatic. I don’t wanna say that everyone could achieve that result if they read my book or something like that. But I do think that his case illustrates a lot of the principles of this kind of obsessive commitment to getting better at things, getting the clear feedback. A lot of the little details of getting these projects right I think was embodied in how he did it. So I am obviously very impressed and surprised that he got that far in a short period of time, but with his work ethic and how he was approaching it, I’m not surprised that he got a lot better at public speaking quickly.

Sam: What was the most surprising thing that you heard after the book came out and you started to get feedback from folks? They were reading it, what was the most surprising piece of feedback that maybe you didn’t expect? 

Scott: Surprising is maybe the wrong word. It makes it sound like it was totally unexpected, but one of the things I’ve really found from talking to people is how much emotions play a role in learning beyond just cognitive stuff. I’ll have these conversations with students sometimes where they’ll be like, ‘what’s the optimal way to schedule this,’ blah, blah, blah, blah. And then you like drill down and you have like these email conversations go back and forth over a couple weeks, and then you drill down and you’re like, oh no, no, they’re not actually doing that much studying because they have this like, anxiety that they’re gonna like fail and it’s gonna ruin their whole life. And so the real problem is not like, what is the right way to schedule flashcards or something. The real problem is that they have this kind of anxiety that keeps them from actually putting in significant amounts of studying time. 

Or people who will be like, I’ve met people who they’re living in another country, for instance, and they’ve been there for years, decades even, and they don’t speak the language, even though it’s a significant handicap there, just like going about their daily life. And it’s not because they don’t have some strategy for learning the language, it’s because they have some psychological block to regularly practicing it and getting in situations where they see, ‘yeah, I’m not that good. I would be better in English, but I can regularly use this.’ And so I think that’s something that, not to say that I neglected it in Ultra Learning, but it’s a dimension of learning I think generally that how much emotions impact it and how much people’s stories they tell themselves about who they are as a learner, what they’re good at, what they’re not good at, how those can really overwhelm any kind of, on the ground, materially, you have a talent, you don’t have a talent, you can learn this, you can’t learn this kind of facts.

Sam: I think that’s a really important point and very relevant right now for talent leaders who have workforces that are made up of a lot of low wage frontline workers. You’re talking about emotions and trauma coming from different educational experiences and now getting a job that maybe, the work that makes up that job’s gonna shift a lot. I think that point hits home for me, cuz I’d imagine that in the years to come the functions that make up jobs are gonna shift rapidly. And I think we need everybody to have the ability to be an ultra learner in order to respond quickly enough. 

Scott: Yeah. And I think as soon as you deal with a social context, there’s so much more than just the intellectual aspect of it. I think whether or not you build the skills that you need depends on what kind of training experiences you have at work, who your peers and mentors are, whether you’re put on projects where you actually have some responsibility and you actually have to like execute on it, or whether you’re relegated to the sidelines or whether you self relegate to the sidelines, because you’re worried about the judgment. And so I think those kinds of social and emotional factors are huge.

And so I definitely think that one sort of element of ultra learning I think that if I were to try to pursue in another book would be some of these aspects just because, as I said, the idea that if you spend a lot of time having real conversations with people in a language you’ll get better at the language is a point nearly everyone accepts. The part that people had difficulty with or that they kind of were scared of is like, you’re gonna go somewhere and you’re gonna be mediocre in this language and you’re just gonna only speak it and commit to it from the beginning is the part that sounds terrifying though. And that’s an effective method. And so I think that’s one of the ways is like, how do you manage those emotions? How do you choose projects so that you can feel like you have a grasp on them and stuff? It’s a big part.

Sam: Sure. Scott, last question. We’re talking a lot about the future of work. These principles and concepts have a direct application today, more than ever. What’s your hope for the future of work?

Scott: Well, I think what I would really hope for is for education and training to have more options. I think the way we’ve structured our society right now is funneling everyone through a four-year degree and now increasingly also masters in professional certification. And it’s no doubt true that a lot of the stuff that is taught in those places is helpful, but at the same time, they often become this barrier so that well-qualified people, people who could succeed, can’t go through that bottleneck. And so they’re stuck in jobs that are much below their potential. And so I’m really hopeful that in the future we’re gonna have more varieties of options for training and education and for people to acquire skills that employers will recognize and say, ‘oh yeah, this person is good. We can hire them to do this job because we can see on their resume that they did this, even if it wasn’t the sort of standard path.’ And so Ultra Learning is not really an answer for everyone, but I hope that it’s an answer for some people.

Sam: Great stuff, Scott, thanks for spending time with us.

Topics Discussed: Future of Work, Education, Training, Learning, Retrieval