The act of remembering can change the memory.
If you remember some event and then I tell you something about the event, later on when
you remember the event, you might remember some original information from the event as
well as some information about what I told you.
And sometimes if you're not able to tell the difference, that information that I told you
gets mixed into the story that you had originally.
So now I give you some more misinformation or you're exposed to some more information,
somewhere else.
And eventually your memory becomes totally detached from what happened.
The following is a conversation with Charon Ranganath, a psychologist and neuroscientist
at UC Davis, specializing in human memory.
He's the author of Why We Remember, Unlocking Memory's Power to Hold On to What Matters.
This is the Lex Friedman Podcast.
To support it, please check out our sponsors in the description.
And now, dear friends, here's Charon Ranganath.
Danny Conwin describes the experiencing self and the remembering self.
And that happiness and satisfaction you gain from the outcomes of your decisions do not
come from what you've experienced, but rather from what you remember of the experience.
So can you speak to this interesting difference that you write about in your book of the experiencing
self and the remembering self?
Danny really impacted me because I was an undergrad at Berkeley and I got to take a class from
him long before he won the Nobel Prize or anything.
And it was just a mind-blowing class.
But this idea of the remembering self and the experiencing self, I got into it because it's
so much about memory, even though he doesn't study memory.
So we're right now having this experience, right?
And people can watch it presumably on YouTube or listen to it on audio.
But if you're talking to somebody else, you could probably describe this whole thing in
10 minutes.
But that's going to miss a lot of what actually happened.
And so the idea there is that the way we remember things is not the replay of the experience.
It's something totally different.
And it tends to be biased by the beginning and the end, and he talks about the peaks,
but there's also the best parts, the worst parts, et cetera.
And those are the things that we remember.
And so when we make decisions, we usually consult memory, and we feel like our memory is a record
of what we've experienced, but it's not.
It's this kind of very biased sample, but it's biased in an interesting and I think biologically
relevant way.
So in the way we construct a narrative about our past, you say that it gives us an illusion
of stability.
Can you explain that?
Basically, I think that a lot of learning in the brain is driven towards being able to make
sense.
I mean, really, memory is all about the present and the future.
The past is done.
So biologically speaking, it's not important unless there's something from the past that's
useful.
And so what our brains are really optimized for is to learn about the stuff from the past
that's going to be most useful in understanding the present and predicting the future, right?
And so cause-effect relationships, for instance, that's a big one.
Now, my future is completely unpredictable in the sense that you could, in the next 10 minutes,
pull a knife on me and slit my throat, right?
I was planning on it.
Exactly.
But having seeds of your work and just generally my expectations about life, I'm not expecting
that.
I have a certainty that everything's going to be fine and we're going to have a great time
talking today, right?
But we're often right.
It's like, okay, so I go to see a band on stage.
You know, I know they're going to make me wait.
The show's going to start late and then, you know, they come on.
There's a very good chance there's going to be an encore.
I have a memory, so to speak, for that event before I've even walked into the show, right?
There's going to be people holding up their camera phones, trying to take videos of it
now because this is kind of the world we live in.
So, that's like everyday fortune telling that we do, though.
It's not real.
It's imagined.
And it's amazing that we have this capability and that's what memory is about.
But it can also give us this illusion that we know everything that's about to happen.
And I think what's valuable about that illusion is when it's broken, it gives us the information,
right?
So, I mean, I'm sure being an AI, you know about information theory.
And the idea is the information is what you didn't already have.
And so, those prediction errors that we make based on, you know, we make a prediction based
on memory and the errors are where the action is.
The error is where the learning happens.
Exactly.
Exactly.
Well, just to linger on Danny Kahneman and just this whole idea of experiencing self versus
remembering self, I was hoping you can give a simple answer of how we should live life
based on the fact that our memories could be a source of happiness or could be the primary
source of happiness.
That an event, when experienced, bears its fruits the most when it's remembered over and over and over and over.
And maybe there is some wisdom in the fact that we can control to some degree how we remember it,
how we evolve our memory of it, such that it can maximize the long-term happiness of that repeated experience.
Okay, well, first I'll say, I wish I could take you on the road with me because that was such a great description.
Can I be your opening act?
Oh my God, no, I'm going to open for you, dude.
Otherwise, it's like, you know, everybody leaves after you're done.
Believe me, I did that in Columbus, Ohio once.
It wasn't fun.
Like, the opening acts, like, drank our bar tab.
We spent all this money going all the way there.
There was only the, everybody left after the opening acts were done.
And there was just that stoner dude with the dreadlocks hanging out.
And then next thing you know, we blew, like, our savings on getting a hotel room.
So we should, as a small tangent, you're a legit touring act.
When I was in grad school, I played in a band.
And yeah, we traveled, we would play shows.
It wasn't like, we were in a hardcore touring band, but we did some touring and had some fun times.
And yeah, we did a movie soundtrack.
Nice.
Henry Portrait of a Serial Killer.
So that's a good movie.
We were on the soundtrack for the sequel, Henry II, Mask of Sanity, which is a terrible movie.
How's the soundtrack?
It's pretty good.
It's badass.
At least that one part where the guy throws up the milkshake, which is my song.
We're going to have to see.
We're going to have to see it.
All right, we're getting back to life advice.
And happiness, yeah.
One thing that I try to live by, especially nowadays, and since I wrote the book, I've been thinking more and more about this, is how do I want to live a memorable life?
I think if we go back to the pandemic, how many people have memories from that period, aside from the trauma of being locked up and seeing people die and all this stuff?
I think it's like one of these things where we were stuck inside, looking at screens all day, doing the same thing with the same people.
And so I don't remember much from that in terms of those good memories that you're talking about, right?
You know, when I was growing up, my parents worked really hard for us.
And, you know, we went on some vacations, but not very often.
And I really try to do now vacations to interesting places as much as possible with my family because, like, those are the things that you remember, right?
So I really do think about what's going to be, like, something that's memorable and then just do it even if it's a pain in the ass because the experiencing self will suffer for that, but the remembering self will be like, yes, I'm so glad I did that.
And do things that are very unpleasant in the moment because those can be reframed and enjoyed for many years to come.
That's probably good advice, or at least when you're going through shit, it's a good way to see the silver lining of it.
Yeah.
I mean, I think it's one of these things where if you have, like, people who you've gone through, since you said it, I'll just say, since you've gone through shit with someone.
And it's like, that's a bonding experience often, you know?
I mean, that can really bring you together.
I like to say it's like there's no point in suffering unless you get a story out of it.
So in the book, I talk about the power of the way we communicate with others and how that shapes our memories.
And so I had this near-death experience, at least that's how I remember it, on this paddleboard where just everything they could have gone wrong did go wrong almost.
So many mistakes were made and ended up, like, at some point just, like, basically away from my board, pinned in a current, like, in this corner, like, not a super good swimmer.
And my friend who came with me, Randy, who's a computational neuroscientist, and he had just been pushed down past me, and so he couldn't even see me.
And I'm just like, if I die here, you know, I mean, no one's around.
It's like you just die alone.
And so I just said, well, failure is not an option.
And eventually, I got out of it and froze and got cut up, and I mean, the things that we were going through were just insane.
But a short version of this is, you know, my wife and my daughter and Randy's wife, they gave us all sorts of hell about this because they were just, like, where are we?
They were ready to send out a search party.
So they were giving me hell about it, and then I started to tell people in my lab about this and then friends, and it just became a better and better story every time.
And we actually had some photos of just the crazy things, like this generator that was hanging over the water, and we're, like, ducking under these metal gratings, and I'm, like, going flat.
And it was just nuts, you know?
But it became a great story, and it was definitely, I mean, Randy and I were already tight, but that was a real bonding experience for us.
And, yeah, I mean, and I learned from that that it's, like, I don't look back on that enough, actually, because I think we often, at least for me, I don't necessarily have the confidence to think that things will work out, that I'll be able to get through a certain thing.
But my ability to actually get something done in that moment is better than I give myself credit for, I think.
And that was the lesson of that story that I really took away.
Well, actually, just for me, you're making me realize now that it's not just those kinds of stories, but even things like periods of depression or really low points.
To me, at least, it feels like a motivating thing that the darker it gets, the better the story will be if you emerge on the other side.
And that, to me, feels like a motivating thing.
So maybe if people are listening to this and they're going through some shit, as we said, one thing that could be a source of light is that it'll be a hell of a good story when it's all over, when you emerge on the other side.
Let me ask you about decisions.
You've already talked about it a little bit, but when we face the world and we're making different decisions, how much does our memory come into play?
Is it the kind of narratives that we've constructed about the world that are used to make predictions that's fundamentally part of the decision making?
Absolutely. Absolutely. Yeah. So let's say after this, you and I decided we're going to go for a beer, right?
How do you choose where to go? You're probably going to be like, oh, yeah, this new bar opened up near me.
I had a great time there. They had a great beer selection.
Or you might say, oh, we went to this place and it was totally crowded and they're playing this horrible EDM or whatever.
And so right there, valuable source of information, right?
And then you have these things like where you do this counterfactual stuff like, well, I did this previously, but what if I had gone somewhere else and said, maybe I'll go to this other place because I didn't try it the previous time.
So there's all that kind of reasoning that goes into it, too.
I think even if you think about the big decisions in life, right, it's like you and I were talking before we started recording about how I got into memory research and you got into AI.
And it's like we all have these personal reasons that guide us in these particular directions.
And some of it's the environment and random factors in life, and some of it is memories of things that we want to overcome or things that we build on in a positive way.
But either way, they define us.
And probably the earlier in life the memories happen, the more defining power they have in terms of determining who we become.
I mean, I do feel like adolescence is much more important than I think people give credit for.
I think that there is this kind of a sense like, you know, the first three years of life is the most important part.
But the teenage years are just so important for the brain, you know.
And so that's where a lot of mental illness starts to emerge.
You know, now we're thinking of things like schizophrenia as a neurodevelopmental disorder because it just emerges during that period of adolescence and early adulthood.
So, and I think the other part of it is, is that, you know, as I guess I was a little bit too firm in saying that memory determines who we are.
It's really the self is an evolving construct.
I think we kind of underestimate that.
And when you're a parent, you feel like every decision you make is consequential in forming this child.
And it plays a role, but so do the child's peers.
And so do, you know, there's so much, I mean, that's why I think the big part of education, I think that's so important, is not the content you learn.
I mean, think of how much dumb stuff we learned in school, right?
But a lot of it is learning how to get along with people and learning who you are and how you function.
And, you know, that can be terribly traumatizing, even if you have a perfect, you know, parents working on you.
Is there some insight into the human brain that explains why we don't seem to remember anything from the first few years of life?
Yeah, yeah.
In fact, actually, I was just talking to my really good friend and colleague, Simona Getty, who studies the neuroscience of child development.
And so we were talking about this.
And so there are a bunch of reasons, I would say.
So one reason is there's an area of the brain called the hippocampus, which is very, very important for remembering events or episodic memory.
And so the first two years of life, there's a period called infantile amnesia.
And then the next couple of years of life after that, there's a period called childhood amnesia.
And the difference is that basically in the lab and, you know, even during childhood and afterwards, children basically don't have any episodic memories for those first two years.
The next two years, it's very fragmentary.
And that's why they call it childhood amnesia.
So there's some, but it's not much.
So one reason is that the hippocampus is taking some time to develop, but another is the neocortex.
So the whole folded stuff of gray matter all around the hippocampus is developing so rapidly and changing.
And a child's knowledge of the world is just massively being built up, right?
So, I mean, I'm going to probably embarrass myself, but it's like if you showed like, you know, you trained a neural network and you give it like the first couple of patterns or something like that, and then you bombard it with another like, you know, years worth of data, try to get back those first couple of patterns, right?
It's like everything changes.
And so the brain is so plastic, the cortex is so plastic during that time.
And we think that memories for events are very distributed across the brain.
So imagine you're trying to get back that pattern of activity that happened during this one moment, but the roads that you would take to get there have been completely rerouted, right?
So I think that's my best explanation.
The third explanation is a child's sense of self takes a while to develop.
And so their experience of learning might be more learning what happened, as opposed to having this first person experience of how I remember I was there.
Well, I think somebody once said to me that kind of loosely philosophically that the reason we don't remember the first few years of life, infantile amnesia, is because how traumatic it is.
Basically, the error rate that you mentioned, when your brain's prediction doesn't match reality, the error rate in the first few years of life, your first few months, certainly, is probably crazy high.
It's just nonstop freaking out.
But the collision between your model of the world and how the world works is just so high that you want whatever the trauma of that is not to linger around.
I always thought that's an interesting idea because, like, just imagine the insanity of what's happening in a human brain in the first couple years.
Just, you don't know anything.
And there's just this stream of knowledge and we're somehow, given how plastic everything is, it just kind of molds and figures it out.
But it's like an insane waterfall of information.
I wouldn't necessarily describe it as a trauma.
And we can get into this whole stages of life thing, which I just love.
Basically, those first few years, there are, I mean, you know, I mean, think about it.
A kid's internal model of their body is changing, right?
It's like just learning to move.
I mean, like, you know, if you ever have a baby, you'll know that, like, the first three months, they're discovering their toes, right?
And it's just nuts.
So everything is changing.
But what's really fascinating is, and I think this is one of those, this is not at all me being a scientist, but it's like one of those things that people talk about when they talk about the, you know, positive aspects of children is that they're exceptionally curious and they have this kind of openness towards the world.
And so that prediction error is not a negative, traumatic thing.
I think it's like a very positive thing because it's what they use, they're seeking information.
One of the areas that I'm very interested in is the prefrontal cortex.
It's an area of the brain that, I mean, I could talk all day about it, but it helps us use our knowledge to say, hey, this is what I want to do now.
This is my goal.
So this is how I'm going to achieve it and focus everything towards that goal, right?
The prefrontal cortex takes forever to develop in humans.
The connections are still being tweaked and reformed, like, into late adolescence, early adulthood, which is when you tend to see mental illness pop up, right?
So it's being massively reformed.
Then you have about 10 years, maybe, of prime functioning of the prefrontal cortex, and then it starts going down again, and you end up being older, and you start losing all that frontal function.
So I look at this, and you'd say, okay, from you sit around episodic memory talks, we'll always say children are worse than adults at episodic memory, older adults are worse than young adults at episodic memory.
And I always would say, God, this is so weird.
Why would we have this period of time that's so short when we're perfect, right, or optimal?
And I like to use the word optimal now because there's such a culture of optimization right now, and it's like I realize I have to redefine what optimal is because for most of the human condition, I think we had a series of stages of life where you have basically adults saying, okay, young adults saying, I've got a child, and, you know, I'm part of this village, and I have to hunt and forage and get things done.
And I need a prefrontal cortex so I can stay focused on the big picture and long-haul goals.
Now, I'm a child.
I'm in this village.
I'm kind of wandering around, and I've got some safety, and I need to learn about this culture because I know so little.
What's the best way to do that?
Let's explore.
I don't want to be constrained by goals as much.
I want to really be free, play, and explore, and learn.
So you don't want a super tight prefrontal cortex.
You don't even know what the goals should be in, right?
It's like if you're trying to design a model that's based on a bad goal, it's not going to work well, right?
So then you go late in life, and you say, oh, why don't you have a great prefrontal cortex then?
But I think, I mean, if you go back and you think, how many species actually stick around naturally long after their childbearing years are over, after their reproductive years are over?
Menopause, from what I understand, menopause is not all that common in the animal world, right?
So why would that happen?
And so I saw Alison Gopnik said something about this, so I started to look into this, about this idea that, you know, really, when you're older in most societies, your job is no longer to form new episodic memories.
It's to pass on the memories that you already have, this knowledge about the world, what we call semantic memory, to pass on that semantic memory to the younger generations, pass on the culture.
You know, even now in indigenous cultures, that's the role of the elders.
They're respected.
They're not seen as, you know, people who are past it and losing it.
And I thought that was a very poignant thing, that memory is doing what it's supposed to throughout these stages of life.
So it is always optimal in a sense.
Yeah.
It's just optimal for that stage of life.
Yeah.
And for the ecology of the system.
So you've got, so I looked into this and it's like another species that has menopause is orcas.
Orcapods are led by the grandmothers, right?
So they're not the young adults, not the parents or whatever, the grandmothers.
And so they're the ones that pass on the traditions to the, I guess, the younger generation of orcas.
And if you, you know, if you look from what little I understand, different orcapods have different traditions.
They hunt for different things.
They have different play traditions.
And that's a culture, right?
And so in social animals, evolution, I think, is designing brains that are really around, you know, it's obviously optimized for the individual, but also for kin.
And I think that the kin are part of this, like, when they're a part of this intense social group, the brain development should parallel that, the nature of the ecology.
Well, it's just fascinating to think of the individual orca or human throughout his life in stages doing a kind of optimal wisdom development.
So in the early days, you don't even know what the goal is and you figure out the goal and you kind of optimize for that goal and you pursue that goal.
And then all the wisdom you collect through that, then you share with the others in the system, the other individuals.
And as a collective, then you kind of converge towards greater wisdom throughout the generation.
So in that sense, it's optimal.
Us humans and orcas got something going on.
It works.
Oh, yeah.
Apex predators.
I just got a Megalodon tooth.
Speaking of apex predators.
It's, just imagine the size of that thing.
Anyway, how does the brain forget and how and why does it remember?
So maybe some of the mechanisms.
You mentioned the hippocampus.
What are the different components involved here?
So we can think about this on a number of levels.
Maybe I'll give you the simplest version first, which is we tend to think of memories as these individual things and we can just access them.
Maybe a little bit like, you know, photos on your phone or something like that.
But in the brain, the way it works is you have this distributed pool of neurons and the memories are kind of shared across different pools of neurons.
And so what you have is competition where sometimes memories that overlap can be fighting against each other, right?
So sometimes we forget because that competition just wipes things out.
Sometimes we forget because there aren't the biological signals which we can get into.
So I would promote long-term retention.
And lots of times we forget because we can't find the cue that sends us back to the right memory.
And we need the right cue to be able to activate it, right?
So, you know, for instance, in a neural network, there is no, you wouldn't go and you'd say, this is the memory, right?
It's like the whole network, I mean, the whole ecosystem of memories is in the weights of the neural network.
And in fact, you could extract entirely new memories depending on how you feed.
You have to have the right query, the right prompt to access that, whatever the part you're looking for.
That's exactly right.
That's exactly right.
And in humans, you have this more complex set of ways memory works.
There's, as I said, the knowledge or what you call semantic memory.
And then there's these memories for specific events, which we call episodic memory.
And so there's different pieces of the puzzle that require different kinds of cues.
So that's a big part of it, too, is just this kind of what we call retrieval failure.
You mentioned episodic memory.
You mentioned semantic memory.
What are the different separations here?
What's working memory, short-term memory, long-term memory?
What are the interesting categories of memory?
Yeah.
And so memory researchers, we love to cut things up and say, you know, is memory one thing or is it two things?
There's two things.
There's three things.
And so one of the things that there's value in that, and especially experimental value in terms of being able to dissect things.
In the real world, it's all connected.
Speak to your question, working memory is a term that was coined by Alan Baddeley.
It's basically thought to be this ability to keep information online in your mind right in front of you at a given time.
And to be able to control the flow of that information, to choose what information is relevant, to be able to manipulate it, and so forth.
And one of the things that Alan did that was quite brilliant was he said, there's this ability to kind of passively store information, you know, see things in your mind's eye or hear your internal monologue.
But, you know, we have that ability to keep information in mind.
But then we also have this separate, what he called a central executive, which is identified a lot with the prefrontal cortex.
It's this ability to control the flow of information that's being kept active based on what it is you're doing.
Now, a lot of my early work was basically saying that this working memory, which some memory researchers would call short-term memory, is not at all independent from long-term memory.
That is, that a lot of executive function requires learning, and you have to have, like, synaptic change for that to happen.
But there's also transient forms of memory.
So, one of the things I've been getting into lately is the idea that we form internal models of events.
The obvious one that I always use is birthday parties, right?
So, you go to a child's birthday party, once the cake comes out and you just see a candle, you can predict the whole frame, you know, set of events that happens later.
And up until that point where the child blows out the candle, you have an internal model in your head of what's going on.
And so, if you follow people's eyes, it's not actually on what's happening.
It's going where the action's about to happen, which is just fascinating, right?
So, you have this internal model, and that's a kind of a working memory product.
It's something that you're keeping online that's allowing you to interpret this world around you.
Now, to build that model, though, you need to pull out stuff from your general knowledge of the world, which is what we call semantic memory.
And then you'd want to be able to pull out memories for specific events that happened in the past, which we call episodic memory.
So, in a way, they're all connected, even though it's different.
The things that we're focusing on and the way we organize information in the present, which is working memory, will play a big role in determining how we remember that information later, which people typically call long-term memory.
So, if you have something like a birthday party and you've been to many before, you're going to load that from disk into working memory, this model, and then you're mostly operating on the model.
And if it's a new task, you don't have a model, so you're more in the data collection?
Yes.
One of the fascinating things that we've been studying, and we're not at all the first to do this, Jeff Sachs was a big pioneer in this, and I've been working with many other people.
Ken Norman, Leila DeVache at Columbia has done some interesting stuff with this, is this idea that we form these internal models at particular points of high prediction error, or points of, I believe, also points of uncertainty, points of surprise, or motivationally significant periods.
And those points are when it's maximally optimal to encode an episodic memory.
So, I used to think, oh, well, we're just encoding episodic memories constantly, boom, boom, boom, boom, boom.
But think about how much redundancy there is in all that, right?
It's just a lot of information that you don't need.
But if you capture an episodic memory at the point of maximum uncertainty for the singular experience, right, you're just, it's only going to happen once.
But if you capture it at the point of maximum uncertainty or maximum surprise, you have the most useful point in your experience that you've grabbed.
And what we see is that the hippocampus and these other networks that are involved in generating these internal models of events, they show a heightened period of connectivity or correlated activity during those breaks between different events, which we call event boundaries.
These are the points where you're surprised or you cross from one room to another and so forth.
And that communication is associated with a bump of activity in the hippocampus and better memory.
And so, if people have a very good internal model throughout that event, you don't need to do much memory processing.
You're in a predictive mode, right?
And so, then at these event boundaries you encode, and then you retrieve and you're like, okay, wait a minute, what's going on here?
Branginath's now talking about orcas.
What's going on?
And maybe you have to go back and remember reading my book to pull out the episodic memory to make sense of whatever it is I'm babbling about, right?
And so, there's this beautiful dynamics that you can see in the brain of these different networks that are coming together and then de-affiliating at different points in time that are allowing you to go into these modes.
And so, to speak to your original question, to some extent when we're talking about semantic memory and episodic memory and working memory, you can think about it as these processes that are unfolding as these networks kind of come together and pull apart.
Can memory be trained and improved, this beautiful connected system that you've described?
What aspect of it is a mechanism that can be improved through training?
I think improvement, it depends on what your definition of optimal is.
So, what I say in the book is that you don't want to remember more, you want to remember better, which means focusing on the things that are important, and that's what our brains are designed to do.
So, if you go back to the earliest quantitative studies in memory by Ebbinghaus, what you see is that he was trying so hard to memorize this arbitrary nonsense, and within a day, he lost about 60% of that information.
And he was basically using a very, very generous way of measuring it, right?
So, as far as we know, nobody has managed to violate those basics of having people forget most of their experiences.
So, if your expectation is that you should remember everything and that's what your optimal is, you're already off because this is not what human brains are designed to do.
On the other hand, what we see over and over again is that the brain does, basically, one of the cool things about the design of the brain is it's always less is more, less is more, right?
It's like, I mean, I've seen estimates that the human brain uses something like 12 to 20 watts, you know, in a day.
I mean, that's just nuts, the low power consumption, right?
So, it's all about reusing information and making the most of what we already have.
And so, that's why, basically, again, what you see biologically is, you know, neuromodulators, for instance, these chemicals in the brain like norepinephrine, dopamine, serotonin.
These are chemicals that are released during moments that tend to be biologically significant, surprise, fear, stress, etc.
And so, these chemicals promote lasting plasticity, right?
Essentially, some mechanisms for which the brain can prioritize the information that you carry with you into the future.
Attention is a big factor as well, our ability to focus our attention on what's important.
And so, there's different schools of thought on training attention, for instance.
So, one of my colleagues, Amishi Jha, she wrote a book called Peak Mind and talks about mindfulness as a method for improving attention and focus.
So, she works a lot with military, like Navy SEALs and stuff, to do this kind of work with mindfulness meditation.
Adam Ghazali, another one of my friends and colleagues, has worked on kind of training through video games, actually, as a way of training attention.
And so, it's not clear to me, you know, one of the challenges, though, in training is you tend to overfit to the thing that you're trying to optimize, right?
So, you tend to, if I'm looking at a video game, I can definitely get better at paying attention in the context of the video game, but you transfer it to the outside world.
That's very controversial.
The implication there is that attention is a fundamental component of remembering something, allocating attention to it, and then attention might be something that you could train.
How you allocate attention and how you hold attention on a thing.
I can say that, in fact, we do in certain ways, right?
So, if you are an expert in something, you are training attention.
So, we did this one study of expertise in the brain.
And, you know, so people used to think, let's say if you're a bird expert or something, right?
People will go, like, if you get really into this world of birds, you start to see the differences in your visual cortex is tuned up, and it's all about plasticity of the visual cortex.
And vision researchers love to say everything's visual, you know?
But it's like, we did this study of attention and working memory and expertise, and one of the things that surprised us were the biggest effects as people became experts in identifying these different kinds of just crazy objects that we made up.
As they developed this expertise of being able to identify what made them different from each other and what made them unique, we were actually seeing massive increases in activity in the prefrontal cortex.
And this fits with some of the studies of chess experts and so forth, that it's not so much that you learn the patterns passively, you learn what to look for.
You learn what's important, what's not, right?
And you can see this in any kind of expert professional athlete.
They're looking three steps ahead of where they're supposed to be.
So, that's a kind of a training of attention.
And those are also what you'd call expert memory skills.
So, if you take the memory athletes, I know that's something we're both interested in.
And, you know, so these are people who train in these competitions, and they'll memorize, like, a deck of cards in, like, a really short amount of time.
There's a great memory athlete.
Her name, I think, is pronounced Yenya Wintersol.
But she, so I think she's got, like, a giant Instagram following.
And so, she had this YouTube video that went viral where she had memorized an entire Ikea catalog, right?
And so, how do people do this?
By all accounts from people who become memory athletes, they weren't born with some extraordinary memory.
But they practice strategies over and over and over again.
The strategy that they use for memorizing a particular thing, it can become automatic, and you can just deploy it in an instant, right?
So, again, it's not necessarily going to, one strategy for learning the order of a deck of cards might not help you for something else that you need, like, you know, remembering your way around Austin, Texas.
But it's going to be these, whatever you're interested in, you can optimize for that.
And that's just a natural byproduct of expertise.
There's certain hacks.
There's something called the Memory Palace that I played with.
I don't know if you're familiar with that whole technique.
And it works.
It's interesting.
So, another thing I recommend for people a lot is I use Anki a lot every day.
It's an app that does spaced repetition.
So, I think medical students and, like, students use this a lot to remember a lot of different things.
Oh, yeah.
Okay, we can come back to this.
But, yeah, go on.
Sure.
It's the whole concept of spaced repetition.
You just, when the thing is fresh, you kind of have to remind yourself of it a lot.
And then, over time, you can wait a week, a month, a year before you have to recall the thing again.
And that way, you essentially have something like note cards that you can have tens of thousands of and can only spend 30 minutes a day and actually be refreshing all of that information, all of that knowledge.
It's really great.
It's really great.

One of the things that I have to solve for myself is how to remember names.
I'm horrible at it.
Yeah.
I think it's because when people introduce themselves, I have the social anxiety of the interaction where I'm like, I know I should be remembering that, but I'm freaking out internally about social interaction in general.
And so, therefore, I forget immediately.
So, I'm looking for good tricks for that.
So, I feel like we've got a lot in common because when people introduce themselves to me, it's almost like I have this, like, just blank blackout for a moment.
And then I'm just looking at them like, what happened?
I look away or something.
What's wrong with me?
So, I mean, I'm totally with you on this.
The reason why it's hard is that there's no reason we should be able to remember names.
Because when you say remembering a name, you're not really remembering a name.
Maybe in my case, you are.
But most of the time, you're associating a name with a face and an identity.
And that's a completely arbitrary thing, right?
I mean, maybe in the olden days, somebody named Miller, it's like they're actually making flour or something like that.
But, you know, for the most part, it's like these names are just utterly arbitrary.
So, you have no thing to latch on to.
And so, it's not really a thing that our brain does very well to learn meaningless, arbitrary stuff.
And so, what you need to do is build connections somehow, visualize a connection.
And sometimes it's obvious or sometimes it's not.
I'm trying to think of a good one for you now.
But the first thing I think of is Lex Luthor.
That's great.
Yeah.
So, I think of Lex Luthor.
Because doesn't Lex Luthor wear a suit, I think?
Like, I know he has a shaved head, though, or he's bald, which you're not.
You've got a great head.
I'd trade hair with you any day.
Right.
But, like, you know, something like that.
But if I can come up with something.
Like, I could say, okay, so Lex Luthor is this criminal mastermind.
And then I just imagine you.
And we talked about stabbing or whatever earlier.
Yeah, exactly.
So, I'm just kind of connected and that's it.
Yeah, yeah.
But I'm serious, though, that these kinds of weird associations, now I'm building a richer network.
I mean, one of the things that I find is if I've, like, you can have somebody's name that's just totally generic, like John Smith or something.
Not that, no offense to people, that name.
But, you know, if I see a generic name like that, but I've read John Smith's papers academically and then I meet John Smith at a conference, I can immediately associate that name with that face because I have this pre-existing network to lock everything into, right?
And so, you can build that network.
And that's what the method of loci or the memory palace technique is all about, is you have a pre-existing structure in your head of, like, your childhood home or this mental palace that you've created for yourself.
And so, now you can put arbitrary pieces of information in different locations in that mental structure of yours.
And then you could walk through the different path and find all the pieces of information you're looking for.
So, the method of loci is a great method for just learning arbitrary things because it allows you to link them together and get that cue that you need to pop in and find everything, right?
We should maybe linger on this memory palace thing just to make obvious because when people were describing to me a while ago what this is, it seems insane.
I just, you literally think of a place like a childhood home or a home that you're really visually familiar with and you literally place in that three-dimensional space facts or people or whatever you want to remember.
And you just walk in your mind along that place visually and you can remember, remind yourself of the different things.
One of the limitations is there is a sequence to it.
So, it's, I think your brain somehow, you need, you can't just like go upstairs right away or something.
You have to like walk along the room.
So, it's really great for remembering sequences but it's also not great for remembering like individual facts out of context.
So, the full context of the tour, I think is important.
But it's fascinating how the mind is able to do that.
When you ground these pieces of knowledge into something that you remember well already, especially visually, fascinating.
And you can just do that for any kind of sequence.
I'm sure she used something like this for the, for Ikea catalog.
Oh yeah, absolutely, absolutely.
And I think the, the principle here is, again, I was telling you this idea that memories can compete with each other, right?
Well, I like to use this example and maybe someday I'll regret this but I've used it a lot recently, is like imagine if this were my desk, it could be cluttered with a zillion different things, right?
So, imagine it's just cluttered with a whole bunch of yellow post-it notes and one of them I put my bank password on it, right?
Well, it's going to take me forever to find it.
I might, you know, it's just going to be buried under all these other post-it notes.
But if it's like hot pink, it's going to stand out and I find it really easily, right?
So, that's one way in which if things are distinctive, if you've processed information in a very distinctive way, then you can have a memory that's going to last.
And that's very good, for instance, for name-face associations.
If I get something distinctive about you, you know, that it's like that you've got very short hair and maybe I can make the association with Lex Luthor that way or something like that, right?
You know, but I get something very specific, that's a great cue.
But the other part of it is, what if I just organized my notes so that I have my finances in one pile and I have my, like, reminders, my to-do list in one pile and so forth, so I organize them.
Well, then I know exactly if I'm going for my banking, you know, my bank password, I could go to the finance pile, right?
So, the method of loci works or memory palaces work because they give you a way of organizing.
There's a school of thought that says that episodic memory evolved from this, like, kind of knowledge of space and, you know, basically this primitive abilities to figure out where you are.
And so, people explain the method of loci that way.
And, you know, whether or not the evolutionary argument is true, the method of loci is not at all special.
So, if you're not a good visualizer, stories are a good one.
So, a lot of memory athletes will use stories and they'll go, like, if you're memorizing a deck of cards, they have a little code for the different, like, the king and the jack and the ten and so forth.
And they'll make up a story about things that they're doing and that'll work.
Songs are a great one, right?
I mean, it's like, I can still remember there's this obscure episode of the TV show, Cheers.
They sing a song about Albania that he uses to memorize all these facts about Albania.
And I can still sing that song to you.
It's just, I saw it on a TV show, you know.
So, you mentioned spaced repetition.
So, what, do you like this process?
Maybe can you explain it?
Oh, yeah.
If I'm trying to memorize something, let's say if I have an hour to memorize as many Spanish words as I can, if I just try to do, like, half an hour and then later in the day I do half an hour, I won't retain that information as long as if I do half an hour today and half an hour one week from now.
And so, doing that extra spacing should help me retain the information better.
Now, there's an interesting boundary condition, which is it depends on when you need that information.
So, many of us, you know, for me, like, I can't remember so much from college and high school because I crammed because I just did everything at the last minute.
And sometimes I would literally study, like, you know, in the hallway right before the test.
And that was great because what would happen is I just had that information right there.
And so, actually, not spacing can really help you if you need it very quickly, right?
But the problem is that you tend to forget it later on.
But on the other hand, if you space things out, you get a benefit for later on retention.
And so, there's many different explanations.
We have a computational model of this.
It's currently under revision.
But in our computer model, what we say is that an easy, maybe a good way of thinking about this is this conversation that you and I are having.
It's associated with a particular context, a particular place in time.
And so, all these little cues that are in the background, these little guitar sculptures that you have and that big light umbrella thing, right?
All these things are part of my memory for what we're talking about, the content.
So, now, later on, you're sitting around and you're at home drinking a beer and you're thinking, God, what a strange interview that was, right?
So, now you're trying to remember it, but the context is different.
So, your current situation doesn't match up with the memory that you pulled up.
There's error.
There's a mismatch between what you pulled up and your current context.
And so, in our model, what you start to do is you start to erase or alter the parts of the memory that are associated with a specific place and time, and you heighten the information about the content.
And so, if you remember this information in different times and different places, it's more accessible at different times and different places because it's not overfitted in an AI kind of way of thinking about things.
It's not overfitted to one particular context.
But that's also why the memories that we call upon the most also feel kind of like they're just things that we read about almost.
You don't vividly reimagine them, right?
It's like they're just these things that just come to us like facts, right?
Yeah.
And it's a little bit different than semantic memory, but it's like basically these events that we have recalled every, you know, over and over and over again, we keep updating that memory so it's less and less tied to the original experience.
But then we have those other ones, which it's like you just get a reminder of that very specific context.
You smell something, you hear a song, you see a place that you haven't been to in a while, and boom, it just comes back to you.
And that's the exact opposite of what you get with spacing, right?
That's so fascinating.
So with space repetition, one of its powers is that you lose attachment to a particular context, but then it loses the intensity of the flavor of the memory.
That's interesting.
That's so interesting.
Yeah.
But, you know, at the same time, it becomes stronger in the sense that the content becomes stronger.
Yeah, so it's used for learning languages, for learning facts, for learning, you know, for that generic semantic information type of memory.
Yeah.
And I think this falls into a category.
We've done other modeling.
One of these is a published study in PLOS Computational Biology, where we showed that another way, which is, I think, related to the spacing effect, is what's called the testing effect.
So the idea is that if you're trying to learn words, let's say in Spanish or something like that, and this doesn't have to be words, it could be anything, you test yourself on the words.
And that act of testing yourself helps you retain it better over time than if you just studied it, right?
And so from traditional learning theories, some learning theories anyway, this seems weird, why would you do better giving yourself this extra error from testing yourself rather than just, you know, giving yourself perfect input that's a replica of what it is that you're trying to learn?
And I think the reason is, is that you get better retention from that error, that mismatch that we talked about, right?
So what's happening in our model, it's actually conceptually kind of similar to what happens with backprop in AI, or neural networks.
And so the idea is, is that you expose, here's the bad connections, and here's the good connections.
And so we can keep the parts of this cell assembly that are good for the memory and lose the ones that are not so good.
But if you don't stress test the memory, you haven't exposed it to the error fully.
And so that's why I think this is kind of, this is a thing that I come back to over and over again, is that you will retain information better if you're constantly pushing yourself to your limit, right?
If you are feeling like you're coasting, then you're actually not learning.
So it's like...
So you should always be stress testing the memory system.
Yeah, and feel good about it.
You know, even though everyone tells me, oh, my memory is terrible, in the moment, they're overconfident about what they'll retain later on.
So it's fascinating.
And so what happens is, is when you test yourself, you're like, oh my God, I thought I knew that, but I don't.
And so it can be demoralizing until you get around that and you realize, hey, this is the way that I learned.
This is how I learned best.
It's like, if you're trying to, you know, star in a movie or something like that, you don't just sit around reading the script.
You actually act it out and you're going to botch those lines from time to time, right?
You know, there's an interesting moment.
You probably experienced this.
I remember a good friend of mine, Joe Rogan, I was on his podcast and we were randomly talking about soccer, football.
Somebody I grew up watching, Diego Armando Maradona, one of the greatest soccer players of all time.
And we were here talking about him and his career and so on.
And Joe asked me if he's still around.
No.
And I said, yeah.
I don't know why I thought, yeah, because that was a perfect example of memories.
He passed away.
I tweeted about it, how heartbroken I was, all this kind of stuff, like a year before.
I know this, but in my mind, I went back to the thing I've done many times in my head, visualizing some of the epic runs he had on goal and so on.
So for me, he's alive.
And part of also the conversation where you're talking to Joe, there's stress and the focus is allocated, the attention is allocated in a particular way.
But when I walked away, I was like, in which world was Diego Maradona still alive?
Because I was sure in my head that he was still alive.
It's a moment that sticks with me.
I've had a few like that in my life where it just kind of...
Like obvious things just disappear from mind.
And it's cool.
Like it shows actually the power of the mind in a positive sense to erase memories you want erased, maybe.
But I don't know.
I don't know if there's a good explanation for that.
One of the cool things that I found is that some people really just revolutionize a field by creating a problem that didn't exist before.
It's kind of like why I love science.
It's like engineering is like solving other people's problems and science is about creating problems.
I'm just much more like I want to break things and create problems.
Not necessarily move fast, though.
But one of my former mentors, Marsha Johnson, who in my opinion is one of the greatest memory researchers of all time, she comes up, young woman in the field, and it's mostly guy field.
And she gets into this idea of how do we tell the difference between things that we've imagined and things that we actually remember?
How do we tell?
I get some mental experience.
Where did that mental experience come from, right?
And it turns out this is a huge problem because essentially our mental experience of remembering something that happened, our mental experience of thinking about something, how do you tell the difference?
They're both largely constructions in our head.
And so it is very important.
And the way that you do it is, I mean, it's not perfect, but the way that we often do it and succeed is by, again, using our prefrontal cortex and really focusing on the sensory information or the place and time and the things that put us back into when this information happened.
And if it's something you thought about, you're not going to have all of that vivid detail as you do for something that actually happened.
But it doesn't work all the time.
But that's a big thing that you have to do.
But it takes time.
It's slow.
And it's, again, effortful.
But that's what you need to remember accurately.
But what's cool, and I think this is what you alluded to about how that was an interesting experience, is imagination is exactly the opposite.
Imagination is basically saying, I'm just going to take all this information from memory, recombine it in different ways, and throw it out there.
And so, for instance, Dan Schachter and Donna Addis have done cool work on this.
Demis Hassabis did work on this with Eleanor McGuire in UCL.
And this goes back, actually, to this guy, Frederick Bartlett, who was this revolutionary memory researcher.
Bartlett, he actually, like, rejected the whole idea of quantifying memory.
He said, there's no statistics in my book.
He came from this anthropology perspective.
And short version of the story is he just asked people to recall things.
You give people stories and poems, ask people to recall them.
And what he found was people's memories didn't reflect all of the details of what they were exposed to, and they did reflect a lot more.
They were filtered through this lens of prior knowledge, the cultures that they came from, the beliefs that they had, the things they knew.
And so what he concluded was that he called remembering an imaginative construction, meaning that we don't replay the past.
We imagine how the past could have been by taking bits and pieces that come up in our heads.
And likewise, he wrote this beautiful paper on imagination saying, when we imagine something and create something, we're creating it from these specific experiences that we've had and combining it with our general knowledge.
But instead of trying to focus it on being accurate and getting out one thing, you're just ruthlessly recombining things without any, you know, any necessary kind of goal in mind.
And I mean, or at least that's one kind of creation.
So imagination is fundamentally coupled with memory in both directions.
I think so.
I mean, it's not clear that it is in everyone, but one of the things that's been studied is some patients who have amnesia, for instance.
They have brain damage, say, to the hippocampus.
And if you ask them to imagine things that are not in front of them, like imagine what could happen after I leave this room, right?
They find it very difficult to give you a scenario of what could happen.
Or if they do, it would be more stereotyped, like, yes, this would happen.
But it's not like they can come up with anything that's very vivid and creative in that sense.
And it's partly because when you have amnesia, you're stuck in the present.
Because to get a very good model of the future, it really helps to have episodic memories to draw upon, right?
And so that's the basic idea.
In fact, one of the most impressive things, when people started to scan people's brains and ask people to remember past events, what they found was there was this big network of the brain called the default mode network.
It gets a lot of press because it's, like, thought to be important.
It's engaged during mind-wandering.
And if I ask you to pay attention to something, it only comes on when you stop paying attention.
You know, so people, oh, it's just this kind of, you know, daydreaming network.
And I thought, this is just ridiculous research.
Who cares?
You know.
But then what people found was when people recall episodic memories, this network gets active.
And so we started to look into it.
And this network of areas is really closely functionally interacting with the hippocampus.
And so, in fact, some would say the hippocampus is part of this default network.
And if you look at brain images of people or brain maps of activation, so to speak, of people imagining possible scenarios of things that could happen in the future or even things that couldn't really be very plausible, they look very similar.
I mean, you know, to the naked eye, they look almost the same as maps of brain activation when people remember the past.
According to our theory, and we've got some data to support this, we've broken up this network into various sub pieces.
Is that basically it's kind of taking apart all of our experiences and creating these little Lego blocks out of them.
And then you can put them back together if you have the right instructions to recreate these experiences that you've had.
But you could also reassemble them into new pieces to create a model of an event that hasn't happened yet.
And that's what we think happens.
And when our common ground that we're establishing in language requires using those building blocks to put together a model of what's going on.
Well, there's a good percentage of time I personally live in the imagined world.
I think of, I do thought experiments a lot.
I, you know, take the absurdity of human life as it stands and play it forward in all kinds of different directions.
Sometimes it's rigorous thoughts, thought experiments, sometimes it's fun ones.
So I imagine that that has an effect on how I remember things.
And I suppose I have to be a little bit careful to make sure stuff happened versus stuff that I just imagined happened.
And this also, I mean, some of my best friends are characters inside books that never even existed.
And I'm, you know, there's some degree to which they actually exist in my mind.
Like these characters exist, authors exist, Dostoevsky exist, but also Brothers Karamazov.
I love that book.
Yeah.
One of the few books I've read.
One of the few literature books that I've read, I should say.
I read a lot in school that I don't remember, but Brothers Karamazov.
But they exist.
They exist.
And I have almost kind of like conversations with them.
It's interesting.
It's interesting to allow your brain to kind of play with ideas of the past, of the imagined, and see it all as one.
Yeah, there was actually this famous mnemonist.
He's kind of like back then the equivalent of a memory athlete, except he would go to shows and do this.
That was described by this really famous neuropsychologist from Russia named Luria.
And so this guy was named Solomon Cheryshevsky.
And he had this condition called synesthesia that basically created these weird associations between different senses that normally wouldn't go together.
So that gave him this incredibly vivid imagination that he would use to basically imagine all sorts of things that he would need to memorize.
And he would just imagine, like, just create these incredibly detailed things in his head that allowed him to memorize all sorts of stuff.
But it also really haunted him by some reports that basically it was like he was at some point, you know, and again, who knows if the drinking was part of this.
But at some point had trouble differentiating his imagination from reality, right?
And this is interesting because it's like, I mean, that's what psychosis is in some ways, is you, you know, first of all, you're just learning connections from prediction errors that you probably shouldn't learn.
And the other part of it is, is that your internal signals are being confused with actual things in the outside world, right?
Well, that's why a lot of this stuff is both feature and bug.
It's a double-edged sword.
Yeah, I mean, it might be why there's such an interesting relationship between genius and psychosis.
Yeah, maybe they're just two sides of the same coin.
Humans are fascinating, aren't they?
I think so.
Sometimes scary, but mostly fascinating.
Can we just talk about memory sport a little longer?
There's something called the USA Memory Championship.
Like, what are these athletes like?
What does it mean to be, like, elite level at this?
Have you interacted with any of them or reading about them?
What have you learned about these folks?
There's a guy named Henry Rodger who's studying these guys.
And there's actually a book by Joshua Foer called Moonwalking with Einstein where he talks about, he actually, as part of this book,
just decided to become a memory athlete.
They often have these life events that make them go, hey, why don't I do this?
So there was a guy named Scott Hagwood who I write about who thought that he was getting chemo for cancer.
And so he decided, like, because chemo, there's a well-known thing called chemo brain where people become like, they just lose a lot of their sharpness.
And so he wanted to fight that by learning these memory skills.
So he bought a book.
And this is the story you hear in a lot of memory athletes is they buy a book by other memory athletes or other memory experts, so to speak.
And they just learn those skills and practice them over and over again.
They start by winning bets and so forth.
And then they go into these competitions.
And the competitions are typically things like memorizing long strings of numbers or memorizing, you know, orders of cards and so forth.
So there tend to be pretty arbitrary things, not like things that would be able, you'd be able to bring a lot of prior knowledge.
But they build the skills that you need to memorize arbitrary things.
Yeah, that's fascinating.
I've gotten a chance to work with something called NBAC tasks.
So there's all these kinds of tasks, memory recall tasks that are used to kind of load up the quote-unquote working memory.
Yeah, yeah.
And to see, the psychologists used it to test all kinds of stuff, like to see how well you're good at multitasking.
We used it in particular for the task of driving.
Like if you fill up your brain with intensive working memory tasks, how good are you at also not crashing, that kind of stuff.
So it's fascinating.
But again, those tasks are arbitrary and they're usually about recalling a sequence of numbers in some kind of semi-complex way.
Do you have any favorite tasks of this nature in your own studies?
I've really been most excited about going in the opposite direction and using things that are more and more naturalistic.
And the reason is that we've moved in that direction because what we found is that memory works very, very differently when you study memory in the way that people typically remember.
And so it goes into a much more predictive mode.
And you have these event boundaries, for instance, and you have – but a lot of what happens is this kind of fascinating mix that we've been talking about, a mix of interpretations and imagination with perception.
And so – and the new direction we're going in is understanding navigation in our memory for places.
And the reason is, is that there's a lot of work that's done in rats, which is very good work.
They have a rat and they put it in a box and the rat goes chase his cheese in a box.
And you'll find cells in the hippocampus that fire when a rat is in different places in the box.
And so the conventional wisdom is that the hippocampus forms this map of the box.
And I think that probably may happen when you have, like, absolutely no knowledge of the world, right?
But I think one of the cool things about human memory is we can bring to bear our past experiences to economically learn new ones.
And so, for instance, if you learn a map of an Ikea – let's say if I go to the Ikea in Austin, I'm sure there's one here – I probably could go to this Ikea and find my way to the – you know, where the wine glasses are without having to even think about it because it's got a very similar layout.
Even though Ikea is a nightmare to get around, once I learn my local Ikea, I can use that map everywhere.
Why form a brand new one for a new place?
And so that kind of ability to reuse information really comes into play when we look at things that are, you know, more naturalistic tasks.
And another thing that we're really interested in is this idea of, like, what if instead of basically mapping out every coordinate in a space, you form a pretty economical graph that connects basically the major landmarks together and being able to use that as, you know, emphasizing the things that are most important.
And the places that you go for food and the places that are landmarks that help you get around and then filling in the blanks for the rest.
Because I really believe that cognitive maps or mental maps of the world, just like our memories for events, are not photographic.
I think there's this combination of actual verifiable details and then a lot of inference that you make.
So what have you learned about this kind of spatial mapping of places?
How do people represent locations?
There's a lot of variability, I think, and there's a lot of disagreement about how people represent locations.
In a world of GPS and physical maps, people can learn it from, like, basically what they call a survey perspective, being able to see everything.
And so that's one way in which humans can do it that's a little bit different.
There's one way in which we can memorize routes, like I know how to get from here to, let's say, if I walk here from my hotel, I can just rigidly follow that route back, right?
And there's another more integrative way, which would be what's called a cognitive map, which would be kind of a sense of how everything relates to each other.
And so there's lots of people who believe that these maps that we have in our head are isomorphic with the world.
They're like these literal coordinates that follow Euclidean space.
And as you know, Euclidean mathematics is very constrained, right?
And I think that we are actually much more generative in our maps of space so that we do have these bits and pieces.
And we've got a small task.
It's right now not yet like we need to do some work on it for further analyses.
But one of the things we're looking at is these signals called ripples in the hippocampus, which are these bursts of activity that you see that are synchronized with areas in the neocortex, in the default network, actually.
And so what we find is that those ripples seem to increase at navigationally important points when you're making a decision or when you reach a goal.
So that speaks to the emotion thing, right?
Because if you have limited choices, if I'm walking down a street, I could really just get a mental map of the neighborhood with a more minimal kind of thing by just saying, here's the intersections and here's the directions I take to get in between them.
And what we found in general in our MRI studies is basically the more people can reduce the problem, whether it's space or any kind of decision-making problem, the less the hippocampus encodes.
It really is very economical towards the points of most, highest information content and value.
So can you describe the encoding in the hippocampus and the ripples you were talking about?
What's the signal in which we see the ripples?
Yeah, so this is really interesting.
There are these oscillations, right?
So there's these waves that you basically see.
And these waves are points of very high excitability and low excitability.
And at least during, they happen actually during slow wave sleep too.
So the deepest stages of sleep when you're just zonked out, right?
You see these very slow waves where it's like very excitable and then very unexcitable.
It goes up and down.
And on top of them, you'll see these little sharp wave ripples.
And when there's a ripple in the hippocampus, you tend to see a sequence of cells that resemble a sequence of cells that fire when, you know, an animal's actually doing something in the world.
So it almost is like a little, people call it replay.
I think it's a little bit, I don't like that term, but it's basically a little bit of a compressed play of the sequence of activity in the brain that was taking place earlier.
And during those moments, there's a little window of communication between the hippocampus and these areas in the neocortex.
And so that, I think, helps you form new memories, but it also helps you, I think, stabilize them, but also really connect different things together in memory and allows you to build bridges between different events that you've had.
And so this is one of our, at least our theories of sleep and its real role in helping you see the connections between different events that you've experienced.
Oh, so during sleep is when the connections are formed?
The connections between different events, right?
So it's like, you see me now, you see me next week, you see me a month later, you start to build a little internal model of how I behave and, you know, what to expect of me.
And with sleep, one of the things that allows you to do is figure out those connections and connect the dots and find the signal and the noise.
So you mentioned fMRI.
What is it and how is it used in studying memory?
And this is actually the reason why I got into this whole field of science.
When I was in grad school, fMRI was just really taking off as a technique for studying brain activity.
And what's beautiful about it is you can study the whole human brain and there's lots of limits to it, but you can basically do it in person without sticking anything into their brains.
And very non-invasive, I mean, for me, being an MRI scanner is like being in the womb.
I just fall asleep.
If I'm not being asked to do anything, I get very sleepy, you know.
But you can have people watch movies while they're being scanned or you can have them do tests of memory, like giving them words and so forth to memorize.
But what MRI is itself is just this technique where you put people in a very high magnetic field.
Typical ones we would use would be three Tesla, to give you an idea.
So a three Tesla magnet, you put somebody in, and what happens is you get this very weak but measurable magnetization in the brain.
And then you apply a radiofrequency pulse, which is basically a different electromagnetic field.
And so you're basically using water, the water molecules in the brain as a tracer, so to speak.
And part of it in fMRI is the fact that these magnetic fields that you mess with by manipulating these radiofrequency pulses and the static field.
And you have things called gradients, which change the strength of the magnetic field in different parts of the head.
So they're all, we tweak them in different ways.
But the basic idea that we use in fMRI is that blood is flowing to the brain.
And when you have blood that doesn't have oxygen on it, it's a little bit more magnetizable than blood that does.
Because you have hemoglobin that carries the oxygen, the iron, basically, in the blood that makes it red.
And so that hemoglobin, when it's deoxygenated, actually has different magnetic field properties than when it has oxygen.
And it turns out, when you have an increase in local activity in some part of the brain, the blood flows there.
And as a result, you get a lower concentration of hemoglobin that is not oxygenated.
And then that gives you more signal.
So I gave you, I think I sent you a gif, as you like to say.
Yeah, we had off-record intense argument about if it's pronounced jif or gif, but we shall set that aside as friends.
We could have called it a stern rebuke, perhaps.
Rebuke, yeah.
I drew a hard line.
I mean, it is true, the creator of gifs said it's pronounced jif, but that's the only person that pronounces jif.
Anyway, yes, you sent a gif of...
This would be basically a whole movie of fMRI data.
And so when you look at it, it's not very impressive.
It looks like these very pixelated maps of the brain, but it's mostly kind of like white.
But these tiny changes in the intensity of those signals that you probably wouldn't be able to visually perceive, like about 1%, can be statistically very, very large effects for us.
And that allows us to see, hey, there's an increase in activity in some part of the brain when I'm doing some task, like trying to remember something.
And I can use those changes to even predict, is a person going to remember this later or not?
And the coolest thing that people have done is to decode what people are remembering from the patterns of activity.
Because maybe when I'm remembering this thing, like I'm remembering the house where I grew up, I might have one pixel that's bright in the hippocampus and one that's dark.
And if I'm remembering something more like the car that I used to drive when I was 16, I might see the opposite pattern where a different pixel is bright.
And so all that little stuff that we used to think of noise, we can now think of almost like a QR code for memory, so to speak,
where different memories have a different little pattern of bright pixels and dark pixels.
And so this really revolutionized my research.
So there's fancy research out there where people really, I mean, not even that, I mean, by your standards, this would be Stone Age,
but, you know, applying machine learning techniques to do decoding and so forth.
And now there's a lot of forward encoding models and you can go to town with this stuff, right?
And I'm much more old school of designing experiments where you basically say, okay, here's a whole web of memories that overlap in some way, shape, or form.
Do memories that occurred in the same place have a similar QR code?
And do memories that occurred in different places have a different QR code?
And you can just use things like correlation coefficients or cosine distance to measure that stuff, right?
Super simple, right?
And so what happens is you can start to get a whole state space of how a brain area is indexing all these different memories.
And it's super fascinating because what we could see is this little like separation between how certain brain areas are processing memory for who was there
and other brain areas are processing information about where it occurred or the situation that's kind of unfolding.
And some are giving you information about what are my goals that are involved and so forth.
And the hippocampus is just putting it all together into these unique things that just are about when and where it happened.
So there is a separation between spatial information, concepts, like literally there's distinct, as you said, QR codes for these?
So to speak.
Let me try a different analogy, too, that might be more accessible for people, which would be like you've got a folder on your computer, right?
And I open it up.
There's a bunch of files there.
I can sort those files by alphabetical order.
And now things that both start with letter A are lumped together and things that start with Z versus A are far apart, right?
And so that is one way of organizing the folder.
But I could do it by date.
And if I do it by date, things that were created close together in time are close and things that are far apart in time are far.
So you can think of how a brain area or a network of areas contributes to memory by looking at what the sorting scheme is.
And these QR codes that we're talking about that you get from fMRI allow you to do that.
And you can do the same thing if you're recording from massive populations of neurons in an animal.
And you can do it for recording local potentials in the brain, you know, so little waves of activity in, let's say, a human who has epilepsy and they stick electrodes in their brain and try to find the seizures.
So that's some of the work that we're doing now.
But all these techniques basically allow you to say, hey, what's the sorting scheme?
And so we've found that some networks of the brain sort information in memory according to who was there.
So I might have, like, we've actually shown in one of my favorite studies of all time that was done by a former postdoc, Zach Rea.
And Zach did the study where we had a bunch of movies with different people in my labs.
There are two different people.
And he filmed them at two different cafes and two different supermarkets.
And what you could show is in one particular network, you could find the same kind of pattern of activity, more or less, a very similar pattern of activity.
Every time I saw Alex in one of these movies, no matter where he was, right?
And I could see another one that was, like, a common pattern that happened every time I saw this particular supermarket nugget, you know?
And so, and it didn't matter whether you're watching a movie or whether you're recalling the movie.
It's the same kind of pattern that comes up, right?
It's so fascinating.
So now you have those building blocks for assembling a model of what's happening in the present, imagining what could happen, and remembering things very economically from putting together all these pieces.
So that all the hippocampus has to do is get the right kind of blueprint for how to put together all these building blocks.
These are all, like, beautiful hints at a super interesting system that makes me wonder, on the other side of it, how to build it.
But it's, like, it's fascinating.
Like, the way it does the encoding is really, really fascinating.
Or I guess the symptoms, the results of that encoding are fascinating to study from this.
Just as a small tangent, you mentioned sort of the measuring local potentials with electrodes versus fMRI.
Oh, yeah.
What are some interesting, like, limitations, possibilities of fMRI?
Maybe the way you explain it is, like, brilliant with blood and detecting the activations or the excitation because blood flows to that area.
What's, like, the latency of that?
But, like, what's the blood dynamics in the brain that, like, how quickly can it, how quickly can the task change and all that kind of stuff?
Yeah.
I mean, it's very slow.
To the brain, 50 milliseconds is, like, you know, like it's an eternity.
Maybe not 50, you know, maybe, like, you know, let's say half a second, 500 milliseconds.
Like, just so much back and forth stuff happens in the brain in that time, right?
So, in fMRI, you can measure these magnetic field responses about six seconds after that burst of activity would take place.
All these things, it's like, is it a feature or is it a bug, right?
So, one of the interesting things that's been discovered about fMRI is it's not so tightly related to the spiking of the neurons.
So, we tend to think of the computation, so to speak, as being driven by spikes, meaning, like, there's just a burst of it's either on or it's off and the neurons, like, going up or down.
But sometimes what you can have is these states where the neuron becomes a little bit more excitable or less excitable.
And so, fMRI is very sensitive to those changes in excitability.
Actually, one of the fascinating things about fMRI is where does that, how is it we go from neural activity to, you know, essentially blood flow to oxygen, you know, all this stuff.
It's such a long chain of, you know, going from neural activity to magnetic fields.
And one of the theories that's out there is, you know, most of the cells in the brain are not neurons.
They're actually these support cells called glial cells.
And one big one is astrocytes.
And they play this big role in regulating, you know, kind of being a middleman, so to speak, with the neurons.
So, if you, for instance, like, one neuron's talking to another, you release a neurotransmitter, like, let's say, glutamate.
And that gets another neuron, starts getting active after you release it in the gap between the two neurons called synapse.
So, what's interesting is if you leave that, you know, imagine you're just flooded with this, like, liquid in there, right?
If you leave it in there too long, you just excite the other neuron too much and you can start to basically get seizure activity.
You don't want this.
So, you got to suck it up.
And so, actually, what happens is these astrocytes, one of their functions is to suck up the glutamate from the synapse.
And that is a massively, and then break it down and then feed it back into the neuron so that you can reuse it.
But that cycling is actually very energy intensive.
And what's interesting is, at least according to one theory, they need to work so quickly that they're working on metabolizing the glucose that comes in without using oxygen, kind of like what, you know, anaerobic metabolism.
So, they're not using oxygen as fast as they are using glucose.
So, what we're really seeing in some ways may be in fMRI, not the neurons themselves being active, but rather the astrocytes, which are meeting the metabolic demands of the process of keeping the whole system going.
It does seem to be that fMRI is a good way to study activation.
So, with these astrocytes, even though there's a latency, it's pretty reliably coupled to the activations.
Oh, well, this gets me to the other part.
So, now let's say, for instance, if I'm just kind of like, I'm talking to you, but I'm kind of paying attention to your cowboy hat or I'm thinking about the right even if I'm not looking at it.
What you'd see is that there would be this little elevation in activity in areas in the visual cortex, you know, which process vision around that point in space, okay?
So, if then something happened, like, you know, sudden the light flashed in that part of, you know, right in front of your cowboy hat, I would have a bigger response to it.
But what you see in fMRI is even if I don't see that flash of light, there's a lot of activity that I can measure because you're kind of keeping it excitable.
And that in and of itself, even though I'm not seeing anything there that's particularly interesting, there's still this increase in activity.
And so, it's more sensitive with fMRI.
So, is that a feature or is it a bug?
You know, some people who study spikes in neurons would say, well, that's terrible.
We don't want that, you know.
Likewise, it's slow, and that's terrible for measuring things that are very fast.
But one of the things that we found in our work was when we give people movies and when we give people stories to listen to, a lot of the action is in the very, very slow stuff.
It's in because if you're thinking about like a story, let's say you're listening to a podcast or something, you're listening to the Lex Friedman podcast, right?
You're putting this stuff together and building this internal model over several seconds, which is basically we filter that out when we look at electrical activity in the brain because we're interested in this millisecond scale.
It's almost massive amounts of information, right?
So, the way I see it is every technique gives you a little limited window into what's going on.
fMRI has huge problems.
You know, people lie down in the scanner.
There's parts of the brain where I'll show you in some of these images where you'll see kind of gaping holes because you can't keep the magnetic field stable in those spots.
You'll see parts where it's like there's a vein, and so it just produces big increases and decreases in signal or respiration that causes these changes.
There's lots of artifacts and stuff like that, you know.
Every technique has its limits.
If I'm lying down in an MRI scanner, I'm lying down.
I'm not interacting with you in the same way that I would in the real world.
But at the same time, I'm getting data that I might not be able to get otherwise.
And so, different techniques give you different kinds of advantages.
What kind of big scientific discoveries, maybe the flavor of discoveries, have been done throughout the history of the science of memory, the studying of memory?
What kind of things have been, like, understood?
Oh, there's so many.
It's really so hard to summarize it.
I mean, I think it's funny because it's like when you're in the field, you can get kind of blase about this stuff.
But then once I started to write the book, I was like, oh, my God, this is really interesting.
How did we do all this stuff?
I would say that some of the, I mean, you know, from the first studies, just showing how much we forget is very important.
Showing how much schemas, which is our organized knowledge about the world, increase our ability to remember information, just massively increase it.
So, studies of expertise showing how experts like chess experts can memorize so much in such a short amount of time because of the schemas they have for chess.
But then also showing that those lead to all sorts of distortions in memory.
The discovery that the act of remembering can change the memory, can strengthen it, but it can also distort it if you get misinformation at the time.
And it can also strengthen or weaken other memories that you didn't even recall.
So, just this whole idea of memory as an ecosystem, I think, was a big discovery.
I could go, this idea of like breaking up our continuous experience into these discrete events, I think, was a major discovery.
So, the discreteness of our encoding of events?
Maybe, yeah.
I mean, you know, and again, there's controversial ideas about this, right?
But it's like, yeah, this idea that, and this gets back to just this common experience of you walk into the kitchen and you're like, why am I here?
And you just end up grabbing some food from the fridge.
And then you go back and you're like, oh, wait a minute, I left my watch in the kitchen.
That's what I was looking for.
And so, what happens is, is that you have a little internal model of where you are, what you're thinking about.
And when you cross from one room to another, those models get updated.
And so, now when you're in the kitchen, you have to go back and mentally time travel back to this earlier point to remember what it was that you went there for.
And so, these event boundaries, turns out, like, in our research, and again, I don't want to make it sound like we've figured out everything, but in our research, one of the things that we found is, is that basically, as people get older, the activity in the hippocampus at these event boundaries tends to go down.
And, but independent of age, if I give you outside of the scanner, you're done with the scanner, I just scan you while you're watching a movie, just watch it.
You come out, I give you a test of memory for stories.
What happens is, you find this incredible correlation between the activity in the hippocampus at these singular points in time, these event boundaries, and your ability to just remember a story outside of the scanner later on.
So, it's marking this ability to encode memories, just these little snippets of neural activity.
So, I think that's a big one.
Um, there's all sorts of work in animal models that I can get into, you know, sleep, I think there's so much interesting stuff that's being discovered in sleep right now.
Um, being able to just record from large populations of cells, and then be able to relate that, you know what I think the coolest thing gets back to this QR code thing, because, like, what we can do now is, like, I can take fMRI data while you're watching a movie.
Or, let's do better than that.
Let me get fMRI data while you use a joystick to move around in virtual reality, right?
So, you're in the metaverse, whatever, right?
But it's kind of a crappy metaverse, because there's always so much metaversing you can do in an MRI scanner.
So, what you're doing is crappy metaversing.
So, now, I can take a rat, record from his hippocampus and prefrontal cortex and all these areas with these really new electrodes to get massive amounts of data,
and have it move around on a trackball in virtual reality in the same metaverse that I did, and record that rat's activity.
I can get a person with epilepsy who we have electrodes in their brain anyway to try to figure out where the seizures are coming from.
And if it's a healthy part of the brain, record from that person, right?
And I can get a computational model.
And one of the brand-new members in my lab, Tyler Bond, is just doing some great stuff.
He relates computer vision models and looks at the weaknesses of computer vision models and relates it to what the brain does well.
And so, you can actually take a ground truth code for the metaverse, basically, and you can feed in the visual information, let's say the sensory information or whatever that's coming in,
to a computational model that's designed to take real-world inputs, right?
And you could basically tie them all together by virtue of the state spaces that you're measuring in neural activity, in these different formats, in these different species, and in the computational model, which I just find that mind-blowing.
You could do different kinds of analyses on language and basically come up with just like, basically, it's the guts of LLMs, right?
You could do analysis on language and you could do analysis on, you know, sentiment analyses of emotions and so forth and put all this stuff together.
I mean, it's almost too much.
But if you do it right and you do it in a theory-driven way, as opposed to just throwing all the data at the wall and see what sticks, I mean, that to me is just exceptionally powerful.
So you can take fMRI data across species and across different types of humans or conditions of humans and what, find, construct models that help you find the commonalities or like the core thing that makes somebody navigate through the metaverse, for example?
Yeah.
Yeah, I mean, more or less.
I mean, there's a lot of details.
But yes, I think, and not just fMRI, but you can relate it to, like I said, recordings from large populations of neurons that could be taken in a human or even in a non-human animal that is, you know, where you think it's an anatomical homologue.
So that's just mind-blowing to me.
What's the similarities in humans and mice?
That's what is smashing pumpkins, well, just rats in a cage?
Is that smashing pumpkins?
Despite all of your rage.
Is that smashing pumpkins, I think?
Despite all of your rage at GIFs, you're still just rat in a cage.
Oh, yeah.
All right, good callback.
Good callback.
See, these memory retrieval exercises I'm doing are actually helping you build a lasting memory of this conversation.
And it's strengthening the visual thing I have of you with James Brown on stage.
It's just becoming stronger and stronger by the second.
It's hot tub.
But animal studies work here as well.
Yeah, yeah.
So, okay.
So, let's go to the—so, I think I've got, you know, great colleagues who I talk to who study memory in mice, you know, and there's some—one of the valuable things in those models is you can study neural circuits in an enormously targeted way because you could do these genetic studies, for instance, where you can manipulate, like, particular groups of neurons.
And it's just getting more and more targeted to the point where you can actually turn on a particular kind of memory just by activating a particular set of neurons that was active during an experience, right?
So, there's a lot of conservation of some of these neural circuits across, you know, evolution in mammals, for instance.
And then, you know, some people would even say that there's genetic mechanisms for learning that are conserved even going back far, far before.
But let's go back to the mice and humans question, right?
There's a lot of differences.
So, for one thing, the sensory information is very different.
Mice and rats explore the world largely through smelling, olfaction, but they also have vision that's kind of designed to kind of catch death from above.
So, it's like a very big view of the world.
And we move our eyes around in a way that focuses on particular spots in space where you get very high resolution from a very limited set of spots in space.
So, that makes us very different in that way.
We also have all these other structures as social animals that allow us to respond differently.
There's language.
There's like, you know, so you name it.
There's obviously gobs of differences.
Humans aren't just giant rats.
There's much more complexity to us.
Timescales are very important.
So, primate brains and human brains are especially good at integrating and holding on to information across longer and longer periods of time, right?
And also, you know, finally, it's like our history of training data, so to speak, is very, very different than, you know, I mean, a human's world is very different than a wild mouse's world.
And a lab mouse's world is extraordinarily impoverished relative to an adult human, you know?
But still, what can you understand by studying mice?
I mean, just basic, almost behavioral stuff about memory?
Well, yes, but that's very important, right?
So, you can understand, for instance, how do neurons talk to each other?
That's a really big, big question.
Neural computation, in and of itself, you'd think it's the most simple question, right?
Not at all.
I mean, it's a big, big question, and understanding how two parts of the brain interact, meaning that it's not just one area speaking.
It's not like Twitter, where one area of the brain is shouting, and then another area of the brain is just stuck listening to this crap.
It's like they're actually interacting on a millisecond scale, right?
How does that happen?
And how do you regulate those interactions, these dynamic, you know, interactions?
We're still figuring that out, but that's going to be coming largely from model systems that are easier to understand.
You can do manipulations, like drug manipulations, to manipulate circuits and, you know, use viruses and so forth,
and lasers to turn on circuits that you just can't do in humans.
So, I think there's a lot that can be learned from mice.
There's a lot that can be learned from non-human primates.
And then there's a lot that you need to learn from humans.
And I think, unfortunately, some of the people in the National Institutes of Health think you can learn everything from the mouse.
It's like, why study memory in humans when I could study learning in a mouse?
And just like, oh, my God, I'm going to get my funding from somewhere else.
So, let me ask you some random fascinating questions.
Yeah, sure.
How does deja vu work?
So, deja vu is, it's actually one of these things I think that some of the surveys suggest that, like, 75% of people report having a deja vu experience one time or another.
I don't know where that came from, but I've polled people in my class and most of them say they've experienced deja vu.
It's this kind of sense that I've experienced this moment sometime before.
I've been here before.
And actually, there's all sorts of variants of this.
The French have all sorts of names for various versions of this.
Chamie vu, parley vu, I don't know.
It's like all these different vu's.
But deja vu is this sense that it can be, like, almost disturbing, intense sense of familiarity.
So, there is a researcher named Wilder Penfield.
Actually, this goes back even earlier to some of the earliest, like, Hewlings Jackson was this neurologist who did a lot of the early characterizations of epilepsy.
And one of the things you notice is in epilepsy patients, some group of them, right before they would get a seizure, they would have this intense sense of deja vu.
So, it's this artificial sense of familiarity.
It's a sense of having a memory that's not there, right?
And so, what was happening was there was electrical activity in certain parts of these brains.
And so, this guy Penfield later on, when he was trying to look for how do we map out the brain to figure out which parts we want to remove and which parts don't we, he would stimulate parts of the temporal lobes of the brain and find you could elicit the sense of deja vu.
Sometimes, you'd actually get a memory that a person would re-experience just from electrically stimulating some parts.
Sometimes, though, they just have this intense feeling of being somewhere before.
And so, one theory which I really like is, is that in higher order areas of the brain, they're integrating for many, many different, you know, sources of input.
What happens is, is that they're tuning themselves up every time you process a similar input, right?
And so, that allows you to just get this kind of affluent sense that I'm very familiar, you're very familiar with this place, right?
And so, just being here, you're not going to be moving your eyes all over the place because you kind of have an idea of where everything is.
And that fluency gives you a sense of, like, I'm here.
Now, I wake up in my hotel room and I have this very unfamiliar sense of where I am, right?
But, you know, there's a great set of studies done by Anne Cleary at Colorado State where she created these virtual reality environments.
And we'll go back to the metaverse.
Imagine you go through a virtual museum, right?
And then she would put people in virtual reality and have them go through a virtual arcade.
But the map of the two places was exactly the same.
She just put different skins on them.
So, one looks different than the other, but they've got same landmarks and same places, same objects and everything, but carpeting, colors, theme, everything's different.
So, people will often not have any conscious idea that the two are the same, but they could report this very intense sense of deja vu.
So, it's like a partial match that's eliciting this kind of a sense of familiarity.
And that's why, you know, in patients who have epilepsy that affects memory, you get this artificial sense of familiarity that happens.
And so, we think that, and again, this is just one theory amongst many, but we think that we get a little bit of that feeling.
It's not enough to necessarily give you deja vu, even for very mundane things, right?
So, it's like if I tell you the word rutabaga, your brain's going to work a little bit harder to catch it than if I give you a word like apple, right?
And that's because you hear apple a lot.
So, your brain's very tuned up to process it efficiently, but rutabaga takes a little bit longer and more intense.
And you can actually see a difference in brain activity in areas in the temporal lobe when you hear a word just based on how frequent it is in the English language.
So, we think it's tied to this basic, it's basically a byproduct of our mechanism of just learning, doing this error-driven learning as we go through life to become better and better and better to process things more and more efficiently.
So, I guess deja vu is just an extra elevated stuff coming together, firing for this artificial memory as if it's the real memory.
I mean, why does it feel so intense?
Well, it doesn't happen all the time, but I think what may be happening is it's such a, it's a partial match to something that we have.
And it's not enough to trigger that sense of, you know, that ability to pull together all the pieces, but it's a close enough match to give you that intense sense of familiarity without the recollection of exactly what happened when.
But it's also like a spatiotemporal familiarity.
So, like, it's also in time.
Like, there's a weird blending of time that happens.
And we'll probably talk about time because I think that's a really interesting idea how time relates to memory.
But you also kind of, artificial memory brings to mind this idea of false memories that comes in all kinds of contexts.
But how do false memories form?
Well, I like to say there's no such thing as true or false memories, right?
It's like Johnny Rotten from the Sex Pistols.
He had a saying that's like, I don't believe in false memories any more than I believe in false songs, right?
So, it's like, and so the basic idea is that we have these memories that reflect bits and pieces of what happened as well as our inferences and theories, right?
So, I'm a scientist and I collect data, but I use theories to make sense of that data.
And so, a memory is kind of a mix of all these things.
So, where memories can go off the deep end and become what we would call conventionally as false memories are sometimes little distortions where we filled in the blanks, the gaps in our memory based on things that we know, but don't actually correspond to what happened, right?
So, if I were to tell you that I'm like, you know, a story about this person who's like worried that they have cancer or something like that, and then, you know, they see a doctor and the doctor says, well, things are very much like you would have expected or like, you know, what you were afraid of or something.
When people remember that, they'll often remember, well, the doctor told the patient that he had cancer, even if that wasn't in the story, because they're infusing meaning into that story, right?
So, that's a minor distortion.
But what happens is, is that sometimes things can really get out of hand where people have trouble telling the difference in things that they've imagined versus things that happen.
But also, as I told you, the act of remembering can change the memory.
And so, what happens then is you can actually be exposed to some misinformation.
And so, Elizabeth Loftus was a real pioneer in this work, and there's lots of other work that's been done since.
But basically, it's like if you remember some event and then I tell you something about the event, later on, when you remember the event,
you might remember some original information from the event, as well as some information about what I told you.
And sometimes, if you're not able to tell the difference, that information that I told you gets mixed into the story that you had originally.
So now, I give you some more misinformation, or you're exposed to some more information somewhere else.
And eventually, your memory becomes totally detached from what happened.
And so, sometimes you can have cases where people, this is very rare, but you can do it in lab, too, or like a significant, not everybody, but a chunk of people will fall for this,
where you can give people misinformation about an event that never took place.
And as they keep trying to remember that event more and more, what happens is they start to imagine,
they start to pull up things from other experiences they've had.
And eventually, they can stitch together a vivid memory of something that never happened.
Because they're not remembering an event that happened, they're remembering the act of trying to remember what happened.
And basically putting it together into the wrong story.
So it's fascinating, because this could probably happen at a collective level.
Like, this is probably what successful propaganda machines aim to do.
It's creating false memory across thousands, if not millions, of minds.
Yeah, absolutely.
I mean, this is exactly what they do.
And so, all these kind of foibles of human memory get magnified when you start to have social interactions.
There's a whole literature on something called social contagion, which is basically when misinformation spreads like a virus.
Like, you remember the same thing that I did, but I give you a little bit of wrong information.
Then that becomes part of your story of what happened.
Because once you and I share a memory, like I tell you about something I've experienced,
and you tell me about your experience of the same event, it's no longer your memory or my memory, it's our memory.
And so, now the misinformation spreads.
And the more you trust someone, or the more powerful that person is, the more of a voice they have in shaping that narrative, right?
And there's all sorts of interesting ways in which misinformation can happen.
There's a great example of when John McCain and George Bush Jr. were in a primary,
and there were these polls where they would do these, like, I guess they were like not robocalls,
but real calls where they would poll voters.
But they actually inserted some misinformation about McCain's beliefs on taxation, I think,
and maybe it was something about illegitimate children or something.
I don't really remember.
But they included misinformation in the question that they asked, like, you know,
how do you feel about the fact that he wants to do this or something?
And so, people would end up becoming convinced he had these, you know, policy things or these personal things
that were not true, just based on the polls that were being used.
So, it was a case where, interestingly enough, the people who were using misinformation
were actually ahead of the curve relative to the scientists who were trying to study these effects in memory.
Yeah, it's really interesting.
So, it's not just about truth and falsehood.
It's like us as intelligent reasoning machines.
But it's the formation of memories where they become, like, visceral.
You can rewrite history.
If you just look throughout the 20th century,
some of the dictatorships with Nazi Germany, with the Soviet Union,
effective propaganda machines can rewrite our conceptions of history,
how we remember our own culture, our upbringing, all this kind of stuff.
And you could do quite a lot of damage in this way.
And then there's probably some kind of social contagion happening there.
Like, certain ideas that may be initiated by the propaganda machine
can spread faster than others.
You could see that in modern day, certain conspiracy theories,
there's just something about them that they are, like, really effective at spreading.
There's something sexy about them to people,
to where something about the human mind eats it up
and then uses that to construct memories as if they almost were there
to witness whatever the content of the conspiracy theory is.
It's fascinating.
Because once you feel like you remember a thing,
I feel like there's a certainty.
It emboldens you to, like, say stuff.
Like, you really, like, it's not just you believe an idea is true or not.
You're like, it's at the core of your being that you feel like you were there
to watch the thing happen.
Yeah.
I mean, there's so much in what you're saying.
I mean, one of the things is that people's sense of collective identity
is very much tied to shared memories.
If we have a shared narrative of the past, or even better, if we have a shared past,
we will feel more socially connected with each other,
and I will feel part of this group.
They're part of my tribe, if I remember the same things in the same way.
And you brought up this weaponization of history.
And, you know, it really speaks to, I think, one of the parts of memory,
which is that if you have a belief, you will find, and you have a goal in mind,
you will find stuff in memory that aligns with it,
and you won't see the parts in memory that don't.
So a lot of the stories we put together are based on our perspectives.
And so let's just zoom out for the moment from, like, misinformation.
It takes something even more fascinating, but not as, like, you know, scary.
I was reading Thanh Viet Nguyen,
but he wrote a book about the collective memory of the Vietnam War.
He's a Vietnamese immigrant who was flown out after the war was over.
And so he went back to his family to get their stories about the war.
And they called it the American War, not the Vietnam War, right?
And that just kind of blew my mind, having grown up in the U.S.
and having always heard about it as the Vietnam War.
But of course they call it the American War, because that's what happened.
America came in, right?
And that's based on their perspective, which is a very valid perspective.
And so that just gives you this idea of the way we put together these narratives
based on our perspectives.
And I think the opportunities that we can have in memory
is if we bring groups together from different perspectives
and we allow them to talk to each other and we allow ourselves to listen.
I mean, right now you'll hear a lot of just jammering, you know,
people going blah, blah, blah about free speech,
but they just want to listen to themselves, right?
I mean, it's like, let's face it, the old days before people were supposedly woke,
they were trying to ban two live crew or, you know, just think about it.
Letty Bruce got canceled for cursing.
Jesus Christ, you know?
It's like, this is nothing new.
People don't like to hear things that disagree with them.
But if you're in a—I mean, you can see two situations in groups with memory.
One situation is you have, like, people who are very dominant
who just take over the conversation.
And they—basically what happens is the group remembers less from the experience
and they remember more of what the dominant narrator says, right?
Now, if you have a diverse group of people—and I don't mean diverse
in necessarily the human resources sense of the word.
I mean, diverse in any way you want to take it, right?
But diverse in every way, hopefully.
And you give everyone a chance to speak
and everyone's being appreciated for their unique contribution.
You get more accurate memories and you get more information from it, right?
Even two people who come from very similar backgrounds,
if you can appreciate the unique contributions that each one has,
you can do a better job of generating information from memory.
And that's a way to inoculate ourselves, I believe,
from misinformation in the modern world.
But like everything else, it requires a certain tolerance for discomfort.
And I think when we don't have much time,
and I think when we're stressed out,
and when we are just tired,
it's very hard to tolerate discomfort.
And I mean, social media has a lot of opportunity for this
because it enables this distributed one-on-one interaction
that you're talking about where everybody has a voice,
but still our natural inclination,
you see this on social media,
is there's a natural clustering of people and opinions,
and you just kind of form these kind of bubbles.
I think that's, to me personally,
I think that's a technology problem that could be solved.
If there's a little bit of interaction,
kind, respectful, compassionate interaction with people
that have a very different memory,
that respectful interaction will start to intermix the memories
and ways of thinking to where you're slowly moving towards truth.
But that's a technology problem
because naturally, left to our own devices,
we want to cluster up in a tribe.
Yeah, and that's the human problem.
I think a lot of the problems that come up with technology
aren't the technology itself,
as much as the fact that people adapt to the technology
in maladaptive ways.
I mean, one of my fears about AI
is not what AI will do, but what people will do.
I mean, take text messaging, right?
It's like a pain in the ass to text people,
at least for me.
And so what happens is the communication becomes very Spartan
and devoid of meaning, right?
It's just very telegraphic.
And that's people adapting to the medium, right?
I mean, look at you.
You've got this keyboard, right?
That's got these dome-shaped things.
And you've adapted to that to communicate, right?
That's not the technology adapting to you.
That's you adapting to the technology.
And I think one of the things I learned
when Google started to introduce autocomplete in emails,
I started to use it.
And about a third of the time, I was like,
this isn't what I want to say.
A third of the time, I'd be like,
this is exactly what I wanted to say.
And a third of the time, I was saying,
well, this is good enough.
I'll just go with it, right?
And so what happens is,
it's not that the technology necessarily
is doing anything so bad,
as much as it's just going to constrain my language
because I'm just doing what's being suggested to me.
And so this is why I say,
kind of like my mantra for some of what I've learned
about everything in memory is to diversify
your training data, basically,
because otherwise you're going to be...
So like humans have this capability
to be so much more creative
than anything generative AI will put together,
at least right now, who knows where this goes.
But it can also go the opposite direction
where people could become much, much less creative
if they just become more and more resistant
to discomfort and resistant to exposing themselves
to novelty, to cognitive dissonance, and so forth.
I think there is a dance
between natural human adaptation of technology
and the people that design the engineering
of that technology.
So I think there's a lot of opportunity
to create this keyboard,
things that on net are positive for human behavior.
So we adapt and all this kind of stuff,
but when you look at the long arc of history
across years and decades,
has humanity been flourishing?
Are humans creating more awesome stuff?
Are humans happier?
All that kind of stuff.
And so there I think technology on net has been,
and I think, maybe hope,
will always be on net a positive thing.
Do you think people are happier now
than they were 50 years ago or 100 years ago?
Yes, yes.
I don't know about that.
I think humans in general
like to reminisce about the past,
like the times are better.
That's true.
And complain about the weather today
or complain about whatever today
because there's this kind of complainy engine
that's just, there's so much pleasure in saying,
you know, life sucks for some reason.
That's why I love punk rock.
Exactly.
I mean, there's something in humans
that loves complaining,
even about trivial things,
but complaining about change,
complaining about everything.
But ultimately, I think on net,
on every measure,
things are getting better.
Life is getting better.
Oh, life is getting better,
but I don't know necessarily
that tracks people's happiness, right?
I mean, I would argue that maybe,
who knows?
I don't know this,
but I wouldn't be surprised
if people in hunter-gatherer societies
are happier.
I mean, I wouldn't be surprised
if they're happier
than people who have access
to modern medicine
and email and cell phones.
Well, I don't think there's a question
whether you take hunter-gatherer folks
and put them into modern day
and give them enough time to adapt.
They would be much happier.
The question is,
in terms of every single problem
they've had is now solved.
There's now food.
There's guarantee of survival,
shelter, and all this kind of stuff.
So what you're asking
is a deeper sort of biological question.
Do we want to be,
oh, Werner Herzog in the movie,
Happy People, Life in the Taiga.
Do we want to be busy 100% of our time
hunting, gathering, surviving,
worried about the next day?
Maybe that constant struggle
ultimately creates a more fulfilling life.
I don't know.
But I do know
this modern society
allows us to,
when we're sick,
to find medicine,
to find cures,
when we're hungry,
to get food much more
than we did even 100 years ago.
And there's many more activities
that you could perform
or create all these kinds of stuff
that enables the flourishing
of humans at the individual level.
So whether that leads to happiness,
I mean,
that's a very deep philosophical question.
Maybe struggle,
deep struggle is necessary
for happiness.
Or maybe cultural connection,
you know,
maybe it's about, like,
functioning in social groups
that are meaningful
and, like, having time.
But I do think this is,
there's an interesting
memory-related thing,
which is that if you look at, like,
things like reinforcement learning,
for instance,
you're not learning necessarily
every time you get a reward,
if it's the same reward,
you're not learning that much.
You mainly learn
if it deviates from your expectation
of what you're supposed to get, right?
So it's like,
you get a paycheck
every, you know,
month from MIT or whatever, right?
And it's like,
you're kind of,
you probably don't even
kind of get excited about it
when you get the paycheck.
But if they cut your salary,
you're going to be pissed.
And if they increase your salary,
you're like,
oh, good, I got a bonus, you know?
And that adaptation
and that ability
that basically you learn
to expect these things,
I think,
is a major source of,
I guess,
it's a major way
in which we're kind of more,
in my opinion,
wired to strive
and not be happy
to be in a state of wanting.
And, you know,
some people talk about dopamine,
for instance,
being this pleasure chemical.
And it's like,
there's a lot of compelling research
to suggest it's not
about pleasure at all.
It's about the discomfort
that energizes you
to get things,
to seek a reward, right?
And so,
you could give an animal
that's been deprived
of dopamine
a reward
and, oh, yeah,
I enjoy it.
It's pretty good.
But they're not going
to do anything
to get it, you know?
And just one of the weird things
in our research
is I got into curiosity
from a postdoc
in my lab,
Matthias Gruber.
And one of the things
that we found
is when we gave people
a question,
like a trivia question
that they wanted
the answer to,
that question,
the more curious people
were about the answer,
the more activity
in these dopamine-related
circuits in the brain
we would see.
And, again,
that was not driven
by the answer per se,
but by the question.
So,
it was not about
getting the information,
it was about the drive
to seek the information.
but it depends
on how you take that.
If you get this
uncomfortable gap
between what you know
and what you want to know,
you could either use that
to motivate you
and energize you
or you could use it
to say,
I don't want to hear
about this,
this disagrees
with my beliefs,
I'm going to go back
to my echo chamber,
you know?
Yeah.
I like what you said
that maybe we're designed
to be in a kind of
constant state of wanting,
which, by the way,
is a pretty good
either band name
or rock song name,
state of wanting.
That's like a hardcore band name.
Yeah, yeah, yeah.
Yeah, it's pretty good.
But I also like
the hedonic treadmill.
Hedonic treadmill
is pretty good.
Yeah, yeah,
we could use that
for like our techno project,
I think.
You mean the one
we're starting?
Yeah, exactly.
Okay, great.
We're going on tour soon.
This is our announcement.
We could build a false memory
of a show,
in fact,
if we want.
Let's just put it all together.
We don't even have to do
all the work to play the show.
We can just create
a memory of it
and it might as well happen
because the remembering self
is in charge anyway.
So let me ask you about,
we talked about false memories,
but, you know,
in the legal system,
false confessions.
I remember reading 1984
where,
sorry for the dark turn
of our conversation,
but through torture
you can make people say anything
and essentially remember anything.
I wonder to which degree
there's like truth to that
if you look at the torture
that happened in the Soviet Union
for confessions,
all that kind of stuff.
How much can you really
get people to really,
you know,
to force false memories,
I guess.
Yeah, I mean,
I think there's a lot
of history of this
actually in the criminal
justice system.
You might have heard
the term the third degree.
If you actually look it up,
historically,
it was a very intense
set of beatings
and, you know,
starvation and physical
demands that they would
place at people
to get them to talk.
And, you know,
there's certainly a lot
of work in the,
that's been done
by the CIA
in terms of enhanced
interrogation techniques.
And from what I understand,
the research actually shows
that they just produce
what people want to hear,
not necessarily
the information
that is being looked for.
And the reason is,
is that,
I mean,
there's different reasons.
I mean,
one is people just get tired
of being tortured
and just say whatever.
But another part of it
is,
is that you create
a very interesting
set of conditions
where there's
an authority figure
telling you something
that you did this,
we know you did this,
we have witnesses
saying you did this.
So now you start
to question yourself.
Then they put you
under stress.
Maybe they're
not feeding you,
maybe they're
kind of like
making you be cold
or, you know,
exposing you to
like music
that you can't stand
or something,
whatever it is,
right?
It's like they're,
they're creating
this physical stress.
And so stress
starts to act on,
you know,
starts to downregulate
the prefrontal cortex.
You're not necessarily
as good at monitoring
the accuracy of stuff.
Then they start
to get nice to you
and they say,
imagine,
you know,
okay,
I know you don't
remember this,
but maybe we can
walk you through
how it could have happened
and they feed you
the information.
And so you're
in this weakened
mental state
and you're being
encouraged to imagine
things by people
who give you
a plausible scenario.
And at some point,
certain people
can be very coaxed
into creating a memory
for something
that never happened.
And there's actually
some pretty convincing
cases out there
where you don't know
exactly the truth.
There's a sheriff,
for instance,
who came to believe
that he had
a false memory.
I mean,
that he had a memory
of doing sexual abuse
based on,
you know,
essentially,
I think it was,
you know,
I'm not going to tell
the story
because I don't remember
it well enough
to necessarily
accurately give it to you,
but people could
look this stuff up.
There are definitely
stories out there
like this
where people
confessed to crimes
that they just
didn't do
and objective evidence
came out later on.
But there's a basic
recipe for it,
which is
you feed people
the information
that you want
them to remember.
You stress them out.
You have an authority
figure
kind of like
pushing this information
on them
or you motivate them
to produce
the information
you're looking for
and that pretty much
over time
gives you
what you want.
It's really tragic
that
centralized
power
can use
these kinds
of tools
to destroy lives.
Sad.
Since there's
a theme
about music
throughout this
conversation,
one of the best
topics
for songs
is heartbreak,
love in general,
but heartbreak.
Why and how
do we remember
and forget heartbreak?
Asking for a friend.
Oh, God,
that's so hard
to...
Asking for a friend,
I love that.
Oh, it's such
a hard one.
Well, so,
I mean,
part of this
is
we tend
to
go back
to particular
times
that are
the more
emotionally
intense
periods.
And so
that's
a part
of it.
And again,
memory is
designed to
kind of capture
these things
that are
biologically
significant.
And attachment
is a big
part of
biological
significance
for humans,
right?
Human
relationships
are super
important.
And sometimes
that heartbreak
comes with
massive changes
in your beliefs
about somebody,
say they cheated
on you or
something like
that,
or regrets
and you
kind of
ruminate
about things
that you've
done
wrong.
There's
really so
many reasons
though,
but,
you know,
I mean,
I've had
this.
My first
pet I had
as,
you know,
was,
we got it
for a wedding
present as a
cat and
got it
after,
like,
but it died
of FIP
when it was
four years
old.
And,
you know,
I just would
see her
everywhere
around the
house,
you know.
We got
another cat
that we got
a dog.
Dog eventually
died of
cancer and
the cat
just died
recently.
And,
you know,
so we got
a new dog
because I
kept seeing
the dog
around and
I was just
so heartbroken
about this.
But I still
remember the
pets that
died.
It just
comes back
to you.
I mean,
it's part of
this.
I think
there's also
something about
attachment
that's just
so crucial
that drives,
again,
these things
that we
want to
remember and
that gives
us that
longing
sometimes.
Sometimes it's
also not
just about
the heartbreak
but about
the positive
aspects of
it,
right?
Because the
loss comes
from not
only the
fact that
the relationship
is over,
but you had
all of these
good things
before that
you can now
see in a
new light,
right?
And so
part of one
of the things
that I found
from my
clinical background
that really I
think gave me
a different
perspective on
memory is
so much of
the therapy
process was
guided towards
reframing and
getting people
to look at
the past in
a different
way,
not by
changing people's
memories or
not by imposing
an interpretation
but just
offering a
different
perspective and
maybe one
that's kind
of more
optimized towards
learning and
appreciation maybe
or gratitude,
whatever it is,
right,
that gives you
a way of
taking,
I think you
said it in
the beginning,
right,
where you can
have this kind
of like dark
experiences and
you can use
it as
training data
to, you
know, grow
in new ways
but it's
hard.
This, I
often go back
to this moment,
this show
Louis with
Louis C.K.
where he's
all heartbroken
about a
breakup with
a woman he
loves and
an older
gentleman
tells him
that that's
actually the
best part,
that heartbreak
because you
get to
intensely
experience how
valuable this
love was.
He says the
worst part is
forgetting it,
is actually when
you get over the
heartbreak,
that's the worst
part.
So, I
sometimes think
about that
because, you
know, having
the love and
losing it, like
the losing it is
when you sometimes
feel it the
deepest, which is
an interesting
way to celebrate
the past and
relive it.
It sucks that you
don't have a thing
but when you don't
have a thing,
it's a good
moment to viscerally
experience the
memories of
something that
you now
appreciate even
more.
So, you don't
believe that an
owner of a lonely
heart is much
better than an
owner of a
broken heart?
You think an
owner of a
broken heart is
better than the
owner of a
lonely heart?
Yes, for sure.
I think so.
I think so.
But I'm gonna
have to, day by
day, I don't
know, I'm gonna
have to listen to
some more Bruce
Springsteen to
figure that one
out.
Well, you know,
it's funny because
it's like after I
turned 50, I think
of death all the
time.
Like, I just
think that, you
know, I'm in
like, I
probably, I
have fewer,
probably a
fewer years ahead
of me than I
have behind me,
right?
So, I think about
one thing, which
is what are the
memories that I
want to carry
with me for the
next period of
time?
And also about
like just the
fact that
everything around
me could be,
you know, I
know more
people who are,
you know, dying
for various
reasons.
And so, I'm
not lots, I'm
not that old,
right?
But, you know,
it's something I
think about a
lot and I'm
reminded of
like how I
talked to
somebody who's
like, you
know, who's a
Buddhist and I
was like, you
know, the whole
idea of
Buddhism is
renouncing
attachments.
Some way the
idea of
Buddhism is
like staying
out of the
world of
memory and
staying in
the moment,
right?
And they
talked about,
you know, it's
like how do
you renounce
attachments to
the people that
you love,
right?
And they're
just saying,
well, I
appreciate that I
have this moment
with them and
knowing that they
will die makes me
appreciate this
moment that much
more.
I mean, you said
something similar,
right, in your
daily routine that
you think about
things this way,
right?
Yeah, I meditate
on mortality
every day.
But I don't
know, at the
same time, that
really makes you
appreciate the
moment and live
in the moment
and I also
appreciate the
full deep
rollercoaster of
suffering involved
in life, the
little and the
big, too.
So I don't
know.
The Buddhist
kind of removing
yourself from the
world or the
stoic removing
yourself from the
world, the world
of emotion, I'm
torn about that
one.
I'm not sure.
Well, you know,
this is where
Hinduism and
Buddhism, or at
least some strains
of Hinduism and
Buddhism, differ.
In Hinduism, like
if you read the
Bhagavad Gita, the
philosophy is not
one of renouncing
the world, because
the idea is that
not doing
something is no
different than
doing something,
right?
So what they
argue, and again,
you could interpret
in different ways,
positive and
negative, but the
argument is that
you don't want to
renounce action, but
you want to
renounce the fruits
of the action.
You don't do it
because of the
outcome, you do it
because of the
process, because the
process is part of
the balance of the
world that you're
trying to preserve,
right?
And of course, you
could take that
different ways, but I
really think about
that from time to
time in terms of
like, you know,
letting go of this
idea of does this
book sell, or
trying to, you
know, like, impress
you and get you to
laugh at my jokes or
whatever, and just be
more like I'm sharing
this information
with you and, you
know, getting to
know you or
whatever it is, but
it's hard, right?
It's like, because
we're so driven by
the reinforcer, the
outcome.
You're just part of
the process of
telling the joke, and
if I laugh or not,
that's up to the
universe to decide.
Yep, it's my
dharma.
How does studying
memory affect your
understanding of the
nature of time?
So, like, we've been
talking about us
living in the
present and making
decisions about the
future, standing on
the foundation of
these memories and
narratives about the
memories that we've
constructed.
So, it feels like
it does weird
things to time.
Yeah, and the
reason is, is that
in some sense, I
think, especially
the farther we go
back, I mean,
there's all sorts of
interesting things that
happen.
So, your sense of,
like, if I ask you
how different does
one hour ago
feel from two
hours ago, you'd
probably say pretty
different.
But if I ask you,
okay, go back one
year ago versus one
year and one hour
ago, it's the same
difference in time.
It won't feel very
different, right?
So, there's this kind
of compression that
happens as you look
back farther in time.
So, it's kind of like
why when you're older,
the difference between
somebody who's, like,
50 and, you know,
45 doesn't seem as
big as the difference
between, like, 10 and
five or something,
right?
When you're 10 years
old, everything seems
like it's a long
period of time.
Here's the point is
that, you know, so
one of the interesting
things that I found
when I was working on
the book, actually,
was during the
pandemic, I just
decided to ask people
in my class when we
were doing the
remote instruction.
So, one of the
things I did was I
would poll people.
And so, I just ask
people, do you feel
like the days are
moving by slower or
faster or about the
same?
Almost everyone in the
class said that the
days were moving by
slower.
So, then, I would
say, okay, so, do
you feel like the
weeks are passing by
slower, faster, or
the same?
And the majority of
them said that the
weeks were passing by
faster.
So, according to the
laws of physics, I
don't think that makes
any sense, right?
Yeah.
But, according to
memory, it did, because
what happened was people
were doing the same
thing over and over in
the same context.
And without that
change in context,
their feeling was that
they were in one long
monotonous event.
And so, but then, at
the end of the week, you
look back at that week
and you say, well, what
happened?
I have no memories of
what happened.
So, it must, the week
just went by without even
my noticing it.
But that week went by
during the same amount
of time as an eventful
week where you might have
been going out and
hanging out with friends
on vacation or whatever,
right?
It's just that nothing
happened because you're
doing the same thing
over and over.
So, I feel like memory
really shapes our sense
of time.
But it does so in part
because context is so
important for memory.
Well, that compression
you mentioned, it's an
interesting process.
Because what I think
about when I was like
12 or 15, I just
fundamentally feel like
the same person.
It's interesting what
that compression does.
It makes me feel like
it's all, we're all
connected, not just
amongst humans and
spatially, but in
terms, back in time.
There's a kind of
eternal nature, like the
timelessness, I guess,
to life.
That could be also a
genetic thing just for
me.
I don't know if everyone
agrees to this view of
time.
But to me, it all feels
the same.
Like you don't feel the
passage of time?
No, I feel the passage
of time the same way
that your students did
from day to day.
There's certain markers
that let you know that
time has passed, you
celebrate birthdays and
so on.
But the core of who I am
and who others I know
are or events, it like
that compression of my
understanding of the
world removes time.
Because time is not
useful for the
compression.
So like the details of
that time, at least for
me, is not useful to
understanding the core of
the thing.
Maybe what it is is that
you really like to see
connections between
things.
This is like really what
motivates me in science
actually too.
But it's like when you
start recalling the past
to, you know, and
seeing the connections
between the past and
present, now you have
this kind of web of
interconnected memories,
right?
And so I can imagine
in that sense, there
is this kind of the
present is with you,
right?
But what's interesting
about what you said
too that struck me is
that your 16-year-old
self was probably very
complex, you know?
And I'm, by the way,
I'm the same way, but
it's like it really is
the source of a lot of
darkness for me.
So, but when like you
can look back at like
let's say you hear a
song that you used to
play like before you
would go do a sports
thing or something like
that, and you might not
think of yourself as an
athlete, but once you
get back to that
mentally time travel to
that particular thing,
you open up this little
compartment of yourself
that wasn't there
before, right?
That didn't seem
accessible for them.
Dan Schachter's lab did
this really cool study
where they would ask
people to either
remember doing
something altruistic
or imagine doing
something altruistic.
And that act
made them
more likely
to want to do
things for other
people.
So, that
act of mental
time travel can
change who you are
in the present.
And we tend to think
of, this goes back
to that illusion of
stability, and we
tend to think of
memory in this very
deterministic way that
I am who I am
because I have this
past, but we have a
very multifaceted
past and
can access
different parts
of it and
change in the
moment based on
whatever part we
want to reach
for, right?
How does
nostalgia connect
into this?
Like this
desire and
pleasure associated
with
going back?
Yeah, so
my friend
Felipe de
Bregard
wrote this
and it just
like blew my
mind where
the word
nostalgia was
coined by a
Swiss physician
who was actually
studying traumatized
soldiers, and so
he described
nostalgia as a
disease, and
the idea was
it was bringing
these people
extraordinary
unhappiness because
they're remembering
how things used
to be, and I
think it's very
complex.
So as people
get older, for
instance, nostalgia
can be an
enormous source
of happiness,
right?
And being
nostalgic can
improve people's
moods in the
moment, but it
just depends on
what they do with
it because what
you can sometimes
see is, nostalgia
has the opposite
effect of
thinking those
were the good
old days and
those days are
over, right?
It's like, America
used to be so
great and now it
sucks, or, you
know, my life
used to be so
great when I
was a kid and
now it's not,
right?
And you're
selectively
remembering the
things that, I
mean, we don't
realize how
selective our
remembering self
is.
And so, you
know, I lived
through the
70s, it
sucked, you
know, it's
like, partly
it sucked more
for me, but I
would say that
even otherwise
it's like there's
all sorts of
problems going on.
Gas lines, people
were like, you
know, worried
about like Russia,
nuclear war, blah,
blah, blah.
So, I mean, it's
just this idea that
people have about
the past can be
very useful if it
brings you happiness
in the present, but
if it narrows your
worldview in the
present, you're not
aware of those
biases that you
have, you will end
up, you can end
up, it can be
toxic, right, either
at a personal
level or at a
collective level.
Let me ask you
both a practical
question and an
out there question.
So, let's start
with the more
practical one.
What are your
thoughts about
BCIs, brain
computer interfaces,
and the work
that's going on
with Neuralink?
We talked about
electrodes and
different ways of
measuring the
brain, and here
Neuralink is
working on basically
two-way communication
with the brain.
And the more
out there question
would be like,
where does this
go?
But more
practically, in the
near term, what
do you think about
Neuralink?
Yeah, I mean, I
can't say specifics
about the company
because I haven't
studied it that
much, but I
mean, I think
there's two parts
of it.
So, one is they're
developing some
really interesting
technology, I
think, with these
surgical robots and
things like that.
BCI, though, has
a whole lot of
innovation going on.
I'm not necessarily
seeing any
scientific evidence
from Neuralink,
and maybe that's
just because I'm
not looking for
it, but I'm not
seeing the evidence
that they're
anywhere near where
the scientific
community is.
And there's lots
of startups that
are doing
incredibly innovative
stuff.
One of my
colleagues, Sergey
Stavisky, is just
like a genius in
this area, and
they're working on
it.
I think speech
prosthetics that are
incorporating decoding
techniques with AI
and movement
prosthetics, the rate
of progress is just
enormous.
So part of the
technology is having
good enough data and
understanding which
data to use and what
to do with it, right?
And then the other
part of it, then, is
the algorithms for
decoding it and so
forth.
And I think part of
that has really
resulted in some real
breakthroughs in
neuroscience as a
result.
So there's lots of
new technologies like
neuropixels, for
instance, that allow
you to harvest
activity from many,
many neurons from a
single electrode.
I know Neuralink has
some technologies
that are also along
these lines, but I
haven't, again,
because they do their
own stuff, the
scientific community
doesn't see it,
right?
But I think BCI is
much, much bigger
than Neuralink, and
there's just so much
innovation happening.
I think the
interesting question,
which we may be
getting into, is I
was talking to
Sergey a while ago
about, you know, so
a lot of language is
not just what we
hear and what we
speak, but also
our intentions and
our internal models.
And, you know, so
are you really going
to be able to
restore language
without dealing
with that part of
it?
And he brought up a
really interesting
question, which is
the ethics of
reading out people's
intentions and
understanding of the
world as opposed to
the more, you
know, the more
concrete parts of
hearing and
producing movements,
right?
Just so we're
clear, because you
said a few
interesting things.
When we talk about
language and BCIs,
what we mean is
getting signal from
the brain and
generating the
language, say you're
not able to actually
speak, it's as a
kind of linguistic
prosthetic.
It's able to speak
for you exactly what
you want it to say.
And then the deeper
question is, well,
saying something isn't
just the letters,
the words you're
saying, it's also
the intention behind
it, the feeling
behind it, all that
kind of stuff.
And is it ethical
to reveal that
full shebang, the
full context of
what's going on in
our brain?
That's really
interesting.
That's really
interesting.
I mean, our
thoughts.
Is it ethical for
anyone to have
access to our
thoughts?
Because right now
the resolution is so
low that we're
okay with it, even
doing studies and all
this kind of stuff.
But if neuroscience
has a few
breakthroughs to where
you can start to
map out the QR
codes for different
thoughts, for
different kinds of
thoughts, maybe
political thoughts,
you know, McCarthyism.
What if I'm getting
a lot of them
communist thoughts
or however we want
to categorize or
label it?
That's interesting.
That's really
interesting.
I think ultimately
this always, the
more transparency there
there is about the
human mind, the
better it is.
But there could be
always intermediate
battles with how
much control does a
centralized entity have
like a government and
so on.
What is the
regulation?
What are the rules?
What are the, what's
legal and illegal?
You know, if you talk
about the police whose
job is to track down
criminals and so on and
you look at all the
history, how the police
could be abused its
power to control the
citizenry, all that
kind of stuff.
So people are always
paranoid and rightfully
so.
It's fascinating.
It's really fascinating.
You know, we talk about
freedom of speech, you
know, freedom of
thought, which is also a
very important liberty at
the core of this country
and probably humanity,
starts to get awfully
tricky when you start to be
able to collect those
thoughts.
But what I wanted to
actually ask you is, do
you think for fun and
for practical purposes,
you'll be able to, we
would be able to modify
memories?
So how difficult is it to,
how far away we are from
understanding the different
parts of the brains,
everything we've been
talking about, in order to
figure out how can we
adjust this memory?
At the crude level from
unpleasant to pleasant, you
talked about, we can
remember the mall and the
people, like location, the
people, can we keep the
people and change the
place, like this kind of
stuff.
How difficult is that?
Well, I mean, in some
sense, we know we can do it
just behaviorally, right?
Behaviorally, yes.
Like tell you, give, you
know, under certain
conditions anyway, it can
give you the
misinformation and then
you can change the
people and places and so
forth, right?
On the crude level, there's
a lot of work that's
being done on a
phenomenon called
reconsolidation, which
is the idea that
essentially when I recall
a memory, what happens
is that the connections
between the neurons and
that cell assembly that
give you the memory are
going to be like more
modifiable.
And so some people have
used techniques to try to
like, for instance, with
fear memories, to reduce
that physical, visceral
component of the memory
when it's being
activated.
Right now, I think I've,
as an outsider looking at
the data, I think it's
like mixed results.
And part of it is, and
this speaks to the more
complex issue, is that
you don't, you need
somebody to actually fully
recall that traumatic
memory in the first
place and in order to
actually modify it, then
what is the memory?
That is the key part of
the problem.
So if we go back to
reading people's
thoughts, what is the
thought?
I mean, people can
sometimes look at this
like behaviorists and go,
well, the memory is like
I've given you A and you
produce B.
But I think that's a very
bankrupt concept about
memory.
I think it's much more
complicated than that.
And, you know, one of the
things that when we
started studying naturalistic
memory, like memory from
movies, that was so hard
was we had to change the
way we did the studies.
Because if I show you a
movie and I show and I
watched the same movie and
you recall everything that
happened and I recall
everything that happened, we
might take a different
amount of time to do it.
We might use different
words.
And yet to an outside
observer, we might have
recalled the same thing.
Right.
So it's not about the
words necessarily.
And it's not about how
long we spent or whatever.
There's something deeper
that is there.
That's this idea.
But it's like, how do you
understand that thought?
I encounter a lot of
concrete thinking that it's
like, if I show a model,
like, you know, the visual
information that a person
sees when they drive, I can
basically reverse engineer
driving.
Well, that's not really how
it works.
I once saw a talk by
somebody or I saw somebody
talking in this discussion
of between neuroscientists and
AI people.
And he was saying that the
problem with self-driving
cars that they had in cities
as opposed to highways was
that the car was OK at, you
know, doing the things it's
supposed to.
But when there are
pedestrians around, it
couldn't predict the
intentions of people.
And so that unpredictability
of people was the problem
that they were having in, you
know, the self-driving car
design, because it didn't
have a good enough internal
model of what the people
were, you know, what they
were doing, what they
wanted.
And what do you think
about that?
Well, I spent a huge amount
of time watching pedestrians,
thinking about pedestrians,
thinking about what it takes
to solve the problem of
measuring, detecting the
intention of a pedestrian,
really of a human being in
this particular context of
having to cross the street.
and it's fascinating.
I think it's a window into
how complex social systems
are that involve humans.
Because, you know, I would
just stand there and watch
intersections for hours.
And what you start to figure
out is every single
intersection has its own
personality.
So, like, there's a history to
that intersection.
Like, jaywalking, certain
intersections allow jaywalking
a lot more.
Because what happens is
we're leaders and followers.
So, there's a regular, let's
say, and they get off the
subway and they start crossing
on a red light, and they do
this every single day.
And then there's people that
don't show up to that
intersection often, and they're
looking for cues of how we're
supposed to behave here.
And if a few people start to
jaywalk and cross on a red
light, they will also, they
will follow.
And there's just a dynamic to
that intersection.
There's a spirit to it.
And if you look at Boston
versus New York, versus a
rural town, versus even Boston,
San Francisco, or here in
Austin, there's different
personalities citywide, but
there's different personalities
areawide, regionwide, and
there's different personalities,
different intersections.
And it's just fascinating.
For a car to be able to
determine that, it's tricky.
Now, what machine learning
systems are able to do well is
collect a huge amount of
data.
So, for us, it's tricky
because we get to, like,
understand the world with very
limited information and make
decisions grounded in this big
foundation model that we've
built of understanding how
humans work.
AI could literally, in the
context of driving, this is
where I've often been really
torn in both directions.
If you just collect a huge
amount of data, all of that
information, and then compress
it into a representation of
how humans cross streets, it's
probably all there.
In the same way that you have a
Noam Chomsky who says, no, no,
no, AI can't talk, can't write
convincing language without
understanding language.
And, you know, more and more you
see large language models without
quote-unquote understanding can
generate very convincing language.
But I think what the process of
compression from a huge amount of
data compressing into a
representation is doing is, in
fact, understanding deeply.
In order to be able to generate
one letter at a time, one word at
a time, you have to understand the
cruelty of Nazi Germany, and the beauty
of sending humans to space.
And, like, you have to understand all
of that in order to generate, like,
I'm going to the kitchen to get an
apple, and do that
grammatically correctly.
You have to have a world model that
includes all of human behavior.
You think an LLM is building that
world model?
It has to, in order to be good at
generating one word at a time, a
convincing sentence.
And in the same way, I think AI that
drives a car, if it has enough data,
will be able to form a world model that
will be able to predict correctly what
the pedestrian does.
But when we, as humans, are watching
pedestrians, we slowly realize, damn,
this is really complicated.
In fact, when you start to self-reflect
on driving, you realize driving is
really complicated.
There's, like, subtle cues we take
about, like, just, there's a million
things I could say, but, like, one of
them determining who around you is an
asshole, aggressive driver, potentially
dangerous.
Yes, yes, I was just thinking about
this, yeah.
Or, like.
You can read it a mile, once you get,
become a great driver, you can see it
a mile away, this guy is going to pull
an asshole move in front of you.
Yeah, exactly.
He's, like, way back there, but you
know it's going to happen.
And I don't know what, because we're
ignoring all the other cars, but for
some reason, the asshole, like, a red,
like, like, a glowing, obvious symbol
is just, like, right there, even in the
periphery vision, because we're, again,
we're usually, when we're driving, just
looking forward, but we're, like, using
the periphery vision to figure stuff out,
and it's, like, a little puzzle that
we're usually only allocating a small
amount of our attention to, at least
a cognitive attention to.
I mean, it's fascinating, but I think
AI just has a fundamentally different
suite of sensors in terms of the
bandwidth of data that's coming in
that allows you to form the
representation and perform inference on
the representation you, using the
representation you form, that for the
case of driving, I think it could be
quite effective.
But one of the things that's currently
missing, even though OpenAI just
recently announced adding memory, and I
did want to ask you, like, how important
it is, how difficult is it to add some of
the memory mechanisms that you've seen in
humans to AI systems?
I would say superficially not that hard, but
then in a deeper level very hard, because
we don't understand episodic memory, right?
And so one of the ideas I talk about in
the book is one of the oldest kind of
dilemmas in computational neuroscience is
what Steve Grossberg called the stability
plasticity dilemma, right?
When do you say something is new and
overwrite your pre-existing knowledge
versus going with what you had before and
making incremental changes?
And so, you know, part of the problem with
going through, like, massive, you know, I
mean, part of the problem of things like if
you're trying to design an LLM or something
like that is, especially for English, there's
so many exceptions to the rules, right?
And so if you want to rapidly learn the
exceptions, you're going to lose the rules.
And if you want to keep the rules, you have a
harder time learning the exception.
And so David Marr is one of the early pioneers
in computational neuroscience.
And then Jay McClelland and my colleague Randy O'Reilly,
some other people like Neil Cohen, all these
people started to come up with the idea that
maybe that's part of what we need in what the
human brain is doing is we have this kind of a
actually a fairly dumb system, which just says
this happened once at this point in time, which
we call episodic memory, so to speak.
And then we have this knowledge that we've
accumulated from our experiences as semantic
memory. So now when we want to, we encounter a
situation that's surprising and violates all our
previous expectations, what happens is, is that
now we can form an episodic memory here.
And the next time we're in a similar situation,
boom, we can supplement our knowledge with this
information from episodic memory and reason about
what the right thing to do is, right?
So it gives us this enormous amount of flexibility
to stop on a dime and change without having to erase
everything we've already learned.
And that solution is incredibly powerful because it
gives you the ability to learn from so much less
information, really, right?
And it gives you that flexibility.
So one of the things I think that makes humans
great is having both episodic and semantic memory.
Now, can you build something like that?
I mean, you know, computational neuroscience people
would say, well, yeah, you just record a moment and
you just get it and you're done, right?
But when do you record that moment?
How much do you record?
What's the information you prioritize and what's the
information you don't?
These are the hard questions.
When do you use episodic memory?
When do you just throw it away?
And these are the hard questions we're still trying to
figure out in people.
And then you start to think about all these mechanisms
that we have in the brain for figuring out some of
these things.
And it's not just one, but it's many of them that are
interacting with each other.
And then you just take not only the episodic and the
semantic, but then you start to take the motivational
survival things, right?
It's just like the fight or flight responses that we
associate with particular things or the kind of like
reward motivation that we associate with certain
things, so forth.
And those things are absent from AI.
I frankly don't know if we want it.
I don't necessarily want a self-motivated LLM, right?
It's like, and then there's the problem of how do you
even like build the motivations that should guide a
proper reinforcement learning kind of thing, for
instance.
So a friend of mine, Sam Gershman, I might be missing
the quote exactly, but he basically said, you know, if I
wanted to train like a typical AI model to make me as
much money as possible, first thing it might do is sell my
house.
So it's not even just about having one goal or one
objective, but just having all these competing goals and
objectives, right?
And then things start to get really complicated.
Well, it's all interconnected.
I mean, just even the thing you've mentioned is the moment.
You know, if we record a moment, it's difficult to express
concretely what a moment is, like how deeply connected it is to
the entirety of it.
Maybe to record a moment, you have to make a universe from
scratch.
You have to include everything.
You have to include all the emotions involved, all the context,
all the things that are built around it, all the social
connections, all the visual experiences, all the sensory
experience, all of that, all the history that came before that
moment is built on.
And we somehow take all of that and we compress it and keep the
useful parts and then integrate it into the whole thing, into
our whole narrative.
And then each individual has their own little version of that
narrative.
And then we collide in a social way and we adjust it and we evolve.
Yeah, yeah.
I mean, well, even if we want to go super simple, right?
Like Tyler Bonnen, who's a postdoc who's collaborating with me,
he actually studied a lot of computer vision at Stanford.
And so one of the things he was interested in is some people who
have brain damage in areas of the brain that were thought to be
important for memory.
But they also seem to have some perception problems with particular
kinds of object perception.
And this is super controversial.
Some people found this effect, some didn't.
And he went back to computer vision and he said, let's take the best
state-of-the-art computer vision models and let's give them the same
kinds of perception tests that we were giving to these people.
And then he would find the images where the computer vision models
would just struggle.
And you would find that they just didn't do well.
Even if you add more parameters, you add more layers, on and on and on,
it doesn't help, right?
The architecture didn't matter.
It was just there, the problem.
And then he found those were the exact ones where these humans with particular
damage to this area called the periorhinal cortex, that was where they were
struggling.
So somehow this brain area was important for being able to do these things that
were adversarial to these computer vision models.
So then he found that the, that it only happened if people had enough time, they could make
those discriminations.
But without enough time, if they just get a glance, they're just like the computer vision
models.
So then what he started to say was, well, maybe let's look at people's eyes, right?
So computer vision model sees every pixel all at once, right?
It's not, you know, and we don't, we never see every pixel all at once.
Even if I'm looking at a screen with pixels, I'm not seeing every pixel all at once.
I'm grabbing little points on the screen by moving my eyes around and getting a very high
resolution picture of what I'm focusing on and kind of a lower resolution information about
everything else.
But I'm, I'm not necessarily choosing, but I'm directing that exploration and allowing
people to move their eyes and integrate that information gave them something that the computer
vision models weren't able to do.
So somehow integrating information across time and getting less information at each step
gave you more out of the process.
I mean, the process of allocating attention across time seems to be a really important
process.
Even the breakthroughs that you get with, uh, with machine learning mostly has to do attention
is all you need as bought attention transformers about attention.
So attention is a really interesting one.
Well, then like, yeah, how you allocate that attention again is like, is at the core of
like what it means to be intelligent, what it means to process the world, integrate all
the important things, discard all the unimportant things.
Attention is at the core of it.
It's probably at the core of memory too.
Because there's so much sensory information, there's so much going on, there's so much going
on to filter it down to almost nothing and just keep those parts and to keep those parts.
And then whenever there's an error to adjust the model such that you can allocate attention
even better to new things that would resolve, maybe maximize the chance of confirming the
model or disconfirming the model that you have and adjusting it since then.
And yeah, attention is a weird one.
I was, uh, I was always fascinated.
I mean, I got a chance to study peripheral vision for a bit and indirectly study attention
through that.
And it's just fascinating how humans, how good humans are looking around and gathering information.
Yeah.
At the same time, people are terrible at detecting changes that can happen in the environment
if they're not attending in the right way.
If their predictive model is too strong, you know, so you have these weird things where
like the machines can do better than the people.
It's not that it's like, you know, so this is the thing is people go, oh, the machines
can do this stuff.
That's just like humans.
It's like, well, the machines make different kinds of mistakes than the people do.
And I will never be convinced unless I, that, you know, we've replicated human, I don't even
like the term intelligence because I think it's a stupid concept, but it's like, I don't
think we've replicated human intelligence unless I know that, uh, the simulator is making exactly
the same kinds of mistakes that people do.
Because people make characteristic mistakes.
They have characteristic biases.
They have characteristic, like, you know, heuristics that we use.
And those, I have yet to see evidence that ChatGPT will do that.
Since we're talking about attention, is there an interesting connection to you between ADHD
and memory?
Well, it's interesting for me because, uh, when I was a child, I was actually told my
school, I don't know if it came from a school psychologist.
They did do some testing on me.
I know for like IQ and stuff like that.
Uh, they, or if it just came from teachers who hated me, but they told my parents that
I had ADHD.
And so this was of course in the seventies.
So basically they said like, you know, he has poor motor control and he's got ADHD.
And so, and, uh, you know, there was social issues.
So like, I could have been put a year ahead in school, but then they said, oh, but he
doesn't have the social, he doesn't have the social capabilities.
So I still ended up being like, you know, uh, an outcast even in my own grade.
But, um, but then like, uh, I was, so then my parents said, okay, well, they got me on
a diet free of artificial colors and flavors because that was the thing that people talked
about back then.
And so, so I'm interested in this topic because I've come to appreciate now that I have many
of the characteristics, if not, you know, full blown, it's like, I'm definitely time
blindness, uh, rejection sensitive, you name it, they talk about it.
It's like impulsive behavior.
I can tell you about all sorts of fights I've gotten into in the past, uh, just you name
it.
Um, uh, but yeah, so ADHD is fascinating though, because right now we're seeing like more and
more diagnosis of it.
And I don't know what to say about that.
I don't know how much of that is, um, based on kind of inappropriate expectations, especially
for children and how much of that is based on true kind of like maladaptive kinds of, uh,
tendencies.
But what we do know is this, is that ADHD is associated with differences in prefrontal
function so that attention can be both more, you're more distractible.
You have harder time focusing your attention on what's relevant.
And so you shift too easily, but then once you get on something that you're interested
in, you can get stuck.
And so, you know, the attention is this beautiful balance of being able to focus when you need
to focus and shift when you need to shift.
And so it's that flexibility plus stability again.
Um, and that's balance seems to be disrupted in ADHD.
And so as a result, memory tends to be poor in ADHD, but it's not necessarily because there's
a traditional memory problem, but it's more because of this attentional issue, right?
And so, um, and people with ADHD often will have great memory for the things that they're
interested in and just no memory for the things that they're not interested in.
Is there advice from your own life on how to learn and succeed from that, from just how
the characteristics of your own brain with ADHD and so on?
And, uh, how do you learn, how do you, uh, remember information, how do you flourish in
this sort of education context?
I'm still trying to figure out the flourishing per se, but education, I mean, being in science
is enormously enabling of ADHD.
It's like you're constantly looking for new things.
You're constantly seeking that dopamine hit and, and, uh, and that's great, you know,
and, uh, you, they tolerate, you're being late for things.
Nothing is really, nobody's going to die if you screw up, you know, but it's nice.
It's not like being a doctor or something where you have to be like much more responsible and
focused.
You can just freely follow your curiosity, which is just great.
Um, but what I would say is, is that like, I'm learning now about so many things like about
how to structure my activities more and basically say, okay, if I'm going to be emails, like the
big one that kills me right now, I'm just constantly like shifting between email and my activities.
And what happens is, is that I don't actually get the email.
I just look at my email and I get stressed.
Cause I'm like, oh, I have to think about this.
Let me get back to it.
And I go back to something else.
And so I've just got fragmentary memories of everything.
Right.
So what I'm trying to do is set aside a time where like, this is my email time.
This is my, you know, writing time.
This is my goofing off time.
And so blocking these things off, you give yourself the goofing off time.
Sometimes I do that.
And I, and I, sometimes I have to be flexible and go like, okay, I'm definitely not focusing.
I'm going to give myself the downtime and it's an investment.
It's not like wasting time.
It's an investment in my attention later on.
And I'm very much with Cal Newport on this.
He wrote deep work and a lot of other amazing books.
He, uh, he talks about tasks, switching as a sort of the thing that really destroys productivity.
So like, you know, switching, well, it doesn't even matter from what to what.
But checking social media, checking email, maybe switching to a phone call and then doing
work and then switching, even switching between if you're reading a paper, uh, switching from
paper to paper to paper because like curiosity and, uh, whatever the dopamine hit from the
attention switch, like limiting that because otherwise your brain is just not capable to
really like load it in and really, uh, do that deep deliberation.
I think that's required to, uh, remember things and to really think through things.
Yeah.
I mean, you probably see this, I imagine in AI conferences, but definitely in neuroscience
conferences.
It's now the norm that people have their laptops out during talks and, you know, conceivably
they're writing, you know, they're writing notes.
But in fact, what often happens if you look at people, uh, we can speak from a little bit
of personal experience is you're checking email and you're like, uh, uh, or I'm working on
my own talk, but often it's like, you're doing things that are not paying attention.
And I have this illusion while I'm paying attention and then I'm going back.
And then what happens is I don't remember anything from that day.
It just kind of vanished because what happens is I'm creating all these artificial event
boundaries.
I'm losing all this executive function.
Every time I switch, I'm getting like a few seconds slower and I'm catching up mentally
to what's happening.
And so instead of being in a model where you're meaningfully integrating everything and predicting
and generating this kind of like rich model, I'm just catching up, you know?
Um, and so, yeah, there's great research by Melina Unkafer and Anthony Wagner on multitasking
and people can look up that talks about just how bad it is for memory and, and, you know,
it's, it's becoming worse and worse of a problem.
So, uh, you're a musician.
Take me through, uh, how'd you get into music?
Like what, what made you first fall in love with music, with, uh, creating music?
I, yeah, so I started playing music just when I was like doing trumpet in school, um, for
school band and I would just read music and play and, you know, it was pretty decent at
it.
Not great, but I was decent.
How'd you go from trumpet to, uh, guitar, to guitar, especially the kind of music you're
into?
Yeah.
So basically in high school, yeah.
So I kind of was a late bloomer to music, but just kind of MTV grew up with me.
I grew up with MTV.
And so then you started seeing all this stuff and then, uh, I got into metal was kind of
like my early genre and I always reacted to just things that were loud and had a beat
like, uh, I mean, ADHD, right.
It's like, uh, uh, like, you know, everything from Sergeant Peppers by the Beatles to like,
uh, um, Led Zeppelin to my dad had both, my parents had both those albums.
So I listened to them a lot.
And then like, uh, the police ghost in the machine.
And, but then I got into metal, Def Leppard and, you know, uh, ACDC Metallica, um, went way
down the rabbit hole of speed metal.
Uh, and that time was kind of like, oh, like, why don't I play guitar?
I can do this.
And I had friends who were doing that and I just never got it.
Like I was, I took lessons and stuff like that, but it was different because when I was
doing trumpet, I was reading sheet music.
And this was like, I was learning by looking, there's a thing called tablature, you know,
this where it's like, you see like a drawing of the fretboard with numbers and that's where
you're supposed to put your, it's kind of like paint by numbers.
Right.
And so, um, I learned it in a completely different way, but I was still terrible at it and I
didn't get it.
It's actually taken me a long time to understand exactly what the issue was, but it wasn't until
I really got into punk and I saw bands, like I saw Sonic Youth, I remember especially, and
it just blew my mind because they violated the rules of what I thought music was supposed
to be.
I was like, this doesn't sound right.
These are not power chords and this isn't just have like a shouty verse and then a chorus
part.
It's not going back.
This is just like weird.
And then it occurred to me, you don't have to write music the way it's people tell you
it's supposed to sound.
I just opened up everything for me and I was playing in a band and I was struggling with
writing music because I would try to write like, you know, whatever was popular at the
time and, or whatever sounded like other bands that I was listening to.
And somehow I kind of morphed into just like, just grabbing a guitar and just doing stuff.
And I realized that part of my problem with doing music before was I didn't enjoy trying
to play stuff that other people play it.
I just enjoyed music just dripping out of me and just, you know, spilling out and just
doing stuff.
And so then I started to say, well, what if I don't play a chord?
What if I just play like notes that shouldn't go together and just mess around with stuff?
And then I said, well, what if I don't do four beats going, no, no, no, no, one, two,
three, four, one, two, three, four, one, two, three, four, whatever I go, one, two,
three, four, five, one, two, three, four, five.
And started messing around time signatures.
Then I was playing in this band with a great musician who was really Brent Ritzel, who was
in this band with me.
And he taught me about arranging songs.
And it was like, what if we take this part and instead of make it go like back and forth,
we make it like a circle?
Or what if we make it like a straight line?
You know, or zigzag, you know, just make it like nonlinear in these interesting ways.
And then next thing you know, it's like the whole world sort of opens up as like the,
and then what I started to realize, especially, so you could appreciate this as a musician,
I think.
So time signatures, right?
So we are so brainwashed to think in four, four, right?
Every rock song you can think of almost is in four, four.
I know you're a Floyd fan.
So think of money by Pink Floyd, right?
Yeah.
Bum, bum, bum, bum, bum, bum, bum.
Yeah.
You feel like it's in four, four because it resolves itself, but it resolves on the last note of,
basically it resolves on the first note of the next measure.
So it's got seven beats instead of eight where the riff is actually happening.
Interesting.
But you're thinking in four because that's how we use it.
We're used to thinking.
So the music flows a little bit faster than it's supposed to.
And you're getting a little bit of prediction error every time this is happening.
And once I got used to that, I was like, I hate writing in four, four, because I was like,
everything just feels better.
If I do it in seven, four, if I alternate between four and three and, and doing all this stuff.
And then it's like, you just, you know, jazz music is like that, you know, they just do so
much interesting stuff with this.
And so playing with those time signatures allows you to like really break it all open.
And just, I guess there's something about that where it allows you to actually have fun.
Yeah.
Yeah.
And it's like, so I'm actually like a very, uh, I, the genre, the, one of the genres we
used to play in was math rock is what they called it.
It was just like, this is so many weird time signatures.
What is math?
Oh, interesting.
Yeah.
So that's, that's the math part of rock is what the, the mathematical disturbances of
it or what?
Yeah.
I guess it would be like, so instead of, you might go like, instead of playing four beats
in every measure, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no,
no, no, no, no, no, no, no, no, no, no.
You know, and just do these things.
And then you might arrange it in weird ways so that there might be three measures of verse
and then one, you know, and then five measures of chorus, and then two measures.
So you could just mess around with everything, right?
What does that feel like to listen to?
There's, there's something about symmetry or, or like patterns that feel good and like
relaxing for us or whatever.
It feels like home and disturbing that can be quite disturbing.
Yeah.
So is that, is that the feeling you would have if you keep messing, if you keep math
rock?
I mean, that's stressing me out, just listening, learning about it.
So, I mean, it depends.
So a lot of my style of songwriting is very much like in terms of like repetitive themes,
but messing around with structure, because I'm not a great guitarist technically.
And so I don't play complicated stuff.
I mean, there's things that you can hear stuff where it's just like so complicated, you know?
But often what I find is, is like having a melody or, and then adding some dissonance
to it just enough, and then adding some complexity that gets, gets you going just enough.
But I have a high tolerance for, for that kind of dissonance and prediction.
I think I have a theory, a pet theory that it's like, basically you could explain most of
human behavior as some people are lumpers and some people are splitters, you know?
And so it's like, some people are very kind of excited when they get this dissonance and
they want to like go with it.
And some people are just like, no, I want to lump every, you know, I don't know, maybe
that's even a different thing, but it's like, basically it's like, I think some people get
scared of that discomfort.
Yeah.
And I really thrive on it.
You know?
I love it.
What's, uh, I, what's the name of your band now?
The cover band I play in is a band called Pavlov's Dogs.
And so, yeah, so it's a, it's a band, uh, unsurprisingly of mostly memory researchers,
neuroscientists.
I love this.
I love this so much.
Yeah.
Actually, one of your MIT colleagues, Earl Miller plays bass.
Plays bass.
So you play, like you play, do you play a rhythm or a leader?
But you could compete if you want.
Maybe we could audition you.
For audition.
Oh yeah.
I'm coming for you, Earl.
Earl's going to kill me.
He's like very precise though.
I'll play triangle or something.
Or is it, we're the cowbell.
Yeah.
I'll be the cowbell guy.
And you guys, what kind of songs do you guys do?
Uh, so it's mostly, uh, uh, seven, late seventies punk and eighties new wave and, and, uh, post
punk Blondie, uh, Ramones, Clash.
Uh, I do, I sing, uh, Age of Consent by New Order and, and Lovell Terrace.
And you said you have a female singer now?
Yeah, yeah, yeah.
Carrie Hoffman and also, um, uh, Paula Croxon.
And, and so they do, uh, they do.
Yeah.
So Carrie does Blondie amazingly well.
And we do like Gigantic by the Pixies.
Paula does that one.
Which song do you love to play the most?
What kind of song is super fun for you?
That song of someone else's?
Yeah.
Cover.
Yeah.
Cover.
Okay.
And it's one we do with Pavlov's dogs.
I really enjoy playing.
I want to be your dog by Iggy and the Stooges, which is perfect.
Cause we're Pavlov's dog.
Yeah.
And Pavlov of course was like basically created learning theory.
So, you know, there's this, but also it's like, but I mean, Iggy and the Stooges, that
song.
So I play and sing on it, but it's just like, it devolves into total noise.
And I just like, uh, fall on the floor and generate feedback.
I was like, I think in the last version, it might've been that or a Velvet Underground
cover in our last show.
I actually, I have a guitar made of aluminum that I got made and I thought this thing's
indestructible.
And so I kind of like was just, you know, moving it around, had it upside down and all
this stuff to generate feedback.
And I think I broke one of the, I broke one of the tuning pegs.
And so I managed, I managed to break an all metal guitar, go figure.
A bit of a big, ridiculous question, but let me ask you, we've been talking about neuroscience
in general.
What do you, you've been studying the human mind for a long time.
What do you love most about the human mind?
Like when you look at it, uh, we look at the fMRI, just the scans and the behavioral stuff,
the electrodes, you know, the psychology aspect, reading the literature on the biology side
and your biology, all of it.
When you look at it, what, what, what is most like beautiful to you?
I think the most beautiful, but incredibly hard to put your finger on is this idea of
the internal model that it's like, there's everything you see and there's everything you
hear and touch and taste, you know, every breath you take, whatever, but it's all connected
by this like dark energy that's holding that whole universe of your mind together.
Right.
And without that, it's just a bunch of stuff.
And somehow we put that together and it forms our, so much of our experience and being able
to figure out where that comes from and how things are connected to me is just amazing.
But just this idea of like that, the world in front of us, we're only sampling this little
bit and trying to take so much meaning from it.
And we do a really good job, not perfect.
I mean, you know, but that ability to me is just amazing.
Yeah.
It's an incredible mystery.
All of it.
It's funny you said dark energy, cause the same in, in astrophysics, you look out there,
you look at dark matter and dark energy, which is this loose term assigned to a thing we
don't understand, which makes out, which helps make the equations work in terms of gravity
and the expansion of the universe.
In the same way, it seems like there's that kind of thing in the human mind that we're
like striving to understand.
Yeah.
Yeah.
You know, it's funny that you mentioned that.
So one of the reasons I wrote the book amongst many is, is that I really felt like people needed
to hear from scientists and like COVID was just a great example of this because like people
weren't hearing from scientists.
One of the things I think that people didn't get was the uncertainty of science and how much
we don't know.
And I think every scientist lives in this world of uncertainty.
And when I was, uh, um, writing the book, I just became aware of all of these things we
don't know.
And so I think of physics a lot.
I think of this idea of like overwhelming majority of the stuff that's in our universe
cannot be directly measured.
I used to think, ha ha, I hate physics.
So this is physicists get the Nobel prize for doing whatever stupid thing.
It's like, there's 10 physicists out there.
I'm just kidding.
Strong words.
Yeah.
No, no, no.
I'm kidding.
It's just the physicists who do neuroscience could be rather opinionated.
So sometimes I like to dish on that.
It's all love.
It's all love.
That's right.
I, this is the ADHD talking.
So, um, uh, but at some point I had this aha moment where I was like, to be aware of
that much that we don't know and have a beat on it and be able to go towards it.
That's one of the biggest scientific successes that I could think of.
You are aware that you don't know about this gigantic section, overwhelming majority of the
universe.
Right.
And I think the more, what keeps me going to some extent is realizing the, changing
the scope of the problem and figuring out, oh my God, there's all these things we don't
know.
And I thought I knew this because science is all about assumptions, right?
So have you ever read the structure of scientific revolutions by Thomas Kuhn?
Yes.
That's like my only philosophy really that I've read, but it's so brilliant in the way that
they frame this idea of like, he frames this idea of assumptions being core to the scientific
process.
And the paradigm shift comes from changing those assumptions and this idea of like finding
out this kind of whole zone of what you don't know to me is the exciting part, you know?
Well, you are a great scientist and you wrote an incredible book.
So thank you for doing that.
And thank you for talking today.
You've decreased the amount of uncertainty I have just a tiny little bit today and reveal
the beauty of memory.
This is a fascinating conversation.
Thank you for talking today.
Oh, thank you.
It's been a blast.
Thanks for listening to this conversation with Sharon Ranganath.
To support this podcast, please check out our sponsors in the description.
And now let me leave you with some words from Haruki Murakami.
Most things are forgotten over time.
Even the war itself, the life and death struggle people went through, is now like something from
the distant past.
We're so caught up in our everyday lives that events of the past are no longer in orbit around
our minds.
There are just too many things we have to think about every day, too many new things we have
to learn.
But still, no matter how much time passes, no matter what takes place in the interim, there
are some things we can never assign to oblivion.
Memories we can never rub away.
They remain with us forever, like a touchstone.
Thank you for listening.
I hope to see you next time.
I hope to see you next time.