Neuroscientist Explains Memory in 5 Levels of Difficulty

Hi, my name is Daphna Shohamy.

I'm a neuroscientist at Columbia University.

And today I've been asked to explain memory

at five levels of increasing complexity.

My research aims to understand how memories are created

and how they shape who we are,

what we do, and the decisions we make.

[gentle music]

We're here today to talk about memory.

When I say the word memory what comes to your mind?

Like I once went on vacation to Dominican Republic.

I think I was like six or five years old.

So on your trip what was the funnest day ever?

There was this pool, I love pools.

And then there's a slide you go down

and there's pool noodles in there too.

And then there's a little tiny sandbox.

It's pretty amazing, right?

Because it happened like a year or two ago.

Our memory is basically a record in our brain

of something that happened in the past,

but that record it created isn't perfect.

So that day, do you remember was the pool at your hotel?

I don't really remember.

Do you remember from that day what color towel you used

when you got out of the pool?

I think the one I used was a dark blue or a light blue or,

or like sort of a teal.

When you were telling me about the sandbox,

it felt like you didn't have to make that guess,

that you could just see the sandbox in your eyes.

I asked you about the towel.

It doesn't come to mind immediately,

but you can stop and think about it

and kind of make a good guess because there are things

you know about the world.

And for those of us who study memory

that's a really strong hint as to how memory works.

And so for you your memory kept all the fun stuff

and all the stuff that was important to you,

but the details like did you have to have a ticket

to get in or how you got there,

all these other less interesting, less exciting aspects,

your brain kind of decided letting go

of all that information.

I think as you get older you start forgetting things more

because you have to make space for new things,

which is why you forget stuff.

Yeah, that's a great insight.

Basically we can't remember everything.

So when you tell us this memory of that vacation

and your brain is able to kind of play almost this movie

of a memory, a moment, a day that you had in your life

a while ago, where do those memories come from?

I do know that there's different parts of the brain.

So there's probably a part of your brain

that remembers a bunch of stuff.

The hippocampus is a part of your brain that if you

kind of took it out and looked at it,

it looks a lot like a seahorse

and it plays a really important role in creating those kinds

of memories that help you two years after your vacation

share with me what happened that day,

all these details of that day.

There's another part of the brain.

It's actually also important for memory

and it's called the amygdala and it is important for keeping

all the kind of emotional processing going for memory.

So memory for things that are really scary,

it's gonna talk to the hippocampus now

and get that memory to be really strong.

Why do you think it might be important

to remember scary things?

Let's say you, you accidentally cut yourself.

Then your brain makes a note

and says don't get too close to sharp things,

or you're going to cut yourself again.

You got it.

And so we don't just remember everything

and we don't remember random things.

We remember the things that matter to us the most.

So Abigail based on our conversation today,

can you tell me what is a memory?

A memory is something stored in your hippocampus

and your hippocampus is a part of your brain.

It's just a big record.

And sometimes you can forget parts of the record

that's not that important to you.

You got it.

[gentle music]

What do you know about memory?

I know when you see something you could like

kind of like picture it,

like in your head you can imagine it.

I remember yesterday I got like orange juice,

but let's say like a year or two, you might think like, oh,

that day I got water.

And on one hand memory is like a record of something

that happened in the past that we can carry with us

and we can like bring it back to mind.

And on the other hand we need to be a little bit suspicious

sometimes because we might get it wrong.

In what world would that memory of the orange juice

maybe be useful?

Let's say you grew up in a place that the orange juice

is just meh, and then you go on vacation going somewhere

and then it might change how you feel about orange juice.

Sometimes, you know, memory is doing something

much simpler but no less important for us,

which is it's helping us figure out what's good

and what's bad.

And if we can remember what's good

and what's bad from what we did in the past,

that can help us make decisions about what to do next time.

So Dylan, you may be surprised to learn this,

but we asked your mom what some of your

favorite candies are.

So you have to make a decision between these two candies

and whichever one you pick,

you actually get to take with you.

I wish I could say both,

but I guess I'll go with pixie sticks.

There you go, there is your decision.

Okay, you can take those.

It took you some time there.

You looked like you were really working hard

at that problem.

And that's interesting because actually we know that from

research that when people have to make a decision between

two things they like equally,

some people like economists think oh those are the easiest

decisions because they're both good options.

But as psychologists and neuroscientists we know

it's the opposite.

One of the reasons is because there's no simple answer of

like I know that one's better I'm just gonna go with it.

And so we think that that's the kind of decision where

memory is especially important because you have to kind of

come up with more information.

Exactly, and then also like after it was like,

maybe like I would've, enjoyed this more than that

and like maybe I should've chose the other one.

And you don't know.

Everything you just said is exactly

what we know from research.

All the thoughts running through your mind,

were really at kind of about a prediction

into the future, right?

You're like, how, how will this taste if,

when it's in my mouth or how will I feel about my decision

down the road?

You know we think of memory as something in the past,

but it's an example of how you use your memories of these

two candies to predict what's gonna happen in the future

so that you can make a decision in the present.

But you did it, you made your way through that torture.

And now you've got to keep the candy.

Am I always gonna like choose pixie sticks

or sour patch kids?

What's gonna like change that?

We fluctuate a lot because we use different kinds of memory

to kind of resolve the uncertainty basically every time.

But also the way we make decisions will change

because our memories change.

What's interesting about your brain as a teenager is that we

know it's actually a phase of life where the brain is

especially sensitive to rewards,

to things that are exciting and positive.

It's a phase of life where those rewards,

whether it's candy or your friends or whatever it is

that's exciting can have an especially powerful control

over the decisions you make and the memories you create.

[gentle music]

Is there any particular memory for you that you feel has

been either kind of influential in your,

in deciding which areas to pursue or that you feel is sort

of emblematic of what it is to have a memory?

I was probably six years old

and my aunt who had stage four ovarian cancer,

she battled it for 20 years.

She got me my first science kit and she asked me to cure

cancer and I will never forget running into my aunt's room

and just opening up the box and seeing that microscope,

it was a tiny little blue one

and the little microscope slides and the little pipettes.

So they just, that memory will never be forgotten

my whole life.

I've got one too.

I was nine and we were at the science expo in San Francisco.

Two scientists were dissecting an eye, a cow's eye.

And I was like that is the coolest thing I've ever seen.

And something about that that carried me forward.

Something happened so long ago,

it left some long lasting trace through neural circuits

and it continues to shape the decisions we make

about what to study and what to do.

Just being able to apply those memories in the future

is just so crucial in everyday life of the humans

and the fact that we're learning how that works.

And I would love to hear more about your side of memory

and the mechanisms that you're studying.

So we're interested in the idea that memory is kind of

a pervasive force that shapes all our behaviors.

And we're trying to understand how different kinds of memory

are organized in different structures of the brain.

And then to understand how those different structures work

together to orchestrate complex cognitive behaviors

like decision-making or reasoning and thinking.

So in your research or you focus more on implicit

or explicit memory?

My work has actually kind of pushed against

that distinction between memories that kind of are

consciously accessible versus unconscious.

When you say that you don't necessarily look at implicit

and explicit memory as anything different,

if you were to take a step back,

what would you define as implicit and explicit memory?

The best way to kind of think about that distinction

really goes back historically to one of the most important

discoveries in memory research.

The patient was referred to famously as patient HM.

The neurosurgeon went in and removed the tissue that

happened to be right around the hippocampus

on both the left side and the right side of HM's brain.

But then they started noticing something odd

in his behavior.

He was not able to create new memories of the experiences

he had after the surgery.

And that led Brenda Milner and her colleagues to report that

the hippocampus was very important for memory,

but a one particular kind, these sort of explicit,

or as now referred to them episodic memories.

But the hippocampus was not necessary for learning skills

like mirror tracing,

things that you can't necessarily articulate

but you just get better at over time.

And it really led to a couple of decades or more even

of an enormous amount of very important work that kind

of kept on breaking memory down further

and further into different types.

Episodic and semantic as both forms of explicit memory

where episodic refers to memory for an event that happened

like what you did yesterday morning

and semantic refers to general knowledge about the world.

Implicit memory is being broken down into a bunch of

different kinds like skills or habits or conditioning.

And when I started graduate school,

many of us felt kind of the next question was really to

understand how do we now understand how they work together.

This was right around when functional magnetic resonance

imaging started becoming a popular tool

for measuring brain activity.

We could scan the entire brain and we could ask questions

about multiple brain regions at once.

And we discovered that what we might expect

during a skill learning task that only the striatum

might show activity,

that we also saw the activity in the hippocampus.

Or when we asked someone to form an episodic memory

that we thought might depend only on the hippocampus

we suddenly also saw activity in the prefrontal cortex.

And so the confluence of these new ideas and questions about

how different forms of memory interact

together with the development of new tools for studying

the human mind and brain allowed us to kind of adjust

our view of memory systems,

to think of them less as multiple completely separate

independent systems and instead to try to understand how

they really work in concert with each other

and give rise to all kinds of behaviors that might not fit

neatly into one category or another.

Are the connectivities that you're seeing using the FMRI

in your studies, are they different when you look at

the implicit and explicit memories?

Are you seeing more activity in the striatum in comparison

to the hippocampus or the prefrontal cortex?

Yeah, you know things are kind of breaking down

in unexpected ways I'd say.

I think there's been a lot of really great work out of

multiple labs showing hippocampal activity related to

behaviors that don't look anything like episodic memories,

but which might depend on episodic memories right?

So for example also when you're making a decision about how

to choose between two good options

that suddenly you might see activity in the hippocampus

related to the choice itself,

others have found that you find activity in the hippocampus

not only when people are forming memories,

but also when they're imagining events that are

gonna happen in the future.

So it kind of forced us to rethink the way we define what

the hippocampus might be doing in a way that can account for

creating memories, thinking about the future,

making decisions and other kinds of behaviors that involve

what we would refer to you in the field

as relational processing.

And that raises all kinds of new questions about

what memory really is.

Reminiscing about our early influences that made us

want to be scientists is that, you know,

of course we know, we know better than to think that those

are necessarily accurate memories.

You know this reminds me of sort of the

classic example from literature when people talk about

memory of course is Marcel Proust's

Madeleine in Remembrance of Things Past.

Where in the book it's this taste for the,

for the protagonist this taste of the crumbs of

the Madeleine cookie and the tea that bring him back

to his childhood.

You know, seven volumes then emerge of memories of,

of that childhood.

Digging further into earlier drafts of Proust's novel

that initially it didn't describe a Madeleine at all.

It's very interesting to see the imperfections

of the human brain and being able to rewrite something

that happened to you and fully believe it yourself.

I think that that's one of the coolest errors

and faults in the human brain that could exist.

You know I think these imperfections,

I interpret them as an indication that the role of memory

is much less about being accurate representation of the past

and much more about being kind of a flexible compass

into the future.

[gentle music]

So what are you studying these days?

Well, I'm preparing my dissertation.

The general topic is going to be how we perceive

different types of social stimuli,

like different people's faces and make judgements about them

and how the way that we report on what we are experiencing

tells us about what's actually going on inside our heads.

What I love about what you were describing is like

you didn't use the word memory,

were how you were talking about kind of perceptions

and social judgments but I think probably we'd both agree

that memory is a big part of that.

Oh of course.

Right, like it reminds me so much of, you know,

what we in the field would refer to as statistical learning.

On one hand we might have like a one-shot memory,

a memory of something that we saw yesterday,

like what we did or what we ate or where we were.

But a lot of our memories and our knowledge

instead are based on many, many, many experiences.

Oh, absolutely.

I think this sort of reminds me of another project

that I'm doing.

It's fairly easy to get different participants to experience

consistent negative emotions to the same stimuli,

but with positive emotions it's so much harder to study.

I might not react the same to like this like cute cat video

than someone else.

And I definitely believe that people's memory

that they're bringing up unconsciously

when they're experiencing something to make

an emotional judgment informs especially our experience

with positive emotion possibly more than negative emotion.

Some of those examples really remind me also some of the

questions about kind of abnormalities in,

in memory or trauma or disruptions in memory

and how they also play out in terms of not just

what people remember but what they do with

those memories right?

As far as the way that my research plays into it

is like the first step in making a judgment

about what to do based on an experience we're calling from

memory is deciding what your emotional response was.

I think what we're still kind of trying to understand

as a field is the more kind of detailed

and bigger picture of where does this model live?

How does it get updated based on experiences?

Why does that sometimes happen and sometimes not?

There are actually people who are born

without a hippocampus.

They have disruptions to episodic memories.

They have trouble after this conversation remembering

what the conversation was about,

but they do have pretty good semantic information.

They have a good sense of knowledge about the world.

And that's interesting for two reasons.

One, it shows this association between the role of the

hippocampus and these two forms of memory,

episodic memories and semantic memory.

The other reason it's interesting is that people often

assume that we get to semantic memory

through episodic memory,

that we encounter one dog and then another dog,

and then another dog and then we learned the concept,

the semantic notion of a dog.

And these people, their profile suggests that you can

learn semantic information.

You can learn general knowledge about the world,

even without ever having the capacity

to build those individual memories.

In people with intact brains,

you might use a particular process,

but if that brain region that subserves that process

is damaged, that it's not that you can't do it anymore

you just now do it a different way.

Right.

It's really hard to tell people like,

just don't use your memory to do this.

And we just sort of have to infer

and guess at what strategies that they're doing.

Is there's a particular research question that the answer

you think is gonna inform the research in your lab

and perhaps in the rest of your field?

You know when I look back at,

some of the work for my own lab and in the field in general,

I really feel like some of the most exciting discoveries

were not an answer to a question that was around before,

but they were discoveries that made us realize

we weren't asking the right question.

I think one example of that even has to do with the

connection between memory and decision-making.

There was a discovery about how the striatum responds to

rewards and to learning that just all of a sudden flipped

the way we thought about the role of reward

and learning and memory,

and may force us to realize there was something

fundamentally connected between these processes.

And that discovery just raised a whole new set of questions

that didn't exist before.

Yeah, I guess then the trick is we need to be designing

studies that will make us most likely to find those sort of

unexpected things, which is kind of funny.

It's like how do you look for something that you don't know

what it is?

Each project should be very focused and rigorous

and know what a study is designed for.

But at the same time we have to keep our minds open,

our eyes open for what else is happening.

Some of the most interesting discoveries

didn't make sense at first.

As someone who is studying social phenomena

I like to hear that.

There you go, exactly.

[gentle music]

Thanks so much for coming, it's great to see you.

It's been a while.

It has been awhile, I'm really excited to be here.

It'd be great to just start by talking about your work

at the broadest level.

To my mind your work has really revolutionized the field

of cognitive neuroscience more broadly.

I've been really interested in how past understandings

of the brain have really focused on,

on pinpointing exactly what each piece of the brain does.

But I think that there's also a broader conversation

happening in the brain,

which is between one region and another.

So it's actually the pattern of connections between these

regions that would allow for the flow of information.

But I think the tools that have become widely available now

from the physics community and mathematics

and computer science are under the umbrella of,

of network science.

So that is a science of networks,

a science of understanding how bits of a system are

interconnected with one another.

Not just comparing two groups of people

or two kinds of two species,

but we can ask even within a single person,

how does the pattern of connectivity in my brain change

as we talk with one another?

How does the information flow change?

I think a lot about traffic on roadway networks

as a good example of this.

Which is, you know, we can understand where the roads are

and perhaps the roads are relatively fixed.

They change over long timescales

and it's similar to the brain.

We really only create large scale connections

over long timescales.

But we can alter the traffic on the roadways really easily.

And that's similar to what we do in the brain

is that we change the way that information is flowing

to allow us to respond to our environment.

It's a great analogy.

You know I remember when I first learned of your work,

for me as somebody who studies learning and memory,

which are processes that are all about rapid updating

that to me was really mind blowing that we can now

start asking questions about dynamics of circuits

on a timescale that's much closer to the timescale

of human thought than anything we had before.

The field is definitely opening up in the kinds of

questions that can be asked now with these approaches.

And I think what you're pointing out is that conceptually

it's where we wanted to go anyway.

So if there's so much orchestration that happens

and a symphony is not a single piece,

it's not a single pattern of,

of harmony between instruments and,

and similar to the brain,

symphony of your brain is not a single pattern.

It's a very dynamically changing pattern.

In the field we often talk of have having a mental model

that we use to make predictions about what's about to happen

and really use that to mean kind of a general understanding

of what's going on.

What does that look like in terms that come from your world

of like the configuration of, of a circuit?

I think this is the most exciting area of memory research

right now I think, which is how people build and remember,

you know, and keep with them models of the world

over long periods of time.

When I think about a mental model,

I think about ideas, potential outcomes, potential events,

potential actions, and how they all depend on one another.

So when we predict what will happen next it would be,

I am currently sitting at this piece of the network

and I know that there are,

there are these possible outcomes.

And so I have to get ready for those, you know,

four possible outcomes.

But that brings this interesting question to the fore,

I think which is how does a network system in our mind

and our brain create a network model of the world outside?

It's almost like there's a reflection in the structure of

what's inside in the hardware and what we're building

in a very abstract way.

And I think that relates to some interesting work that's

coming out about the hippocampus too.

We now think of the hippocampus so much in terms of memory

but really some of the earliest work in neuroscience

on the hippocampus talked about the role of the hippocampus

in spatial navigation.

And it really exposed I think one of the deepest questions

in memory research,

or at least from the neuroscience perspective is like,

well what is the connection?

What is the connection between

spatial processing and memory?

And why is it that neurons in the hippocampus

when an animal is navigating a maze,

why is it that there's spatial specialization there?

I really do think that this notion of a mental model,

a generative model is where those,

where memory and space meet.

That makes a lot of sense.

And I think that it makes me wonder how we think about

places in the abstract, what is a cognitive place, right?

And what is, how do we map out the cognitive spaces that we,

that our mind moves within in the same way

or in a similar way to the spaces

that we physically walk in.

But I think it opens a lot up,

a lot of questions about how individuals

make their own maps, right?

And then how the way that they choose to make that map

will then affect the way that they may respond in the future

to a new context or environment.

It's really interesting.

And I think it also really speaks to the kind of pervasive

and circular role of memory, right?

Because the maps are dependent on memories,

they're built from memories and experiences.

So there's this kind of dialogue with memory creating maps,

maps creating experiences, experiences creating memories,

affecting the maps,

and really makes me think even of a question

a friend asked me recently about moods and mental health

and this feeling of like how do some people get kind of

trapped in a state of mind where all they can see

is a particular way forward.

But then something happens in the world that shifts them

to another model.

Which is actually really interesting because you can make

different maps in different physical spaces too, right?

As you walk into a new room

you make a new map of that room.

So what are the doorways basically for mental models?

And I think that often we sort of fall into this pattern of,

of thinking about memory as, as always additive,

that we always just keep gathering more information

and update what we currently have.

We don't as often speak about deleting

pieces of information or,

or perceptions or something that we thought was a,

was a useful fact but actually it's not a fact.

Memories as models or models first

and memory as part of what helps build the model

starts generating new predictions.

Exactly the kind you just brought up

or other ways of thinking about things because an efficient

model needs to let go of information, right?

I just heard a colleague of mine joking about the fact that

the principles of neuroscience textbook

keeps growing and growing.

And that if we had enough strength

in theoretical neuroscience,

if our models of neuroscience were good enough,

that book should shrink back.

We know that memory retrieval is essentially construction.

And so you don't need to store all those separate memories.

You need to construct them.

But if you have a good model,

let go of all those details and just construct them

to feel as if it's a memory.

But really all you're doing is using the model

to fill the details.

It's kind of like the matrix, you know.

Right.

Getting way out there, but,

but it is a way in which if you have the right model,

it should be an efficient one.

I'm wondering what you, what you think about what all this

means for kind of a redefinition of what memory is,

in many ways a revision or expansion of that notion.

Would you be willing to try to help me think about

how to define it?

I think it's information certainly that,

that is stored in the brain

and that can then affect our behavior.

I think getting your behavior in there somewhere

seems, seems important.

But also is it just a record of the past or is it, you know,

our perceptions of, of what happened in the past,

or how we've changed how we think about that past event

or have we deleted that event completely

and then gathered this higher level structure

or mental model that then allows us to behave differently

in the future?

So it's more alive, it's more changing.

It's more, I think it's bubbling out and gathering,

gathering and growing and becoming more flexible,

but it's also sort of bubbling in,

in the sense of deleting and removing and forgetting.

So yes, I think that what we need to add perhaps

is that changeability and perhaps the behavioral relevance

of that changeability. I don't know, what do you think?

I agree.

I think, you know, as you were speaking I was thinking

maybe the most important word

to replace there is record.

Record implies an etching of something as it was.

And maybe what we need instead is that it's a model,

it's an approximation,

it's an abstraction that can help us create the illusion of

constructing your record.

But that it's the model that's the important thing

that drives the whole thing.

Yeah, yeah I agree.

I think that's perfect. Yep.

I hope you learned something about memory.

It plays a role in shaping the changing preferences

of things we encounter in our lives.

Memory is fundamental to everything we do, to who we are,

to what we order for breakfast,

to how we think about our past, to how we plan our future.