Decoding the Brain

[Music] one useful way to organize our understanding of reality is to think in terms of things that are either very big like stars in black holes very small like molecules atoms and subatomic particles or very complex aggregates of many particles that are highly ordered or structured tonight we're going to focus on one of these three categories the complex the human brain now it is not uncommon to hear the brain described as the most complex structure in the entire universe but as we have such a meager knowledge of what might be out there i for one

am not ready to make that commitment and there's no need to the brain is complex and its functions not just wondrous but the very substrate of who we are as individuals and as a species so the pull to understand the brain is strong this program is called decoding the brain because the work of all the scientists we'll be speaking with in one way or another is contributing to a deeper understanding of the cascading electrical processes crackling through our brains which somehow result in our experiences our emotions and our behaviors in short we will explore various

ways that new tools that allow us to directly probe brain processes are helping to reveal that most complex of objects here on earth the human brain [Music] our first guest is michael holasa who is the class of 1958 career development professor at mit's department of brain and cognitive sciences and is also an associate investigator at mit's mcgovern institute for brain research his aim is to understand the basic architecture and the functional connections in the brain that underlie cognition and he has identified basic circuit mechanisms for attention and decision making he joins us from his home

outside of boston welcome michael thank you brian great to be here our next guest is edward chang who is the joan and samford wild chairman of the department of neurological surgery at the university of california san francisco he co-directs the center for neural engineering and prosthesis which brings together engineers and neuroscientists to develop technology that can restore function to patients with neurological disabilities he joins us from san francisco welcome eddie hi brian our next guest is michael cahanna who is the edmond j and louise w khan term professor of psychology at the university of pennsylvania

where he is the principal investigator at the university's computational memory lab he studies human episodic memory and is currently working on building a prosthetic device to enhance it he joins us from his office in philadelphia thank you brian welcome michael happy to be here our next guest is helen mayberg who is the director of the center of advanced circuit therapeutics and a professor of neuroscience neurology neurosurgery and psychiatry at the icann school of medicine at mount sinai she has done pioneering work to uncover the neural circuitry of depression and to treat it via deep brain

stimulation she joins us from her office in new york welcome helen nice to be here brian finally we have yuri birjaki who is the biggs professor of neuroscience at new york university of medicine he is the author of the brain from inside out in which he proposes new paradigm for the study of the brain and he is currently decoding the underlying patterns by which the brain operates he joins us from his office in new york welcome yuri thank you brian so thank you all for joining us and let me start with a basic question that's

really to the group i'd love to hear all of you chime in if you have a perspective on it which is this do any of you have a preferred metaphor for thinking about the working of the mind and what do you think about the computer metaphor is it a is it a reasonable way of thinking about the brain for some purposes or is it a metaphor that should be retired so maybe we'll begin with uh eddie chang you know from where i stand it's it's actually very hard for me to think about the brain like

a computer because when i go to the operating room and see it every week it doesn't look anything like a computer its substrate is fundamentally different but we use the analogy all the time and so in our work it is helpful to frame some questions in terms of the information processing of the mind in in the framework of a computer but there's so much more that goes on that it's generated from the internal model that's inside the mind in the substrate of the brain that is far beyond what modern computers do right now in fact

i think the analogy for computers in the future might be more like the mind as opposed to the other way around michael halasa yeah that's a great question brian and i sort of echo what eddie already mentioned i should say that um you know one of the things that i think about quite a lot is is the idea of scales in the brain and this multi-scale organization and the vertical integration across these scales and i think some of these scales computer science can be extremely valuable in providing the language to describe them i think what

we're potentially lacking is a vertically integrated theory to connect all of these scales in a way that would be helpful to to connect them um to real world applications so while the metaphor may work for certain scales at certain times i think we're we're just missing a a unifying theory for for neuroscience to make sense of all of these ideas sure helen thoughts on that well i think it's interesting because as a neurologist studying a psychiatric disorder like depression one wanted to start with modules and to think about modular organization but i think like michael

says you know thinking at different scales is really critical and i think maybe the one metaphor of depression to this conversation is the nature of it feels like a black hole as kind of an attractor state and as soon as you enter that metaphor into the conversation the idea of a computer kind of quantifying that or giving us an organizational structure becomes a little tougher so um i'm really interested in how yuri answers this question well that's a good lead in yuri metaports are useful and we don't have anything else the way how the brain

works is it always compares we know the unknown with the help of the known so we always ground something that we don't understand with the help of something we think we do such as a computer and this is a this is a a useful metaphor but we have a lot of other things that are blocking this and the reason why we are taking this this way is because uh is the computer world that took the initiative and says oh you know this is like the brain and the reason why the computers and computer scientists think

like about the brain is because they took the wrong brain model and the wrong brain model is that the brain is a passive device just like a computer into which you shovel information and the computer has no choice the human mind doesn't have the choice this is what called the tabula rasa or the blank slate model and this is not by chance you know that that one of the giants of the the of ai and computer scientist alan turing explicitly said that the brain the newborn brain is like a white paper onto which we have

to this prescribe things and this is where we we fill it up now just by saying that the the real problem here is that the if you follow this recipe from the brain and ai then the the complexity of the human brain of any brain or the ai system depends on the amount of information you put in and i can tell you later why i think it doesn't so that's a fundamental difference the brain is a self-organized system whereas the computer is a man-made machine right yeah so we will have a time to discuss that

in more detail a bit later on but michael kahana thoughts on the metaphors of choice for thinking about thinking yeah so those are all great points and uh i love yuri's point about a self-organizing system the metaphor that got me interested in memory was from condensed matter theory spin glass systems or complex thermodynamic systems where a system could be in multiple states i mean a simple example would be water it could be solid liquid or gas but even more complex systems exist in nature and so that metaphor i found very appealing as a way of

thinking about the brain differently from the computer analogy and in line with the idea of a self-organizing system that you already mentioned with that as a preamble let's jump in to some of the astounding really remarkable work that you all have been pioneering eddie i'm going to start with you your focus at least for some time has been on understanding the neural circuitry of speech with i gather one of the goals being to to be able to have some sensors that can have some understanding of the electrical and other properties that are happening in the

brain and from those signals be able to get some understanding of what the patient is trying to say so can you give us a sense of what your approach has been and what you have found sure so our approach has been to understand the neural code for our ability to articulate words and speak and what i mean by that is the patterning of electrical activity that's generated by the brain activity in the temporal lobe and in the mortar cortex that's above it it's a very precise coordination of neural activity that gives rise to words and

sentences and so for the past decade we've worked with patients of ours in in san francisco at the university of california san francisco who have volunteered as part of medical treatments that they've had for treatment of their seizures when implanted electrodes were placed directly on the brain surface with very very small sensors maybe the ones you're referring to to record that electrical activity at the order of millimeters and milliseconds and with those kind of recordings we've been able to decipher and correlate those electrical brain patterns with individual consonants and vowels as well as syllables in

order to give rise to words so if i was one of those patients you would ask me to imagine saying words and you'd monitor what goes on inside my head when i'm doing that yeah a lot of what we focused on actually was not so much about imagining speaking but actually the actual production of words so what i mean by that is um when i'm speaking these words right now how does the neural activity at one particular spot about a millimeter wide control the movements of my tongue in a very particular trajectory to create a

duh sound or the sight that's right next to it that controls my lips when we make the sun so there is a map that controls basically all of the very precise movements of the vocal tract and we study that when people are speaking normally but also in other conditions for example when they are imagining or um sometimes even speaking different languages and is it important that you're getting the patient to actually say the words the physical act of speech as opposed to the mental act of imagining a word do those give radically different signals if

you try to measure them yeah great question we've looked at that pretty carefully and it turns out that imagining speaking is very very different than actually speaking so the act of speaking uh is a fully intentional and volitional event whereas when you're just thinking about a thought or or even imagining the words it's a different neural activity pattern that's much more subtle and complex and that we have a lot more work to figure out how that works but the actual production of of of words has become much more clear over the last five years and

is it fairly consistent across distinct patients if everybody intends to articulate cat or the same signal by and large is generated yeah that's right so everyone has a little bit of variability in the absolute position of where these locations are in the brain but they are confined to the sensory motor cortex of of the part of the brain that controls the parts of our vocal tract and when you look at the precise position of all of those neurons it can be highly variable from one individual to another but when you look at the patterns at

a higher order they're actually really remarkably similar and so do you have kind of a like a dictionary or a map yeah both uh basically it's a map of the different articulators of the vocal tract and the the target sounds that we try to speak when we articulate words and we call it a dictionary because for example in english there's only about 40 phonemes different speech sounds that give rise to every possible syllable in word so a dictionary is a very useful way of thinking about it is that you've got this elemental set of units

that by themselves have no meaning but in combination can give rise to every possible meaning so you basically just go through that that exhaustive or fairly exhaustive list of elementary building blocks map those out and then you can stick them together i gather in your dictionary to get to generate more or less anything that somebody would say yeah exactly and when when you when i you know when when you say exhaustive something that's very important to think about that is just in the couple of minutes that we've already spoken we've gone through most of that

dictionary not just once but a couple of times so just by simple conversation just by simple speaking through um you know random sentences even you can sample all of this and what's really incredible about it is that as people generate new sentences we're resampling from that that core dictionary that core set of units and recombining of ways to give rise to completely new meanings and sentences and then i presume then you can test this on on new subjects and determine how well your system does at decoding the signatures in the brain that are being read

through the implants yeah that's right so we can essentially take that code and um in the same person or in a different person with some modifications of that code essentially translate that brain activity for a completely new sentence and reconstruct just from the brain activity alone what someone wanted to say or actually said and how well do you do i think we're starting to get to the point where things are starting to sound intelligible so for example if you wanted to translate brain activity into words and sentences that you actually hear with your ears using

speech synthesis using a machine algorithm to translate the brain activity to actual sounds that you hear through speech synthesis we're starting to get to the point where it's intelligible which for us isn't a really important milestone there's a lot of room for improvement but where we are right now is that you can actually make out the words even though they're kind of fuzzy the print that you are seeing is signal rolling wheels the proof that you are seeking is not available in books she feel the video must mess any process ship building is a most

fascinating process so we're really excited in terms of achieving that milestone yet there's still a lot more to do to make it better that's fascinating work eddie michael gahanna let's let's turn to you there's not a person among us who wouldn't love the prospect that we hear in sci-fi scenarios of one day being able to enhance our memories through some kind of medical intervention and i know that you have begun work in that direction so just to get us going how well do we understand the process the biological process by which memories are formed in

the brain well i would say that we still are at a very very early primitive stage in our understanding of the biological processes of the formation of memories but we do have a very good ability to decode neural signals that predict whether or not memories will be stored and whether or not they will be accurately retrieved so you're treating the memory imprinting process as a kind of a black box i guess and saying we don't really understand how it's happening in the brain but we're just going to observe it and find the patterns do you

feel that ultimately you're going to need to understand the process itself that's happening inside of our brains or can we just treat it like that black box well i think that you need to proceed on multiple uh tracks if you want to address uh the one in 12 americans for example who have significant memory loss then you want to move forward before you wait to have a full understanding or even a good approximate understanding of the underlying processes governing the storage and retrieval of memories but i think that the most interesting process in the brain

is the process by which we retrieve memories rather than the process by which we store memories because memory retrieval you might have a memory that's stored really well i mean you've experienced this thing many times and uh you clearly this this memory pops into mind often and then you find yourself in a situation where you need that information you just can't find it right so something is going on the memory is there and and if you just think about the fact that we walk around with vast stores of knowledge of experience of memories in our

mind and yet at any given moment our ability to access that information is really quite limited it makes you really wonder about the process of memory search how do we search our memories how do we retrieve them and we can't just look at every single we can't go through the library and look at every index card we need a mechanism by which to to find those memories so tell us about the work that you've done in that regard in trying to understand that in more detail and also i gather in trying to develop approaches that

might enhance the ability of an individual to retrieve those memories well so i think most of us wonder why is one person's memory good and another person's memory is bad why can't i remember things the way i used to when i was much younger but there is a fascinating paradox or mystery about memory which is that within a given person at a given time let's say today my memory is going to fluctuate between periods of better and worse function there will be a dynamic process by which my memory varies in my ability to access those

memories that i know in my ability to create new memories and we can prove with detailed experimental studies that that variability is not due to some un unexplained external experimental factors but it's something in the brain it's something in my head that makes memory good at times and not good at other times so now if you can decode that physiologically if you can use electrodes to measure electrical signals the brain is after all an electrical network if you can measure those electrical signals and forecast when you're going to have a memory lapse using the electrical

signals recorded from the brain then the idea would be that maybe you could make the brain look more like its best when it is at its worst and so the idea is that if you first can decode or forecast this complex system that's varying and then you can pinpoint the moments of a memory lapse and somehow coax or nudge the system from this worse state to this better state the better state is a state that we know the brain can express then perhaps you could meaningfully improve memory and that's the kind of therapeutic approach that

my my team is taking and so are you working with human subjects and subjecting them to various memory tests and seeing when they're doing well and when they're not and recording what's happening inside their head and building up the correlation in that way is that is that the approach that's exactly right we have a patient who is undergoing the same kind of neurosurgical evaluation as what dr eddie chang described will play in instead of playing games where they speak or listen to speech they're going to play memory games in my in my work and we'll

track moments of good memory and moments of poor memory and then we'll try to see whether we can build a mathematical model so that tomorrow when we come back to the patient's bedside and they play the game we can predict when their memory is going to be good and when it's not moment to moment and now if that mathematical model is providing a reasonably accurate forecast prediction we can now use very gentle electrical stimulation of the brain to try to coax the brain into a better memory state and so what we do is we track

those fluctuations in memory in real time so we determine at any moment whether you're about to forget something whether you're going to have trouble remembering something and then we apply electrical stimulation at the moment when memory is predicted to fail aligned with the kind of stimulation that would generally push it in the direction of a better forecast of good memory and in that way you can significantly improve memory in many patients how significant well you know it's variable from person to person the if if you asked uh how significant in terms of percent memory improvement

it's about 19 improvement in different studies between 15 and 20 percent in our latest work 19.2 percent um if you ask what does it mean for your memory to be 19.2 better like what does that number mean 19.2 percent that is about half a little bit more than half of the deficit that would be incurred in a patient who had a moderate to severe traumatic brain injury so in other words that would be on average restoring half of the loss in memory in such a patient and really what that means is that for let's say

a third of the patients it would be almost fully restoring that loss for a third it doesn't do anything and for a third it's about a a half restoration of that loss of memory fascinating let me move from that to uh michael halassa who michael you've been focusing on focusing right you've been focusing on trying to understand what it is to have attention and and whether for instance another metaphor that we've certainly all used which is kind of this this spotlight of attention you know that's that's the thing that the brain is illuminating you know

that's the thing that we're focused on what have you learned is that is that a good metaphor for thinking about the things that hold our attention or is there another view of awareness and attention that's emerged from your research so brian i'm gonna take a roundabout way to answer your question having followed you know eddie and and michael kahana in in what they talked about so uh uh they they they talked about action you know on from from eddie's perspective you know articulation moving um and and and memory and and and i think there is

a case to be made that one of the primary purposes of having a brain is to plan our next action right without without without the ability to move and act on the world it makes no sense to have a brain in the first place that's why plants don't have a brain and um you know as you go up in evolution organisms that are better able to plan the sequences of their actions have bigger brains have bigger internal models of how they control those uh those actions over prolonged periods of time and there's a temporal hierarchy

there and and ultimately they have a pretty good way of using these stored patterns of past inputs that we call memory right and my work has been on this um sort of intermediate uh level of organization between temporally speaking between action immediate action and long-term memory which is this idea of cognitive control one of one of one of the aspects by which we control um our cognition is through attention how do how we how we prioritize incoming sensory information how we filter them and then how we sort of format them in a way that would

be appropriate for both short-term and long-term action planning so we talk about making models of the world which is vital right because look we're inundated by information all the time i mean right now i'm just sitting here in this theater and the electromagnetic waves that are impinging on my eye you've got these oscillating electric fields and magnetic fields trillions of times a second if i was sensitive to even that information at that level of description my brain would be overwhelmed right to trying to cope with that data so what does the brain do to to

cope with this influx of information so i think that's a fascinating question and i think it's helpful to you know coming from mit i think it's helpful to invoke david marr mar stipulated that we can think of the brain as we can think of a computer as uh and i'm not saying the brain is a computer i'm just saying this is a helpful metaphor for approximating the types of things that we do as neuroscientists to understand the brain there's a computational level that the high level objective that we're trying to accomplish for any particular neural

process the algorithmic level which is what are the series of steps that the brain could take in order to to to sort of perform this computation and then the implementational level how is this actually implemented in neural hardware in spiking neurons right and for any particular neural operation i think thinking about things in this particular way and this in that sort of division is helpful and an understanding ultimately may be how we link these different levels of description to one another where it becomes natural to talk about you know the algorithms of the mind implemented

in neural hardware of spiking neurons the work that we've you know done in my lab has focused on the implementation to the algorithmic level so we we record from individual spiking neurons in the brain in areas like the prefrontal cortex this this front part of the brain and that's where we think a lot of cognitive control operations are generated and the way by which we imagine they control things like what memory should we retrieve or what things should we pay attention to is by sending what we call these top-down instructions to downstream uh neural circuits

to change their operation in such a way that we dampen the noise on the things that we don't care about we amplify the signals on the things that we care about and you know we retrieve the memories that we currently are are looking for we upload the memories that we care about and so on is is it accurate to say that the brain then from a hardware perspective and perhaps an algorithmic perspective too has kind of gatekeepers that information is trying to make it up to the place of attention and awareness but those gatekeepers are

only letting some things through is that a reasonable way of thinking about it that's a great analogy and i would say that it's not just in the domain of sensory systems i would say that this is in the domain of cognition as well i think that even and i i'm gonna make uh you know sort of a conjecture here that um even when we are trying to retrieve you know generative hypotheses about what's going on in the world we we i would say most people would be going through this filtering process in order to sort

of maximize the likelihood of what's out there right i mean these are statistical inference machinery that we have in our brain to infer what's going on in the outside world and we have these things automatically running in the background all the time to arrive at the world as we see it out there i think in certain disorders for example like psychosis schizophrenia etc these these kinds of inferential filtering mechanisms at the level of cognition uh are impaired and and that's why we end up seeing i think i mean again this is my sort of leading

hypothesis of what's going on in some of these illnesses is that these filtering operations in that domain are impaired and you end up with folks you know gen and concluding that the fbi is after them aliens are landing in their backyard etc fascinating so so helen we've seen a lot of discussion to this point regarding patterns in the brain that give us insight into words and speech in eddie chang's work insight into memory retrieval michael kahana's work issues of attention and awareness gatekeeper aspects of the brain and michael halas work your work is also as

i understand it focused upon patterns in the brain but your focus has been on the patterns that correspond to depression so can you give us a sense of of what that work has revealed is there is there a kind of uniform mental brain pattern that one can associate with that kind of an affective disorder so i think it's important first to kind of define the term i mean depression is really a very common and serious medical illness and i'll make the argument that it's a brain illness we can put people in scanners and we can

get maps of what is the topography the functional topography of brain areas that are abnormal and it turns out there's not just one spot none of these complex systems but even in these syndromes if you have problems with mood and drive and thoughts and actions it's no big surprise that you see multiple brain areas acting together or actually not acting together that are abnormal and those can be mapped and one can subsequently map how the dynamics of the brain changes with treatments and our work on decoding so to speak or actually our use of causal

manipulations in the brain not just with medication or therapy but in real time with electrodes helps us to see that you can go from no capacity at all to within seconds minutes have a total shift of actually how you can see the outside world when you are stuck internalized can you tell us a little bit more about exactly that so if you have um a patient who is depressed clinically diagnosed as depressed presumably the patterns of the brain are sufficiently recognizable that simply by looking at that data you would recognize that that would be the

appropriate diagnosis can you go that far in lining them up well i think that you know we like to think so that we work in groups of patients you know our team has taken the approach of diagnose depression by standard of practice throw people in a scanner take a resting state picture of the brain treat people with different kinds of treatments for you know three months you're either well or you're not go back to the beginning and see if you can tell the difference of how people responded it's working now that you can start to

look at an individual patient to make a decision about which way to treat and we and others are trying to look at that kind of treatment specific biomarker but it's the same network in the brain but in different states so the question is is every patient different is or is there one pattern there's not one pattern because we don't know what the instigator is of depression and by the time a patient is ill there are many maladaptive patterns that you're seeing and so you're really trying to wade through all of this noise to get at

the kernel that might be helpful and at that point is you know as yuri said the self-organizing system is totally self-organized to try to adapt and now you actually have to figure out what are the state you're in you're now lost so how do you take someone lost and figure out what you can do to move them back into a place where they can retrain and be found and and that's really it's become much more clear in our work with implants what that really means that was harder to see when people were less sick so

with the implants and you mentioned this this really stunning result of in minutes you mentioned sometimes you can see a change in a person you just tell us a little bit about that how are you stimulating that person was it through some kind of electrode in the brain and and what was the response like in some of these remarkable cases so a there have been many now so i started this um 15 years ago in toronto with a team and neurosurgery team andres lozano sid kennedy a psychiatrist we basically took the hypothesis based on this

imaging on this map that we had that really told us there were brain areas that were working together but their synchrony was um dysregulated and we basically had reason to believe that one area deep in the brain seemed to be the ringleader it was really focus the subclosal cingulate connections to midline thalamus like michael talks about its connections to the hippocampus that my kahana talks about its connections to the frontal cortex to the hypothalamus to other parts of the brain stem it seemed to be focused in a place that if it didn't change it wreaked

havoc on everything else we didn't take epilepsy patients we took depressed people who had failed multiple drugs psychotherapy electroconvulsive shock therapy they're pretty much at the end of the line they were in that attractor state they are stuck and they can't get out and we basically said can we leverage the technology that's used in parkinson's implant with anatomical precision in this area and in the white matter that connects it to these other areas in this putative circuit and actually when we stimulate at high frequency it's like we release the lockdown of this region and people

kind of come out of it so as we got better and better at it with our imaging we can plan the surgery precisely within millimeters place it at this convergence of these multiple pathways that we know interact with each other and we know that from animal studies we know that from epilepsy studies and when we apply the high frequency stimulation we can predict that someone will emerge and what they describe is that the negative turns off and it's as though they kind of come back online they can pay attention they feel connected they feel like

they can move so the paralysis kind of goes away and and that happens very rapidly and if you give the stimulation even in the operating room 10 15 minutes so that we can record and actually see what happens we leave the operating room and they'll stay well you know several weeks without further stim and then we actually turn it on and give stimulation and we have patients that have these chronic implants have this pacemaker on all the time and people are better and go about their business so that we've reset something very fast and then

we've allowed the brain again i'll use this self-organizing principle to re-learn who you are with a brain that doesn't get stuck yeah that's remarkable helen that a a little electrical nudge at the right spot in the brain can have such a dramatic long-term effect now michael kahana you've also found that the right electrical nudge can have a big impact elevating the capacity of patients to retrieve memories so michael when you add these tiny little electrical impulses to the brain i'm i'm assuming the patient doesn't feel it they only feel the repercussions of that stimulation so

this is truly you are manipulating the brain the patient doesn't know when you're doing it but yet they will show a difference on these memory exams that shows a marked difference from before and after what we do is we apply stimulation these stimulation protocols randomly on certain lists of certain memory games the game might be like 30 seconds or a minute trial with or without the patient doesn't know the experimenter doesn't know and that's how we gauge the beneficial effects and so do you think that the ability to read these signals and even to manipulate

these signals which is what you're doing now will this ultimately lead to a full understanding of the process of memory retrieval and i would love to memory formation as well is this the root toward that goal yeah i mean i think that that's a really critical question that we want to know the answer to meaning if you have people let's assume that we can create a therapy that's whose benefit is sufficiently meaningful that many people who need the therapy will get the therapy and the risk is low and the benefit is high and now you

have people walking around with devices that decode variable memory states in the brain and transmit those signals encrypted to a cloud which is able to analyze those signals and build up a massive database of information about how the brain operates when you're studying and recalling information or even just acting doing your regular activities of daily life once you have that massive database you could imagine that you could begin to test theories out and improve and refine theories much like you know maybe the hubble telescope or some other devices give you insights into the cosmos this

kind of technology could give us tremendous insights into an understanding of memory encoding and memory retrieval eddie the technology you've developed synthesizes speech based on the intention to form words have you applied this in situations when the individual is speech impaired for for some reason has it been able to work in those circumstances right well that's a great question so for the past five years five ten years we've really focused on what that basic science that basic neuroscience of this code for all of the different consonants and vowels and movements of the uh the vocal

tract to give rise to speech and about two years ago um because of the results of that that foundational work we started a clinical trial actually called the bravo trial in which we enrolled the first participant it was a it was a young gentleman who at the time when he enrolled in study was 35 years old and the reason he was a participant in this study was that 15 years ago he was in a car accident that led to complications of a brain stem stroke with the inability to move his arms and legs as well

as to speak and so uh for us that was a very important development in the last two years to move a lot of what we had studied um in the context of people who are normally speaking to a clinical context in the situation of someone who is fully paralyzed who has the full capacity and intent to speak in the cerebrum in the cerebral cortex but couldn't because of a disconnection in the brain stem [Music] and we were able to translate that brain activity in into four words the proof of principle is there and that it

is possible to really leverage that rich rich information of those patterns of neural activity to give rise to things that are so human and so intimate like words and speech yeah it's fascinating work michael halasa unlike eddie and helen and mike kahana i understand that you work primarily with animals rather than with humans so how do you go about that so we we train them on decision making tasks in which they uh need to retrieve a reward it's it's what we call uh alternative choice tasks so we on every and we do it trial by

trial so on every trial we give the animal the option of choosing between two options and the animal is required to do this based on appropriately deploying attention so on every trial for example it gets a visual or an auditory target and it has to pick the appropriate one we train it to be sensitive to a cue beforehand so we cue it on every trial and we can make that cue sort of ambiguous so basically we can we can make this process of attention a decision-making process decide what to pay attention to based on ambiguous

input and that allows us to study how the brain decides to what to pay attention to and both the filtering at the level of the cognitive operation and the filtering at the level of the sensory operation and those animals typically is that mice or dogs or cats yeah so so a lot of the work that we have published so far are in mice and the reason why we choose mice is because they are they're mechanistically tractable we can go in and turn circuits on and off uh using a technology called optogenetics and we can talk

about that later and then in more recent studies uh we've been using uh this organism called the tree shroom tepaya and and that's a that's an organism that's sort of intermediate between uh rodents and primates and that allows us the same kind of mechanistic accessibility but we can achieve high level cognition and study things like causal inference in these in these animals if schizophrenia for instance aligns with the theoretical description that you're imagining somehow the gatekeeper not doing its work does that suggest a particular treatment or or course of action or is it too early

for that kind of a diagnostic i can tell you where my intuition comes from so my intuition comes from studying a particular circuit from the frontal cortex the prefrontal cortex which is particularly developed in humans and and its connection to a structure called the meteorothalamus the part of the thalamus uh or essential structure in the brain in schizophrenia patients those connections you know the structural integrity of these connections are impaired and that's been known for a long time in in my lab and others in the community we've been studying this particular network in in um

in animals and what we've been finding is that a lot of these uh cognitive filtering operations sort of deciding which evidence you should use in order to deploy your attention goes through those circuits almost the algorithm is is is going through different refresh cycles between these two structures and in that you know in that um uh process the thalamus really acts as a filter for how reliable the information that you're getting is so the sort of the idea that patients and animals when we when we um uh impair those struggles when we inhibit those those

connections end up jumping to unlikely conclusions basically leads us to conclude that you know this is a reasonable hypothesis to have about these impaired inferential mechanisms in schizophrenia zuri so we've heard from a lot of researchers the four researchers on the discussion here tonight eddie talking about looking at the brain finding the patterns that decodes to language the michaels on memory and attention hell and on other patterns that allow us to understand aspects of depression and even have suggested ways to address that situation that have been remarkably effective all of that i believe you would

describe as the outside-in approach to understanding the brain you know we as third-party observers we objective scientists are studying some aspect of patterns inside the brain and with that third-party perspective trying to do something to the brain in order to get the pattern to change or to read the pattern or to do something with a pattern i gather your view is that yes this is profound work but i think that you suggest that there's another way of thinking about it that ultimately you think is the way forward give us a sense of of what that

is and you know historically research in the brain or on the brain has been working its way in from the outside world hoping that such systematic approach will take us to the middle and all through the middle to the output which is the action now of course people didn't start out as a neuroscientist early thinkers were thinking about the importance of psyche the soul the mind and so on and they were speculating and that speculation was codified by christian philosophers and then the british empiricist the bottom line of all of this is that the brains

the human brains are there to learn the truth of the world that is outside in its entity and its beauty and in the process we realize that we need to explain various things and so we made up terms such as attention memory decision making creativity imagination scratch pad memory working area that can go on and on and on and on there are many many many words so when neuroscience entered the scene it seemed that we already have a task we are on the right road we have a road map what we have to do namely

to find homes to all of these terms and see how they work i think you agree with me that this is a naive approach because those boundaries that we made up in our minds and those boundaries that smart people a thousand years ago hundreds of years ago they made up as a term cannot really have a mechanism with the same boundaries inside the brain now just to give you an example this is uh this outside inner brow which is also you used to be called the tabula rasa or the black slate model and so on

with the understanding that you just have to shovel knowledge into the brain because the brain is a receiver that is there ready to absorb all the information that comes from outside so for example when we inspect an elephant which has a shape size smell and so on then the goal is to figure out how the brain combines or binds those attributes into a representation or into a symbol and it becomes a symbol of the elephant there is a fundamental problem with this approach which is the attributes of the elephant or of any object for the

sake is not in the object those attributes are made by the brain so that's where we are right now we are at is what i can call a blind alley and we are trying to find our way out from this blind area so what do you think is is the way out i mean i'm not sure that everybody would agree so i don't want to put words in anybody's mouth but if one agrees that this is a blind alley what do you think is the way forward what what is what are we missing is it

is it a radical change of perspective do you just need to add something to the mix in order to have a more complete framework well an alternative what i call the brain-centric view the one i'm promoting is that we learn from our actions the task of the brain if there is a task you know evolutionary tasks is to serve the body and the consequences predict the consequences of our actions now let me explain the difference between the two frameworks in practical neuroscience terms so what we have been doing for decades is that we present something

to the brain be it an experimental animal or a human brain and we record from inside the brain and it could be an electrical pulse that we are picking up in michael kahana's lab or it could be a bald signal that that images get and then we correlate the two and then we say oh wow there's a nice correlation between a moving pattern and the neurons firing a particular way in the visual cortex for example we can go further on and the eddy showed a very nice example that we can train a a computational algorithm

we say we make these systems smart it's so smart that now it can we can reverse the process we are recording from brain activity and from the previous learning we can reconstruct what would be for example the music what would be the language from the brain patterns there is a catch here the catch is that the experimenter is in a privileged situation only the experimenter has access to the world outside there and the activity of the brain inside here neurons in the brain have no clue what's going on in the outside world neurons will just

get action potentials from their partners and they have no clue whether those action potentials mean anything outside in other words this approach does have no grounding as i mentioned you know grounding is a process where the unknown is grounded to the known and there is only one source of knowledge that neurons can have besides what's coming in from the outside world and this is action on output now action of course is a is a typical way you think you are moving your arms or you moving your eyes or do something like that but action is

also the action on your heart the action on your your your endocrine system and thought is also an action now the good news is that every action system in the brain informs it sends out an output not only to the body but it also informs the rest of the brain that i have sent out an output now neurons in the visual codecs for example can deal with two types of information what is that is coming from outside the what is coming from here so this is the way and the only way how i can say

that i am the actor i am the agent of my own actions so this is a fundamental difference between the two approaches and i would say let me give you an example no amount of staring a stick in water or no amount of argument between you and me we would agree that the stick is broken or not but a little movement a little action there immediately will reveal to the brain that the stick is not broken the source of knowledge in the traditional outside in model is the assumed richness of the world waiting to be

observed by a camera-like brain in contrast in the inside out framework knowledge comes from action blaze exploration and everything is compared and valued from the organism's point of view no exploration no knowledge so so i so i gather and i apologize if this is a course summary but you want to kind of close the loop where the brain does receive stimuli from the external world but it also acts on the external world which then affects the stimuli we receive from the external world and that's this loop of of information that is required for the brain

to have grounding for it to know that there's an external world to know what these neural firings are actually corresponding to in the external sense but what is the definition of action and i ask for the following reason so as you noted action in the most straightforward is just moving the arms and the legs and salsa horse but you also mentioned it can be the action of one neuron sending an outward signal within the brain itself so can action be a completely inner process or does it need to access the external world it doesn't have

to need to to to have any access to the external world i'll give you an example when you are born before you are born we all kick you know baby kicks are very very important and the baby kicks correlate with the apgar score and the intelligence quotient 20 years later and so so they are very important what they do is help the brain to build a body map and it's a very fascinating mechanism namely that every single time there is this dumb teacher that some muscle activity is is there it sends back a signal to

the future sensory somatosensory representation now of course this is a not a random process because the skeletal system confines how those muscles can move so now the agonists and the antagonists are represented together in time when i happen to move my left arm or i happen to move my my finger as a baby or as a fetus and and those timing effects because their proximity in time they allow that neurons when they grow in they will make the connections appropriately so this is how we make a map of the body and we build it up

and update it every day because every week the distance between your finger and your nose is it varies so it has to be updated so this is an example of how a actual system can teach and make sensation meaningful no amount of sensation you can have your camera uh your eyes everything and you can give any amount of information through sensors to a robot that robot just remains dumb without testing the the distance between me and the tree and the between me and the mountain i have no clue that the mountain is bigger or smaller

than a tree so only through this action-based calibration can sensation make sense so i just want to comment on that because again it's observation from the patients but despite the fact that i describe this shift from an total internal focus in these depressed patients to the capacity to interact outside themselves and to get the feedback and what we see over time is exactly what yuri is talking about it takes time to recalibrate one's own internal body and its relationship to the outside world and you're watching over months we actually saw a patient for a 10-year

follow-up last week who when she tried to describe in retrospect every stage of recovery she was realizing that updating her own external internal map had been an ongoing process and that when she tried to identify yeah she remembered in the operating room but that was minor compared to actually taking a year or two to really trust her representation of her internal state the outside world and to feel that there was dynamical predictions that she could make and learn and so i'm i'm uh i'm a fan of how yuri is conceptualizing it even at if at

the neuron to neuron level we have no idea but the the metaphors that patients use the experience that they have is there is no depression per se it's just the absence of capacity to do those ongoing updates that probably is how we should really look at what we see so let me jump off from that to one final question as we're running out of time here i want to go back to uh eddie chang and a question that i'll ask eddie but one that i'd like you all to weigh in on if you have a

perspective on it eddie you you noted that you go into the operating room and you look at the brain itself and you know that it's more than a computer right you're not looking at a motherboard with chips it's something beyond a computer but but from your years of of staring at the brain and manipulating the brain and trying to decode the brain where do you come down on on the question that of course has been kicked around for for hundreds if not thousands of years is consciousness right we really haven't spoken about consciousness here and

some of you may roll your eyes at the idea of consciousness and if that's your response i'd be happy to hear it but when it comes to consciousness is it within this physical structure is that all that there is or do you find that it's even inadequate for that purpose that you think there has to be something beyond the physical in order for us to have the kinds of inner worlds that happen inside of our heads yeah brian thanks for um thanks for asking that question and from a physician's perspective we we think about consciousness

in fairly concrete terms in terms of whether someone is awake or not awake there's some very very important medical definitions that we use to define and there are specific parts of the brain for example the brain stem and the thalamus the areas of michael halassis studies very intensely and how they interact with the cortex that need to be intact in order to have conscious awareness there's a lot of other aspects of conscious awareness and subjective perception and our appreciation of the environment and and that kind of awareness but in terms of the basic fundamental idea

of being awake versus asleep from my understanding the way i see it is that it's an emerging property that results from the interactions of some very very core structures in the brain including the thalamus and other subcortical areas you can see that when people have brain injuries in some of these areas they may never fully wake up from those strokes or traumatic brain injuries where in other parts of the brain there's no effect whatsoever even that you can detect in many cases so the location the substrate the hardware is very fundamental to this the operation

implementation of how the neurons in these particular brain locations give rise to conscious awareness is is a fascinating question that a lot of people are working on now and it's very much related to all of our work michael halasa how about your views consciousness purely physical something else will will a computer ever be able to have the kind of inner worlds that we do i mean i think it's a great question brian i think you mean by consciousness you mean phenomenal consciousness the the experience of i mean the thing that's happening right between my ears

right now what it is to be me what is it to have a perspective what is it to have you know the redness of red the qualitative experience of the outside world i mean i think we're missing something really fundamental in our description of science altogether to be able to arrive at a at an answer to the question would a computer have consciousness or not does an aunt have a consciousness does anybody else other than me uh have consciousness i i think you know we we just don't know i mean we don't really have the

the the theoretical framework to be able to answer a question of this of this nature i mean i'd like to think that you know other people other than me are do have an internal experience and do have do see green and red in in ways that are similar to me and by extension i can imagine dogs and cat doing the same thing and if and i do believe fundamentally that the brain is generating processes that can be approximated in in artificial uh uh sort of in machines and therefore i don't see a reason why machines

won't be able to experience things that are similar to that but again i can't tell you how yeah helen any thoughts on the physical basis of conscious awareness versus something else um the short answer is no and i think you know we these are surrogate discussions and when we have a readout um i'm not even sure that we would know that we know that that's what we're talking about but um i'll leave that to philosophers and neuroscientists that think they're studying it yuri the the dream that we dream together is reality so that's a nice

metaphor and we can go this way but you ask concrete questions whether my consciousness that is brian's consciousness has something to do with your brain and only with the brain my answer is definitely no and this is a connection between the computer and the your brain and your eyes are rolling now i can see your brain is calibrated not by the physical world only but also by my brain and brains of many many many many others so this reflection of your actions back to you or back to the brain is what gives rise to something

that you know we are we are talking about and and of course you know you can ask also whether computers can have it uh well right now what we do is all that information that give give to the computers usually there's one computers don't or robots don't communicate with each other so it is an unfair comparison between you know a few billion people's knowledge compared to one of few machines that thousands and thousands of people to put together the competition or the real thing would be that we have at least a million robots running around

in the world and they would team up against us and then would they collaborate and things like that that's where we would talk about the funny part you know whether we could be outsmarted or not right michael khan i'm giving you the last word consciousness anything you want to share i think that the most interesting things that the brain does are the things that it's not consciously aware of it's all that stuff behind the veil of consciousness that is working in the background and that may underlie for example why somebody might ultimately become depressed right

or or why their their memories of events that happened long ago are are doing stuff to affect their future behavior they're not even aware of it i think we want to understand all of that stuff the inside stuff that sometimes doesn't come out immediately but may come out at an inopportune moment uh i i'd really like to understand those processes well i i just like to reiterate what michael kahana said and several of you that what is me me is my memories if you take away my memories am i aware of my surroundings god knows

it's kind of a you know you are a zombie you can react and it's hard to tell whether you are the same brian then uh five minutes ago but without your past you you cannot have that kind of consciousness that philosophy is not talking about yeah no i mean the fact that collections of particles can hold on to a lifetime of memories as we were talking before giving us a consistent identity through time is of course the essence of what it means to be human so from this human being to you five human beings from

these brain patterns to yours i thank you so much for this conversation on the brain it was deeply fascinating and illuminating thank you for joining us thank you thank you [Music] you