The Reality of Reality: A Tale of Five Senses

[Music] we have a fascinating discussion tonight we're gonna leave you maybe rethinking your five senses and how real they are do we see an independent reality or is what we're seeing something our brains are constructing because we need to predict or we need to feel safe or we need to feel less fearful or because we've been programmed to see or hear or smell or taste or touch what we're seeing just to survive as a species so let's get started first with our first panelist he is a world-renowned neuroscientist he specializes in both biology and psychology

he's the founder of the very aptly named lab of misfits a perception based creative agency everyone please give a warm welcome to beau lotto well it's great really really lovely to be here it's such a wonderful group of people are gonna be here so I feel really honored to be sitting here with them actually I have a personal aim which is I want you to know less at the end then you think you know now all right because nothing interesting begins with knowing it begins with not knowing and we brilliant if you didn't know what

the most fundamental level so to start with what is our greatest fear right what is our greatest fear think about it and it's the dark right we hate not knowing we hate uncertainty right we evolved to take what is unknown and make it knowable right it's one of the biggest sources of emotional stress is the not known which then leads on to actually what is one of our greatest desires all right what is one of our greatest desires and instead of telling you I want you to feel our greatest desire so we're all going to

stand up all right and we're going to conduct a pee of music together its Strauss you all know this piece of music alright and we're gonna connect it together and I want you to feel what our greatest need is are you ready yes all right ready 1 2 3 if give the music louder come on conduct you know where it's going [Music] [Music] Oh can you feel it right what we need is closure okay you can sit down this is because we evolved to predict all the information I'm going to show you that all the

information that falls onto your eye into your ears onto your skin is instantly meaningless it's useful but it's not meaningful it doesn't come with instructions doesn't tell you what to do so evolved to predict and so in the next slide you can see that here your brain is predicting the colors of this cube you see a dark brown tile at the top and a light orange tile at the side yes yeah that is your perceptual reality you'd bet your life that they are different when they are exactly the same right that's your physical reality and

now if we change the sight again that's your perception allottee nothing's changing on the screen what's changing is the meaning of the information not the information right you're seeing the historical significance of data not the data what this also means is that what history gives you right are your assumptions and biases right that's what history gives you because your brain is the functional structure of a brain is a physical manifestation of your past interactions with the world and not just your history but the history of your family of your culture of your evolutionary history most

of your life happened without you even there okay and if if everything you're doing is grounded in your biases that come from your history then it means you can actually change what you see by changing your biases so if we start this in emotion you'll see that your brain assumes that you look down on two surfaces that's because we evolved in a ground plane so you're seeing it spin from left to right but now change your assumptions imagine you're looking up at that central plane then stand it down on it and all flip how many

can get it to flip right sometimes it helps if you blink blow your eyes look around it and suddenly it will go on opposite direction how many get a flip now right all you're doing is you're changing your assumptions Oh okay so quick question actually which direction is actually rotating how many say left right how many say right to left Sammy say I don't care right what if I tell you there's no motion on the screen at all nothing's moving would you believe me no nothing's moving you're looking at an animation an animation is a

series of still images are slightly different from each other right your brain is taking that small change and seeing in the meaning of the change not to change if you take those exact same images and put them in a random order you see a flickering right so you're taking a series of still images perceiving them to move and then flipping it from one direction to the other depending on how you think about it in this sound sequence I'm going to tell you that nothing's going to change but you'll decide what you want to hear you

decide if you want to hear brainstorm or grieve okay it's up to you and flip it back and forth let's turn it up how many can flip it you're deciding what do you right these perceptions and these biases don't just stay inside your head right you project them out into the world so here you have two triangles in a circle completely meaning the shapes until we put them into motion all right and you're going to project a meaning onto them you hate one of the triangles right you're feeling bad for the other one you're worried

for the circle right this is a horror film you can hear the soundtrack and we're gonna stop it there and you're all wondering what happens to the circle and the answer is nothing it's a circle right and what we do for objects we also do for other people because where we can measure their what and there where we can never measure their why right so you project a meaning onto them based on your biases your assumptions then this will come in we'll talk a lot about tonight so what then is the fundamental challenge of the

brain it's this and again instead of telling you I want to show you I want you to hold up your finger to me ideally this one okay and line it up to me and put it towards you or way for you until me and your finger the same size right do you have that okay we're not the same size okay but your brain has no physical access to the world other than through the data that is getting on to its receptors and at that point your eyes and your brain would tell you that me and

your finger the same size right because light sound etc conflate multiple aspects of the world in this case size and distance so the thing that is loud and far away can generate the exact same stimulus something that's quiet enough close and this is the fundamental challenge of every brain what to solve the problem of uncertainty okay so with that said I think we're finished with the introduction right and again I hope you will know less at the end of all of our conversations than you know now but understand more so thank you very much yeah

Thank You beau I'm worried about the circle I know it's just a circle but it looked like it was in danger away he gets away thank God angles was good and clearly trying to help rescue the circle at any rate all right we've got a great illustrious panel for all of us tonight so let's get right to meeting the rest of our panelists our next participant studies the brain and the mind professor of cognitive and computational neuroscience at the University of Sussex in England he has published more than 100 papers exploring consciousness perception and the

self please welcome Anne Ilsa do past risks and rewards determine how we act in the present our next participant studies how we compare experience to evidence she holds a PhD from Columbia she did her postdoc fellowship at Princeton University and today she runs her own lab at NYU please welcome Christine Constantinople all right we're gonna ask all of you can you make the case against reality well that's the title of the forthcoming book from our next participant who argues that we have evolved thanks to our ability to hide the truth from our own perceptions he's

the professor of cognitive sciences at UC Irvine please welcome Donald Hoffman and finally feelings of hunger feelings of fear feelings of safety even love can all apparently according to our next participant be aroused by our sense of smell but how our next participant is professor of biochemistry molecular biophysics and neuroscience at Columbia University please welcome Stavros Lombardo's all right I know most of you got to see Bo's little experiments on stage and his sort of point of view that people can be tricked into thinking that they're seeing something that they're not actually tell me what

your reactions are to that and how your own philosophies on this whole thing differ or or align so I think a beautiful set of demos but and we tend to think of illusions as exploiting tricks in the visual system right so an illusion as somehow revealing a quirk in how we see that if we could somehow fix it you know we things would be better and we'd see the world more more accurate as it is and I think maybe you were making this point but but what all the sorts of things you were demonstrating to

me show is is not that these are quirks in the visual system this is how all of our perception is all the time and it's it's this this kind of flip where it's intuitive to think that you know perception is some kind of readout of an external reality that with the eyes are a windows out onto the world and we read it out and sometimes we make mistakes but really perception is not coming from the outside in it's coming from the inside out always and all the time it's this it's this pro this kind of

a neuronal fantasy that is tied in some way to whatever is out there and it's the process of understanding how that relationship works that that's perception and the perception is not about representing the world as it is so you believe that there is an independent reality but it is influenced by what you yourself are processing with your own senses and your own experience yeah I think that probably is something every no if I jump in front of a bus it's not gonna go well regardless of whether I read canal it's there and but the way

we experience things from colors to shapes to to maybe even the structure of space and the way in which our perceptions have formed those however real they seem aren't a kind of just opening out of the brain to the world we we impose this structure on I mean sensory data doesn't have color yes its electromagnetic radiation is its color all our sensory inputs a colorless shape shapeless smell death odorless we construct not only the specific contents of what we perceive but the modalities themselves the structure the the invite the whole shape and organization of our

perception comes more from within than from without Donald you're nodding yes I think that evolution shaped us not to see reality as it is we tend to think that our perceptions our visual system is like a camera it just takes a picture of the truth and we're just seeing the truth as it is the wonderful demonstrations are a clear example of a very general principle as the Neal was saying that we're constructing what we see but there's this other twist we usually want cognitive neuroscientists it's pretty well received that we're constructing what we see but

the assumption is that evolution has shaped us so that our constructions match the truth and I'm gonna argue against that that are the evolution shaped us with a user interface that hides the truth nothing that we see is the truth the very language of space and time and objects is the wrong language to describe reality and that that's very useful to not see the truth you can control reality if you don't know what reality is very very usefully and so it's that kind of is that's a very strong position on this so we don't see

reality as it is and that's why we're here that's what keeps us alive we don't see reality as it is but then how do you account for the fact that so many of us see the same reality sometimes I mean clearly there are many many occasions when we differ in our own perceptions great the one example that bol Otto gave us quite a few examples so when we all saw those two squares as as orange and brown we were not seeing reality as it is and we all agreed the reason we agree is not because

we all see the truth is because we all have the same interface shape by natural selection we agree we have consensus because we're members of the same species and we all have the same user interface it's like all Apple users have the same desktop so the things are very very similar but if you have a you know PC then it's a very different kind of desktop so we agree not because we see the truth but because we have the same interface and Christine you found in your research that basically as a species as Donald just

refers to we're all and not just us other species as well sort of desperate to see patterns desperate to be able to predict because as both said in as opening there's nothing we fear more than the unpredictable than chaos yeah that's right so in the lab we study we're interested in kind of the neural basis of decision biases so if you present people with the same option repeatedly they won't always answer the same way and there's multiple reasons for that first of all our brains and our sensory organs are imperfect so there's some noise that's

associated with the processing of those options but there's also a decision biases that depending on what we previously have experienced or choices that we've previously made they'll shape our future decisions and we actually study these things in rats because we have a very sophisticated experimental toolset that allows us to interrogate the neural basis of these biases with cellular resolution and just to give an example we conducted a study where we presented rats with different Gamble's or safe amounts of reward so these were thirsty rats so they were working for water rewards and what we found

is that if the rats gambled and they won they were more likely to gamble again and humans do this so this is in economics referred to as the hot hand fallacy so you know by in finance or recreational gambling if you gamble and win then you think you have a hot hand and you'll gamble again so then we were able to leverage the tools that we have in rodents specifically we used viral and genetic techniques to express ion channels in a particular part of the brain called the orbitofrontal cortex which is behind the eye and

when we shine a laser we were basically able to turn off or on cells in that brain area with you know millisecond precision and if we inactivated the orbitofrontal cortex we found that we eliminated the hot hand bias in our rats and so we were able to again kind of leverage the tools that we have in rats to identify a brain circuit that's causal to kind of this really bad bias right where the rats think that they're more likely to win but they're not all right Stavros I'd like you to weigh in on all this

I study a faction which transformed chemical structures into illusions of smells so by default this sense create an illusion of reality because chemical structures do not have a smell that's our brain transformers transformed this destruction to something that we think as a small and that's very helpful because we can navigate into our environment and animals use it to avoid predators to find food but it's not an accurate representation of these chemical structures so but we all for example recognize the smell of smoke and assign that to danger right so there are some there are some

evolutionary selected smells that have common meaning to a lot of us and this is also very common in animals mice are very scared of fox urine without ever smelling it but the majority of smells do not have the same meaning to all of us actually we have a lot of genetic polymorphism that makes us understand the smell of things very differently with each other I'm sure many of you in the audience remember recently there was that whole issue on all the local newscasts where there was a dress and everybody was there was a lot of

difference of opinion as to whether or not people when they look at the dress saw I thought it was blue or thought it was gold all right in the audience couldn't pop the lights who thinks they're you're seeing a blue dress and who thinks you're seeing a gold dress Wow okay now let's go to the second slide alright can you explain this any of our panelists on why we're seeing the same dress in you know dramatically different colors so I love this kind of phenomenon when this came out there was kind of vigorous to be

among vision neuroscientist about what the basis was and I think the general consensus although I wouldn't say it's resolved is that this particular picture which someone just like took with their iPhone is kind of weird because the illumination is ambiguous right so you could interpret the case that the dress is cast in shadow and that bright light in the background is casting a shadow on the dress there's you other people can interpret it that the that Bret light and the background is illuminating the dress right and so there's a very kind of two possible sources

of illumination and there was a really nice study that was done that was published in the Journal of vision in 2017 where I thought the experimenters really did a good job of demonstrating or fortifying this hypothesis so they did the following experiment they photoshopped the dress onto an image of a woman that was in unambiguous light settings so in one case she was in a situation where it was a photo that she was clearly illuminated and then there was a different photo and they showed a different group of subjects the dress photoshopped on an image

where she was clearly in shadow and then they showed them the dress and they asked them what it looked like and basically all of the subjects that saw it in light thought it was blue and black and the subjects that saw it in shadow thought it was golden white and so the you know it just shows that your prior the expectations that you have here about the illumination really bias your perception and this goes to Neal to what you were just saying that you know people do see different things and the thing I love about

this is that there's a phenomenon and vision that people have known about forever which is called color constancy so for instance if you just take a piece of white paper from in here to outside if it was still sunny you know it still looks white and this may be like of course of course it does but this is actually quite surprising because the light that's hitting your eyes is very different when you take the piece of paper from in here out there because this surrounding illuminate different now of course vision is a is trying to

keep track of things that they're invariant doesn't care what the light is that's hitting your eyes it's not like a light meter and so what the visual system does is it discounts the illumina it takes into account the ambient light and so we've known about this for a very long time but what had not been not didn't seem to have occurred to professional vision scientists that there could be individual differences in how this happened and so this this example you know when the dress hit-the hit-the me i remember i was teaching a course at sussex

and it came out like halfway through the course by the time I got back to my office there's like 10,000 like emails and I thought it was a hoax at first I thought it's definitely blue and black and I was really relieved when the fifth person in my lab said no it's white and gold and so now all right okay there is something to investigate here but it was a lovely example of how you know science can sometimes just come out of the wild ray it was it was a cultural meme that just took off

and as you say there's now a small industry there versus other hypotheses of you know how come this this difference comes about and basically I think the thing is if you see it as as blue and black it means you spend too much time indoors yes go on at the same time because being a color neuroscientist we were doing lots of interviews on why this was happening and actually we started shifting the topic the interview because I found it far more interesting in a way that people find it so difficult that this is happening they

I mean really people started getting in conflicts so where we can accept that we might speak other languages and I might understand the words you say but surely my colors are your colors but I love the fact that actually created that sense of sort of humility and doubt in people well it's a graphic example of something that I don't think anybody ever considers we all think the grass is green the sky is blue that flower is purple you know the the chair the cushions on the seats are red they're might be other like I might

hear something differently but we don't think that when it comes to colors and shapes I'm not talking politics I'm talking about concrete things people don't consider the fact that we actually may see concrete things very differently I mean you should talk about politics that because it's actually good as you can give people a demonstration that even something as simple as that can be perceived very differently it becomes much easier I think to appreciate that we can legitimately hold very different world views even when faced with objectively the same situation and that's even setting aside the

fact that you know you we were all asked to look at that dress right but in normal vision and just as we can select which news channels we reaching into we we choose where to look we choose what data to sample so that makes it even less surprising that given that what we perceive is shaped by the data we sample from the world and all that data on all those different channels is all you know even though everybody likes to say it's not fake news this is the real news it's all through a person there's

no such thing as you know person free or journalist free news it all comes through us so and reflects words we chose pictures we chose anyway all right let's move on yes it's very memorable because you're factory based how many of you when you eat asparagus your pee smells horrible some people do not smell it and it's because there is a single mutation on the receptor the olfactory receptor is possible for detecting that gentle so about 10% of the population does not have a problem eating asparagus a 10% of the population cannot smell it but

90% of the population can't but it's there right it's not let that 90% of the population is there sample food don't create that metabolite but that's a different group I love Stavros that you brought up asparagus me that's really very classy of you let's move on let's move on to the sense of by the way one last question before we leave this part a sense of smell you all have talked a little bit about our species and as we've evolved have we evolved has our survival depended on relying on since more than any other I

think primates are highly visual animals as ever I thought yeah sense of sight and you know because I work with rats right and they have kind of a less developed or unless they have lower acuity in their visual system because they live at night or they were awake at night they live in tunnels but they kind of rely on other modalities one of which we don't have right so they have these whiskers that they use and I studied these for my PhD thesis a very affectionate about rat whiskers but they basically they use them the

way that we use our fingertips to actively explore tactile environments and they have a tremendous amount of acuity in terms of their sensitivity with their whiskers basically this the acuity that we have in the human fingertip and so you know what it's like to be a rat where you're navigating around with your whiskers and you're smelling and they have an amazing odor sense that we can't even imagine their subjective experience is probably quite different because they rely on these different modalities okay it's also important to remember though that our senses are integrated right so it's

it's our official system make sense of light using sound as well or using touch if you're wearing a backpack the visual angle of a slope appears steeper than if you're not really yes it's not because I'm just thinking I gotta climb that thing with this thing on my vest because again again none of these are actually in some sense accidents of perception they're only accidents if you think the visual system evolved to see the world accurately to see it not accurately then to make a mistake these are not mistakes it shows what the visual systems

or the sensory systems in general trying to do they're trying to see something useful not accurate it feels like that the tip of the plane looks higher but it can't write because the planet you your frame of reference in the plane has not changed it says your vestibular system that's making it seem as if so there's that and then when you're wearing a backpack it's like the effort required to climb up that you're actually seeing the meaning of it which is it's going to require more effort oh so I was right yes oh okay first

time I've been riding outside okay let's move on to the ear because the ear actually unlike the ID ear can actually according to you guys actually makes sound because it's a complex network of structures that receives sound waves and and sets in motion sort of a domino effect but like our eyes our ears can mislead us and you're gonna show us another example of that and again it's not misleading in a sense right it's always trying to generate something that was useful right so we're going to well do a little demonstration and it's again it's

on sound and it's to show the context of how everything your brain does is contextual I mean your brain evolve not to see it you know not to even detect absolutes but detect relationship so actually quickly if if you right now if you're as you're looking at me your eyes are actually moving you don't know it they're called the cause of my crisis code so you can actually stop your eyes from moving by covering one eye and taking your finger and putting the tear duct and forcing your eyeball to the back alright you can do

it at home with your kids okay and what happens so your eyes open right what happens the whole world disappears because you're not there's your brain is geared to find change in space and time because you've chopped your eyes your stuff your eyes from moving right so anyway so time is also a context so what we're going to do now is I'm going to give you an example of how time is a context but it's the time of the experience of a meaning of something not things so when we think about how history is encode

is not the history of what the thing was because you have no access to it or what it turned out to be because you still have no access to it it's the history of your past meanings of that thing and what I mean by meanings I mean behavioral significance alright so if we play the first sound string nothing actually does anyone here what does anyone hear anything in that shout it out if you hear anything Wayne Queen you don't hear Cody do you hear the word Queen and that you are anticipating that is Queen playing

but did you heady words know some people do it's a psychology test we'll find out okay so now what we're gonna do is we're gonna use some words and read along yes are you getting better at hearing it yes yes notice you're getting better at hearing something that doesn't actually exist you're becoming delusional right because you're hearing the history of what you heard before not the history of what it turned out to be but the history of your past responses to it your history of your past perceptions so what does that sound it's this play

backwards [Music] so that makes obviously similar points to vision but it also makes a point about what the histories that is shaping our brain which is that's that feedback is the history of our past perceptions in this case it was a suggestion by reading this case of course there's a correlation there is you know some correlation between those words and that sound string it's not random right but that sound string doesn't those words don't exist in that say nobody would have come up with it's fun to smoke marijuana if you hadn't put those words oh

my actually I mean we did an interesting experiment this is a site but we had two groups because we do work on conflict resolution so we had two groups and we had them come up with it with a sentence that they heard right and they all this one group all convinces each other of the sentence that they heard we didn't show them it's fun to smoke Marilyn that they came up with a sense and they swore they could hear it the other group came up with a completely different sentence and they swore they could hear

it the two groups could not hear each other's words right that's culture right none of which actually existed but they swore that it did all right which brings us of course to the heart of the debate which is what whether or not what we really see whether or not we what we hear or feel or smell or sense in some way in the world is actually independent of us at all and I know you have varying degrees of thought on this Donald you and I were talking a couple of hours ago downstairs in the greenroom

about the fact you sort of a radical idea that it's it's all it's 100% our perception and there is no right so we independent reality so to speak yeah we tend to think that we're just taking a picture of what's really there and what we're seeing here is that there's a huge amount of construction going on that we're constructing our interpretations of these sounds we're constructing the colors that we see they're useful constructions there is I think an objective reality there is something that exists independent of us the question is did evolution by natural selection

shape us to see that reality or not and I would argue that it shaped us explicitly not to see any of that objective reality and for a good reason I think that what we see is more like a desktop interface so suppose that you're writing an email and the icon for that email that you're writing is blue and rectangular and in the middle of this desktop screen does that mean that the email itself in your computer is blue and to angular and in the middle of your computer well of course not anybody who thought that

completely misunderstands the point of the desktop interface it's not there to show you in this metaphor the reality the circuits the software the diodes all that mess if you had to toggle voltages to craft an email your friends would never hear from you it would just be too hard right and so what what evolution has done for us it's given us a user interface it lets you control reality without knowing anything about reality if we had to actually know the circuits and software none of us could actually use the computer but what we have is

a nice clean desktop with simple icons that lets us control the truth even though we're complete most of us have no idea how circuits work inside the computer what's what's inside there and that's what evolution has done for us it lets us connect of reality to answer your question there is an objective reality and evolution has explicitly hid it from us completely there's a term called unbelt it's a german term which is basically it describes each animals unique understanding of the world that every species sees what it needs to survive and we might see something

different from a zebra or from a lion or from an otter or a fish and you have a really interesting example of that involving a beetle oh yes so in the outback of Australia there's a jewel beetle and as you can see it's dimpled glossy and brown the the females are flightless the male's fly looking of course for eligible females and when he finds one he alights and mates but it turns out in the outback there's another species Homo sapiens and the the males of that species like full beer bottles and they're not interested in

empty beer bottles and they toss them out into the outback and if we go to the next slide we can see that those bottles are dimpled and glossy and just the right shade of brown to tickle the fancy of these male beetles and they swarmed all over the bottles trying to mate so they have full-bodied contact with these bottles and they can't figure it out that's the classic case so the mail leaving the female for the bottle so the species what's interesting about this is the male Beatles for who knows how many hundreds of thousands

of years had successfully found the females and and made it and you would think well that means that evolution gave them an insight into what it means to be a female well apparently not apparently they had a little trick a little hack a female as anything dimpled glossy and brown and that worked apparently the bigger the better too so but so and that that hack that little trick was good enough in their niche all you needed to do was disturb the niche with a few beer bottles and the whole species could have gone extinct 99.99%

of all species that have been on this planet are now extinct we all have interfaces that keep us alive in our niche and they're just there if the niche gets perturbed too much we're gone and so how does rest of the panel how does that relate to the human species and the way we've evolved in the way our senses and our perceptions of our senses have evolved I think we have our own well too I mean we think returns to point made earlier it's not that the unveil is some sort of selection of of the

real world it is a construction of it too so it's it's not we don't just sub sampling this this this external reality we're generating things we come back to color because we regenerate colors from things that are essentially colorless and so as do we as humans render dangerous things more colorful I mean I would give me some concrete examples of how we as a species have evolved to use though ours are those senses and our perceptions of our senses to fur and survival what all of it is like that no I think everything that we

perceive is shaped by its it's evolutions role to keep us alive I mean we there's a tempting way to think about perception as you know we take in information we form the perception and then our brains being some kind of internal computer decides what we should do now in order to maximize the chance of the chance of survival and it sort of follows in that direct but really I think it goes in the other direction that perception of any sort for any species evolved fundamentally to keep us alive to regulate our physiology we've got a

better chance of staying alive and that is then how we perceive everything whether it's our body whether it's the outside world and whether it's other people's mental states anything we perceive is fundamentally shaped by this biological imperative to keep us alive it's not just a few things here and there that we might see differently for that purpose there's a reason for why we do this which is if if we could actually see the world actually wouldn't actually be useful and because the world changes and so we have to adapt and so because our brain evolved

continually get feedback and interact with the world and construct perceptions were able to continually adapt our perceptions with the changing world and for instance we can complexify them and therefore become more adaptable so your brain evolved to evolve this adapted to adapt and if there was only one thing to see well that would be it and that would be a great idea if the world were static but the world changes so we have to change with it you actually have a great the next set of slides and a little experiment oh yes about change yes

this way this will challenge some of your beliefs about what you see most of us think that we see the world in great detail and we can go ahead and put up the two flashing slides here so most of us think that when we look around we see the whole world in fairly high detail I'm showing you two images that image the blank screen and then another image there are some changes do you see any change don't tell anybody if you see it but how many people see change between that image and that image how

many how many can see a change okay do you see that there in the lower right there's a big Bush that appears and disappears right okay did you notice on the in the green lawn on the left side that there's a little patch of green that appears and disappears did you okay and there's another one above the house there's a tree in the middle that appears and disappears so now once the now that you've seen them you can't help but see them every time it flips you can see it going over and over again before

that you probably looked and they look like exactly the same images what's going on well we think that we're seeing the whole world in high resolution in fact all you can see is about your thumb at arm's length so the width of your thumb is the the radius of a circle that's all the high resolution you have and what happens as you look around you put that thumb at different places in the world and you get high resolution at those places and so you have the illusion that you're seeing the whole world in high resolution

but in fact most of us highly blurred you only have high resolution in one little part of the visual world and it turns out this is not just an abstract scientific fact it's really important for marketing and advertising what you're really doing in marketing and advertising is trying to get people to put your thumb on their product right that's the goal and not on the competition's product now it turns out we actually understand rules by which you move that thumb around move your attention around and we can actually manipulate people to put attention on our

products and not on someone else's product so this is actually very useful information I remember reading about Charles Spence at Oxford who won the ignoble prize because of he has worked with potato chips it was very very important research crunching the sound of plastic bag the potato chip bags and eating potato chips and he was able to show that if you fed if we fed all of you stale potato chips according to his experiment but we were crunching up a fresh bag the sound of that bag would make you think would trick you into thinking

you were eating fresh not stale potato chips we're gonna make all of you eat some stale potato chips and Justin really just kidding but seriously it does talk about how those senses work together and how they can mislead one another well I mean I think kind of a common theme is relying on expectations versus evidence right in the sensory world and something that's kind of come up is that sometimes the evidence is bad right like if you have low resolution for most of the visual field then the quality of that evidence is not good right

and if it's very dark and you're trying to make what a shape is the quality of the sensory evidence is not good and so in that case it's good to rely on you know you might do better if you rely on your expectation more and there's actually a whole kind of field of computational neuroscience and a useful quantitative framework called Bayesian inference that explicitly describes just in an in a very elegant equation how one might combine what's called your prior your expectations and sensory evidence and if the quality of the evidence is bad then in

those cases if you'll do better if you rely on your prior more versus if it's you know if you're looking for a long time at an image at high resolution in light then the quality of your sensory evidence is very good and so in that case you don't need to rely on your prior very much you could just rely on the evidence to some of our as our senses work faster than others yes which ones work fast um so audition is the fastest right Sam Addison sation is pretty fast although temperature and pain sensation is

a little bit slower olfaction and gustation because they're chemically mediated tend to be slower there's also speed in relation to the quality of the image in terms of how natural it is how natural the image is to natural in statistics so you can describe the images that fall onto your retina coordinates and sistex and so if the image is like a natural image we can we tend to perceive it as faster we tend to perceive faster than if we're if it's a we warp the statistics so it's not natural even though the same amount of

informations there if an image matches if the data essentially that it matches your priors or you will see it faster you'll expose you persuade if it matches to what if you lean over to smell a rose and it and it smells sort of like what you've always smelled roses to smell like it'll that that will register faster you're saying yeah it's almost like perception is this continual cycle of error correction yeah you have a best guess about what's going on and you test it against the data and and then you keep updating and and when

they align then your brain will settle on a more stable best gassen and that's what you perceive but the key point is what you're perceiving is the best guess you're not suddenly like ah now I perceive the sensory data it's a really important point because not only you're not foreseeing their sense data you're still not perceiving the world because the only way to do that is to have God that tells you your perception is right and this is how actually how neural networks are trained when they use something called back prop so they'll they train

them with with data and they state the network makes a prediction and then the computer programmer says you're off by this amount and then they update the weights in the network but we don't have that we don't have something or someone that says actually you're off by this amount in terms of accuracy what you get is behavioral feedback right you miss behaviorally but you don't necessarily miss in terms of your accuracy in terms of what it's actually they're just curious how also does anxiety or trauma or high emotion affect your senses and how they I

mean I can think of traumatic instances in my life and I can remember exactly what a you know smelled like or felt like or for me it was smell like it the smell of wet cement takes me back to Okinawa when I was six years old and my dad went to Vietnam which was very traumatic so I still like to this day you know decades later when I smells wet cement it takes me back there if you ever had Robin oysters rotten oysters if you get sick from oysters the thought of carving them again is

revolting so your experience or your internal state really affects how you perceive can it cause you to miss your senses to misfire so we put instruments on empowerment so I could put you in different states of power high-power low-power in three minutes by getting to imagine a time that you were stressed and in control or stressed and out of control and you imagine in as much detail as possible you write about etc and then what we can do is we can show you these illusions like the cube illusion and when you're in a state of

low power the actual strength of illusion increases right and it's as if your brain uses context more and use it more uses it more indiscriminately I could then show you six images they're kind of random dot pattern images and in three of them I've hidden a structure and if you're in a low-power state and say how many how many of those six contain a structure you'll say all six but if you're in a high power state you're more likely to say three so in a sense you become more global you start seeing things that don't

actually exist you start finding connections that don't actually exist and this pervades even to the lowest levels of perception another example happens it's wired interest evolutionarily in our evolutionary past there were animals out there that could kill us and it turns out that in this example that we showed where you you couldn't find the the changing trees and so forth it turns out in those displays were very good at finding an animal that changes so we're wired we're actually pre-wired for these things that could have actually hurt us in the past and once again we

use the snap marketing and advertising we can very very cleverly trigger the visual system to look at your product by giving subtle pictures that are like an animal but not not exactly like an animal we can pack into you and tap into this pre-wired evolutionary anxiety about animals and grab attention I thought that was super interesting about how kind of the your feelings of vulnerability can adjust how much you rely maybe on your expectations versus evidence there's also translational implications for this so in the field of computational psychiatry there's an a hypothesis that schizophrenia may

result from kind of people relying not enough on their expectations and actually if you present schizophrenic patients with some of the visual illusions that Beau presented many of them don't see them the way that we do and see them they seem stronger form I remember understood that in psychosis you tend to have well it's a big it's a big debate right you can have more active priors - you tend to see patterns and things where patterns don't exist yet cause a characteristic of delusions and psychosis right and so then the implication is or the hypothesis

right is and is that part of schizophrenia is pathology may result from kind of miss specifying the prior right not doing you know where where things are going wrong is like how they're constructing what those expectations are and that's kind of a promising entry point from a clinical perspective if then we can identify which parts of the brain are involved in constructing those expectations and priors then maybe that would be a good point for an for intervention for clinical interventions and people with children with autism see the illusions as being weaker right as well but

also just the notions of priors I'm wondering um every perception I think requires bias the shape of your ears you consider to be a prior right you are funneling certain frequencies and not to others based on history of experience if you go down deep in the sea you have fish that actually I have only a single receptor tuned to the bioluminescence of the other fish and then as you go more shallow you get two receptors and three receptors so in some sense every aspect of yourself is actually in something that is a prior right it's

it's evolved according to that history we can never leave your history so the idea is stepping outside the box is a silly idea by the way because all you do is you step inside a new box but you can never leave biases and assumptions you can expand them this brings us to a topic that actually Christie knew I think you brought up a little bit earlier which is synesthesia how one sense can sometimes trigger another but talk about how the senses are interrelated that way how they can collaborate together and how they can conspire against

each other sometimes just in this false cues so synesthesia is example where people will hear or see or experience a perception that is not directly related to the stimulus so they and their a tend to be very specific where if you play a note on a cello the person might see the color red as an example so that's one example of cross sensories where they kind of it's not I don't know if you described this going wrong and some sense I would love to know what it's like what's also really interesting about synesthesia is that

the perceptions the relationships don't change what do you mean so the if your if your sena seats you will have a certain relationship between a note or a number or and a color for instance and it will always be and will always be that so Kandinsky many people thought he was a Cinna seat he wasn't he was thinking about it more metaphorically because he would change his relationships throughout time so he was thinking about it in a creative sense so he wasn't we would consider a truce in seat four percent of the population has synesthesia

right and does this mean that they walk through their day you know either been several works of fiction about you know characters who see colors around people they can see jet make a judgment calling on a character based on the aura the color I mean that is obviously fiction what is reality common form of synesthesia is called grapheme-color synesthesia and this is where people when they see letters or numbers and graphemes they will have an Associated experience of color it doesn't mean that sir to see a black eye it doesn't mean that the black a

is red they still see the black a as a black a but they will have an additional experience of redness that may be spatially around where the black a is but quite often not quite often somewhere else or without any special location at all but the key thing and as Bo said the key thing is it will be the same red so if I showed you a black a and and sort of asked you to pick a color to associate with it you might pick a particular and I first you did do that again and

half an hour's time you might remember that you picked red but you wouldn't pick the same exactly the same shade of red but for a sinister will pick from a color palette of thousands or millions of colors the same shade they won't get it wrong so that's one of the diagnostic signatures of synesthesia it's very very difficult to fake that and it shakes one of these things that makes us think it's perceptually real so people do have these reliable automatic experiences and you're in this way there's so there's an argument that actually we're all born

synesthetic and that in early development there's much more intermingling of the senses and that we we learn through development and and just just being alive that our senses begin to separate out William James one of the founders of psychology talked about the the perceptual world of the infant as a blooming buzzing confusion that then resolves into what we think of as independent sensory modalities but of which we've been already talking about a much less independent than than we actually believe and of course at one end of the scale synesthesia just shades into into metaphor we

tend to think of high notes as being lighter than low notes which are dark and again notes don't have luminance so there's there's something that synesthesia is not this weird additional extra thing I think it's it's an amplification of an interaction among the senses that that we all have but it is specific in the sense that again if you have this grapheme-color thing for instance you have it and then you can do things and you will experience the world in a way that people were not synesthetic do not we can give people actually an experience

this sort of a conceptual synesthesia the whole audience if you want sure so I want you to imagine so you can close your eyes keep them open us up to you I'm also gonna read your minds by the way okay I want to read all your minds essentially good okay so I want you to imagine that you're gonna draw a black on white drawing simple line drawing and the first shape I want you to have it be I don't know seven extensions but they're but they're pointed okay does everyone have that in your mind so

seven points on this now what I want to do is now imagine another shape also seven extensions but they're kind of rounded more like a cloud okay do you have those two shapes in your mind okay they don't have any work names to them right they're not circles diamonds are squares right they don't have any names they're arbitrary you drew them right we're going to give you two sounds that are meaningless Kiki and boo-boo okay also has no meanings right which of those shapes is Kiki in which of those shapes is boo-boo how many of

you say that the sharp one is Kiki um how many say the rounded one is booboo like 98% of the population row there's actually a male female difference in this we've discovered a anyway so you in some sense Xperia you now just experienced a moment to synthesise if I give you the words love and hate which of those words which of those shapes is love is that the rounded one is love yeah and the shark one is hate yeah depending on the state of your relationship okay right and if I say hate and I prick

your skin I activate the same part of your brain that has to do with pain right using this assumption of pain so you're actually combining things that actually don't belong to each other right so in a sunsets synesthesia what is what does it mean coming from that when you compare the contour like the conscious world of the conscious self you have a great I love this video the swing distance is up there yeah so this is Sussex campus where I work and there are a few too many dogs not usually that many dogs Bowie you

mentioned about neural networks how we train them and we train and we train you on networks to recognize objects and images by labeling them and saying what's there this is actually one of those neural networks run backwards and so we tell the network there's a dog and then we update the image until the network is satisfied and then we sort of string it together in a movie and you can put it on a virtual reality headset and then you'll be so fused in this slightly weird slightly psychedelic world where there's dogs emerging organically out of

all parts of the parts of the scene what does that demonstrate the point of that is to demonstrate that to sort of give a feeling for what it would be like were your perceptual priors your expectations to overwhelm the sensory data Christine you talked about perception as a process of Bayesian inference where you have expectations and you have data and you're continually updating your priors based on the data if you have overly strong priors then you know will overwhelm the data maybe not get updated that much so this is an example of what your perception

might be if you had a very strong perceptual priors to see dogs everywhere and now the argument is that's what's going on all the time but this is just moving the knee a little bit more to the extreme to the extreme and whether it's a good and a good analogy for us psychotic hallucination probably not more like a psychedelic hallucination but it gives us a way of actually trying to simulate what it might be like but to have these different kinds of experiences in a way that maps onto a dramatically different life experience would alter

that what your expectation is of what you would see during the day versus everybody else for example right yeah I mean it's it's it's a peculiar thing to do if you're familiar with that I mean honestly come to Sussex it's not full of dogs all the time if you're familiar with that environment it's it's it is peculiar it's a peculiar dissonance and through the use of virtual reality as well by putting yourself on a headset allowing you to look around that is a very that adds a lot and when I first tried it I was

very surprised because I thought you look at it on a screen it's that kind of interesting but when you can interact visually with a scene when you can look around and explore it the sense that that it's actually more real is is surprisingly strong you have another interesting video with a student and a a fake hand yeah yeah and this is an example of how you can confuse the senses now I think that this is an example of the another assumption that we typically make about perception is that perception is all about the world that

perception is trying to represent to the world something whether it's accurate or not and what's doing the perceiving well that's the self you know the self is somewhere between behind the eyes in the skull in the body somewhere the self is that which does the perceiving and another way to think about it is that know the self itself is a perception what we see here is an experiment that shows how something we take for granted which is what is our body so normally normally I said don't try this at home but seriously try this at

home it's it's lots of fun so what's happening here is somebody you have this sir it's called the rubber hand illusion and there's that there is a fake hand there the the guy in the blue t-shirt can see it his real hand is hidden out of sight so he's looking at this fake hand the experiment has strokes both hands in synchrony and so what's happening the guy is looking at the fake hand he's seeing something that looks like a hand and is roughly where her hand should be he's feeling touch on that hat because his

real hand is being stroked that's enough evidence in this sort of Bayesian way for the brain to make a best guess mm-hmm that that fake hand is part of his body now it doesn't feel exactly like its hand in fact it a lot of individual differences in that that depend on various things but it's it's an uncanny feeling and you can tell what the brain is really assimilating that because when you stab the fake hands you get this as you could see quite a dramatic a startle response and you know this is something we'd like

to do at science festivals too children will have like you know full-body versions of this where you can where you can have kids stab mannequins in the stomach and the point underlying it is is that yes we tend to assume that yeah this is my body in the rest of the world it is something else but our our perception of what our body is is an active construction just as much as my perception of what I happen to be drinking is or or whether something is changing or not that's also why veterans for example if

somebody who's lost a limb will still feel like oh I think I feel something right there's those phantom limb syndrome where people will still feel pain in an inexistent limb you think well and in fact an amazing way to to treat that is to actually and this guy yes Ramachandran in San Diego years and years ago just use the clever arrangement of mirrors we can now do this with VR you give people like a fake an you give them the sense that their limb is back and they know it isn't but even the fact that

they know it isn't if they can piss if there's a perception of this limb that the response to their movements often the phantom pain will go away but and there are so many other weird there's a condition my favorite bizarre condition is called somatic periphery Nia which is a condition where I perceive that part of my body is actually belonging to somebody else so I would experience that this arm is actually your arm and you know you think well hurry up that'd be yeah and so you have cases people who have this which will often

end up throwing themselves out of bed because they you know they feel that their limb doesn't belong to them it's sort of the opposite of the phantom limb syndrome these all sound very strange but what they all illustrate is that our perception of what our body is is as much up for grabs as anything else and it doesn't just stop at the body the perception of ourself in general the set of memories that constitute me as an individual over time the experience of willing of free will of intending to do an action that's also a

perception and emotion is a perception so the self isn't this in the homunculus in there like little me which would of course then necessitate an infinite regress of minimis inside you know we perceive ourselves as much as we perceive the world what do you think why don't we have I know we talked about the five senses but why do like why is it balance a sense like how do we come up with these five senses balance is a sense right so we have vestibular organs in our inner ear that actually measure things like velocity acceleration

and balance yeah so I guess we have a sixth sense great the idea we have five is some sort of historical accident Aristotle I think is probably to blame for that is it but you know and so much more is tubular we the most important senses that we often overlook are called interoceptive senses so these are these are sense the sense of the body from within you know a large part of the brain is dealing with sensory signals that reporting the internal state of the body what do you mean like blood pressure hunger gastric distension

all this stuff and when we feel an emotion that's a perception well it's so many emotions often have physiological facts I mean anxiety causes your stomach to churn in your heart to race and your breathing to become rapid and shallow yeah you could say that the experience of an emotion is the perception of those changes in physiology so it's the the changes in physiology what's it's the chicken of the egg which is it so this goes back to this guy William James again said hey it's not that we cry because we're sad or that we

run because we're afraid it's the other way around so we so when we feel sad it's because we're perceiving our body in the state of crying or in or in a particular physiological state when we feel fear the the experience of fear is perceiving the changes in adrenaline the changes in heart it wouldn't feel afraid if I didn't run you wouldn't feel afraid if your body wasn't in the state of being at least prepared to run an example is that if someone in some feeling low you can ask them to smile and their brain looks

at the mouth and says that's a bit weird but last time my body did this I felt this way and you will actually start feeling a bit happier so you know a brain evolved in a body about in a world you can actually get from cells not what's called an extended phenotype I mean where does the spider and in the web begin because the the brain the the spiders man is also using the web as its way of sensing the world in it and someone who's blind is walking with a walking stick actually starts getting

cells that are responding to the end of the walking stick so the walking stick in a sense becomes part of them so just to further iterate your point yes we're gonna continue well into the night talking now about a whole host of things unfortunately we are out of time but I want to give each of the five of you a chance to sum up what your sense of the senses are and how going forward whether it be the technological advances the scientific advances or even just the way we change our thinking about what's reality and

what is just the perception could change in the next few years stavros why don't we start with you well I would like to end by saying that our brain really is a very complicated connection collection of cells and its role is to take the physical world and transform it into cells that fire so imagine this very complicated physical world it only transforms into firing cells right so really the perception is is some form of illusion that works for us so the more we harness the understanding of how this firing leads to perception the more wouldn't

understand it I think right I would agree that the perception is illusion that works for us but I would take it a step further and say that the brain is part of that illusion so that the brain itself is an illusion that neurons don't exist except when we see them just like the different colors on the dress only exist when we perceive them and so that that there's some reality and we're utterly ignorant about what that reality is that's actually creating our experiences and so forth one of the experiences that we create is the brain

itself but brains and neurons although I do study them and I think it's very important to do neuroscience and for everyday neuroscience is perfectly fine to talk about neural activity having causal powers strictly speaking it's false and we're going to need to find a deeper understanding of reality that gives rise to what we call space and time and the brain well I disagree with that you know I but I take the spirit of your sentiment and I think it's an interesting perspective and you know we study kind of the neural instantiation of decisions and perception

and it's known in humans if you electrically stimulate areas and certain parts of the brain you can make people see things or perceive things and so I feel comfortable asserting that the activity of neurons and the brains observes perception but I think that there's a lot of really interesting work to be done to understand how experience and biases and kind of our internal expectations shape what it is that we see and feel so my closing thing is it's a combination of two phrases that I borrowed from other people one from psychologists Chris Fritz and the

other from a friend of mine in New York Barbara Brinkman and I would say that perception as a controlled hallucination and what we call reality is when we agree about our hallucinations when our who when they align when they line up that's it well I'll sort of I think I'll finish with how I started do you now know less at the end and you thought you know the beginning right there's actually tremendous power in that so what where I like to think about perceptions once you understand perception and as you now do and maybe did

before you it choirs you to enter conflict in a new way right conflict is the only way in which we learn so the way we enter conflict we usually enter conflict with the aim to convince the other person that's wrong and to shift them towards us the problem is they're trying to do exactly the same thing prove that we're wrong to shift down towards you right and so notice the conflict set up to win but not learn and you only ever learn if you move and so I think the power of understanding perception is it

requires you to enter conflict to the question and to conflict with doubt enter comfort to the uncertainty because imagine what would happen if we enter conflict with a question set of an answer right because everything that you're perceiving is grounded in your history which means everything that the other person's perceiving is also grounded in their history right so it gives you a basis from which at least to sit with the aim of understanding and I think that for me is the power of understanding perception alright thank you so much [Music] [Applause] [Music]