Meta Just Achieved Mind-Reading Using AI

this video is brought to you by brilliant welcome to another episode of Cold Fusion it's the year 2054 and the United States has just federally launched a new police unit it's special precisely because the police will arrest people who commit crimes in the future a concept known as pre- crime this is of course the plot to the 2002 movie Minority Report such a world where crimes are known before they're committed is disturbing it sits in between the Comforts of security and low crime but also complete government knowledge and control it's scary but at the end

of the day it's just a concept in a movie or is it this next breakthrough is a lot deeper all the way down to the very tissue of our brains where researchers have used AI to translate brain scans into text UT researchers have essentially created a device called a semantic decoder that can read a person's mind by converting brain activity into a string of text next in May of 2023 at the University of Texas at Austin a device was created that can turn a person's brain activity and thoughts into a conscious understandable stream of text

the company meta also recently did the same but that's not all just last month meta also unveiled an AI system that can analyze a person's brain waves and predict what that person is looking at if that's not enough they were able to do this in real time basically the system was mind reading has a multi-billion dollar social media conglomerate just created a telepathy device with the help of generative AI on the one hand this is a potential disaster for privacy but on the other hand such Technologies could help many who can't speak due to illness

or injury it appears that these days every major technological innovation raises more questions than answers we're not quite yet at the capabilities of Minority Report but reading brain waves and interpreting what people are thinking is a step in that direction in this episode we'll take a look at all of this so get ready for a wild ride let's start by diving deep into the university mind reading AI then we'll get onto what meta is doing you're going to want to hang around for that one because it's world changing [Music] stuff you are watching cold fusion

[Music] TV UT's newest artificial intelligence tool could be a game changer D for some folks our real hope uh is that this could help people who have lost the ability to communicate for some reason Jerry Tang a doctoral student in computer science and Alexander Huth an assistant professor of Neuroscience and computer science led the research at the University of Texas at Austin the team that developed the mind reading device they published their study in the Journal of Nature Neuroscience the mind reading device is called a non-invasive language decoder while language decoders are not new using

them with the recent breakthroughs in generative AI certainly is older versions could recognize only a few words here or there or at best a couple of phrases Jerry tang and his team have managed to blow this out of the water their device can reconstruct continuous language from perceived speech imagined speech and Silent videos a technology like neuralink on the other hand would be an invasive method it would provide clearer data but has higher risks associated the UT Austin team created a new system called a semantic decoda it works by analyzing non-invasive brain recordings through functional

magnetic resonance imaging or fmri the decoder reconstructs what a person perceives or imagines into continuous natural language it sounds incredible but before we get too excited let's understand the limitations of a non-invasive system and the challenges that the researchers had to overcome while invasive methods such as electrodes implanted into the brain have demonstrated notable achievements in the past non-invasive systems are still in their early stages of development trailing behind in vital areas the team chose the non-invasive fmri because it has excellent spatial specifity meaning that it can pinpoint neuroactivity in the brain with great accuracy

however its major limitation is its temporal resolution or in other words how quickly it captures changes in brain activity the blood oxygen level dependent or bold signal that the fmri measures is notoriously slow taking about 10 seconds to rise and fall in response to neural activity now imagine trying to decode natural language where we speak at a rate of one to two words per second so how did they tackle this problem to address this researchers employed an encoding model which predicts how the brain responds to natural language to train the model they recorded brain responses

while subjects listen to 16 hours of spoken narrative stories by extracting the semantic features and using linear regression they built an accurate model of the brain's responses to different word sequences it's uh basically a model that has been trained with over 15 hours of data per person to take in their brain activity when they're sitting in an MRI scanner and to spit out the story that they're listening to here is where it becomes a little more interesting to ensure the words that were decoded had proper English grammar and structure the research SE archers Incorporated a

generative neural network language model or generative AI interestingly they used gpt1 an earlier version of the now famous chat GPT models this model can accurately predict the most likely words to follow in a given sequence but here's the catch with countless possible word sequences it would be impractical to generate and score each one individually this is where a clever algorithm called beam search comes into play let me briefly explain how this works first it analyzes brain activity and then generates word sequences one word at a time it starts with a group of the most probable

word sentences as it analyzes brain activity further and identifies new words it generates more possible continuations for each sequence in the group this approach gradually Narrows down possibilities and the most promising options are kept for each step by keeping the most credible sequences the decoder continuously refines its predictions and determines the most likely words over time an example would be necessary here consider the scenario if a person hears or thinks the word Apple the decoder learns the corresponding brain activity when the person encounters a new word or sentence the algorithm utilizes learned associations to generate

a word sequence matching the language stimulus's meaning for example if a person hears or thinks I ate an apple the algorithm May generate I consumed a fruit or an apple was in my mouth the generated word sequence may not be an exact replica but it captures the semantic representation so with the help of an ingenious encoding system generative Ai and a clever search algorithm the scientists were able to circumvent the limitations of invasive fmri [Music] scans so with the challenges tackled it was time for the researchers to put the system to the test and see

what it could do the team trained decoders for three individuals and evaluated each person's decoder by analyzing the brain responses while listening to to new stories the goal was to determine if the decoder could understand the meaning of the information and the findings absolutely remarkable the decoded word sequences not only captured the meaning of the information but often replicated the exact words and phrases this shows that semantic information or in-depth meaning can indeed be extracted from brain signals measured by [Music] fmri the researchers also wanted to find out whether different parts of the brain have

similar or different information about language essentially they were looking for how the decoding process works across cortical regions cortical regions are specific areas in the outer layer of the brain known as the cerebral cortex each region corresponds to distinct sensory motor or cognitive processes the researchers compared predictions from decoding word sequences in these regions and they discovered something extraordinary they found out that diverse brain regions redundant ly encode Word level language representations basically our brains have backup systems for language understanding and usage it's like having several copies of the same book if you lose one

copy or it gets damaged you can still read the book with another copy so it suggests that even if one brain region is damaged others can process the same information preserving our language abilities and that's a pretty astonishing [Music] find the researchers went even further they tested their system on imagine speech which is crucial for interpreting silent speech in the brain participants were told to imagine stories during fmri scans and the decoder successfully identified the content and meaning of their stories the researchers were ecstatic about their new discoveries and intrigued by the prospect of decoding

other types of information as well so they explored cross modal decoding researchers say they were surprised the decoders still worked even when the participants weren't hearing spoken language even when somebody is seeing a movie you can kind of the model can kind of predict uh word descriptions of what they're seeing this study is a significant advancement for brain computer interfaces that don't require invasive methods however the study highlighted the need to include motor features and participant feedback to improve the decoding process but this story isn't over as promised meta has come into the picture and

they've come to raise the bar of what we thought was possible although in the movie Minority Report it was psychics that helped reveal the intentions of an individual it's AI That's looking to do the same here in real life but how does AI work anyway well now there's a fun and easy way to learn about that and many other stem subjects with today's sponsor brilliant brilliant's course on artificial neural networks is perfect for that I've used it to brush up on some background context when I was making AI episodes brilliant has interactive stem courses in

anything from maths and computer science to General Science and statistics whether you just want to brush up on learning or need a refresher for your career brilliant has you covered you can get started free for 30 days and the first 200 people to sign up get 20% off an annual plan visit the URL brilliant.org coldfusion to get started thanks and now back to the episode on October 18th 2023 meta pushed this field of research even further instead of using fmri which has a resolution range of one data point every few seconds meta used meeg technology

another non-invasive neuroimaging technique which can take thousands of brain activity measurements per second with this they created an AI system capable of decoding visual representations in the brain the system can be deployed in real time to reconstruct from brain activity the images perceived and processed by the brain at each instant I told you this was going to get wild the system consists of three parts an image encoder a brain encoder and an image decoder the AI was trained on a public data set of Meg recordings acquired from healthy Volunteers in meta's research it was found

that quote brain signals best align with modern computer vision AI systems like Dino V2 this AI learned imagery by itself and wasn't supervised by human labels meta states that quote self-supervised learning leads AI systems to learn learn brain-like representations the artificial neurons in the algorithm tend to be activated similarly to physical neurons in the human brain in response to the same image this functional alignment between such AI systems and the Brain can be used to guide the generation of images similar to what the participants see in the scanner using this technique meta managed to read

the minds of people and interpret what they were looking at on the left is the original image shown to the participant on the right is what the AI thinks the person is looking at and remember this resulting image was only determined by the brain waves the AI had no idea what the person was actually looking at the resulting images could nail broad object categories but were unstable in the finer details meta claims that their research is ultimately to guide the development of AI systems designed to learn and reason like humans we'll see about that envisioning

the future of brain reading technology is challenging but the potential could be V last for instance it could help those who are mentally conscious but are unable to speak or communicate thinking further a field imagine if a smartphone's volume notifications and music selection would adapt to a user's mood or brain activity and smartphones and other devices could be commanded or queried just by thoughts alone it would be like speaking to Google Assistant but with your brain next sense which originated from Google's parent company alphabet's X division has been working on something interesting since 2020 the

startup is currently developing earbuds with the remarkable ability to capture human brain signals these earbuds have the potential to Monitor and diagnose brain conditions like epilepsy depression and Sleep Disorders they can also detect seizures assess medication Effectiveness improve deep sleep and provide valuable insights for comprehensive brain assessment now there's obviously a long way to go with brain scanning technology but it's clear that we're at the very start of a new era of interpreting the human brain thanks to generative AI on the negative side however the sum of all of this technology could be a privacy

catastrophe imagine meta or Google being able to know what you're looking at or what you're even thinking in order to better Target their advertising or to sway public opinion in one way or the other who knows what this technology will evolve into in the next 30 Years in the very same era that Minority Report is set in with meta getting involved there are natural concerns for me mind decoding technology in research is one thing but having popular wearable technology such as the meta VI headset or video camera glasses could be a future Temptation for the

firm meta were never known for their care of user privacy and consent though still to achieve true mind reading or telepathy the decoding technology would need to overcome a lot of challenges and limitations accuracy and resolution must increase the system would need to understand tone emotion and contextual nuances of thoughts second it would would need to work in both directions allowing a person to both send and receive thoughts from one person to another so true telepathy remains firmly rooted in fiction and perhaps that's for the best so back to the university researchers for a second

the scientific world has been stunned the findings of the semantic Dakota are undeniably remarkable tonight we turn to news in one specific way that artificial intelligence technology is changing our world and it is absolutely remarkable these neuroscientists at the University of Texas in Austin say they've made a major breakthrough they figured out how to translate brain activity into words using artificial intelligence Dr Alexander Huth the lead researcher expressed his surprise to the guardian saying quote we were kind of shocked that it works as well as it does I've been working on this for 15 years

so it was shocking and exciting when it finally did work end quote the Breakthrough captured widespread attention from the media and scientists alike Professor Tim Beren from the University of Oxford praised the study's technical press highlighting its potential to decipher thoughts and dreams and reveal new ideas from subconscious brain activity he said to the guardian quote these generative models are letting you see what's in the brain at a new level it means that you can really read out something deep from the fmri end quote when you stop and think about it an artificial brain interpreting

the internal happenings of a human brain makes total sense but it's just something that I thought would be a long way off it's just my opinion but I think that these AI Technologies truly shine when they're applied to scientific research and studies like this that's when we get to harness the full power of artificial intelligence so there you have it brain decoders that utilize the power of AI to interpret human thoughts and generate coherent language and imagery well at least in a controlled environment for now the scanners are still bulky and expensive and the training

process is long and tiresome however as with all technology this is only going to get better of course there are concerns companies could misuse this for their own selfish gains but for the physically impaired the possibilities are almost endless personally I found this quite fascinating but what are your thoughts on this how do you guys think that this is going to shape the future do you think that it's good or bad to be able to read or at least interpret someone's thoughts or do you just think that this technology should just never see the light

of day it's an interesting discussion so feel free to discuss in the comment section below all right so that's about it from me thanks for watching if you want to see anything else on science technology business or history feel free to subscribe to Cold Fusion it's free my name is toogo and you've been watching cold fusion and I'll catch you again soon for the next episode cheers guys have a good [Music] one [Music] [Music] cold fusion it's new thinking