Sal Khan: How AI Will Revolutionize Education

Hello and welcome to tonight's Commonwealth Club World Affairs Program. My name is Adam Lashinsky. I'm a columnist with the Washington Post and the San Francisco Standard and your moderator for this evening. Now, it's my pleasure to introduce tonight's guest, Sal Khan. Sal is the founder and CEO of the Khan Academy. Oh, please, please. By all means. By all means. So much for any hard questions I was going to ask you. I would get a hard time from this from this room. As I was saying, Sal is the founder and CEO of the Khan Academy, a

nonprofit providing free world class education for anyone, anywhere. Sal believes education is a human right and his interest in education began as an undergraduate at MIT. In his new book, Brave New Words How I Will Revolutionize Education and Why That's a Good Thing. Sal explores what technology means for students and educators and the social and ethical implications that I also will have on society. The Wall Street Journal called his book readable, a readable and cheery view of the academic apocalypse, and Bill Gates posted very recently, No one understands where education is headed better than Sal

Khan, and I can't recommend brave new words enough. Sal, welcome back to the club. And now please add one more round of applause. I hope I'm not too clever in asking you my first question, which is how much of GPT wrote your book? Very little. I will say that I definitely used it for ideas showing some of the chapter titles, but I don't think any of Egypt's ideas made the final cut. Okay. So and yet the book is is a very articulate and cheery defense of and I think explanation of why I will be a

valuable tool specifically for education. So, you know, share the high level thesis for everybody. Oh, and I'm sorry, I meant to do a little exercise. We can see you could could I just see a show of hands for anyone in the room who has used Khan Academy at all in their life enthusiastically, about maybe a third of the book. And one more show of hands. How many people in the room have sort of aggressively or actively or frequently used GPT or things like it a lot. Okay. All right. I wanted you to I wanted us

to have a sense of people's interest level or knowledge level, a high level thesis. Why might this be why might be a good thing for education? Yeah. You know, a lot of folks, and I've always used to talk about how Khan Academy got started as just a fun family story, but I now tell the story because it really does inform what our true North is and what I hope I could be, at least in education. A lot of folks know I started Khan Academy almost exactly 20 years ago. Original background in tech. But after business

school I found I was an analyst at a hedge fund and 2004 my family was visiting me from New Orleans, which is where I grew up in initially New Jersey for our wedding and then Boston and just came out conversation. My 12 year old cousin Nadia needed help with math. I tutor her remotely. When she goes back home, it starts to work out. Word spreads in my family, free tutoring is going on, and before I know it, you know, 1015 cousins every day after work I'm working with. And you know, even those early days, 24,

25, I saw a common pattern. The reason they were struggling wasn't because they weren't bright. It wasn't because they weren't hardworking, wasn't because they didn't have good teachers. It was because they had gaps in their in their learning. They they would be pushed ahead. Maybe they were in an algebra class. They were a little bit shaky on their decimals or their negative numbers or whatever. And then I as a tutor, I could diagnose that, I could work with them and it's something that had been very hard if I had to work with 30 of them

at once. So it was when I was able to do one on one or I was able to do small group. And then as I got more cousins, I actually I did get more because of those tutoring more and then there's a few have been produced since then. But the I said, Hey, you know, I have a background in technology and I did. I've, I've always been fascinated by could you use technology in some ways to scale up learning? And so I said, Well, okay, what if my cousins need more practice? I'll write a little

question generator. I is their tutor, their teacher. I want to monitor what they're doing, where their gaps are. So we could focus during when we got on the phone with each other. And then and that was the first Khan Academy had nothing to do with videos. And then in 2006, a friend suggested that. And by this point we had moved out here to Northern California. A friend suggested I make videos. I gave that a shot. And my cousins, I mean, they famously told me they like me better on YouTube than in person. And and what

they were saying, I believe they still appreciated me in their life, but they really liked having a on demand version, infinitely patient. No shame if they had to review something that they should have mastered many years ago. Whenever they want, pause, repeat, double speak. It has to be. And I kept going. And you know, you fast forward from that period from 2006 all the way to now or especially even right before we started exploring generative AI and kind of it was much more than me. Now 160 million users and it's in 50 languages and it's

in almost every country where a team of over 300 folks now. And but everything we've been working on is kind of using technology in order to scale what I was able to do with with Nadia. You know, I tell our team all the time, we said our mission is a nonprofit, is free world class education for anyone, anywhere. And there's a very natural tension and intentional tension there of world class. We want to be truly great. To be world class means personalized. Meet the student where they are. I always say, and I mentioned in the

book Alexander the Great had Aristotle as his personal tutor, as his teacher. That's what world class looks like. But two or 300 years ago when we had mass public education, which was a very good thing for society and did all sorts of great things, we we had to make a compromise. Well, we can't give everyone a personal tutor. We'll batch students together in groups of 25, 30, 35, move them together at a set pace. And so there's always been that free that tension between free and world class. But maybe and this I didn't predict generative.

I would happen as fast as it's happened. Maybe if you intelligently use technology, you can start to resolve that tension a bit, get more personalization. And so in summer of 2022, Sam Altman and Greg Brockman, two of the founders of Openai, send me an email and they say, Hey, we're training our next model. We would love to show it to you. We think this is going to be the model. That's the guy that really wakes people up. I was skeptical. I had seen GPT one, two and three and they were cool as technologies, but they

didn't seem ready for primetime. But what they showed me initially and then the rest of our team ended up being GBG four and that one seemed like, you know, the track that Khan Academy was going in terms of how good of a personal support it could provide for teachers and students was going to get better and better over time, but it was probably going to reach some limit. But what we saw with these this new generation of models could do that much further. So the thesis is a tutor for every student and a teaching assistant

for every every teacher. And so before we dive into this statistics, you made two important points in the book because you said you were skeptical. At first, as I was reading your book, I found myself being skeptical. I think you anticipate a lot of the skepticism and then you and then you answer it and you talk about how you've been thinking through it. But the two points you made that I'd like you to share to to to articulate or elaborate for for the room, is that one the pacing issue, that the pacing isn't right for

everybody. And the fact is, okay, you know, we don't know if this A.I. thing is going to work out, but how crazy are we about what we're doing right now? Talk about that if you. Yeah, and it was yeah. If you think about what's going on in most schools, in almost all schools, even the fanciest private schools, etc., students are best together. You know, if you have more resources, there's usually a smaller class size, but it still might be 15 or 20 students. And in some larger schools, it's, you know, 30, 35 students and you're

going at a set pace. And a teacher, let's say we're covering basic exponents. There's some lecture homework, lecture homework. Now, classes are more active learning than when we were in school, but that's essentially what happens. And then after a couple of weeks of that, there's a test. And on that test, let's say you get a 90%, I get a 75%. Even though that test identified gaps and the test doesn't even measure everything, it's just samples some questions. The whole class will then move on to the next concept and that it's usually going to be now

negative exponents or logarithms, somehow expecting the student who got a 75% on the more basic to not master the more advanced. And then that's even before you talk about things like the summer slide and kids forgetting things, etc. etc. and so the the the the reason why smaller class sizes at the ideal personalized tutoring, the reason why I was able to help my cousin so much was because we were we had the luxury of being able to address what those gaps are. The stats I I've seen and the pandemic has made things worse. Before the

pandemic, the average classroom, the average math classroom and it's not just math, but math is maybe where it's most pronounced. In a classroom of 30, you had three grade levels of students in that. So and every teacher knows this, like, okay, those kids are ready to run. These kids have some real gaps that if someone just filled it in, they would also be ready to run. And then there's a bunch of kids in the middle. So I'm going to try to, you know, go to the average after the pandemic, the spread. I've seen stats, that

is, it's typical now for sixth grade levels of kids to be sitting in the same classroom. So how do you keep all of those kids engaged at a level that's appropriate for them? And it's completely the opposite of if you think about being excellent at something, if you imagine what's happening and think about someone in sports, think about someone who's a who's a musician, that's not what you're going to do. You're not going to give a lecture about playing piano and just hope that the 230 people at a time, you're going to watch that student

play piano. It's like, okay, you know, you got to work on this scale a little bit. Your fingering needs to be a better maybe composition and adjust to it. And so that's that's what the real the real ideal is. And your hypothesis is that in a tutor, if it works well, can, can close that gap because it can give each student who has access to it the attention, the tutoring, the nurturing, the criticism with a small C that that they need. Yeah. And I don't want to make I think anytime you talk about I and

I think I feel this way, too, sometimes it feels like, hey, this is this going to go after people? Is this going to be after people's jobs? And I want to make very clear and even in the early days of Khan Academy, well before the AI conversation became a big conversation, people would say, oh, you're you're giving you're doing YouTube videos. That's kind of like a lecture. And you're doing these exercises that go to students face. Is this a replacement for teachers? And now people are making that that saying, hey, is AI a replacement for

many jobs now? And I've been very clear and I say this because I truly believe it, not just because it's the right thing to say. If I had to pick between an amazing teacher and no technology or amazing technology and no teacher, I would take amazing teacher and no technology any time now. What the hope is, is with AI and even pre AI technology, but especially with the AIS. Yes. When a student you know, one of the criticisms of Khan Academy over the years, like the videos can be great. They're on demand, there's bite sized,

there's so many of them. Students can figure out what they need. But what if a student has a question? Well, now you can give the AI context of what's going on and have a have a conversation about it. Now, there's a lot of questions about cheating, but this is all it's more of an engineering question. You know, the A.I. that I talk about that we work on it kind of academy. We call it Conmigo, It is Socratic. If you say, Why should I care about learning this algebra or learning this American history, it'll say, Well,

what do you care about? And then actually remember, once you say, I want to be a professional athlete, I want to be a lawyer, I want to be a doctor, whatever. And then it'll it'll connect it to you. It'll personalized for you. Yeah. I recently just did another example. I'm always I'm a learner still myself. I still make videos on Khan Academy and I had a gap in my now I was like, Why do why does Supernova why do they explode? You would think when a star runs out of fuel, they would collapse. So I

go on to Conmigo and I say, Why does the supernova explosion say, okay, before we go into that, can you explain to me can Sal explain to the eye what you think a supernova is? So can Socratic re push you in the way that you would expect a strong tutor to. So can you can you explain in a way that non software coders can understand? How did you teach the AI to have this Socratic method? This is a this is a feature I assume, right, that that doesn't necessarily exist in GPT. Four. Oh yeah. It's

a really interesting thing. And I remember even that first weekend when I believe we were the first people outside of Openai to have access and this was months before chat GPT existed and even Chatbot wasn't built on GPT four, it was built on GPT 3.5. But it, you know, the term that I get a lot of people know now is prompting that you can give directions to the AI and there are things called system prompts where it's happening behind the scenes. So if you're the user you don't see that the AI is getting that instruction,

but the person, the people making the application can give that, that instruction to the A.I.. But even if you were to go to chat GPT and if you were to prompt it yourself and you would say you are you, the AI are a Socratic tutor, you will not answer questions, but you will ask leading questions. You can even give it examples, etc. etc. Just doing that, it'll actually already be pretty good. Now if you're going to make an application that's going to be used by millions of children in real school settings, you have to put

a lot more work into it to make sure that it handles edge cases, etc., etc. But that's essentially the essence. And it's interesting because we've talked to all of the leading AI folks now, you know, Openai, Microsoft, Google, we're close to all of them. And it's interesting because we're like, oh, a little kind of academy. We're just trying to do what we can to make this useful for people. But like you helped us appreciate the power of prompting because a lot of the technology companies are making the AIS, you know, more and more complex, more

and more powerful. But there's a lot of the art to to prompting now. There's more things that you can do on the back end, make multiple calls to the AI, make, you know, to make the math better. You know, the memory is a whole other thing that we're doing behind the scenes. But at the essence, these things are the word and the AI community steerable. And that's actually what made GPT four. Produce steerable. Steerable like you're doing a car and that's what made GPT four different than some of the other models before it was that

you could give it this type of instruction and for the most part it will, it will listen to. It and with Conmigo to tell everybody what that means. Yeah, it it's an interesting story how we decided to name it. We initially were going to call it companion or a lot of puns on. Companion on. On. But then I think some of our. Yes, well, what happened was some of our colleagues in Latin America told us that that was a euphemism in Latin America for a not so good thing. And so so then the focus group

this, yes, it was good to follow. So then they they they suggested Conmigo. And, you know, we thought it was nice because it's a it's a pun on Conmigo with me and also with maybe conmigo so that's what stuck. But what I want to be clear and and we're getting to the issue of safeguards or guardrails but with with conmigo you said you design it, you set the bells and whistles, the features. So if I'm in a class that's using it, I can't. I. Can I unsaid it? What can I say? The conmigo told you,

asking me what I think about this. Just tell me the answer, please. Yeah. So as soon as we start having access this technology and you could imagine the debates that we had inside of our organization, especially, you know, we're a nonprofit of everyone who joins Khan Academy. We're here for social benefit. So many of us, and we're excited about this technology. But it had real issues. The math isn't perfect. It can make up facts, is often referred to as hallucinations by by the A.I.. That is the technical term. But then it was very obvious to

us, well, this could be a cheating tool. This could it's you can have open ended conversations with it. How what what safeguards can you put on it? How do you make it transparent? And what I encouraged our team and I refer to this in the book a lot I use this term educated bravery, which, you know, all of these risks and fears are not reasons to run away or stop doing it. There are reasons to build features that can mitigate what these risks are. So to your question, can we go? Will not and you know,

if you say tell me the answer says I'm here to be your tutor. I'm not here to tell you, you know, etc., etc.. All the conversations for an under 18 user, it's observable by a teacher or a parent. If the student explicitly tries to do something dangerous, if they say, I want to build a bomb, I want to harm myself, or we have a second day that will actively notify parents, teachers and administrate orders just so they know, Hey, this the thing is, it looks sometimes it has false positives. It's okay. And my my daughter

recently told Conmigo, hey, I want some ideas for our project. I really want to beat the other team. It took that a little bit, literally notified her teacher. The teacher said, okay, I don't think she wants to act physically, beat them, but it's a good safety. But but we do see coming across, you know, every now and then there are kids that are every now and then saying things that, okay, you know, their teacher, the principal should know about that. And then there's a whole series of guardrails around protecting student privacy that's extra sensitive. We

don't none of the interactions that the students have with the AI can be used to train the general public model. We d we anonymize personally identifiable information as a as a secondary precaution there. So there's a lot of care in making sure that and this is before we talk about all but the teacher side of the equation, which we think is just as powerful. And where where is what's the status of conmigo? How much is it being used? And are you going to charge money for it and that sort of thing? Yeah, it's this is

an interesting thing. You know, as I said, our mission is free world class education for anyone, anywhere. And as I always point out, when I'm fundraising for Khan Academy, which I do a lot of, it's not free to build. So we've historically been mostly philanthropically supported so that we can create these resources and send it out to the world. But what's been interesting about generative AI, you know, we immediately there's is multiple there's different models. There's models like GPT 3.5 and there's equivalents from Google and other players as well that are pretty low cost. But

they are there, I would say, for what we are trying to do, they are not good enough. And then there's what you'll oftentimes hear the term frontier models. The frontier models are the best models that are out there. And currently we think that you need to use these frontier models in order to have a quality experience that you could credibly put in front of teachers and students. Now, the problem, especially even when we launched in March of 2023, these are not cheap. A lot of folks might like the computation to run these models. A year

ago, we estimated that it would cost 10 to $20 a month for a user to run this. Now, that might still be interesting because if a student can really learn and even if it's 100 or $200 a year, that could still be. But that was obviously a tension with our mission of can we give it away for free to people? So when we launched as part of our pilot, we said, okay, we need a resource that we found some pilot school districts were help helped core resource it. We found some philanthropist who helped co resource

it, but we did have to charge something to some of these school districts just so we didn't we didn't, we didn't go bankrupt. The good news is, is since then the computation costs and the models, the costs have come down. The models have gotten more efficient. We're now looking at what I was saying, 10 to $10 a month. It's now closer to 10 to $20 a year and it's getting cheaper. I'm hoping this time next year it's going to be in the single digit dollars. And so now it is getting in the range that we

can start to give it away for free. I was actually just in Seattle this morning. We made an announcement with Microsoft. Microsoft has donated a lot of compute. There's several parts of the partnership so that we can give these tools for free to every US teacher. And I'm hoping we're going to figure out ways to give more and more free to every to every student. But that is a major issue. How do you pay for the computation and. The are you seeing for profit competitors yet? Yeah. Oh, you know, every hour it feels like there's

five new startups that are you know, saying we are a tutor, we're going to do this or that. It's interesting. It I think obviously I'm biased open AI's intuition when they reached out to us, they said, look, this model AI is going to start opening people's minds. It's going to be exciting, but it's also going to be scary. And they said, We want to lead with social positive use cases. We thought of education. We wanted to lead with organizations that folks trust that have the technical capability. We thought of Khan Academy. And it is it

is the case that, you know, it's very easy to imagine a lot of bad use cases of AI. And it's funny. I mean, everyone who reads the book kind of says, oh, you're being very, very optimistic. Yeah. Not not to call you a marc Andreessen or anything, but you're you're you come off as a techno optimist. And in the book. And I've been thinking and, you know, I don't want to be overly defensive of that because it feels like everyone has walked away from the book with that. But what I what I try to point

out is I too, am afraid of a lot of the clear negative use cases I am afraid of. I mean, if we thought things like phishing attacks and fraud were bad five years ago, they're about to get ten times worse. I mean, it's just in the car. I was telling my wife who's here, I was like, we need a code word for our family because we might get a phone call from someone who sounds like our mom or our children saying, Hey, you got to bail me out or you got to, you know, send some

money, etc.. So I'm scared about that. I'm scared about deepfakes. I'm scared about misinformation. You know, there was a case recently in a school district where disgruntled employee made a deepfake of, I think, either the superintendent or the principal making racist comments. And the person almost, you know, got fired for that. There's bullying happening where people are creating deepfakes of classmates doing horrible things. So these are very, very scary things. But what I point I try to point out this out in the book, it's very easy these and, you know, most dinner parties where people

are talking about A.I., they get into this like, oh, this is this is scary stuff. This is going to be and I say, look, this isn't a flip of a coin at all. Technology in human history is an amplification of human intent. A knife can be used to kill, a knife can be used to build things that can be used to prepare food. It's an amplification of human intent. AI is no different, although it does start to question who are we? And it goes after things that we thought were uniquely human. But when I say

we know the bad actors are going to do the bad stuff, so they're going to amplify their intent. So the question is, those with positive intent, are we just going to wring our hands and say, well, yeah, this stuff, I'm just going to I want nothing to do with it? And then the only people are going to be using these very powerful tools are going to be the bad actors. And we say, No, let's put some real positive intent behind it. And I think there are really exciting areas. If we have the right guardrails in

education, in health care, it's going to be driving productivity in probably every industry. Now we just have to think about what's going to happen around dislocation. How do we make sure people are prepared for what's about to happen? But I'm a I'm a big believer of good people should not run away from it because it has it has some bad uses. Yeah. I mean, on the on on the positive side, you're saying if you have to choose between being a techno optimist or a Luddite, you're going to choose the former. But you also run the

risk of you. You're making some something of a guns. The guns don't kill people. People kill people. Argument. Well well I think the the argument I think I will do some very dangerous things I don't want to pretend and and I do think that we need to we need to be very smart about how we address it even further. I think it's going to be a national security issue now that the national security this is going a little bit out of out of the scope of the book. But the the the way I think about

addressing it is, you know, obviously people who think about defense are like, okay, how do we put guardrails around it? How do we make sure that, you know, bad actors can't use this technology to undermine our safety and all of that. But the way that we're going to the the best chance we have of A.I. being a net positive for humanity because Pandora's Box is open, it's not it's not going to be put back in, I still believe is for good actors with good intent to put as much energy behind it. It's not a flip

of a coin. It is. It's going to be as good as we as we make it. And how how involved are you in the conversation about U.S. government regulation? And do you have a, you know, a high level, high level bit on that? I'm a little involved with it. You know, President Biden was out here in San Francisco several months ago, and then he had a little roundtable of of and I you know, Artie Prabhakar, who's, you know, works on science and technology in the Biden administration, she says, like, well, I'm inviting you out because

you're the only optimist. You know, like, you're the only one who has a positive use case, you know? So we literally I remember we were in that room and, you know, people are talking about really scary things, which I agree are real risks, like imagine someone creating deepfakes of a run on a bank that can create a, you know, a global a financial contagion like that. These are real risks we have to protect against. And, you know, after seven people talked about these, like apocalyptic things, I was like, oh, but it could also really help

kids learn and it could also help teachers with great lesson plans. And now we have to put guardrails. We have to make sure it can't be used for cheating and we have to make it transparent to the user. And I think there are ways and we're building and this is the other thing. I don't want to be someone who just philosophize and just I'm a techno optimist. I'm just going to say nice things. I want to I want to build these things. I want to show people that these are real things, that real. To your

earlier question, we are now there's tens of thousands of real teachers and students that are using this, too. And, you know, I encourage people to do a web search on on Conmigo teacher stories. These are real teachers getting a lot of benefit. It's saving them time. It's allowing them to personalize their classes more, their students who are feeling better supported, more confident than they ever had, not just in math and the humanities. They're getting more support in writing, not less. And actually. I'm sorry. Interrupt you. We will you tell the story about how I can't

remember who was a student or if it was you who used Conmigo to have a dialog with Jay Gatsby and how that why that was effective. Yeah, well, you know, the book actually starts with This was Christmas, Christmas holidays of 2022. I was just running experiments and our our daughter the I was like, Hey, I've prompted the AI so it'll write a story with you, not for you. So she was writing a story, the story and it was about a social media influencer who got stuck on a desert island without an internet connection and that

which my daughter was writing with, with the AI. She would write a couple of sentences and then they said, Oh, what if this happened? And my like, Oh, how about if this happened? Well, anyway, the protagonist, Samantha, got stuck on this island and she's having something of an anxiety attack. The character in the story, because it's so beautiful on the desert island she can't shared on Instagram. And and my daughter, she's like, Oh, I want to talk to Samantha. And I was like, Oh, that's an intro. I talked to a character in a story that

you were writing with and try it out. So talk to Samantha and then some. They then said, Hey dear, this is Samantha. How can I share how beautiful this island is? And and my daughter started saying, Hey, it's not you should just experience it. You don't have to share everything just for it to like, like my daughter started becoming the therapist for a character in a story that my daughter was writing with an AI. And I was like, This is this is like this is like the future. This is this is beyond anything science fiction

imagined. And so I went to work the next day and, you know, we were working on Conmigo, and I told our team like, we should have activities not only where you can write stories, but what if students could talk to AI simulations of literary characters of of historical figures? And there's a lot of questions about how to do that, right, etc.. But I think we're threading the needle. So one of the activities on me goes, You could talk to literary characters and we have an online high school called Khan World School that's run with Arizona

State University, has kids from all over the world in it, and they were amongst the first to use Conmigo and a young a young woman named Sohn V from India. She in the school in Khan World School. When they read books, they have to do video reviews of of of the books. And, you know, she she'd done some research about why Jay Gatsby, you know, looks at the green light and all of that and you know that's obviously been studied to that. But she wanted a little bit more texture. So she goes on Conmigo and

she she talks to the AI simulation of Jay Gatsby and she has a whole question about what is and talked about Daisy Buchanan and love and things that are on the can I mean, long conversation. And then after several hours of like a real conversation, she apologizes to Jay Gatsby for taking up all of his time. And he's like, oh, no worries, You know, old sport. You know, you know, this is what I'm here. But it was it was a it was actually a substantive conversation. But, you know, that's the kind of thing it really

made it come alive for her in ways that are almost impossible otherwise. So you are sending in great questions. I'm going to I'm going to start asking them very shortly. But but but you touched on the one thing that I found myself struggling and struggling and struggling with, and it's this question of creativity. And I think you argued your point articulately very well, even though I'm not all the way there. But but your your point is that let's not have a let's not have an idealistic view of what creativity is in the first place. And

and I can help us be creative, not just allow us to not be creative. I don't know if I'm butchering your way of saying, Yeah. I'll go back to technology is an amplification of human intent and AI is like that. And so if your intent is to be a creative person, it will amplify that. Because I point out if Adam's a creative person and I want to be a creative person, when Adam walks into the room, I don't become less creative. We start riffing on ideas. We start talking about what if we did this? What

if we all imagine the most creative times in our life is when we were around other creative people? If we think about the most creative times in human history is when creatives got together, they hung out at coffee shops in Paris, they hung out and, you know, and the art studios of Florence, wherever they were, they worked on startups in Silicon Valley. A great television shows get written in a writer's room. Exactly. So creative people, when a creative person enters the room, it's not a zero sum game. I become more creative, and I personally have

seen this with the I. You can use it as a really good partner to riff on. I riff on ideas. Now, if a student is lazy and says I just want to like, I don't want to think you could, yes, you could go to chat GPT and then say, Here's some ideas and like, Oh, copy and paste etc. Now Conmigo going to do that, It's going to push you on your thinking and it's also going to make it transparent to the teacher if you're trying to just kind of call it in and go down the

path of least resistance. And then this is why teachers, one of the many reasons why teachers and adults and parents are so important because it's like, hey, you're kind of just just trying to coast here. That's not what the use of these tools. So I think transparency is super important. But yes, I think, one, it can amplify creativity because it gives more people access. You know, even when I want to brainstorm, even even on Khan Academy stuff, a lot of my team members aren't available. They're in meetings, they're doing this, they're doing that. But I

really want to brainstorm what can we build? And I have found it to be a good thought partner. And then I'll, you know, there's a and this is a more philosophical like, you know, what is creativity? A lot of people will say these large language models, you know, they're trained off of this huge amounts of content and they're predicting the next word. And, you know, and now it's not just about language. They can predict the next image or the next pixel and things like that. But at the end of the day, you know, creativity is

we are creating what we think are our ideas. We have didn't come out of the vacuum. They were they came out of things that we've all been exposed to. And then the most creative people, whether you think of an Albert Einstein or a great artist, they were able to connect dots with things that they had already been exposed to in ways that people didn't before. And I suspect that these the eyes will will be able to do this. There's there's already some research. You you know, there's a debate happening whether A.I. is just going to

kind of asymptote to what's already been done or will in some ways be able to surpass. But there's already people using A.I. to start proving, you know, going through the frontiers in mathematics and physics, which which implies that there's real creativity happening. Here's the one where the elemental one I think there where I'm not there is you wrote and Bill Gates wrote approvingly about the AI approach to the blank paper or the blank page problem. I just don't know how to get started. So it's not that I'm going to cheat, but I'm going to ask

the AI to to to get started for me. And then I'm going to jump in and I think if we if we abandon the ability to stare terrified at the blank page and overcome that terror, which you're saying the A.I. can help you overcome that terror. But I think we're missing something. Well, I think it depends what you're hoping to have happen. I 100% agree. If if for my own children, for anyone else's children, I want them to face that blank page. I want them to be able to put on paper. And so what we've

told teachers and we're also building for this is you do need to do more in class writing now or in class, anything in class design image generation, so that you can see kids do things from scratch and write grammatically, etc.. Now, at the other extreme, if what you're trying to and this is maybe more for high school or college students, if the goal isn't the grammar, it isn't the sentence constructs construction. If it's the research you did, you went and interviewed people. Yeah. Then maybe, you know, you can take your interview notes and then it

could help you be more productive, which is important too, because in the workplace, those who can produce a great thing faster or of it are going to be are going to have an advantage. And then there's a middle ground. And this is what we're building at CONMIGO, which is, yes, you do want students to be able to work independently on something, but it's truly their own work. So we are building it so that the student can work with the I the axis of writing coach. And when the student ready to submit what the teacher gets

isn't just the final output, it gets the whole narrative with the AI. So the AI says, Hey, I worked on this essay for four and a half hours with Adam. He had trouble with the thesis statement, by the way. Here's the whole transcript. If you went to chat GPT or if you went to your older sister, or if you went to one of these many things that existed before A.I., were there happy that, you know, write your essays for you online and if you copy and paste that in our I will tell the teacher, I

don't know where this essay came from or this is not consistent with the work that Adam's been doing in class. So here you have a situation where I will actually undermine not just A.I. cheating, but all the types of cheating that existed well before, I it, you know, I write about there's a whole chapter on cheating in the book. And I point out cheating was rampant. Well, before I if anything, I just put a spotlight on or in some ways made it a little bit more egalitarian in terms of the cheating. Okay. I'm going to

I'm going to go to your questions and please keep sending them up, if you like. Is there a danger that most of the benefits of AI in education become sequestered in elitist institutions and households? How might this influence inequality? So that is, I think, a very legitimate danger. And, you know, it's interesting because you'll you'll hear questions, one of fears of AI in education. And then you have a question like this, which I agree with, is like, well, what if only the elite are able to leverage it? They're able to leverage it so that their

kids can learn faster. And I can tell you in my own household, you know, our lab school, con lab school, you know, down in the South Bay where my kids go are world school. The kids are learning a lot more, not just because of the AI, they're getting supports and other things as well. But but but they're off to the races. They're they're able to go on kind of the, you know, you know, we all see the cartoon like the magic School bus and all of that. They're having experiences akin akin to that. And so

the reason I tell this, the team at Khan Academy all the time, the reason why it's important for us as a not for profit with the social mission to be in this game is so that we can make sure that that does not happen so that all students that we are going to work very hard to make this as equitable as possible. Because if we if we if we come out of the mix, you're either going to have people who just focus on the elite and are going to try to just charge people who can

afford it. And and then you're going to have people make maybe cheaper approximations that for for those who who can't afford it. And then that goes. But we try to do both. We want to be world class. We want to be the best, even for folks. You know, one of the reasons why Khan Academy really got on folks radar back in 2009, 2010 is Bill Gates famously said, I am using it with my children and it was a huge signal to the world that he can afford. Anything. Anything. But he still chose to use this

free tool, this this tool that is also free for kids in villages in India or kids in the inner city in San Francisco. And so we want that same reality to happen with with with these even more powerful tools. Great. The next question, I am a high school teacher, and by the way, this high school teacher prints perfectly. You win the award for penmanship. And I teach students how to how to get prepared for AP tests. In the tests, students, students have to write essays and analyze data. But if the students constantly use A.I. to

do their homework or practice their essay, how can they be ready to be successful in the test? How can I be useful here? Yeah, well, this is exactly what I'm excited about. The the A historically a student doing open ended things and getting feedback on it. It was a very labor intensive or expensive process. You know, any teachers in the room, whether it's a lab report or an essay to grade those papers to give feedback and there's amazing teachers doing it, you know, doing an amazing job. But imagine I just met a seventh grade teacher

who has six class periods, 30 kids in each class period. She assigns an essay on, say, The Great Gatsby, 180 papers to grade over. If you told me that all I had to do over the next week and had no other work was grade 180 papers on The Great Gatsby by seventh graders, I would tell you, No, thank you. I will not be able to do that. That is very hard. So one is what's amazing is large language models and related technologies are are able to process student open ended responses. And I don't think and

I'm very clear about this book, I don't think it should be the final grader, but I think it can help teachers and rate the students all along a rubric, give evidence why the students are doing this on that rubric. ET cetera, etc.. So I think we can actually get more free response support for students in the classroom and give once again more transparency to these teachers so that they know it's not being done by their by their, you know, fraternity brother or by, you know, that they're not cutting corners somehow. But functionally, how does this

work? It has to be submitted as as a digital tech straight to text written on a in a computer. So right now on Conmigo, that is the case. But the models and we shouldn't call them large lag. I mean, they are large language models, but now people there are multimodal models so they can take imagery and their ability to parse handwriting now is phenomenal in certain cases, better than better than yours because better. Than mine, better than the low bar. But, you know, there's a there's a demo that's gone somewhat viral of myself and my

site, like open. I invited us down to their offices a couple of weeks ago before they released their most recent model. And this is one where it is capturing real time video. And in that demo, my son is doing a trigonometry problem and with lighting on, you know, and I give my son a lot of credit because he knows trigonometry, but he had to pretend like he doesn't know what a hypotenuse is. And so all of his friends have been dragging on him. So I felt the need to defend him right now. But sweet. But

but when you see that there was no typing involved, it was talking naturally. And it was. And it was writing. Yeah. So I actually think there's one of the things that we are working on. We have a, we have a project on like the future of assessment and think about what's happened in assessment, especially standardized assessment over the last 30 or 40 years. You want to standardize it, you want to make it scalable so that you can give it to millions of kids. You essentially eliminated open ended things. You've eliminated writing, you've eliminated free response,

you've eliminated those blue books that we all remember growing up. And so it's not a surprise that when you even go to some of the top universities in the country, professors will say, your kids know how to write anymore because they're because that's not measured anymore. We're now in a world where I see and once again, this isn't conjecture. We're literally building this in the next year or two, you're going to see standardized assessments where the student work, whether it's math and they're showing their work, or it could even be them drawing a diagram, designing

something in science. It could be them writing out something in the humanities or labeling a map or drawing a map. It can actually be processed at scale. So now you can and actually you can then give those insights to the teacher, because historically, if you're a parent or teacher, you said, Oh my, the child's in the 52nd percentile. You know what? What do you do with that information? But now you could say they're in the 52nd percentile. And here's an example where they really struggled with the distributive property and that here's their work. And so

I think that will enrich the experience, not not kind of turn it into something superficial. You know, it's really interesting, too, because it's a far more sophisticated version of the the you know, I think the thing we heard about ad nauseum for ten or 15 years of helping radiologists read, you know, X-rays or other kinds of scans. Right. And that's great and all. But it was the only thing that that anyone would talk about. Someone wants to know, did you have a favorite teacher or professor? And if so, why did you like them so much?

Oh, yeah. I mean many, and I'll go through it fast. I'm probably going to miss some folks, but you know, I remember a second grade. Ms.. Ms.. Roussel And this was kind of a I guess for like we were gifted enrichment program. And I remember when I went in, she's like, Well, what do you care about? Like, what are you into? And I remember I was seven years old and I remember this I don't remember much from, you know, whatever in 1980, 83. And and I said, Oh, I like puzzles, I like drawing, I like.

And she's like, okay, we'll do puzzles, will draw, will do. And I was afraid to tell other people about this enrichment program. I thought it was some racket that was going on in the school. But that and there were just some kids drawing and there's some other kids playing. We're in the world is Carmen, San Diego and some other kids were, you know, doing these mind but bender puzzles but that experience over and I had that teacher for for several years. Mr. So that was a powerful experience for me. I mean, when I think about

creativity and when I think about discovering passions and having that space, and why did I, why did I love because it it didn't feel like I was getting it felt a lot closer to Aristotle and Alexander the Great than kind of being in a factory model and being a, you know, Yeah, in many times if you're not if you're kind of in the middle someplace or you're not going to get a lot of attention, you know, there were other classes where I don't think the teacher really knew who who I was. You fast forward to

fifth grade. I remember Miss Alice. You would run her fifth grade social studies class like a like a like a humanities seminar at graduation. I remember she would peel an orange and she would just keep asking questions. She would never answer questions. So why do you think this? Why do you think fifth grade and fifth grade? And that still informs me. Even then, when I think about Conmigo, I'm like, Conmigo. And you should be a little bit like Ms.. Ellis. You should just keep asking questions. All right. Well, let's let Those two have all the

glory. Okay? Okay. I have more. I have. More. I know Hernandez Miss Kennedy, Dr. Santana does more. Clearly. We're were. Fortunate. A fortunate student was. Yeah. You are in charge of redesigning the US's educational system from scratch. What would it look like? Oh, so great question. So I will start with things that are, I would say not technology related. The first thing I would do is move to what I would call a competency based system. So this is a background. Our system today is a time based system. Like even if you look, you know, most

states do this in California, you have the ADA requirements. If you want to go to the University of California system, they say you need to take at least three years of math. You need to take two or three years of foreign language, four years of English. They're not saying that you need to master this level of math. Yeah, you need to. And the reality is, in the time based system that we have, you know, most students in America take and I'll focus on math, but the same thing is happening in the humanities. They take classes

called Algebra one, Algebra two, geometry. Almost all students take those classes and then many take a trigonometry calculus 60 to 70% of kids who go to college. So it's about half kids. It's about half of all students who even end up going to college. But of that with caught the top half, it's not necessarily the top half, but of that half of students, 60 to 70% when they go to college, colleges give them a placement exam. And that placement exam for 60 to 70% of them says you are not ready to learn algebra yet. So

the system has students with these gaps, the same gaps that I saw with my cousins. And we see throughout the system, they didn't know exponents well, they didn't know decimals well. They're struggling with that algebra equation. But hey, I just got to keep moving you along, keep moving you along. And what happens is you just keep watering it down, keep it seat time based. And the reality is you sat in the math class for so many years, but how many kids actually learned algebra? And I would make an argument it's much more important to master

basic algebra than to have seen trigonometry and remember that it involves some words, but not really knowing what it is or how to apply it. So number one, I would move to a competency based system and then I would and then I would say, well, you know, nationally, internationally, anyone who reaches this level of competency, we will give you the equivalent of a high school diploma, a college diploma, etc.. Then I would work with industry and graduate schools to say this is what you care about. And then I would get the education system, you know,

universities would and schools would then be about preparing people for these like and it would be very transparent no matter how fancy or old a university is. If your students aren't reaching that level of competency, who cares what you're doing? Well, if someone else can figure out an innovative way to to get students that level of competency, then that that could be so. So that's the first thing I would do. And then the other thing I would do is I would kind of create and obviously this is what we're working on, on Khan Academy is

a you know, what we say is we want to raise the ceiling. If you're in a class of 2530, you have an amazing teacher. We want to make it more personalized, but we also want to raise the floor. So if you are in rural Alaska, you don't have access to a great calculus class. You don't have to a great English lit class. We think the technology now exists that we can we can give everyone at least access to that. So I would do the combination competency based system, make those credentials matter, and then I would

give them a safety net system that and then the the traditional schools could also use that technology to just make sure that they're able to deliver. Your competency based idea strikes me as somewhat similar to the health care professionals who say, you know, that we shouldn't be treating symptoms, we should be treating people's overall health, and we know that's not going to happen. And a skill 0 to 10, How likely is it, do you think, that your competency based dream will be realized in your lifetime? Let's say? You know, what's interesting is it kind of

is happening behind the scenes. It's just that the the traditional degrees aren't fully recognizing if if you want to be, I'll pick up you know let's take software engineering now there's such a shortage of software engineers. Some of these big tech companies, if you go to so I'll say two cases, they're very young people graduating from some of the top universities in the country with computer science, software engineering, electrical engineering degrees, but they don't have the skills to actually be productive in the workplace. And then there's other folks who are taking these these competency based

sometimes are called bootcamps, but you're learning some of these skills. It's variable on how long it takes you. Some people, it might take you six months, some people it might take you two years. But once you get there, those students are getting the best jobs. Western Governors University, a lot of folks don't know about it, depending how you account for it, either the largest or second largest university in the country, it's competency based. They are. So, you know, I ask people, where do you think Apple hires the most engineers from Amazon, hires the most engineers

from, and most people say, oh, at MIT, Stanford, you know, they come to Western Governors University competency based and the average time to get a four year degree in computer science is 2.7 years because. Where is that. It's online. I see. It's online. Yeah. It was started by ten Western governors about 30 years ago. Interesting, because they wanted and so it's primarily been for career changers for people in their thirties and forties who want to. But think about if you take that type of a model and then you, you know, you could do it for

use different use cases, you pair it with an I'm a big fan of in person I'm a big you know, my wife's here. I met her at college. I love the quad, I love throwing the Frisbees. I love the late night conversations about philosophy. I think there's a world where. You kept that clean. Your list of things that you loved about college. I had a I had a very great experience, actually. If I, you know, I sometimes regret that. But the but but, but the but but, you know, what I say is use technology for

what technology could be good for. But when human beings get together in a room or on a campus optimized for that. Here's a speaking of of higher learning, here's a loaded question Do you think I applied in academia will restore diversity of viewpoint? I I'll give you an example that I think makes me optimistic. I guess that's the theme here. I'm optimistic I'm guilty we had a reporter you know, we have an activity on Conmigo called Tutor Me in the Humanities. We had a reporter come to our offices super skeptical about Will and I be

able to maintain some level of balance on politically charged issues. I said, don't take my word for it. Try it out with Conmigo. We've taken great care here. So the reporter took on the persona. Maybe it's the reporter's point of view of someone who's very pro-gun control, goes on to Conmigo and says, I can't believe there are people who still think the Second Amendment makes sense, guns are killing people, etc., etc., etc. made a very strong argument. Conmigo Now, I suspect if you go in 50% of American classrooms, you'll have a teacher will say, yes,

you're absolutely right. I don't I don't get what the rest of the country is thinking and the other half of the country. How dare you say that This is a fundamental right. What Conmigo did is, I think what is the right answer, which is it? Asked the reporter before we even get into now, why do you think the Second Amendment was originally placed in the Constitution? So the reporter was taken aback. It's like, oh, no one's asking. I is asking my opinion. Okay. So The reporter's like, Oh, well, it was right after the Revolutionary War.

We didn't have, you know, the British weren't allowing us to have arms. And then the reporter said, you know, but but that was a different time. And now it doesn't make sense anymore. Arms are no longer muskets. They're, you know, somehow and can be served before we get there. Your understanding of the historical context is pretty good. Why do you think it's persisted? So once again, it's pushing the students critical thinking. It's not trying to enforce or rebut their point of view. And that's what we need more of in schools. Here's a related question. As

a history teacher, I've been showing my students how to use I as an aide in analyzing primary sources. We've made growth and experiencing the discomfort of the unknown. But how do we teach students how to prompt AI when they are unsure how to ask for what they need? This is a really interesting question because I'll tell you very transparently, one of the problems we have seen was we've rolled out Conmigo into classrooms. We are seeing a lot of students not know how to ask a question. And, you know, this question is more of like a

more sophisticated how do you analyze primary sources. But yeah, we're seeing students who are I remember I visited a classroom in the early it's not that long ago. It's about a year ago. And it was it was I think it was a fourth grade classroom. And the students, when they were faced with a chat, they would write stuff like what? Or ID K or Ha. And and I told the teacher I was like, You know what? Okay, I'm sorry. This is probably not age appropriate for. And the teacher says, No, no, no, no. This is

so important for these students because they do that to me. They, they raise their hand and they don't know how to they've lost the ability to articulate what they want and just think about it. I mean, you know, I just talked about learning literature and learning, coding and learning. But what if you go through life and you don't know how to articulate what you need? And what the teacher said is this is giving those students practice and they don't have to feel embarrassed of raising their hand in front of the whole class. Just computer telling.

Them it's a it's a and the teacher can see transparently that there and then the teacher can get feedback and then the I can say, hey, can you can you can you explain a little bit more? Why are you saying what? Why are you saying that? Tell me what part you don't understand the I can also I mean, I don't want my daughter in her conmigo account. Has Conmigo talked to her in Gen-z slang? Interesting. I thought that was my daughter being silly, but it actually she responds better to. And are you. Are you saying

are you saying straight out are you confident that the that the eye will not roll its eyes or condescend to the to the child? Well, you know. Are we already there or is that the hope? Well, a lot of the stuff that we're not talking about is we do. And I think most people who are dealing with AI, especially these things that can be very open ended in the software world, you call it Red Hat testing, where you you act as know, you try to make the AI do the worse imaginable things. You try to

make the AI put it in the hardest possible scenarios and make sure that it responds okay. And I won't say that it is perfect. I mean, we have definitely discovered some, some indications of potential bias, etc., etc.. But in terms of context tending or we have not we have not seen that because we're prompting it. You are empathetic, right? You are you you are listeners. You are like I mean, these are the things that and and it does take on that type of persona. If open a I approached you to be the voice of GPT

four would you why or why not? Or have they. You know that Scarlett Johanson said, Oh, all of us wanted that a little bit, but I probably b I don't want to think about it a little bit, but I think there is something, you know, I part of. I'm. A little worried worried about. I will tell you, I won't I won't pretend that I'm completely above vanity. You know what part of the joy of creating Khan Academy videos over the years that I've made, like, you know, 7000 videos is, you know, the first videos were

before I had children. And, you know, my wife knows I used to say, like, imagine one day our kids could be learning physics from me. Our grandkids could you know, this was even before, you know, all of your kids might have been using it. And so there was something I thought very exciting about that. You know, we all I mean, most of us, I think, think about a legacy and I was like, well, that could be a really fun legacy. You know, maybe if one day long after I'm gone, if folks could still be learning,

you know, most of this content is pretty evergreen. If people could still be learning, you know, Newton's laws for me in the year 2300, how cool would that be? So it sounds like you want the gig. It would be interesting. It would be interesting. I want to say it's really maybe this maybe this doesn't surprise you at all, but there are a lot of teachers in the room tonight. Yeah. Oh, a lot. Yeah. They came. They came to see you. This person asks. Google democratized information. Jenny. Jenny democratizes knowledge. So is the role of education

to teach values and ethics? And will apprenticeships become more important? This is what I believe. What do you think? The person writes? Yeah, I I've I've been a something of a traditionalist. Even when the Internet came out and Google came out, a lot of people would even at the calculator, people said, Oh, kids don't need to know arithmetic anymore. I strongly disagree. You need to be able to walk into the supermarket. It's just 30% off. You need be able to make sense of that. You need to be able to, you know, calculate bills you need.

And and that fluency isn't even it. You know, maybe if there's a calculator and you could do a little bit faster but that fluency helps you form connections even subconsciously when you're not around. Same thing people said, Oh, Google exists. You don't need to know content knowledge anymore. I'm a traditionalist, you know, I've started some schools and in progressive progressive education, I just did air quotes for those listening. Like, you know, it's become very in vogue to say, Oh, it's more about it's more about the hands on or the constructive constructivist learning. It's less about

the content knowledge. I'm a big believer that it is both that you need to have that content knowledge in order to make connections. And so similarly, in an AI world, the people who are going to be amplifying their intent the most are not the people who don't have the content knowledge or don't have the skills that the eye can do, or they're going to be the people who can hang with the AI, who know enough to know how to steer the AI, who know how to put the pieces of the AI in place. So I

tell everyone it's even more imperative that students have their written and oral communication, that they have strong, critical thinking, they have strong math fluency, they have strong content knowledge and broad content knowledge. Of course, you know, the philosophical, the ethical portion of it. And then also how do you leverage these tools to amplify what you what you want to do in the world? This person asks, How do you imagine conmigo scaling across the public? Public higher education sector? For example, the 116 California community colleges in California serve traditional transfer students, workforce, career education, and in

nontraditional aged adult learners, And this person helpfully included their email address. So I'll give this to you afterwards. You know, it's fascinating is, you know, I tell the story about when you start talking to open air back in summer 2022 and we we signed all these non-disclosure agreements and we started working on Conmigo trying to put all these guardrails in place, make it safe, make it pedagogically positive. And then in November, chat came out and I you know, I slack the openly I leadership team. I'm like, what's going on? You have us all lawyered up

and you just lost something. And they said we just put a chat interface on an older model and the whole world kind of exploded. And I was really scared because it was very clear it was a cheating. It could be a cheating tool. It was not built for education. There were no guardrails put on it. And rightfully a lot of school districts and colleges started just banning chat. GPP And so I was worried. I was like, Oh, we're really working on it's very it could be a very positive technology that people are going to throw

out the baby with the bathwater. The good news is, by the time that we launched in March of 2023, most I actually think almost all educators I've met very few who are now like, Oh, I want nothing to do with AI. I think they every educator, we we talk to every school leader, every university president, every professor saying, no, this is going to be part of kids futures. They need to know how to use this. But we have a real problem. How do we expose them to the technology? How do we make sure it has

a positive benefit on their learning without it undermining their learning? But in some ways, that is going to drive, I think, faster adoption than otherwise, because even Khan Academy in the past, it was always an additive thing. It wasn't an emergency that you had to use Khan Academy, but now, especially in higher ed and high schools are like, Well, kids are cheating on their essays. What do we do? Can you give us a tool that has more guardrails that can undermine the other forms of cheating? So we're seeing you know, we started with a few

tens of thousands a year ago. We're now at about 100,000 real teachers and students in classrooms. This coming back to school, we're going to be approaching about a million. These are in US public schools. We're going to have several hundred thousand in Brazil. We're looking at ways to have several hundreds of thousands to millions in India soon. And this is working with the school systems, with the ministries of education. I got it recently. A country is thinking, I mean, that former question about what would you do if you had to architect the education system. I,

in a strange way, is is making countries even start to think about some of these questions in a more holistic way. We're at the point in the program where we only have time for one more question. It's going to be mind, How are you thinking? It's my prerogative, Rich, How are you thinking about intellectual property rights? And maybe, you know, quickly, how does an academy already deal with intellectual property rights? And The New York Times is suing Openai because it believes that open air is is stealing its intellectual property, along with a lot of other

people's intellectual property. Yeah. So this is a very interesting philosophical and on the kind of academy side, we have always said, hey, our content is there for if it can make the world better. And we've historically said, hey, it's noncommercial if someone wants to use it and there are some people who have used it as pseudo commercial, but we're like, Hey, if you're using it as a positive end, but you. Do pay to create it. I meant on the creation side. The Oh yeah, well we create all of our own content or we work with

partners to create, to create that content. Now, a really interesting question. I know there's this lawsuit about an artist you could ask open A.I. or you could ask Judge, but can you create an image in the style of can you write an article, the style of Adam Lashinsky And it will be informed because it would have trained on your articles or on that artist's paintings. But it's an interesting philosophical debate because it's not like these eyes somehow have stored your articles in them or somehow have, you know, scraped the picture of this artist. It these

models are modeled after the human brain. The human brain. You have 100 billion on the order of a hundred billion neurons. You have you know, we're talking trillions of synapses that connect it. They have different strengths depending on what that brain, depending on what we are exposed to. This is what's happening in these models. So as it gets exposed to your op ed or it gets exposed to this artist painting these the synaptic, the the model, I mean, these are numbers, really, but they represent synaptic connections between neurons are changing. So it's not like you

can dig into that A.I. and you can see that. And so the courts are going to decide whether whether. But I will say something very quick, because people are are are kind of throwing stones on both directions. On one thing, they're saying, hey, I don't want my stuff to be used to train the model. But they're also saying, hey, is this model being trained on on a representative sample of of humanity? Right. And or are and I'll give another example of imagine if it was 1990 before most of us saw the, you know, the Internet

and etc., and you said, hey, there's going to be the thing that is going to that you're going to be able to create content and publish it to anyone. But there's going to be these multibillion dollar, $2 trillion companies that are going to scrape that content and they're going to make trillions. They're going to make billions, hundreds of billions of dollars by surfacing your content to users when they want to find it. They're also going to feed some ads. You'd be like, Oh, that's horrible. But what actually, I still. Think it's. Horrible. Well, no, you

know, now you're paying people so that Google finds your content. You are paying, you're trying to search engine optimize so that Google finds your stuff. Yes, I understand. So I, I don't know how it's all going to play out, but I do think people should should really think about what they want. Do they do they as more and more people go to and once again, it is the genie is out of the bottle as more people are going to go using chat GPT or using copilot on Microsoft or using Gemini and Google to make sense

of the world. Do you want your content somehow represented or do you not want it represented? Is a is an interesting question. Well, on that note, I think everyone would agree with me that no way I could be as engaging or as articulate as Sal Khan. Oh, we we encourage everyone to purchase a copy of sales book either in the lobby here or at your local bookstore. If you want to continue to support the Commonwealth Club's efforts in making virtual and in-person programing possible, please visit Commonwealth Club dot org. And even though I don't have

a gavel, this is the end of our evening. Thank you very much and take care. Thank you. Thanks. Out of of.