Brave New Words - Bill Gates & Sal Khan

hi everyone s here from Khan Academy and as some of you all know I have released my second book Brave New words about the future of AI and education and work it's available wherever you might buy your books but as part of the research for that book I did some interviews with some Fascinating People which you are about to watch I'm very excited to introduce you to Our Guest who probably needs no introduction Bill Gates Bill thanks for joining us great to see you so I know AI is very front of mind for you uh

obviously all of us at Khan Academy have been thinking a lot about AI in education and you know the the focus of this is AI in education and work and I'll start with a question that I'm getting from a lot of teenagers and folks in college and their parents is given what you know about what's happening in artificial intelligence and maybe in other Technologies well like what advice would you give to someone who's trying to figure out what skills should they focus on or not focus on what should they major in what are the jobs

going to look like or not look like well the AI uh will make people more productive and you know in some cases like writing software you know there's a lot of unmet demand for good software you can make it higher quality software exactly what the job market looks like is very hard to predict but the deeper skill set you know whe whether it's medical consultation or uh scientific thinking or customer support uh you know the more value that's going to have even in a world where your productivity will be enhanced by having this sort of

almost free uh worker you know is looking over your shoulder giving you advice uh giving giving you lots of feedback on what it is that you're doing so I don't think there's anything like you know don't become a barber or a radiologist that we have in mind because this the new capabilities are very horizontal in nature it's not a a topic specific uh set of advances so it's it's kind of like everyone's going to have a an intern or a research assistant or an analyst so maybe the answer is don't be the research assistant or

the intern or the analyst learn to manage them yeah and and of course jobs like uh you know helping handicapped elderly uh helping to motivate students uh you know there's such an infinite demand for those things even in the rich world but even more dramatically so in developing countries that you know jobs that require deep human connection and empathy uh you know we'd say it's a long ways off uh before there's a a substitute for those things you know we have two revolutions kind of in parallel we have this uh chat BT enabled sort of

reading and writing White Collar enhancement but in the meantime people shouldn't underestimate that the blue collar sort of Robotics eventually leading to a humanoid robot that's quite general purpose in terms of the physical skills arms legs uh the things that it can do that's also happening and so the good news is that we're going to have more Leisure Time uh that a lot of the shortages that we have on medical advice or when we get confused being able to be straightened out you know we have this supply of these new capabilities and you know the

unique roles that humans take on or you know how they'll take their extra spare time and and make that fulfilling uh are our challenges you know what you just touched on is interesting because I think people have always theorized that hey we're going to get more productive we're going to be able to do more with less time and so we'll have more Leisure which is you know I think utopian for a lot of folks but it looks like the patterns of the past of productivity improvements are if anything that the the hyperproductive are working harder

that they're they're they they're because they get so much leverage from from every minute um that it's not necessarily leading to that do you think something's different about this wave where maybe things like the four-day or maybe even eventually a three-day work week will will become common or do you think people are just going to be able to do more with less and hire fewer people than they do today the analogies from the past are worth understanding but the changes in the past were generational in nature you know so the mechanization of the primary occupation

which was growing food that happens gradually uh and so you know Grandpa was a farmer and you know your parents less so and then you know now I don't know many Rich World Farmers I know a few but not many uh and so you didn't have to change midcareer and you know the it is true that demand for things uh that we wouldn't have expected emerged and so it's comforting to look at that history but the total level of capability here and the speed at which it's going to suffuse into the economy and change job

markets you know five years the effect will be very substantial I'd say in 10 years it's going to be quite dramatic and so I don't think we can completely dismiss the idea of uh uh this change will uh finally get us to the point where the tradeoff between labor and Leisure uh actually becomes important and you know instead of taxing labor like with payroll taxes uh you know I called it a robot tax when I talked about it and people thought it was strange but we'll shift to where we essentially subsidize labor and tax you

know completely on the the capital and and profit side which is a that alone is a pretty big shift that government will need to make it's a fascinating idea it makes a ton of sense because obviously if we have a more abundant Society more productive Society but you know let's take an investment bank or consulting firm a lot of the traditional work that people are building and the size of the organization are analysts doing spreadsheets and making PowerPoint presentations and a lot of that stuff's going to get automated very fast and so this is a

world where you subsidize these firms to say no keep hiring interns keep hiring these Junior analysts um we'll Subs so have a sense of purpose as opposed to these young people not having something to do not building their skill set and never being able to become the the manager yeah if you get to the extreme where you know the purpose that jobs provide uh isn't the central organizing principal Society then you get into you know some deep philosophical questions about you know what should priorities and what what is the purpose in an abstruse way you

know the Boston Book superintelligence touched on that I I don't really think it's that's been written about all that well um you know life 3.0 did a good job of saying okay what is the role of the intelligence in these futuristic scenarios lot of new books coming out right now I haven't uh been able to catch up to them all uh probably some of those will be will be helpful I'm I'm interviewing you right now for one of them perfect this and and um just going back to I guess more uh uh Basics so to

speak obviously a lot of the folk we talk a little bit about work but now going back to the core of Education which I know you care deeply about we've chatted many times about education right now there's a bit of an emergency going on in education people like kids are going to cheat using this technology how what what do you have an answer for and we're hearing this from every you know faculty across grade levels across what would you do if you were a a a history Professor or a writing professor at a high school

or a or college right now well to have a pure test of somebody's ability to write on their own or calculate on their own you'll need to create a an environment where they they don't have access to the software tools our efforts to detect whether software has helped you write something uh you know we'll catch the sloppiest you know sort of the equivalent of cotton paste from Wikipedia but in the end uh we won't be able to tell because there'd be such a variety of tools and the variety of prompts uh will make that quite

difficult so the good news here is that for the first time if you write on a topic the AI can give you feedback uh it can help you with the first draft it can uh help compare what you've done to to something else it can create a program of study of things that you might try to write about and so the level of Engagement where it can uh help you whenever you're stuck really is something new it goes even beyond the fact that there's great YouTube videos about all the sciences and history this gets into

the realm of Uh custom to tutors and feedback on reading and writing so the Paradox that now we have this tool that if you want to learn it makes it easier but testing somebody's uh capability we're going to have to use new techniques and sadly in some cases it might make people feel like wow wow if it's so good uh you know do I still need to push myself uh as hard we're you know you and I are seing both for personal fulfillment and to play uh a role in contribution in a in a job

market the higher level your skills are you know the more uh it it will retain a substantial value so the you know helping kids learn how to uh get feedback on their writing um I think that's imperative because we we do a very poor job of that today so what I'm hearing is is that not only is there not less of a reason there's more of a reason to accelerate part the earlier part of our conversation which is those intern level entry level things are quickly going to expect you to be more of the architect

more of the editor more of the manager and if you can't make that leap sooner than later the good news is these same tools can be used to do that which I agree with as well and obviously we're working on a lot of that what would you uh so if if there was a profession I mean we get emails from folks maybe you do too of like what do I do this fall like what and it sounds like you would do more in-class Proctor things for assessment and would you still do the traditional term paper

or would you change the rules of engagement on that well yes in the workplace people are going to be encouraged to come up with the best product they can where it's them plus the AI and so there's no reason not to have that be one of the uh skills that you practice is hey you know let's do a business plan and sure you know learn how to use the very best AI tools but you know I will view the if I just have the AI do it without any of your value added you know that's

going to be like a c minus uh are worse uh you have to get in there and you know bring in a little more accuracy different sources of knowledge uh and you know have have unique uh value added I mean you understand the audience that it's going to it's even hard still for the computer to understand all those different context so yes uh you're going to have two kinds of writing uh writing you do completely without the help of the computer and writing you do where you're encouraged to make it be the best assistant possible

no that makes sense you know we talked to Kevin Roose who writes for the New York Times also famously wrote about his interactions with Sydney and all of that uh and his point is there's writing and there's journalism the writing the AIS are getting very good at but the journalism making the contacts being able to interview people form bonds Etc and that there's an analog to everything if you're an entrepreneur doing user testing you know although the AI might might be able to do pieces of that related to this whole practice feedback you know you

you touched on the word assessment I'm curious what you where you think assessment is going obviously for many years scalable assessment was fairly limited Scantron multiple choice that's what we did because we couldn't do richer assessment it started to drive what does or does not happen in the classroom do you think that's about to change well there's a controversy over a assessment uh independent of the AI you know should SAT scores be part of college admissions you know I do think uh proving that your math skills and your you know basic understanding of you know

history and uh science is strong you know that's pretty important and that when you want to consolidate knowledge the fact that you'll be asked very broad questions about it is very motivational to making it accessible not just okay tomorrow I'll know it in a simplistic form rather I know how to access it and combine it with my other knowledge so I'm still somebody who thinks that tests uh can help you know you know where you are in a subject area and you know it's a meritocratic thing that uh you really want to be able to

prove that you have some cap capability certainly when I go to a doctor I'm glad they've been subject to some uh uh examination and and test of their competence now dealing with the fact that if you have a disadvantaged background even if you're on a slope to get really good that looking at you just at that moment in time you know may not maybe shouldn't be directly comparable that's an interesting challenge partly that's why I love the idea of the computer tutor because uh it can be encouraging it can come it can understand what your

misconceptions are uh and it will you know if we can make it available very broadly it means that the sophistication that say my parents you know provided to me as college graduates is less unique uh in a world where the the AI tutor is super helpful in in a huge variety of subjects but you know I'll stick up in many cases you know having a objective assessment assessment of your skills um you know I I think once you move away from that you're in a uh a very difficult place even in terms of your own

discipline of do I understand this topic or not yeah no I I I agree and you know whenever someone says oh I'm against standardized assessment I'm like which part are you against the standardized part or the assessment part and it's usually hard for them to be against either of those but I think what they are implicitly saying is they feel like it's maybe not fully capturing where the student is there's only certain things you can measure maybe it's not capturing their slope to your point it's a snapshot in time do you think that AI in

the next five years 10 years might solve this by being able to give richer assessment and you know assessment could be where you are in math it could be where you are in writing it could even be things like job interviews that's a form of assessment uh career progression do you think it's going to play a big role there well I think given that we still have certain imperfections in the AI uh people can still be skeptical when we say that a interactive assessment with the AI discussing a topic like your understanding of history or

economics that that we can make that equivalent to the very best sort of uh Professor who would engage you deeply and and I think we will get to that level and that is superior to just a set of standardized questions uh that you know have to have sort of a a straightforward answer and you know your your richness of understanding the topic uh and there is you know there are people who are better at multiple choice type questions and test things you know I I think you and I probably would have done well educationally but

I think partly we are good at tests and and I felt bad for kids who you know the test I could tell they were intimidated you know when that you know multiple choice you have to answer all of these in an hour would come out so by accommodating a more flexible dialogue I think different styles of learning uh will will be appreciated different a broader set of skills will be appreciated but we still have to prove that uh to people we're not we're not at that level of matching a a great uh Professor discussing a

topic with you yet and then being able to actually give a good assessment of it exactly we've seen we can emulate the actual discussion part quite good already but that the strong assessment I think there's but but probably going to happen my guess yes it's going to happen I mean it's it when you have experts who are good at something that's linguistic in nature and as you gather those data sets and constantly give feedback to them you know even with today's mechanisms which uh can be improved but even just with this we can develop uh

something like a human expert and I mean with that it's funny I mean this conversation as you know even a year ago would have all seemed like science fiction and this is all uh changing so quickly I I know I feel this way and I feel like I'm in the middle of it but still things are changing so fast you also are probably seeing even more I want to ask you a long-term question because who knows what the world's going to be like in even 20 years but let's say five years which is I think

what most people should be kind of planning for in their lives whether they run a company whether they're a young person for their career what do you think the world of education and work is going to look like like paint a picture are we are we going to be able to have a video conference with I mean I I don't want to Cloud your what would it look like how's it going to be dramatically different or not than than the world we're in right now well there'll be a certain set of jobs where you're just

moving documents around and looking at documents say you know accounting or medical claims where the depth of understanding in those jobs is pretty straightforward and some of those jobs will just be purely automated then there'll be a ton of jobs where they're still being done by humans uh you know like making greeting cards um and I went back and watched the movie her uh because it raises uh in a pretty I thought greeting cards were definitely going to be automated because now you can have mass customization of greeting cards well anyway yeah you you probably

can do it by yourself uh to you know take you know Dolly 10 uh and say Hey you know here's how I want to make fun of my sister remind her about this thing from the past and here's some photos to stick in that so yeah you might you might eliminate the professional there because it's very uh you and the AI can do it in a very bespoke way but there'll be a lot of jobs left that are just more productive and that you know where you have this kind of co-pilot that's taking the you

know okay convert the PowerPoint to a Word document and boom you know that draft is available for you to uh have as as the starting point and you know so that extra productivity will create you know new opportunity to take things like Medical Care and make it better education and make it better care of the elderly and and make it better uh you know productivity is a good thing and people have been boning and there's many books about oh you know we had electricity and then since then everything hasn't been quite at least the way

it reflects into economic figures as dramatic as what happened there this you will see in the economic figures in a big way both the robotic side of it and the uh llm reading and writing uh part of it as well so in five years we will uh be in a different space you know hopefully we won't you know elections will still be uh uncontested and you know deep fak won't won't make the problems we already have there even more difficult uh it is interesting that government at a time when you know it's not as trusted

as we'd like will have to step forward and change tax policies and have retraining policies and you know be part of the dialogue of setting rules you know the entire industry is saying you know there's a role for government here but uh you know we have at at a time where trust in that and the divisiveness is is very challenging so I hope uh you know that we haven't suffered that much from the negatives in the The Five-Year time period And by then uh you know Society will have reoriented itself and say okay how do

we avoid this being used as a an attack vehicle how do we make sure it's used in an equitable way you know certainly the dreams I have about you know health and education poor countries uh you know I think it'll take us five years to kind of get it out and get into widespread use five years is a very short uh period of time but it lays a foundation where I hope by being more productive as a society many of the challenges of Aging or climate uh or you know people expecting government to do more

than it's capable of doing that will say wow you know we're making obesity drugs better better we're making Alzheimer's drugs better and you'll get this sort of positive view of the current system and making not making it work better in an incremental way versus uh the divisiveness or you know even considering radical Alternatives that uh I think would be pretty scary so Innovation here because it will surprise people it's not expected does have a lot of positive upside yeah that leads to the last question which is how are you feeling you know I my view

on AI changes almost daily as I read more and I think more and have conversations like this are you feeling net positive are you really worried about certain things within your positivity are you net negative how how are you feeling about it and what what are the things you're thinking most about you know I'm thrilled that when I you know sit with the malaria team and we talk about uh how we're going to have you know better ways of killing mosquitoes that you know these software things can help with or you know we sit and

talk about kids who've decided they're not good at math that we haven't made much progress uh you know for those kids uh you know just having all the ability to try out their knowledge you know that wasn't enough to draw them in uh and so that that issue of motivation and the fact the AI tutor one of its excellent things will be understanding the right level the right encouragement the to make it somebody feel like yeah this mathematical understanding of the world is something that can work for me and make my life better even Beyond

uh what sort of job opportunities it enables so I I'm very thrilled about that the world that we're delivering it into and its ability to have you know complex factual debates and you know different countries try out different regulations I would say that the political challenge uh that where everyone says government has this big big role to play to take advantage of the positives uh for example changing the tax structure and to mitigate negatives you know deep fakes bias uh AI enabled cyber attacks uh you know are is our our governance structures mature enough agile

enough and can they consensus enough uh that we'll manage this opportunity in a good way and there are definitely days uh that I worry uh we won't the government side of this thing won't step up to what's necessary yeah well I could talk to you for several more hours maybe days about this but Bill thanks for taking your time out for this this is super valuable no it's fun to be on this journey together and uh you know we'll uh be great partner and help you realize your vision great well we appreciate that so much

it is exciting times let's see what happens all right thanks all right thanks Bill