AI Required: Teaching in a New World with Ethan Mollick | ASU GSV 2023

[Music] all right we're going to keep the AI conversation going we are really excited to welcome Ethan Malik next to Stage um Ethan is a professor at the Wharton School of Business at uh UPenn he teaches Innovation and Entrepreneurship and he also leads Wharton interactive which is an immersive approach to education using gaming and simulations and it's all about democratizing access to business education so we're really thrilled to have him here his work has been covered in Publications like the New York Times and CNN and many others and he also advises a number of startup

companies and today he'll be talking about ways that AI can help make instructors more effective while saving time and effort and what students learn when AI is mandatory so we're really excited for his talk and without further Ado please welcome Ethan Malik foreign [Music] I'm Ethan Malek I'm a professor at Wharton the business school University of Pennsylvania I also run who yes I also run Ward interactive which is our attempt to build games for business training and simulations I've been doing that for over a decade and I have slipped into AI as a result of

that so doing Interactive Learning for a long time working in AI adjacent spaces and I have been doing a lot of early experimentation and research on the topic so that's what I want to talk with you about today so let's let's start the conversation um so among many things that I've been experienced with AI I've been required in the class I want to talk to you a little about that but I want us to have some ground rules about what I think AI is going to look like in classrooms in the near future and so

the answer is it's going to be undetectable ubiquitous and transformative and all these images by the way I did in mid-journey in three seconds right so um if they all have the right number of fingers now that problem is solved but it's pretty amazing you generate these kind of Graphics instantly right but um undetectable in that we know that you cannot detect AI plagiarism right not not just in the short term it's possible in the long term uh it's not possible there's both theoretical reasons why it's not possible but also I found that when I

prompted my students to actually do a good job writing AI based essays they become undetectable just two or three rounds and it's impossible for any of these systems to find out what they are so you're not going to be able to detect a plagiarism and of course who knows what plagiarism is anymore right is working with the AI to come with an outline that you don't use plagiarism ask it for 30 ideas for your essay and picking the one you like best plagiarism asking to rewrite a paragraph that you didn't like before I've had students

come to me and say English my fourth language and I don't do very well in English and people don't take me seriously as a result and now I just use AI to do all the writing and now I'm taken seriously is that bad is that plagiarism we don't know we have to Grapple with some of these questions in a serious way ubiquitous in that what makes AI so interesting is it the tool available to over a billion people right now and if you're rich you have no better access than if you don't have a lot

of money right the same tools available everywhere you can use gpt4 the latest model free through Bing um hint being the purple creative mode that's what uses GPD for and you can actually use this system right now to do everything into the hands of every teacher everywhere right so everything that you guys are doing building edtech there's now a universal AI in everybody's hands it is ubiquitous not just among students but among teachers not everyone's figured that out yet but they will soon and it's transformative in that we don't quite know what to do with

this right I was on the stage yesterday I was saying the original definition of the singularity you may have heard of the AI Singularity right the idea of a singularity is past that point we can't predict what's going to happen and that's where we are right now if you ask me and I teach at a business school what the future of jobs looks like two to three years out I don't really know I know that we're seeing 20 to 80 performance improvements at white color tasks with AI 20 to 80 percent steam power was 25

I don't know what we do with that information right I don't know whether people do more work than they did before where the nature of jobs changes I can't give you an answer to that but we don't know what's going to happen a lot is going to happen in the near future it's transformative to education too I think that we have a real Advantage here on the education side for those who are Educators that I think we're going to be fine I think that uh what we just saw from The Amazing speech by Sal Khan

is that there are tools that will make education better and more effective we'll figure out there'll still be a need for classrooms at least in the medium term we're fine we'll figure this stuff out but is going to transform things right and if you haven't seen this this is the trend in AI test scores and keep in mind GPD 3.5 that's chat gbtv that was released back at the end of November and gpd4 just came out this the bar exam this is the percentile it scored among all human students right not the percent right it

went from five percent right to scoring 85 percent better than 85 percent of people on the bar exam at the 99th percent of the GRE verbal at the 80th percentile in quantitative just past the neurology neurosurgeon bar exam sorry a qualifying exam a paper just showed a couple days ago this is changing the way we think of testing right if we're testing people on these kind of characteristics the AI is passing that what does that mean again past Singularity I can't give you the answer but it certainly forces us to Grapple with this in some

kind of useful interesting way so things are changing things are changing fundamentally how do we start to deal with that I'm going to propose three levels of adjustment we're going through and I know some of you are Educators some of you from edtech companies I want to kind of talk to all of that so the first thing we can do is we can expect more from students that's the first level so I actually have mandatory AR in my class I want to show you my AI syllabus question and I'll make sure you have access all

this so you don't have to read it all don't worry but I required everyone to use AI for everything okay so I gave them the basic rules it lies to you a lot you're responsible for its content right it it rewards good work if you don't do good effort into putting them together prompts you won't get good outcomes from those prompts right it's a tool and it's one and you have to acknowledge so they have to list all the prompts they use and write a reflection on what worked it with Dent so it completely changed

how the class worked and and I'm lucky enough to teach a business class right so it's about outcomes and doing things if it was a fundamental writing class I think some of the solutions you saw earlier and again we'll solve those blue books working outside of class working with those AI tutors all those things will help us solve these problems but embracing AI is really powerful I one class for example before AI I had the students come with a theoretical product design it's about designing new products and create and test a really early prototype write

an outline of the results with AI in the class it became come with a product design build the product using laser cutters build a working app create a marketing material create Graphics write a full report all of that right was and for people who've never done any of this before right and so here's it was they were building board games in this class here some happy students with the outputs they came up with this was for just a four week long class right like this was not that many sessions together so we can expect more

I don't accept papers that aren't good anymore because the minimum you can do is turn into good paper so why am I going to force you to write a bad one right so we I can increase our standards of what we can accomplish what can we accomplish in a Grade what can we accomplish with students what can people do when they're Unleashed in this way that's the first change that I think we can think about making the second one is to better pedagogy and less work what I mean by this well so um this is

a chart of peer-reviewed articles in pedagogy so the science of teaching and it turns out the last couple decades have been incredibly Rich for us to learn how people learn and those rules are not going to change the way humans learn isn't going to change it turns out though that many of the ways that we are teaching people are not actually well aligned with the science right it turns out like lectures are a very bad approach to teaching which is bad for me as a college professor right and there's a whole bunch of techniques that

don't work very well or don't scale well that are really effective well one of the cool things about the AI is it allows us to more easily Implement these kind of complex pedagogical approaches so this is just a couple examples and I'll put a link at the end you can give I give away the prompts for all these things so and you could just experiment by the way there's no magic in prompt crafting just use the AI enough and you will figure out how to make good prompts there's not anyways trying to sell you courses

don't worry about it it's figure edible right um but this is an example of creating and explanations right explain photosynthesis to elementary school kids using analogies right it's like baking a cake the plant the plant needs specific ingredients sunlight and water and carbon dioxide and a kitchen tool chlorophyll to make a delicious cake glucose or imagine it's a solar powered toy car any level I want I can now create explanations I can create customized information if I said my this kid is interested in becoming a chef growing up it would change all the explanations of

Chef based explanations this is all something you can do right now right you don't need a tool to do this you don't need an edtech technology you can literally go and just say I'd like to explain this concept to my class my class is interested in you know in whatever TV show they want Mr Beast whatever they care about and you can have the explanations in that context it's very easy to do right and simply low stakes testing we all hate it it's the single best approach though to getting people to learn right not just

for assessment but turns out tests do lots of great things for us mentally it increases our ability to recall recall related facts interleaving and it could be annoying to generate all kinds of testing material it does it for you right and with just the right prompt you'll get good testing material even other things like another thing that the pedagogical research shows is really important is interleaving things so if I'm teaching one lesson I need to mention previous lessons where people forget it right that could be a hard thing to do as you move from topic

to topic you can literally just ask the AI I was teaching about the American history last week now I'm teaching about World War One create some questions some connections and questions I can bring up about that does it for you right so this is even without moving to a new world of tutoring your job gets easier it does better work for you and this is leaving aside all this sort of sketchy of things if it grades better than my Tas do but I'm still grading by hand for now but there is this all of this

new kind of technology and you can do all of this right you don't have to wait for someone to provide this to you this is available to you right now right and you can save time and effort as an instructor and do better teaching as a result and like I said I'll give you I've got lists of prompts but you can experiment with whatever you want you guys are all experts and professionals you can build from that and what's great is we know what we need to do to have that sort of personalized instruction piece

we need to give people the right prior knowledge we need to give them appropriate challenges I mean one of the things that I've learned is somebody who's you know using games to teach is it's not you know you can't just give people fun right they unfortunately people have to be challenged to work that's how we we learn through Challenge and so scaling that challenge is really hard you end up with students who are bored and students who are over who are overworked now we can actually personalize that level right give me an exercise for most

advanced students give me an easier exercise all of that's now easily doable adaptive feedback how do we give people information as they go you saw some of that in the Khan Academy and how do we provide more support we can start to do that in a classroom if we shift more the instruction to tools like the ones you saw earlier that lets us as classroom teachers do a better job interacting with the students bringing things together making meaning out of our work rather than just counting on it to be delivering content really exciting sets of

changes right and one that again you guys can Embrace as Educators or as people who are education adjacent because this gives you the ability to do that you just get to experiment you get to be better at your job it's a pretty exciting outcome right you don't have to buy anything to do this which is also kind of amazing right um okay third thing this is what I've been thinking about a lot at this conference right which I think we've entered a completely new phase of Ed Tech and I think some people have figured that

out and other people are about to figure it out pretty soon so let me tell you about that okay so I am an ed Tech I you know I didn't really think of myself in this conference as an edtech person but I guess I'm an ed Tech person because I'm running an edtech startup inside a business school right so we're not really trying to you know I'm not raising Capital I've already done the startup thing I teach startups I don't want to do it again myself um having been there actually I invented the paywall in

the late 90s I still feel bad about that so I'm trying to make it up to all of you um but um but in any case um let's talk about what we're doing so I've been working with uh really amazing team people over the course of 10 years more active in the last five to build these massive games that teach entrepreneurship and business education I think this is super important because it turns out small amounts of business education absolutely transform people's lives we have all these controlled studies that show that I really want to figure

out how do we teach at scale and Wharton I can teach the classroom right but not in this way so we've built this whole gaming platform that does instructional teaching and it's there's a lot of clever stuff I want to show you a little bit how it works right both because I want to brag but also because I it'll be contextually useful later so we've done a lot of intellectual thinking about how you do this and by the way if you want to you can go to board interactive I'll give you the site you can

play a bunch of our games for free including one that gives is a teaching simulation and another one that you run a small business but we're building all these complex games and we're building this around this pedagogical approach there's there's me I probably should have used a different person's picture but there's two of me um Beard's a little better cut there but anyway um the idea is that every one of our simulation starts with the idea of built-in lessons so not just me but we brought in experts from around the world to teach lessons in

this case is an entrepreneurship simulation and we're actually gonna have people run a fake company in real time over the course of six class sessions so it's like a 20 hour experience with all the teaching is built in which is really cool right so you actually have videos and information of all these kind of famous people about how to run a company and then we actually built like fake Gmail fake slack and you actually get email messages from people you have to respond to them people get angry if you don't do things I've had um

for our science fiction based one I had the person who won the Hugo award this year helped me out in writing it which was pretty exciting because it's it's you know my nerd Hearts things that's the sort of oscars of Science of fiction um and um so all of this is kind of uh built in and so what happens is you're faced with these challenges you have to solve all these problems even built in fake Zoom meetings let's see if this will work here um so you actually and you know literally get Zoom calls and

you're hiring someone in this case you have to choose what answer you want to send to them so I've been building this stuff out for a long time they work really well and then there's all kinds of cool feedback that you get right you do your Zoom meetings and then afterwards you end with an elevator pitch and you have to pitch your idea in this entrepreneurship case and you get 35 pages of feedback right details grading information you get it's all gamified so there's scores and all kinds of cool stuff there and you know you

get these detailed reports everything you do right or wrong you get feedback on so really been awesome working really hard on this and it's working right 3 000 instructors are using it we've got incredibly High completion rates I do moocs and stuff too people love this stuff it works really well love to talk more about it um and you know this has been a lot of my life's work in the short term but I've been using AI a lot recently so I decided to do something I decided to ask um AI a question what if

I just used gpt4 okay so again I've got a 15 person team they're all brilliant I've got creative fiction writers and I've got coders all these people working together pedagogy experts what what if I just ask GPT hey chat GPT I want to do a deliberate practice about how to conduct negotiations you're my negotiation teacher simulated scenario in which you have to engage in negotiation you'll feel the role of one party I'll be the other uh you'll ask for a response for each step in the scenario and then you'll give me details what they do

and you'll grade my response at the end if I do well you'll give me a harder task if I do badly you'll do give me an easier task and it just started doing it right it's not 100 there but it's 90 percent of the way to what would have taken us and we're very good at this weeks to build right uh months to build with the whole team I can do with a paragraph right and we're experimenting right now with how we can do this with three or four paragraphs and right now we have a

version of the game that we've just worked on this week where you see a little video of me talking and then we fed everything I have spoken about about the topic into the AI and you can now ask me questions in the AI and then the next phase you we spin up a AI that is your counterparty negotiate with and you negotiate with that person and then a third AI appears and grades you on your performance Right End to End taking it is five paragraphs of work right a little bit fine tuning but we don't

need a huge team it's it's done right and so my feeling about this um you know like um so and you know so there's some AI art for you so I think we're on the edge of a real transformation like a very very serious one and that's kind of what I want you to be thinking about here right which is a lot of you are Ed Tech providers and suddenly we've democratized education technology right in a really fundamental way any of you can experiment with this I'm you've got this paragraph We should be figuring out

how to share them among ourselves as Educators but if your advantage is that I I you know was we had this very elaborate simulation system and now we can do this with a prompt what does that mean for your education system right it means a lot of empowering teachers to do stuff but what does it mean for the edtech providers here in the room and I think that's a kind of harder question right because not just simulations I have managed to get gbt to do Harvard case studies with um two paragraphs of of text like

write a Harvard case study by the way Google is not winning in Ai and then write the teaching note for it and then write the board plan of what I'd write on the board write the assignments that I give to the students and the essays give me be the rubrics for the assignment create a fake assignment grade the assignment what would the students want to ask me about the assignment what are the first questions they should ask simulate a classroom of diverse students asking me questions have them ask me the questions and let me practice

it right send in sentence send it it just doesn't right is it 100 correct no of course it hallucinates and makes stuff up is it pretty amazing right yes right and as anyone who's an instructor for the room knows you can be teaching a class and get everything exactly right and your students learn something completely different than what you were saying right so it's I'm a little less worried about the lying in this piece because it's not usually harmful lying there are facts wrong but every teacher gets things wrong too and will get better at

facts right this is the worst AI you're ever going to use is the one available to you right now it's only going to get better I don't know if you have better faster and we all have to worry about that or whether it's a better slower but it's certainly not going to get worse than it is now and it's certainly going to get better so don't get comfortable it's like oh it doesn't do math it will do math right it's not going to be a problem you know it's not creative actually it already is maxing

out our creativity tests that we give to humans like the association tests and things like that but it'll get better at that too so we can't really draw rules about what it's doing and what it's not doing it's already doing a huge amount of Education work again wonderful for your instructor worrying if you're an edtech provider I think right um okay so um I want to talk just about a few things right final thoughts here which is we've never had like this is the democratization moment we've been waiting for we have an edtech tool that

is available to everybody everywhere right with the right paragraph you know we've been so much work has gone into it was peripherally involved with like the one laptop for child program at MIT which is a great idea we're gonna give everybody laptops all over the world but it was all this other systems had to be in place it kind of failed because there was no there wasn't the right technology around it at the time there wasn't the right support we now have a tool that really is in everybody's hands and can work as a tutor

you put in a math problem it will help you work through it you say please help me out it will help you what do we do with that it's a really fundamental rethinking of what it means to do education it also means you can build your own edtech solution right whoever in your school is the most creative person or university can just come up with a whole bunch of lessons and send them to you right it used to be that you're going and sharing lesson plans online share a prompt and now you're in much better

shape and it's interactively building stuff for you your programming in English Pros it's crazy when you do that I can't program at all in non-stisical programming languages like I only know nerd programming and I have written and python programs in the last week because I just asked chachibd um you know come up with a way for it's automatically download thing scores from canvas assignments from canvas grade those assignments using GPT according to rubric and then re-upload in the canvas and it spits up python code and I just run it and it does the thing right

um again for my students I've not done that with you guys yet actively I'm looking for cameras um but the point is is that we don't need to wait for newer AI this is not about like there's a lot of obsession over what happens next it doesn't matter the singularity for us as Educators and as edtech providers is already here right we already have a huge change in the world of work a huge change in the world of Education nobody has direct answers for you there's no rule book so what I would recommend that we

think about doing is as Educators is to experiment with these systems to use them as much as possible to embrace them to ask your students to embrace them and it will require change but we have some more room to change in education than in other fields to do that and I think we have a lot of power to do that and we really have a tool now that changes the nature of the game right and I think we're going to realize a lot of old methods don't work but we can adapt right we already know

I think a lot of you can already picture a classroom in the future where people are using tools like the online video and these tutors outside of class that coming to class and doing really satisfying activities where you can pay the attention to them as an educator that you need right we can do these things it's not going to be impossible to adjust but we don't know what that future looks like and the best thing you could do right now is stay Nimble by trying these things out yourself so if you're not a huge user

of chat gbt I think everybody needs 10 or 20 hours in this at a minimum and I think you should be using it for every task you do you'll find it's bad at some things good at some things that's the only way to learn there is no rule book you shouldn't wait for a memo to come down just start doing it right and you will figure it out figure out how to be the person that this matters to so I hope you enjoyed this if you want to do any of our simulations um I think

that's on screen yeah you can actually go to interactive.worton and I've got like a little blog which I'm just always trying to publish like there's a bunch of academic papers and stuff on how to do this stuff that you can build from too thank you guys very much it's a really exciting future I'm looking forward to figuring out what we all discover thank you [Music]