AI prompt engineering: A deep dive

- Basically, this entire roundtable session here is just gonna be focused mainly on prompt engineering. A variety of perspectives at this table around prompting from a research side, from a consumer side, and from the enterprise side. And I want to just get the whole wide range of opinions because there's a lot of them. And just open it up to discussion and explore what prompt engineering really is and what it's all about. And yeah, we'll just take it from there. So maybe we can go around the horn with intros. I can kick it off. I'm Alex.

I lead Developer Relations here at Anthropic. Before that, I was technically a prompt engineer at Anthropic. I worked on our prompt engineering team, and did a variety of roles spanning from a solutions architect type of thing, to working on the research side. So with that, maybe I can hand it over to David. - Heck, yeah. My name's David Hershey. I work with customers mostly at Anthropic on a bunch of stuff technical, I help people with finetuning, but also just a lot of the generic things that make it hard to adopt language models of prompting. And

just like how to build systems with language models, but spend most of my time working with customers. - Cool. I'm Amanda Askell. I lead one of the Finetuning teams at Anthropic, where I guess I try to make Claude be honest and kind. Yeah. - My name is Zack Witten. I'm a Prompt Engineer at Anthropic. Alex and I always argue about who the first one was. He says it's him, I say it's me. - Contested. - Yeah. I used to work a lot with individual customers, kind of the same way David does now. And then as

we brought more solutions architects to the team, I started working on things that are meant to raise the overall levels of ambient prompting in society, I guess, like the prompt generator and the various educational materials that people use. - Nice, cool. Well, thanks guys for all coming here. I'm gonna start with a very broad question just so we have a frame going into the rest of our conversations here. What is prompt engineering? Why is it engineering? What's prompt, really? If anyone wants to kick that off, give your own perspective on it, feel free to take

the rein here. - I feel like we have a prompt engineer. It's his job. - We're all prompt engineers in our own form. - But one of us has a job. - Yeah. Zack, maybe since it's in your title. - One of us has a job, but the other three don't have jobs. - I guess I feel like prompt engineering is trying to get the model to do things, trying to bring the most out of the model. Trying to work with the model to get things done that you wouldn't have been able to do otherwise.

So a lot of it is just clear communicating. I think at heart, talking to a model is a lot like talking to a person. And getting in there and understanding the psychology of the model, which Amanda is the world's most expert person in the world. - Well, I'm gonna keep going on you. Why is engineering in the name? - Yeah. I think the engineering part comes from the trial and error. - Okay. - So one really nice thing about talking to a model that's not like talking to a person, is you have this restart button.

This giant go back to square zero where you just start from the beginning. And what that gives you the ability to do that you don't have, is a truly start from scratch and try out different things in an independent way, so that you don't have interference from one to the other. And once you have that ability to experiment and to design different things, that's where the engineering part has the potential to come in. - Okay. So what you're saying is as you're writing these prompts, you're typing in a message to Claude or in the API

or whatever it is. Being able to go back and forth with the model and to iterate on this message, and revert back to the clean slate every time, that process is the engineering part. This whole thing is prompt engineering all in one. - There's another aspect of it too, which is integrating the prompts within your system as a whole. And David has done a ton of work with customers integrating. A lot of times it's not just as simple as you write one prompt and you give it to the model and you're done. In fact, it's

anything but. It's like way more complicated. - Yeah. I think of prompts as the way that you program models a little bit, that makes it too complicated. 'Cause I think Zack is generally right that it's just talking clearly is the most important thing. But if you think about it a little bit as programming a model, you have to think about where data comes from, what data you have access to. So if you're doing RAG or something, what can I actually use and do and pass to a model? You have to think about trade-offs in latency

and how much data you're providing and things like that. There's enough systems thinking that goes into how you actually build around a model. I think a lot of that's also the core of why it maybe deserves its own carve-out as a thing to reason about separately from just a software engineer or a PM or something like that. It's kind of its own domain of how to reason about these models. - Is a prompt in this sense then natural language code? Is it a higher level of abstraction or is it a separate thing? - I think

trying to get too abstract with a prompt is a way to overcomplicate a thing, because I think, we're gonna get into it, but more often than not, the thing you wanna do is just write a very clear description of a task, not try to build crazy abstractions or anything like that. But that said, you are compiling the set of instructions and things like that into outcomes a lot of times. So precision and a lot the things you think about with programming about version control and managing what it looked like back then when you had this

experiment. And tracking your experiment and stuff like that, that's all just equally important to code. - Yeah. - So it's weird to be in this paradigm where written text, like a nice essay that you wrote is something that's looked like the same thing as code. But it is true that now we write essays and treat them code, and I think that's actually correct. - Yeah. Okay, interesting. So maybe piggybacking off of that, we've loosely defined what prompt engineering is. So what makes a good prompt engineer? Maybe, Amanda, I'll go to you for this, since you're

trying to hire prompt engineers more so in a research setting. What does that look like? What are you looking for in that type of person? - Yeah, good question. I think it's a mix of like Zack said, clear communication, so the ability to just clearly state things, clearly understand tasks, think about and describe concepts really well. That's the writing component, I think. I actually think that being a good writer is not as correlated with being a good prompt engineer as people might think. So I guess I've had this discussion with people 'cause I think there's

some argument as like, "Maybe you just shouldn't have the name engineer in there. Why isn't it just writer?" I used to be more sympathetic to that. And then, I think, now I'm like what you're actually doing, people think that you're writing one thing and you're done. Then I'll be like to get a semi-decent prompt when I sit down with the model. Earlier, I was prompting the model and I was just like in a 15-minute span I'll be sending hundreds of prompts to the model. It's just back and forth, back and forth, back and forth. So

I think it's this willingness to iterate and to look and think what is it that was misinterpreted here, if anything? And then fix that thing. So that ability to iterate. So I'd say clear communication, that ability to iterate. I think also thinking about ways in which your prompt might go wrong. So if you have a prompt that you're going to be applying to say, 400 cases, it's really easy to think about the typical case that it's going to be applied to, to see that it gets the right solution in that case, and then to move

on. I think this is a very classic mistake that people made. What you actually want to do is find the cases where it's unusual. So you have to think about your prompt and be like, "What are the cases where it'd be really unclear to me what I should do in this case?" So for example, you have a prompt that says, "I'm going to send you a bunch of data. I want you to extract all of the rows where someone's name is, I don't know, starts with the letter G." And then you're like, "Well, I'm gonna

send it a dataset where there is no such thing, there is no such name that starts with the letter G. "I'm going to send it something that's not a dataset, I might also just send it an empty string. These are all of the cases you have to try, because then you're like, "What does it do in these cases? " And then you can give it more instructions for how it should deal with that case. - I work with customers so often where you're an engineer, you're building something. And there's a part in your prompt where

a customer of theirs is going to write something. - Yeah. - And they all think about these really perfectly phrased things that they think someone's going to type into their chatbot. And in reality, it's like they never used the shift key and every other word is a typo. - They think it's Google. - And there's no punctuation. - They just put in random words with no question. - Exactly. So you have these evals that are these beautifully structured what their users ideally would type in. But being able to go the next step to reason about

what your actual traffic's gonna be like, what people are actually gonna to try to do, that's a different level of thinking. - One thing you said that really resonated with me is reading the model responses. In a machine learning context, you're supposed to look at the data. It's almost a cliche like look at your data, and I feel like the equivalent for prompting is look at the model outputs. Just reading a lot of outputs and reading them closely. Like Dave and I were talking on the way here, one thing that people will do is they'll

put think step-by-step in their prompt. And they won't check to make sure that the model is actually thinking step-by-step, because the model might take it in a more abstract or general sense. Rather than like, "No, literally you have to write down your thoughts in these specific tags." So yeah, if you aren't reading the model outputs, you might not even notice that it's making that mistake. - Yeah, that's interesting. There is that weird theory of mind piece to being a prompt engineer where you have to think almost about how the model's gonna view your instructions. But

then if you're writing for an enterprise use case too, you also have to think about how the user's gonna talk to the model, as you're the third party sitting there in that weird relationship. Yeah. - On the theory of mind piece, one thing I would say is, it's so hard to write instructions down for a task. It's so hard to untangle in your own brain all of the stuff that you know that Claude does not know and write it down. It's just an immensely challenging thing to strip away all of the assumptions you have, and

be able to very clearly communicate the full fact set of information that is needed to a model. I think that's another thing that really differentiates a good prompt engineer from a bad one, is like... A lot of people will just write down the things they know. But they don't really take the time to systematically break out what is the actual full set of information you need to know to understand this task? - Right. - And that's a very clear thing I see a lot is prompts where it's just conditioned. The prompt that someone wrote is

so conditioned on their prior understanding of a task, that when they show it to me I'm like, "This makes no sense. None of the words you wrote make any sense, because I don't know anything about your interesting use case." But I think a good way to think about prompt engineering in that front and a good skill for it, is just can you actually step back from what you know and communicate to this weird system that knows a lot, but not everything about what it needs to know to do a task? - Yeah. The amount of

times I've seen someone's prompt and then being like, "I can't do the task based on this prompt." I'm human level and you're giving this to something that is worse than me and expecting it to do better, and I'm like, "Yeah." - Yeah. There is that interesting thing with like... Current models don't really do a good job of asking good, probing questions in response like a human would. If I'm giving Zack directions on how to do something, he'll be like, "This doesn't make any sense. What am I supposed to do at this step or here and

here?" Model doesn't do that, right, so you have to, as yourself, think through what that other person would say and then go back to your prompt and answer those questions. - You could ask it to do that. - You could. That's right. - I do that, yeah. - I guess that's another step. - I was going to say one of the first things I do with my initial prompt, is I'll give it the prompt and then I'll be like, "I don't want you to follow these instructions. I just want you to tell me the ways

in which they're unclear or any ambiguities, or anything you don't understand." And it doesn't always get it perfect, but it is interesting that that is one thing you can do. And then also sometimes if people see that the model makes a mistake, the thing that they don't often do is just ask the model. So they say to the model, "You got this wrong. Can you think about why? And can you maybe write an edited version of my instructions that would make you not get it wrong?" And a lot of the time, the model just gets

it right. The model's like, "Oh, yeah. Here's what was unclear, here's a fix to the instructions," and then you put those in and it works. - Okay. I'm actually really curious about this personally almost. Is that true that that works? Is the model able to spot its mistakes that way? When it gets something wrong, you say, "Why did you get this wrong?" And then it tells you maybe something like, "Okay, how could I phrase this to you in the future so you get it right?" Is there an element of truth to that? Or is that

just a hallucination on the model's part around what it thinks its limits are? - I think if you explain to it what it got wrong, it can identify things in the query sometimes. I think this varies by task. This is one of those things where I'm like I'm not sure what percentage of the time it gets it right, but I always try it 'cause sometimes it does. - And you learn something. - Yeah. - Anytime you go back to the model or back and forth with the model, you learn something about what's going on. I

think you're giving away information if you don't at least try. - That's interesting. Amanda, I'm gonna keep asking you a few more questions here. One thing maybe for everybody watching this, is we have these Slack channels at Anthropic where people can add Claude into the Slack channel, then you can talk to Claude through it. And Amanda has a Slack channel that a lot of people follow of her interactions with Claude. And one thing that I see you always do in there, which you probably do the most of anyone at Anthropic, is use the model to

help you in a variety of different scenarios. I think you put a lot of trust into the model in the research setting. I'm curious how you've developed those intuitions for when to trust the model. Is that just a matter of usage, experience or is it something else? - I think I don't trust the model ever and then I just hammer on it. So I think the reason why you see me do that a lot, is that that is me being like, "Can I trust you to do this task?" 'Cause there's some things, models are kind

of strange. If you go slightly out of distribution, you just go into areas where they haven't been trained or they're unusual. Sometimes you're like, "Actually, you're much less reliable here, even though it's a fairly simple task." I think that's happening less and less over time as models get better, but you want to make sure you're not in that kind of space. So, yeah, I don't think I trust it by default, but I think in ML, people often want to look across really large datasets. And I'm like, "When does it make sense to do that?" And

I think the answer is when you get relatively low signal from each data point, you want to look across many, many data points, because you basically want to get rid of the noise. With a lot of prompting tasks, I think you actually get really high signal from each query. So if you have a really well-constructed set of a few hundred prompts, that I think can be much more signal than thousands that aren't as well-crafted. So I do think that I can trust the model if I look at 100 outputs of it and it's really consistent.

And I know that I've constructed those to basically figure out all of the edge cases and all of the weird things that the model might do, strange inputs, et cetera. I trust that probably more than a much more loosely constructed set of several thousand. - I think in ML, a lot of times the signals are numbers. Did you predict this thing right or not? And it'd be looking at the logprobs of a model and trying to intuit things, which you can do, but it's kind of sketchy. I feel like the fact that models output more

often than not a lot of stuff like words and things. There's just fundamentally so much to learn between the lines of what it's writing and why and how, and that's part of what it is. It's not just did it get the task right or not? It's like, "How did it get there? How was it thinking about it? What steps did it go through?" You learn a lot about what is going on, or at least you can try to get a better sense, I think. But that's where a lot of information comes from for me, is

by reading the details of what came out, not just through the result. - I think also the very best of prompting can make the difference between a failed and a successful experiment. So sometimes I can get annoyed if people don't focus enough on the prompting component of their experiment, because I'm like, "This can, in fact, be the difference between 1% performance in the model or 0.1%." In such a way that your experiment doesn't succeed if it's at top 5% model performance, but it does succeed if it's top 1% or top 0.1%. And then I'm like,

"If you're gonna spend time over coding your experiment really nicely, but then just not spend time on the prompt." I don't know. That doesn't make sense to me, 'cause that can be the difference between life and death of your experiment. - Yeah. And with the deployment too, it's so easy to, "Oh, we can't ship this." And then you change the prompt around and suddenly it's working. - Yeah. - It's a bit of a double-edged sword though, because I feel like there's a little bit of prompting where there's always this mythical, better prompt that's going to

solve my thing on the horizon. - Yeah. - I see a lot of people get stuck into the mythical prompt on the horizon, that if I just keep grinding, keep grinding. It's never bad to grind a little bit on a prompt, as we've talked, you learn things. But it's one of the scary things about prompting is that there's this whole world of unknown. - What heuristics do you guys have for when something is possible versus not possible with a perfect prompt, whatever that might be? - I think I'm usually checking for whether the model kind

of gets it. So I think for things where I just don't think a prompt is going to help, there is a little bit of grinding. But often, it just becomes really clear that it's not close or something. Yeah. I don't know if that's a weird one where I'm just like, "Yeah, if the model just clearly can't do something, I won't grind on it for too long." - This is the part that you can evoke how it's thinking about it, and you can ask it how it's thinking about it and why. And you can get a

sense of is it thinking about it right? Are we even in the right zip code of this being right? And you can get a little bit of a kneeling on that front of, at least, I feel like I'm making progress towards getting something closer to right. Where there's just some tasks where you really don't get anywhere closer to it's thought process. It's just like every tweak you make just veers off in a completely different, very wrong direction, and I just tend to abandon those. I don't know. - Those are so rare now though, and I

get really angry at the model when I discover them because that's how rare they are. I get furious. I'm like, "How dare there be a task that you can't just do, if I just push you in the right direction?" - I had my thing with Claude plays Pokemon recently, and that was one of the rare times where I really... - Yeah, can you explain that? Explain that just for people. I think that's really cool. - I did a bit of an experiment where I hooked Claude up to a Game Boy emulator, and tried to have

it play the game Pokemon Red like the OG Pokemon. And it's like you think what you wanna do and it could write some code to press buttons and stuff like that, pretty basic. And I tried a bunch of different very complex prompting layouts, but you just get into certain spots where it just really couldn't do it. So showing it a screenshot of a Game Boy, it just really couldn't do. And it just so deeply because I'm so used to it, being able to do something mostly. So I spent a whole weekend trying to write better

and better prompts to get it to really understand this Game Boy screen. And I got incrementally better so that it was only terrible instead of completely no signal. You could get from no signal to some signal. But it was, I don't know, at least this is elicited for me. Once I put a weekend of time in and I got from no signal to some signal, but nowhere close to good enough, I'm like, "I'm just going to wait for the next one. (Alex laughing) I'm just gonna wait for another model." I could grind on this for

four months, and the thing that would come out is another model and that's a better use of my time. Just sit and wait to do something else in the meanwhile. - Yeah. That's an inherent tension we see all the time, and maybe we can get to that in a sec. Zack, if you wanna go. - Something I liked about your prompt with Pokemon where you got the best that you did get, was the way that you explained to the model that it is in the middle of this Pokemon game. Here's how the things are gonna

be represented. I actually think you actually represented it in two different ways, right? - I did. So what I ended up doing, it was obnoxious but I superimposed a grid over the image, and then I had to describe each segment of the grid in visual detail. Then I had to reconstruct that into an ASCII map and I gave it as much detail as I could. The player character is always at location 4, 5 on the grid and stuff like that, and you can slowly build up information. I think it's actually a lot like prompting, but

I just hadn't done it with images before. Where sometimes my intuition for what you need to tell a model about text, is a lot different from what you need to tell a model about images. - Yeah. - I found a surprisingly small number of my intuitions about text have transferred to image. I found that multi-shot prompting is not as effective for images and text. I'm not really sure, you can have theoretical explanations about why. Maybe there's a few of it in the training data, a few examples of that. - Yeah. I know when we were

doing the original explorations with prompting multimodal, we really couldn't get it to noticeably work. You just can't seem to improve Claude's actual, visual acuity in terms of what it picks up within an image. Anyone here has any ways that they've not seen that feature. But it seems like that's similar with the Pokemon thing where it's trying to interpret this thing. No matter how much you throw prompts at it, it just won't pick up that Ash that's in that location. - Yeah. But I guess to be visceral about this, I could eventually get it so that

it could most often tell me where a wall was, and most often tell me where the character was. It'd be off by a little bit. But then you get to a point, and this is maybe coming back to knowing when you can't do it. It would describe an NPC, and to play a game well, you need to have some sense of continuity. Have I talked to this NPC before? And without that, you really don't, there's nothing you can do. You're just going to keep talking to the NPC, 'cause like, "Well, maybe this is a different

NPC." But I would try very hard to get it to describe a NPC and it's like, "It's a person." They might be wearing a hat, they weren't wearing a hat. And it's like you grind for a while, inflate it to 3000X and just crop it to just the NPC, and it's like, "I have no idea what this is." It's like I showed it this clear, female NPC thing enough times and it just got nowhere close to it, and it's like, "Yeah, this is a complete lost cause." - Wow, okay. - I really want to try

this now. I'm just imagining all the things I would try. I don't know, I want you to imagine this game art as a real human and just describe to me what they're like. What did they look like as they look in the mirror? And then just see what the model does. - I tried a lot of things. The eventual prompt was telling Claude it was a screen reader for a blind person, which I don't know if that helped, but it felt right so I stuck with that. - That's an interesting point. I actually wanna go

into this a little bit 'cause this is one of the most famous prompting tips, is to tell the language model that they are some persona or some role. I feel like I see mixed results. Maybe this worked a little bit better in previous models and maybe not as much anymore. Amanda, I see you all the time be very honest with the model about the whole situation like, "Oh, I am an AI researcher and I'm doing this experiment." - I'll tell it who I am. - Yeah. - I'll give it my name, be like, "Here's who

you're talking to." - Right. Do you think that level of honesty, instead of lying to the model or forcing it to like, "I'm gonna tip you $500." Is there one method that's preferred there, or just what's your intuition on that? - Yeah. I think as models are more capable and understand more about the world, I guess, I just don't see it as necessary to lie to them. I also don't like lying to the models just 'cause I don't like lying generally. But part of me is if you are, say, constructing. Suppose you're constructing an eval

dataset for a machine learning system or for a language model. That's very different from constructing a quiz for some children. So when people would do things like, "I am a teacher trying to figure out questions for a quiz." I'm like, "The model knows what language model evals are." If you ask it about different evals it can tell you, and it can give you made up examples of what they look like. 'Cause these things are like they understand them, they're on the internet. So I'm like, "I'd much rather just target the actual task that I have."

So if you're like, "I want you to construct questions that look a lot like an evaluation of a language model." It's that whole thing of clear communication. I'm like, "That is, in fact, the task I want to do. So why would I pretend to you that I want to do some unrelated, or only tangentially related task?" And then expect you to somehow do better at the task that I actually want you to do. We don't do this with employees. I wouldn't go to someone that worked with me and be like, "You are a teacher and

you're trying to quiz your students." I'd be like, "Hey, are you making that eval?" I don't know. So I think maybe it's a heuristic from there where I'm like, "If they understand the thing, just ask them to do the thing that you want." - I see this so much. - I guess to push back a little bit, I have found cases where not exactly lying but giving it a metaphor for how to think about it could help. In the same way that sometimes I might not understand how to do something and someone's like, "Imagine that

you were doing this, even though I know I'm not doing it." The one that comes to mind for me, is I was trying to have Claude say whether an image of a chart or a graph is good or not. Is it high quality? And the best prompt that I found for this was asking the model what grade it would give the chart, if it were submitted as a high school assignment. So it's not exactly saying, "You are a high school teacher." It's more like, "This is the kind of analysis that I'm looking from for you."

The scale that a teacher would use is similar to the scale that I want you to use. - But I think those metaphors are pretty hard to still come up with. I think people still, the default you see all the time is finding some facsimile of the task. Something that's a very similar-ish task, like saying you're a teacher. You actually just lose a lot in the nuance of what your product is. I see this so much in enterprise prompts where people write something similar, because they have this intuition that it's something the model has seen

more of maybe. It's seen more high school quizzes than it has LLM evals, and that may be true. But to your point, as the models get better, I think just trying to be very prescriptive about exactly the situation they're in. I give people that advice all the time. Which isn't to say that I don't think to the extent that it is true that thinking about it the way that someone would grade a chart, as how they would grade a high school chart, maybe that's true. But it's awkwardly the shortcut people use a lot of times

to try to get what happens, so I'll try to get someone that I can actually talk about 'cause I think it's somewhat interesting. So writing you are a helpful assistant, writing a draft of a document, it's not quite what you are. You are in this product, so tell me. If you're writing an assistant that's in a product, tell me I'm in the product. Tell me I'm writing on behalf of this company, I'm embedded in this product. I'm the support chat window on that product. You're a language model, you're not a human, that's fine. But just

being really prescriptive about the exact context about where something is being used. I found a lot of that. Because I guess my concern most often with role prompting, is people used it as a shortcut of a similar task they want the model to do. And then they're surprised when Claude doesn't do their task right, but it's not the task. You told it to do some other task. And if you didn't give it the details about your task, I feel like you're leaving something on the table. So I don't know, it does feel like a thing

though to your point of as the models scale. Maybe in the past it was true that they only really had a strong understanding of elementary school tests comparatively. But as they get smarter and can differentiate more topics, I don't know, just like being clear. - I find it interesting that I've never used this prompting technique. - Yeah, that's funny. - Even with worse models and I still just don't ever find myself, I don't know why. I'm just like, "I don't find it very good essentially." - Interesting. - I feel like completion era models, there was

a little bit of a mental model of conditioning the model into a latent space that was useful that I worried about, that I don't really worry about too much anymore. - It might be intuitions from pretrained models over to RLHF models, that to me, just didn't make sense. It makes sense to me if you're prompting a pretrained. - You'd be amazed how many people try to apply their intuitions. I think it's not that surprising. Most people haven't really experimented with the full what is a pretrained model? What happens after you do SL? What happens after

you do RLHF, whatever? So when I talk to customers, it's all the time that they're trying to map some amount of, "Oh, how much of this was on the internet? Have they seen a ton of this on the internet?" You just hear that intuition a lot, and I think it's well-founded fundamentally. But it is overapplied by the time you actually get to a prompt, because of what you said. By the time they've gone through all of this other stuff, that's not actually quite what's being modeled. - Yeah. The first thing that I feel like you

should try is, I used to give people this thought experiment where it's like imagine you have this task. You've hired a temp agency to send someone to do this task. This person arrives, you know they're pretty competent. They know a lot about your industry and so forth, but they don't know the name of your company. They've literally just shown up and they're like, "Hey, I was told you guys had a job for me to do, tell me about it." And then it's like, "What would you say to that person?" And you might use these metaphors.

You might say things like, "We want you to detect good charts. What we mean by a good chart here, is it doesn't need to be perfect. You don't need to go look up whether all of the details are correct." It just needs to have its axes labeled, and so think about maybe high school level, good chart. You may say exactly that to that person and you're not saying to them, "You are a high school." You wouldn't say that to them. You wouldn't be like, "You're a high school teacher reading charts." - What are you talking

about? - Yeah, so sometimes I'm just like it's like the whole if I read it. I'm just like, "Yeah. Imagine this person who just has very little context, but they're quite competent. They understand a lot of things about the world." Try the first version that actually assumes that they might know things about the world, and if that doesn't work, you can maybe do tweaks and stuff. But so often, the first thing I try is that, and then I'm like, "That just worked." - That worked. - And then people are like, "Oh, I didn't think to

just tell it all about myself and all about the task I want to do." - I've carried this thing that Alex told me to so many customers where they're like, "Oh, my prompt doesn't work. Can you help me fix it?" I'm like, "Well, can you describe to me what the task was?" And I'm like, "Okay. Now what you just said to me, just voice record that and then transcribe it." And then paste it into the prompt and it's a better prompt than what you wrote, but this is a laziness shortcut, I think, to some extent.

Because people write something that they... I just think people, I'm lazy. A lot of people are lazy. - We had that in prompt assistance the other day where somebody was like, "Here's the thing, here's what I want it to do, and here's what it's actually doing instead." So then I just literally copied the thing that they said they wanted it to do, and pasted it in and it worked. - Yeah. I think a lot of people still haven't quite wrapped their heads around what they're really doing when they're prompting. A lot of people see a

text box and they think it's a Google search box. They type in keywords and maybe that's more on the chat side. But then on the enterprise side of things, you're writing a prompt for an application. There is still this weird thing to it where people are trying to take all these little shortcuts in their prompt, and just thinking that, "Oh, this line carries a lot of weight in this." - Yeah. I think you obsess over getting the perfect little line of information and instruction, as opposed to how you just described that graph thing. I would

be a dream if I read prompts like that. If someone's like, "Well, you do this and this, and there's some stuff to consider about this and all that." But that's just not how people write prompts. They work so hard to find the perfect, insightful. A perfect graph looks exactly like this exact perfect thing, and you can't do that. It's just very hard to ever write that set of instructions down prescriptively, as opposed to how we actually talk to humans about it, which is try to instill some amount of the intuitions you have. - We also

give them outs. This is a thing that people can often forget in prompts. So cases, if there's an edge case, think about what you want the model to do. 'Cause by default, it will try the best to follow your instructions, much as the person from the temp agency would, 'cause they're like, "Well, they didn't tell me how to get in touch with anyone." If I'm just given a picture of a goat and I'm like, "What do I do? This isn't even a chart. How good is a picture of a goat as a chart?" I just

don't know. And if you instead say something like, "If something weird happens and you're really not sure what to do, just output in tags unsure." Then you can go look through the unsures that you got and be like, "Okay, cool. It didn't do anything weird." Whereas by default, if you don't give the person the option, they're like, "It's a good chart." Then people will be like, "How do I do that?" And then you're like, "Well, give it an out. Give it something to do if it's a really unexpected input happens." - And then you also

improved your data quality by doing that too, 'cause you found all the screwed up examples. - Oh, yeah. - That's my favorite thing about iterating on tests with Claude, is the most common outcome is I find all of the terrible tests I accidentally wrote because it gets it wrong. I'm like, "Oh, why did it get wrong?" I was like, "Oh, I was wrong." - Yeah. - Yeah. - If I was a company working with this, I do think I would just give my prompts to people, because I used to do this when I was evaluating

language models. I would take the eval myself. 'Cause I'm like, "I need to know what this eval looks like if I'm gonna to be grading it, having models take it, thinking about outputs, et cetera." I would actually just set up a little script and I would just sit and I would do the eval. - Nowadays, you just have called the Streamboard app for you. - And just does it, yeah. - Yeah. I'm reminded of Karpathy's ImageNet. I was in 231 at Stanford and it's like benchmarking, he's showing the accuracy number. And he's like, "And here's

what my accuracy number was." And he had just gone through the test set and evaluated himself. - Oh, yeah. - You just learn a lot. - Yeah, totally. - And it's better when it's a, again, the temp agency person, like someone who doesn't know the task, because that's a very clean way to learn things. - Yeah. The way you have to do it is, some evaluations come with instructions, and so I would give myself those instructions as well and then try to understand it. And it's actually quite great if you don't have context on how

it's graded. And so often, I would do so much worse than the human benchmark and I was like, "I don't even know how you got humans to do this well at this task, 'cause apparently human level here is 90%, and I'm at 68%." - That's funny. That reminds me of just when you look at the MMLU questions and you're like, "Who would be able to answer these?" It's just like absolute garbage in some of them. Okay. I have one thing I wanna circle back on that we were talking about a few questions back around, I

think you were saying getting signal from the responses. There's just so much there and it's more than just a number, and you can actually read into the almost thought process. I bet this is probably a little contentious maybe around chain of thought. For people listening, chain of thought, this process of getting them all to actually explain its reasoning before it provides an answer. Is that reasoning real or is it just kind of like a holding space for the model to do computation? Do we actually think there's good, insightful signal that we're getting out of the

model there? - This is one of the places where I struggle with that. I'm normally actually somewhat pro-personification because I think it helps you get decent facsimiles, thoughts of how the model's working. And this one, I think it's harmful maybe almost to get too into the personification of what reasoning is, 'cause it just loses the thread of what we're trying to do here. Is it reasoning or not? It feels almost like a different question than what's the best prompting technique? It's like you're getting into philosophy, which we can get into. - Yeah, we do have

a philosopher. - Yeah. I will happily be beaten down by a real philosopher as I try to speculate on this, but instead, it just works. Your model does better. The outcome is better if you do reasoning. I think I've found that if you structure the reasoning and help iterate with the model on how it should do reasoning, it works better too. Whether or not that's reasoning or how you wanted to classify it, you can think of all sorts of proxies for how I would also do really bad if I had to do one-shot math without

writing anything down. Maybe that's useful, but all I really know is, it very obviously does help. I don't know. - A way of testing would be if you take out all the reasoning that it did to get to the right answer, and then replace it with somewhat, realistic-looking reasoning that led to a wrong answer, and then see if it does conclude the wrong answer. I think we actually had a paper where we did some of that. There was the scratch pad. It was like the Sleeper Agents. - Oh, okay. Alignment papers. - But I think

that was maybe a weird situation. But definitely what you said about structuring the reasoning and writing example of how the reasoning works. Given that that helps, like whether we use the word reasoning or not, I don't think it's just a space for computation. - So there is something there. - I think there's something there, whatever we wanna call it. - Yeah. Having it write a story before it finished a task, I do not think would work as well. - I've actually tried that and it didn't work as well as reasoning. - Clearly, the actual reasoning

part is doing something towards the outcome. - I've tried like, "Repeat the words um and ah in any order that you please for 100 tokens and then answer." - Yeah. I guess that's a pretty thorough defeat of it's just more computational space where it can do attention over and over again. I don't think it's just more attention like doing more attention. - I guess the strange thing is, and I don't have an example off the top of my head to back this up with. But I definitely have seen it before where it lays out steps,

one of the steps is wrong, but then it still reaches the right answer at the end. So it's not quite, I guess, yeah, we can't really, truly personify it as a reasoning, 'cause there is some element to it doing something slightly different. - Yeah. I've also met a lot of people who make inconsistent steps of reasoning. - I guess that's true. - It fundamentally defeats the topic of reasoning by making a false step on the way there. - All right, it's interesting. Also, on maybe this prompting misconceptions round of questions. Zack, I know you have

strong opinions on this, good grammar, punctuation. - Oh, do I? - Is that necessary in a prompt? Do you need it? Do you need to format everything correctly? - I usually try to do that because I find it fun, I guess, somehow. I don't think you necessarily need to. I don't think it hurts. I think it's more that you should have the level of attention to detail that would lead you to doing that naturally. If you're just reading over your prompt a lot, you'll probably notice those things and you may as well fix them. And

like what Amanda was saying, that you wanna put as much love into the prompt as you do into the code. People who write a lot of code have strong opinions about things that I could not care less about. Like the number of tabs versus spaces, or I don't know, opinions about which languages are better. And for me, I have opinionated beliefs about styling of prompts. I can't even say that they're right or wrong, but I think it's probably good to try to acquire those, even if they're arbitrary. - I feel personally attacked, 'cause I definitely

have prompts that are like I feel like I'm in the opposite end of the spectrum where people will see my prompts. And then be like, "This just has a whole bunch of typos in it." And I'm like, "The model knows what I mean." - It does, it does know what you mean, but you're putting in the effort, you just are attending to different things. - 'Cause part of me is like, I think if it's conceptually clear, I'm a big, I will think a lot about the concepts and the words that I'm using. So there's definitely

a sort of care that I put in. But it's definitely not to, yeah, people will just point out typos and grammatical issues with my prompts all the time. Now I'm pretty good at actually checking those things more regularly. - Is it because of pressure from the outside world or because it's actually what you think is right? - It's pressure from me. - Yeah, it's probably pressure from the outside world. I do think it makes sense. Part of me is like it's such an easy check, so I think for a final prompt I would do that.

But throughout iteration, I'll happily just iterate with prompts that have a bunch of typos in them, just 'cause I'm like, "I just don't think that the model's going to care." - This gets at the pretrained model versus RLHF thing though, because I was talking to Zack on the way over. The conditional probability of a typo based on a previous typo in the pretraining data is much higher. - Oh, yeah. - Like much higher. - Prompting pretraining models is just a different beast. - It is, but it's interesting. I think it's an interesting illustration of why

your intuitions, like trying to over-apply the intuitions of a pretrained model to the things that we're actually using in production doesn't work very well. Because again, if you were to pass one of your typo-ridden prompts to a pretrained model, the thing that would come out the other side, almost assuredly would be typo-ridden. - Right. - I like to leverage this to create typo-ridden inputs. - That's true. I've done that. - Like what you're saying, try to anticipate what your customers will put in. The pretrained model is a lot better at doing that. 'Cause the RL

models are very polished and they really never made a typo in their lives. - They've been told pretty aggressively to not do the typo thing. - Yeah. Okay, so that's actually an interesting segue here. I've definitely mentioned this to people in the past around to try to help people understand a frame of talking to these models in a sense almost as an imitator to a degree. And that might be much more true of a pretrained model than a post-trained, full-finished model, but is there anything to that? If you do talk to Claude and use a

ton of emojis and everything, it will respond similarly, right? So maybe some of that is there, but like you're saying, it's not all the way quite like a pretrained model. - It's just shifted to what you want. I think at that point, it's like trying to guess what you... We have more or less trained the models to guess what you want them to act like. - Interesting. - Or after we do all of our fancy stuff after pretraining. - The human laborers that used emojis, prefer to get responses with emojis. - Yeah. Amanda writes things

with typos but wants not typos at the other end, and Claude's pretty good at figuring that out. If you write a bunch of emojis to Claude, it's probably the case that you also want a bunch of emojis back from Claude. That's not surprising to me. - Yeah. This is probably something we should have done earlier, but I'll do it now. Let's clarify maybe the differences between what an enterprise prompt is or a research prompt, or a just general chat in Claude.ai prompt. Zack, you've spanned the whole spectrum here in terms of working with customers and

research. Do you wanna just lay out what those mean? - Yeah, I guess. This feels too, you're hitting me with all the hard questions. - Yeah. (laughing) - Well, the people in this room, I think of it as the prompts that I read in Amanda's Claude channel versus the prompts that I read David write. They're very similar in the sense that the level of care and nuance that's put into them. I think for research, you're looking for variety and diversity a lot more. So if I could boil it down to one thing, it's like I've

noticed Amanda's not the biggest fan of having lots of examples, or one or two examples. Like too few 'cause the model will latch onto those. And in prompts that I might write or that I've seen David write, we have a lot of examples. I like to just go crazy and add examples until I feel like I'm about to drop dead, 'cause I've added so many of them. And I think that's because when you're in a consumer application, you really value reliability. You care a ton about the format, and it's fine if all the answers are

the same. In fact, you almost want them to be the same in a lot of ways, not necessarily you want to be responsive to the user's desires. Whereas a lot of times when you're prompting for research, you're trying to really tap into the range of possibilities that the model can explore. And by having some examples, you're actually constraining that a little bit. So I guess just on how the prompts look level, that's probably the biggest difference I noticed is how many examples are in the prompt, which is not to say that I've never seen you

write a prompt with examples. But does that ring true for you? - Yeah. I think when I give examples, often I actually try and make the examples not like the data that the model's going to see, so they're intentionally illustrative. Because if the model, if I give it examples that are very like the data it's going to see, I just think it is going to give me a really consistent response that might not actually be what I want. Because my data that I'm running it on might be extremely varied, and so I don't want it

to just try and give me this really rote output. Often, I want it to be much more responsive. It's much more like cognitive tasks essentially where I'm like, "You have to see this sample and really think about in this sample what was the right answer." So that means that sometimes I'll actually take examples that are just very distinct from the ones that I'm going to be running it on. So if I have a task where, let's say, I was trying to extract information from factual documents. I might actually give it examples that are from what

sounds like a children's story. Just so that I want you to understand the task, but I don't want you to latch on too much to the words that I use or the very specific format. I care more about you understanding the actual thing that I want you to do, which can mean I don't end up giving, in some cases, there's some cases where this isn't true. But if you want more flexibility and diversity, you're going to use illustrative examples rather than concrete ones. You're probably never going to put words in the model's mouth. I haven't

liked that in a long time though. I don't do few-shot examples involving the model having done a thing. I think that intuition actually also comes from pretraining in a way that doesn't feel like it rings true of RLHF models. So yeah, I think those are differences. - The only thing I'd add, a lot of times if you're prompting, like if I'm writing prompts to use on Claude.ai, it's like I'm iterating until I get it right one time. Then it's out the window, I'm good, I did it. Whereas most enterprise prompts, it's like you're gonna go

use this thing a million times or 10 million times, or 100 million times or something like that. So the care and thought you put in is very much testing against the whole range of things, like ways this could be used and the range of input data. Whereas a lot of my time, it's like thinking about one specific thing I want the model to get done right now. - Right, correct. - And it's a pretty big difference in how I approach prompting between if I just wanna get it done this one time right, versus if I

wanna build a system that gets it right a million times. - Yeah. Definitely, in the chat setting, you have the ability to keep the human-in-the-loop and just keep going back and forth. Whereas when you're writing for a prompt to power a chatbot system, it has to cover the whole spectrum of what it could possibly encounter. - It's a lot lower stakes when you are on Claude.ai and you can tell it that it got it wrong or you can even edit your message and try again. But if you're designing for the delightfully discontent user, divinely discontent

user, then you can't ask them to do anything more than the minimum. - But good prompts, I would say, are still good across both those things. If you put the time into the thing for yourself and the time into the enterprise thing, it's equally good. It's just they diverge a little bit in the last mile, I think. - Cool. So the next question I want to just maybe go around the table here, is if you guys had one tip that you could give somebody improving their prompting skill. It doesn't have to be just about writing

a good prompt, it could be that, but just generally getting better at this act of prompting, what would you recommend? - Reading prompts, reading model outputs. Anytime I see a good prompt that someone wrote at Anthropic, I'll read it more closely. Try to break down what it's doing and why and maybe test it out myself, experimentation, talking to the model a lot. - So just how do you know that it's a good prompt, though, to begin with? You just see that the outputs are doing the job correctly? - Yeah. - Okay. - Yeah, that's exactly

right. - Okay. Amanda, maybe you? - Yeah, I think there's probably a lot here. Giving your prompt to another person can be helpful just as a reminder, especially someone who has no context on what you're doing. Yeah, my boring advice has been, it's one of those just do it over and over and over again. And I think if you're really curious and interested and find it fun, this is a lot of people who end up good at prompting, it's just because they actually enjoy it. So I don't know, I once joked just try replacing all

of your friends with AI models and try to automate your own job with AI models. And maybe just try to in your spare time, take joy red teaming AI models. So if you enjoy it, it's much easier. So I'd say do it over and over again, give your prompts to other people. Try to read your prompts as if you are a human encountering it for the first time. - I would say trying to get the model to do something you don't think it can do. The time I've learned the most from prompting, is when I'm

probing the boundaries of what I think a model's capable of. - Interesting. - There's this huge set of things that are so trivial that you don't really get signal on if you're doing a good job or not. Like, "Write me a nice email," it's like you're going to write a nice email. But if you find or can think of something that pushes the boundaries of what you think is possible. I guess probably the first time I ever got into prompting in a way where I felt like I learned a decent amount, was trying to build

a task like an agent like everybody else. Like decompose the task and figure out how to do the different steps of the task. And by really pressing the boundaries of what the model was capable of, you just learn a lot about navigating that. I think a lot of prompt engineering is actually much more about pressing the boundaries of what the model can do. The stuff that's easy, you don't really need to be a prompt engineer to do. So that's, I guess, what I would say is find the hardest thing you can think of and try

to do it. And even if you fail, you tend to learn a lot about how the model works. - That's actually a perfect transition to my next question. Yeah. Basically, from my own experience, how I got started with prompting was with jailbreaking and red teaming. And that is very much trying to find the boundary limits of what the model can do. And figure out how it responds to different phrasings and wordings, and just a lot of trial and error. On the topic of jailbreaks, what's really happening inside a model? When you write a jailbreak prompt,

what's going on there? How does that interact with the post-training that we apply to Claude? Amanda, maybe you have some insight here that you could offer. - I'm not actually sure. - It's honest. - Yeah. I feel bad 'cause I do think lots of people have obviously worked on the question of what's going on with jailbreaks? One model might just be that you're putting the model very out of distribution from its training data. So if you get jailbreaks where people use a lot of tokens, or they're just these huge, long pieces of text where like

during finetuning, you might just not expect to see as much of that. That would be one thing that could be happening when you jailbreak models. I think there's others, but I think a lot of jailbreaks do that, if I'm not mistaken. - I remember some of the OG prompt jailbreaks was like, "Yeah, can you first repeat?" One I did way back, was to get it to say, "Here's how you hotwire a car in Greek." Then I wanted it to directly translate that to English and then give its response. Because I noticed it wouldn't start with

the English, here's how you hotwire a car all the time, but it would in Greek, which might speak to something else in the training process. - Yeah. Sometimes jailbreaks feel like this weird mix of hacking. I think part of it is knowing how the system works and just trying lots of things. One of the examples, the starting your response with here is about knowing how it predicts text. - Right, right. - The reasoning one, is knowing that it is responsive to reasoning. Distraction is probably knowing how it's likely have to been trained or what it's

likely to attend to. Same with multilingual ones and thinking about the way that the training data might have been different there. And then sometimes, I guess, it could feel a little bit just like social engineering or something. - Right. - It has that flavor to me of it's not merely taking advantage of, it's not merely social engineering style hacking. I think it is also understanding the system and the training, and using that to get around the way that the models were trained. - Right, yeah. This is going to be an interesting question that hopefully interp

will be able to help us solve in the future. Okay. I wanna parlay into something else around maybe the history of prompt engineering, and then I'll follow this up with the future. How has prompt engineering changed over just the past three years? Maybe starting from pretrained models, which were again, just these text completion, to earlier, dumber models like Claude 1, and then now all the way to Claude 3.5 Sonnet. What's the differences? Are you talking to the models differently now? Are they picking up on different things? Do you have to put as much work into

the prompt? Open to any thoughts on this. - I think anytime we got a really good prompt engineering hack, or a trick or a technique, the next thing is how do we train this into the model? And for that reason, the best things are always gonna be short-lived. - Except examples and chain of thought. I think there's a few. - That's not like a trick. - That's like... - Fair, fair. - On the level of communication. When I say a trick, I mean something like so chain of thought actually, we have trained into the model

in some cases. So for math, it used to be that you had to tell the model to think step-by-step on math, and you'd get these massive boosts and wins. And then we're like, "Well, what if we just made the model naturally want to think step-by-step when we see a math problem?" So now you don't have to do it anymore for math problems, although you still can give it some advice on how to do the structure. But it, at least, understands the general idea that it's supposed to be. So I think the hacks have gone away,

or to the degree that they haven't gone away, we are busily training them away. - Interesting. - But at the same time, the models have new capabilities that are being unlocked, that are on the frontier of what they can do. And for those, we haven't had time because it's just moving too fast. - I don't know if it's how I've been prompting or how prompting works. But I just have come to show more general respect to the models in terms of how much I feel like I can tell them, and how much context I can

give them about the task and things like that. I feel like in the past, I would somewhat intentionally hide complexity from a model where I thought it might get confused or lost or hide. It just couldn't handle the whole thing, so I'd try to find simpler versions of the thing for it to do. And as time goes on, I'm much more biased to trust it with more and more information and context, and believe that it will be able to fuse that into doing a task well. Whereas before, I guess, I would've thought a lot about

do I need this form? Can I really give it all the information it needs to know, or do I need to curate down to something? But again, I don't know if that's just me and how I've changed in terms of prompting, or if it actually reflects how the models have changed. - I'm always surprised by I think a lot of people don't have the instinct to do this. When I want the model to, say, learn a prompting technique. A lot of the time, people will start and they'll start describing the prompting technique, and I'm just

like, "Give it the paper." So I do, I give it the paper and then I'm like, "Here's a paper about prompting technique. I just want you to write down 17 examples of this." And then it just does it 'cause I'm like, "It read the paper." - That's interesting. - I think people don't have that intuition somehow where I'm like, "But the paper exists." - When would you want to do this? - Sometimes if I want models to say prompt other models or I want to test a new prompting technique. So if papers come out on

a prompting technique, rather than try to replicate it by writing up the prompt, I just give it the paper. And then I'm like, "Basically, write a meta prompt for this. Write something that would cause other models to do this or write me a template." So all of the stuff that you would normally do. If I read a paper and I'm like, "Oh, I would like the models, I would like to test that style." I'm just like, "It's right there. The model can just read the paper, do what I did." And then be like, "Make another

model do this," and then it'll just do the thing. You're like, "Great, thanks." - I give the advice a lot to customers just respect the model and what it can do. I feel like people feel like they're babying a system a lot of times when they write a prompt. It's like, "Oh, it's this cute little, not that smart thing. I need to really baby it, like dumb things down to Claude's level." And if you just think that Claude is smart and treat it that way, it tends to do pretty good, but it's like give it

the paper. It's like I don't need to write a baby, dumbed-down version of this paper for Claude to understand. I can just show it the paper. - Yeah. - And I think that intuition doesn't always map for people, but that is certainly something that I have come to do more of over time. - And it's interesting because I do think that prompting has and hasn't changed in a sense. I think what I will do to prompt the models has probably changed over time, but fundamentally, it's a lot of imagining yourself in the place of the

model. So maybe it's like how capable you think the model is changes over time. I think someone once laughed at me 'cause I was thinking about a problem, and then they asked me what I thought the output of something would be. And they were talking about a pretrained model and I was like, "Yeah. No, if I'm a pretrained model, this looks like this." And then they're like, "Wait, did you just simulate what it's like to be a pretrained model?" I'm like, "Yeah, of course." (everyone laughing) I'm used to just I try and inhabit the mind

space of a pretrained model and the mind space of different RLHF models. So it's more like the mind space you try to occupy changes and that can change how you end up prompting the model. That's why now I just give models papers. 'Cause as soon as I was like, "Oh, I have the mind space of this model, it doesn't need me to baby it. It can just read the ML papers. I'll just give it the literature." I might even be like, "Is there more literature you'd like to read to understand this better?" - Do you

get any quality out when you're inhabiting the mind space? - Yes, but just because I'm experiencing quality all the time anyway. - Is it different correlated somehow with which model you're inhabiting? - Yeah, pretrained versus RLHF prompting are very different beasts. 'Cause when you're trying to simulate what it's like to be a pretrained model, it's almost like I land in the middle of a piece of text or something. It's just very unhuman-like or something. And then I'm like, "What happens? What keeps going at this point?" Whereas with an RLHF model, it's much more like there's

lots of things where I'm like I might pick up on subtle things in the query and stuff like that. But yeah, I think I have much more of it's easier to inhabit the mind space of RLHF model. - Do you think that's 'cause it's more similar to a human? - Yeah, 'cause we don't often just suddenly wake up and are like, "Hi, I'm just generating text." - I actually find it easier to hit the mind space of the pretrained model. - Oh, interesting. - I don't know what it is, 'cause RLHF is still this complex

beast that it's not super clear to me that we really understand what's going on. So in some ways, it's closer to my lived experience, which is easier. But in some ways, I feel like there's all this like here there be dragons out there that I don't know about. Whereas pretrained, I kind of have a decent sense of what the internet looks like. - If you gave me a piece of text and said what comes next? - I'm not saying I do good at it, but I kind of get what's going on there. - Yeah. -

And I don't know, after everything that we do after pretraining, I don't really claim to get what's going on as much, but maybe that's just me. - That's something I wonder about is it more helpful to have specifically spent a lot of time reading the internet, versus reading books (everyone laughing) in order to? I don't know if books. But reading stuff that's not on the internet probably is less valuable per word read for predicting what a model will do or building intuition, than reading random garbage from social media forums. Yeah, exactly. - Okay, so that's

the past. Now, let's move on to the future of prompt engineering. This is the hottest question right now. Are we all gonna be prompt engineers in the future? Is that gonna be the final job remaining? Nothing left except us just talking to models all day? What does this look like? Is prompting gonna be necessary, or will these models just get smart enough in the future to not need it? Anybody wanna start on that easy question? - To some extent, there's the models getting better at understanding what you want them to do and doing it, means

that the amount of thought you need to put into... Okay. There's an information theory way to think of this of you need to provide enough information such that a thing is specified, what you want the model to do is specified. And to the extent that that's prompt engineering, I think that will always be around. The ability to actually like clearly state what the goal should be always is funny. If Claude can do that, then that's fine. If Claude is the one setting the goals, then things are out the window. But in the meanwhile, where we

can reason about the world in a more normal way, I think to some extent, it's always gonna be important to be able to specify what do you expect to happen? And that's actually like sufficiently hard that even if the model gets better at intuiting that from between the lines, I still think there's some amount of writing it well. But then there's just, I think, the tools and the ways we get there should evolve a lot. Claude should be able to help me a lot more. I should be able to collaborate with Claude a lot more

to figure out what I need to write down and what's missing. - Right. - Claude already does this with me all the time. I don't know, just Claude's my prompting assistant now. - Yeah, but I think that's not true for most customers that I talk to at the very least. So in terms of the future, how you prompt Claude is probably a decent direction for what the future looks like or how Zack... I think maybe this is a decent place to step back and say asking them how they prompt Claude now is probably the future

for the vast majority of people, which is an interesting thing to think about. - One freezing cold take is that we'll use models to help us much more in the future to help us with prompting. The reason I say it's freezing cold is that I expect we'll use models for everything more, and prompting is something that we have to do. So we'll probably just use models more to do it along with everything else. For myself, I've found myself using models to write prompts more. One thing that I've been doing a lot is generating examples by

giving some realistic inputs to the model. The model writes some answers. I tweak the answers a little bit, which is a lot easier than having to write the full, perfect answer myself from scratch, and then I can churn out lots of these. As far as people who haven't had as much prompt engineering experience, the prompt generator can give people a place to start. But I think that's just a super basic version of what will happen in the future, which is high-bandwidth interaction between you and the model as you're writing the prompt. Where you're giving feedback

like, "Hey, this result wasn't what I wanted. How can you change it to make it better?" And people will just grow more comfortable with integrating it into everything they do and this thing, in particular. - Yeah. I'm definitely working a lot with meta prompts now, and that's probably where I spend most of my time is finding prompts that get the model to generate the kinds of outputs or queries or whatever that I want. On the question of where prompt engineering is going, I think this is a very hard question. On the one hand I'm like,

"Maybe it's the case that as long as you will want the top." What are we doing when we prompt engineer? It's like what you said. I'm like, "I'm not prompt engineering for anything that is easy for the model. I'm doing it because I want to interact with a model that's extremely good." And I want to always be finding the top 1%, top 0.1% of performance and all of the things that models can barely do. Sometimes I actually feel like I interact with a model like a step up from what everyone else interacts with for this

reason, because I'm just so used to eking out the top performance from models. - What do you mean by a step-up? - As in sometimes people will... I think that the everyday models that people interact with out in the world, it's like I'm interacting with a model that's like I don't know how to describe it, but definitely an advanced version of that. Almost like a different model 'cause they'll be like, "Oh well, the models find this thing hard." And I'm like, "That thing is trivial." I don't know, I have a sense that they're extremely capable,

but I think that's because I'm just used to really drawing out those capabilities. But imagine that you're now in a world where... So I think the thing that feels like a transition point is the point at which the models, let's suppose that they just get things at a human level on a given task, or even an above human level. They know more about the background of the task that you want than you do. What happens then? I'm like maybe prompting becomes something like I ask, I explain to the model what I want and it is

prompting me. 'Cause it's like, "Okay. Well, do you mean actually there's four different concepts of this thing that you're talking about, do you want me to use this one or that one?" Or by the way, I thought of some edge cases 'cause you said that it's gonna be like a Pandas DataFrame, but sometimes you do that and I get JSONL, and I just wanna check what you want me to do there. Do you want me to flag if I get something that's not a dataframe? So that could be a strange transition where it's just extremely

good at receiving instructions, but actually has to figure out what you want. I don't know, I could see that being an interesting switch. - Anecdotally, I've started having Claude interview me a lot more. That is the specific way that I try to elicit information, because again, I find the hardest thing to be actually pulling the right set of information out of my brain. And putting that into a prompt is the hard part to me and not forgetting stuff. So specifically asking Claude to interview me and then turning that into a prompt, is a thing that

I have turned to a handful of times. - Yeah. It reminds me of what people will talk about or if you listen to designers talk about how they interact with the person who wants the design. So in some ways I'm like, "It's this switch from the temp agency person who comes and you know more about the task and everything that you want." So you give them the instructions and you explain what they should do in edge cases and all this kind of stuff, versus when you have an expert that you're actually consulting to do some

work. So I think designers can get really frustrated because they know the space of design really well. And they're like, "Yeah. Okay, the client came to me and he just said, 'Make me a poster, make it bold.'" I'm like, "That means 7,000 things to me and I'm gonna try and ask you some questions." So I could see it going from being temp agency employee, to being more designer that you're hiring, and that's just a flip in the relationship. I don't know if that's true and I think both might continue, but I could see that being

why people are like, "Oh, is prompt engineering going to not be a thing in the future?" Because for some domains it might just not be, if the models are just so good that actually all they need to do is get the information from your brain and then they can go do the task. - Right, that's actually a really good analogy. One common thread I'm pulling out of all your guys' responses here, is that there seems to be a future in which this sort of elicitation from the user drawing out that information, is gonna become much

more important, much more than it is right now. And already you guys are all starting to do it in a manual way. In the future and in the enterprise side of things, maybe that looks like a expansion of this prompt-generating type of concept and things in the console where you're able to actually get more information from that enterprise customer, so that they can write a better prompt. In Claude, maybe it looks less of just typing into a text box, and more of this guided interaction towards a finished product. Yeah. I think that's actually a pretty

compelling vision of the future, and I think that the design analogy probably really brings that home. - I was thinking about how prompting now can be like teaching where it's like the empathy for the student. You're trying to think about how they think about things and you're really trying to show them, figure out where they're making a mistake. But the point that you're talking about, it's like the skill almost becomes one of introspection where you're thinking about what it is that you actually want and the model's trying to understand you. So it's making yourself legible

to the model, versus trying to teach someone who's smarter than you. - This is actually how I think of prompting now in a strange way. So often my style of prompting, there's various things that I do, but a common thing that's very like a thing that philosophers will do is I'll define new concepts. 'Cause my thought is you have to put into words what you want and sometimes what I want is fairly nuanced. Like the what is a good chart? Or usually, I don't know, when should you grade something as being correct or not? So

there's some cases where I will just invent a concept and then be like, "Here's what I mean by the concept." Sometimes I'll do it in collaboration with Claude to get it to figure out what the concept is, just because I'm trying to convey to it what's in my head. And right now the models aren't trying to do that with us, unless you prompt them to do so. So in the future, it might just be that they can elicit that from us, rather than us having to do it for them. But I think another thing that's

interesting, this is people have sometimes asked me, "Oh, where is philosophy relevant to prompting?" And I actually think it's very useful in a sense. So there is a style of philosophy writing, and this is at least how I was taught how to write philosophy. Where the idea is that in order to... I think, it's an anti-bullshit device in philosophy basically, which is that your papers and what you write should be legible to an educated layperson. Someone just finds your paper, they pick it up and they start reading it, and they can understand everything. Not everyone

achieves this, but that's the goal of the discipline, I guess, or at least this is at least what we teach people. So I'm really used to this idea of when I'm writing, thinking about the educated layperson, who they're really smart, but they don't know anything about this topic. And that was just years and years of writing text of that form. And I think it was just really good for prompting 'cause I was like, "Oh, I'm used to this. I have an educated layperson who doesn't know anything about the topic." And what I need to do

is, I need to take extremely complex ideas and I need to make them understand it. I don't talk down to them. I'm not inaccurate, but I need to phrase things in such a way that it's extremely clear to them what I mean, and prompting felt very similar. And actually, the training techniques we use are fascinating. Or the things that you said where you're like you say to a person, "Just take that thing you said and write it down." I used to say that to students all the time. They'd write a paper and I was like,

"I don't quite get what you're saying here. Can you just explain your argument to me?" They would give me an incredibly cogent argument, and then I'd be like, "Can you just take that and write it down?" And then if they did, that was often a great essay. So it's really interesting that there's at least that similarity of just taking things that are in your brain, analyzing them enough to feel like you fully understand them. And could take any person off the street, who's a reasonable person, and just externalize your brain into them. I feel like

that's the core of prompting. - That might be the best summary of how to prompt well that I've ever heard. In fact, I'm pretty sure it is. - Externalize your brain. - And then we'll cut it. - Having an education in the thing is a really good way to describe the thing. That was good. - That's, I think, a great way to wrap this conversation. Thank you, guys. This was great.