“Prompt Design & Engineering for GPT-3”: Ashley Pilipiszyn with OpenAI (Video + Transcript)

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at OpenAI and check out open jobs at our trusted partner companies.
• Watch all Elevate 2021 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!

Transcript

Sukrutha Bhadouria: We will move on to our next talk. Our next talk is going to be given by Ashley. Ashley leads OpenAI’s developer ecosystem and creative application strategy, where she helps accelerate developers and startups build new applications with positive impact. She has also helped lead the launches of OpenAI’s research and commercial products, including Usenet, Jukebox, Rubik’s Cube, Multi-Agent, Image GBT, GPTC API, CLIP and so, oh my goodness. That was a lot. Welcome, Ashley.

Ashley Pilipiszyn: Excellent. Thank you so much for having me. Let me go ahead and share my screen. All right. Excellent. Let me bump this over here.

Ashley Pilipiszyn: Okay, great. Well, thank you everybody for joining this session. I am very excited to walk you through prompt design and engineering with GPT-3. As mentioned, my name’s Ashley and I’m the technical director at OpenAI. So, just a quick introduction here. If you haven’t heard of OpenAI before. So, we are an AI research and deployment company with the mission to ensure that artificial general intelligence benefits all of humanity.

Ashley Pilipiszyn: And what’s unique about us, is we’re actually made up of three distinct pillars focused on engineering startup, research lab, and safety and policy group. And so, a little bit of background here in the lead up to GPT-3. So, nine months ago, we launched our very first commercial product, which was the OpenAI API.

Ashley Pilipiszyn: And this has really become our core platform for accessing our latest AI models. And unlike most AI systems that maybe you’ve interacted with before that are typically designed for one use case, our API actually provides a general purpose text in, text out interface, which I’ll walk you through in a live demo in just a bit.

Ashley Pilipiszyn: And so, this enables our users to try it on virtually any English language task. Since launching, we’ve already seen 200 production-ready applications built using the variety of capabilities that GPT-3 offers. And so, what we’ve seen is actually this incredibly new ecosystem of applications. Spanning things from legal to HR, game development, customer support, productivity, science and education, and both new companies being developed and startups as well as other companies integrating the API. So, a little bit about GPT-3. So, this model doesn’t have a goal or objective other than predicting the next word.

Ashley Pilipiszyn: And so, the key thing to take away here, and this is going to be key as we begin to dive into this prompt design, is it is not programmed to do any specific task. So, this single API can perform as a chat bot. It can perform as a classifier. It could do summarization because at its root level, it’s able to understand what those tasks look like purely from a text perspective. So, really the best way to really… If there’s one thing to take away about GPT-3, it is really just trying to predict the very next word based on all of the previous text it’s seen beforehand.

Ashley Pilipiszyn: So, prompt design and engineering. What do you need to take away here? So, if you have ever played the game charades, this is actually a really great exercise for figuring out how to program with GPT-3. Because what essentially you’re trying to do, again, if it’s just trying to predict the kind of task that you’d like it to perform, you basically want to provide enough context, but not have to give all the information at once. And so, you want to be able to just provide some guidelines about what you’d like GPT-3 to do.

Ashley Pilipiszyn: So, for example, if you want to do classification, want to be able to provide some information about what you’d like done and then maybe a couple of examples. And then try to even provide some counterexamples as well. And so, I’ll show that in just a second. Before we dive in, I just want to highlight some of the settings that are going to come up. There are things called Temperature and Top P. These again, back to thinking about prediction. So, these are not necessarily creativity dials, but they’ll control randomness.

Ashley Pilipiszyn: Another thing we offer is “Best of.” And so, again, GPT-3 in the API is trying to think, “Okay, what is the best response here?” And so, what is the highest average value of the tokens being generated. Frequency, we also… Basically it’s saying, “Okay, we don’t want to repeat what’s already being generated.” And then the Presence setting is also trying to figure out, “Okay, do we want to change topics here and being able to move forward from that?”

Ashley Pilipiszyn: So, we can come back to that, but I’m going to go ahead and move over into… This is the OpenAI beta site. And so, let’s just move this down here. So, this is the Playground setting. So, here on the right hand side, you’re going to see all of these settings that I was just talking about. So, for example, you can determine what the response links will be and to generate with. As I mentioned, this is the Temperature setting. So, we have it currently set to 0.7. So, that’s a pretty standard setting. We also have the Frequency Penalty, the Presence Penalty, and Best of, which I had mentioned. We won’t dive into these just quite yet.

Ashley Pilipiszyn: So, what we have here is what’s known as a prompt library. And what we’ve done is, actually with our developer community, figured out what are some of the best prompts that people are able to get really good results on and what are those settings?

Ashley Pilipiszyn: So, for example, let’s say we want to summarize for a second grader. If you’ve ever received an NDA or any type of legal documents. Actually I, myself, am not a lawyer. And so, many times if I’m reading a legal document, I really don’t know what the essence of that document is really saying. So, actually this prompt, Summarize for a 2nd grader, is really helpful because essentially it is transforming more dense text and simplifying that into maybe how you would explain that to a second grader.

Ashley Pilipiszyn: So, the prompt here. This is actually talking about Jupiter. So, it’s saying that it’s the fifth planet from the sun, the Roman God it’s named after, et cetera. So, again, as I was talking about before, you’re providing the example, so you’re already telling GPT-3 here, “My second grader asked me what this passage means.” You’re already putting that context of putting it into something that a second grader understand, then you’re separating it here. And then you’re actually putting a content that you would like summarized. And then you’re telling GPT-3, okay, you’d like it to be rephrased in plain language a second grader can understand. Here, it will also tell you, “What are some of the ideal settings for a prompt like this.” So, let’s go to Playground.

Ashley Pilipiszyn: And just a second, there we go. So, then all the settings, everything pops up in my Playground setting. And so, here the prompt is, and let me bump this up and let me hit submit. So, “Jupiter is a big ball of gas. It’s the fifth planet from the sun. It’s bright. You can see it in the sky at night. It’s named after the Roman God, Jupiter.” That’s pretty good. It pulled out kind of all the main pieces that we’d want from the prompt and the original text.

Ashley Pilipiszyn: Now, the cool thing here is, too, let’s say you don’t want to use Jupiter… Or figure out more about the solar system, but let’s say you did want a section of a legal document. What you could do is you can just edit these prompts right in your Playground. So, you could delete this and go ahead and delete this as well. And then you could go ahead and copy and paste your own text in there as well, because you’re still retaining those key guidelines. Again, imagine if this is a game of charades or even if you’re working with a coworker and you’re trying to give a set of instructions. So, the key instructions here are asking the second grader–saying, “My second grader asked me what the passage means,” and you want it rephrased. But you can always insert different types of content here.

Ashley Pilipiszyn: So, let’s do another example. So let’s go back to the prompt library. So, a very cool thing we also understand. Remember how I said GPT-3 is focused on text. However, it is able to transform text into emojis. Which actually, thanks to one of our developers who discovered this, we were actually not aware of this capability beforehand. So, if you want to convert a movie title into emoji, you could give some examples. So, Back to the Future might be, you know, boy, man, a car, and a clock. Batman might be a man and a bat. Game of Thrones will be some arrows and some swords. And again, you’ll have the settings on here to get you started.

Ashley Pilipiszyn: So, we can open this up again in Playground. And so, let’s see what we’ll come back for Spider-Man. So, it’s got some spiders, some webs, and that’s pretty good. Let’s see if… What it might come back with if we try it again. All right. So, it looks like it’ll repeat itself on that one. But also, you can begin to combine some of these as well.

Ashley Pilipiszyn: So, you can imagine using chat. So, obviously chat bots are a really popular application. And as I mentioned before, you can think about in customer support scenarios, you can think of in all different types of applications.

Ashley Pilipiszyn:Many of us have already interacted with chat bots before. So, let’s say you want to customize your chat bot. So, the base prompt here is, “This is a following conversation with an AI assistant. The assistant is very helpful, creative, clever, and very friendly.” And so, we’ll begin this dialogue. So saying, “Hello, who are you? I’m an AI created by OpenAI. How can I help you today?” Let’s say, “What movie do you recommend I watch this week?” And we’ll set AI. And submit, oops. My apologies.

Ashley Pilipiszyn: Looking at works of Christopher Nolan. Interstellar, Inception, The Prestige. That is actually a little bit freaky. Christopher Nolan is one of my favorite directors and I love, actually, all three of those movies. So, very spot on actually. But you can begin to actually customize these even more. So for example, let’s say, “The assistant is very creative, clever, very friendly, and an expert on sci-fi.” So, let’s say, “Which books should I add to my reading list?” The Left Hand of Darkness. The Gate to Women’s Country, The Ship Who Sang. Interesting.

Ashley Pilipiszyn: So anyways, you can begin to play around and begin to add that additional context. So, for example, we’ve seen people say, “Okay, this AI chat bot is a science teacher or a bookstore clerk,” and you can begin to actually create these various personas to kind of probe GPT-3, or nudge GPT-3 into the direction, or have that context that you would like it to have. So, let’s do one more.

Ashley Pilipiszyn: So, I mentioned earlier before, Classification. So, you can imagine this being a really useful example. Whether you think of product classification, here is an example of a list of companies and the various categories that they’ll fall into. So, if we open this up in Playground.

Ashley Pilipiszyn: So, again, we’re telling GPT-3, “Okay, Facebook. You want the tags, social media, technology.” LinkedIn will also have that, but maybe enterprise and careers. McDonald’s, you’ve got food, fast food, logistics. And so, this is an opportunity also to create different types of tags. So, let’s see. Logistics transportation. Let’s add… What’s another one. See what comes back for TikTok, social media entertainment. So, that’s pretty good.

Ashley Pilipiszyn: But you can imagine again, applying this to a variety of different products. So, let’s say you’re building a different kind of app for different types of clothing or different types of foods. These kinds of things. And so, you can begin to actually add all of these different capabilities together. So, let’s say for example, the chat bot from the previous example also was able to then help you classify the different products you had in your application.

Ashley Pilipiszyn: And so, as I had shown before for the different startups we’ve seen, et cetera, all the different applications you’re seeing with GPT-3, all boil down to these prompts. And so, your ability to actually help GPT-3 understand, “Okay, what is the end result that you’re trying to get GPT-3 to do,” is really where a lot of interesting things can happen. And so, some of the best applications we’ve seen have been ones where you actually combine these capabilities. So, not just doing a single classification or a single chat bot, but actually being able to integrate those because that’s where GPT strengths lie. As I said before, GPT-3 can do a lot of different things. It’s not programmed to do one or the other, but it actually is very good at, essentially, multitasking.

Ashley Pilipiszyn: So, with that, I wanted to… I’m not sure if any questions have come through, but I wanted to leave a time for just a few questions. But I know this was a very, very rapid fire, deep dive into prompt design and engineering. If A), you have any questions, please feel free to email me. If you are interested in getting access to GPT-3 or building an app or product with GPT-3, again, please email me. I’d be delighted to discuss and very excited to have more people join our developer ecosystem and build with GPT-3. So, thank you so much. And I’d be happy to take any questions with the remaining time.

Angie Chang: There’s some questions in the chat. Most of them were like, “How do we get access to GPT-3?”

Ashley Pilipiszyn: Okay.

Angie Chang: You just answered that question, but if you would like to look in the chat, there’s a question about how OpenAI overcame bias about, for example, food suggestions, American versus Western food, or summarizing New York Times, Wall Street Journal, short article or headline. Let’s see if you can answer in three minutes.

Ashley Pilipiszyn: Okay, awesome. So, and I can not see the exact question, but I think… So, on the question of bias. So, excellent question. It is a, first of all, a very big industry-wide issue at OpenAI, especially we’re really focused on addressing this. Especially with our safety and policy work.

Ashley Pilipiszyn: Actually, I highly recommend checking out if you haven’t last week, we released a new research release about multimodal neurons in our latest clip model, which is our most powerful vision model.

Ashley Pilipiszyn: And the reason I bring this up is because, this is kind of demystifying what’s happening underneath the hood with these AI models. Because obviously, these models are trained on all of the internet. And so, they’re basically integrating what they’re learning from us on the internet. And so, what this multimodal analysis allows us to do, is actually peek under the hood and understand, “Okay, so how are these associations being made?”

Ashley Pilipiszyn: And this allows us to figure out, “Okay, then how can we begin to address these,” by identifying where these associations are happening. And so, this is really borrowing a lot from neuroscience. So, but to address bias in the case of prompt design and engineering.

Ashley Pilipiszyn: There is an opportunity actually to address some of this in text form as well. And so, whether it’s modifying your prompts. So, I think the example was for like foods or recipes, being able to provide a little bit more context to be able to help nudge where you’d like GPT-3 to go. And this actually will help with giving examples as well.

Ashley Pilipiszyn: So, actually one quick example that might help address this is… Question/answer. So quickly, what you can do in a situation like this is you also can provide, for example, a question that’s rooted in truth. I would get the answer. If you ask me a question that’s nonsense, or it doesn’t have a clear answer, I’ll respond with unknown. And so, you can also provide facts or essentially give those examples of how you’d like GPT-3 to respond. So, that’s another way again, is through that prompt as well.

Ashley Pilipiszyn: And then the second question… Angie, I forget what was the second question on?

Angie Chang: It was on headlines, for summarizing media company headlines.

Ashley Pilipiszyn: Oh yes, yes. So, summarization, I guess more broadly. So, GPT-3 is excellent at summarizing. Actually, it can do data parsing and summarization. And so, if I’m understanding the question correctly, could you take a variety of headlines and then summarize a bunch of different headlines and what’s the TLDR main takeaway from that? GPT-3 would be very good at that. Pretty much summarizing, again any text, it will be quite strong at.

Angie Chang: Great. Thank you so much, Ashley. That’s all the time we have today. I know people will be definitely signing up to join the GPT-3 beta and trying it out. And thank you for leaving your contact information on the slide..

Ashley Pilipiszyn: Yes.

Angie Chang: Where you can get in touch with Ashley directly.

Like what you see here? Our mission-aligned Girl Geek X partners are hiring!

• See open jobs at OpenAI and check out open jobs at our trusted partner companies.
• Watch all Elevate 2021 conference video replays!
• Does your company want to sponsor a Girl Geek Dinner? Talk to us!