ChatGPT used for Dungeons & Dragons-like game called QuestGPT! Techniques: reflection, step by step

Welcome to the Prompt Engineering Podcast, where we teach you the art of writing effective prompts for AI systems like ChatGPT, mid Journey Dolly, and More. Each week we explore prompting techniques, interviews with experts and newbies, and tips on selling your prompts. Here's your host, Greg Schwartz.

Welcome to the Prompt Engineering Podcast. I'm your host, Greg Schwartz. So we have a guest with us today. Go ahead and introduce yourself.

You'll notice Bruno is like the very technical one, and I'm very much the touchy feely one with prompt engineering. And it's two different approaches, but I'll try my best to describe it and Bruno can then explain what the heck I'm trying to say essentially.

Yeah, I feel like sometimes there's this prompt design like part, there's also this application part, and sometimes just the application is just more interesting than the prompting techniques. 

So I I have two individuals from QuestGPT, and I met them at EthDenver, which was this very fun and exciting conference and hackathon and great experience except for the fact that the wifi basically never worked. So it was hard to build anything at the event, which was a very weird experience for a hackathon.

But anyway, we don't need you to talk about that. ahead and introduce yourselves.

I'm Adam Boyle. Nice to meet everyone. Nice to meet you, Greg. I met Bruno at Eve Denver at the Hackathon. I have a background in solution engineering. I'm essentially a demo guy. I'm a highly paid Vanna White. I say click this button, do this button. I've been selling sort of AI machine learning software for a long.

And this ChatGPT stuff has just changed the game across the board and just building cool stuff now with Bruno. Nice to meet you all.

I am Bruno. I have a background in neuroscience and I've been doing software development for quite some time now. C T O at the community one where we developer products for community management. I'm working with Adam now. It's nice to be here.

Awesome. So Adam and Bruno, how how did you first learn about ChatGPT and prompt engineering?

I must have heard it through the internet just going nuts. I was definitely late to it Overall. Just slowly but surely, my YouTube started to get full with how to create side hustles with ChatGPT cause I'm a little bit of a side hustler myself.

I had a a best man speech I had to write, so I had a wedding coming up. And I was like, oh, maybe this is, this could be useful. And that was my first for Foray into ChatGPT. I was just trying to write a speech and it took while and I really test the limits, but that's how I got familiar with it and that's when I really started to learn what it was all about.

Oh, that's awesome that is that is the first speech, particularly best man speech, but even just speech, that I've heard of being written with ChatGPT. So I have 

I have to ask then: how did it go?

it went really well. It was, at first it was for fun, it was like, okay, write me a best man speech. And I was blown away by actual just how good of a decent speech it was. But then I was curious, what this thing was capable of.

So I was, I told ChatGPT make it a little, make it subtly obvious that I'm in love with the bride. And then ChatGPT was like, started to show some jealousy and I'm like, make it more. Make it a little bit more angry. And, slowly but surely, you learned how to interact with ChatGPT. And it was like, you don't deserve her.

Really? You don't? I do. And then I really pushed the boundaries and I was like, write it in Trump's voice. Trump's giving the speech,

Oh wow.

And then I was, I wanted to break it. I was like, write it as if you were a Tyrannosaurus Rex and you're hungry. And it was still bringing in jokes where it's I'm hungry and I bite when I'm hungry.

Just kidding. Ha. So that's rarely where I saw how far he could stretch this thing. And then I used it to even write a couple jokes. That went really well. I was pretty surprised. There's one particular joke at, I have to give ChatGPT the credit for it, but it said, Mike, I'm not gonna say that you're the best guy ever, but every day are you trying to become a better person than you were the day.

Like also, no, you're not all, you know also. No. And it was pretty good and and it really helped. It really did. But I used ChatGPT, I used some other things online and then I did honestly write a little bit of myself. But it was useful. I've been using it ever since.

That's amazing. 

Wow. Bruno, how about you?

Yeah, it's such a cool story. I 

That's okay. 

this. I basically been using I've been using it for summarizing papers and trying to do a lot of things with much less time. 

Cool. Okay. Yeah, that that, that's definitely gonna be a hard story to beat. I don't think I've heard one that good before. So well done, 

Thank you. Yeah, I was just lazy. It was a meaningful moment in my life and I didn't wanna do it myself. Technology, ai. 

So we're building a game master a storyteller that can develop interesting, engaging and fun story for people to play with. A lot of people want to get like into games like Dungeons and Dragons

Especially with the movie that just came out. 

Yeah. And but there's so much prep to do and sometimes people just don't wanna do it.

And if you're not in interested in being the storyteller, then it's hard to find a storyteller. So we're trying to like fill this void. Especially for people who want to like solar role play. That's also like a very untapped market. But also like for multiplayer experiences.

So that's what the QuestGPT is. One of the pitfalls that can sometimes encounter when building like these conversations, these stories because it's a loop thing where we also take the, player has said in the past as part of our prompt. Is that the AI may learn patterns that you don't want it to learn during the conversations, so you have to take steps to mitigate that happening.

Look for patterns that you do not want or constrain it so that it'll only 

keep the ones that you really want 

What's an example of a a pattern that you wouldn't want else. either in QuestGPT or or something else?

For example once we had a we had a hard time debugging this because we thought this was a bug, basically the way we were designing this prompt sometimes the DM thought was talking to. The, I thought it was talking to itself when generating something, sometimes we would just ask it something.

I repeat the same 

I think you were here

like, uh, during some of these tests.

Oh yes. I do remember seeing that in some of the play tests. , I remember one the was walk walk in the room And it says like you walk in the room, and that was it. Yeah, what happened? 

and I noticed it getting almost worse or, I see what you mean by re reinforcing these patterns. Cuz it would happen once and it wouldn't be a big deal and it would just be like maybe a blip or we thought it was hallucination or I thought it was a bug as well, but then it would happen more and more often.

But that was really just ChatGPT or the G P T model teaching itself that, oh, this is acceptable, let's keep doing it essentially.

Stuck. 

Wow. It got stuck in kind of the wrong thing. That's so interesting. 

So we basically the way we feed the data, we have to change it. So we made impossible this kind of patterns to happen and we. So when you talk to the dm, you only see one agent interacting with you, but there's more than that behind it, right? So we're also looking for patterns that we want to exclude from its memory, so it, it doesn't reinforce themselves. 

I find that sort of one of the more just larger, broad pitfalls of ChatGPT and these models is that the second it gets a bad idea in its. It's like really hard to get it out. If I include one bad sentence, one bad thing about the thing I'm trying to describe it will poison the well, so to speak for all the responses soon after.

So there is this concept of like N shot learning, right? When you want the AI to do a specific thing, give it examples . Ideally you don't need to give it any example. That's zero shot, right? And the number of examples is the N.

So like one shot, one example, two shot, two examples, like to run of the, of a task. And you just add more examples. The more you want to constrain or make sure that the AI. Has a very well-defined idea of what it's supposed to do and how it's supposed to do it. So if you introduce like bad patterns, it also treats them as like an one shot N shot.

So you end up reinforcing the wrong thing and you need to be very careful, especially if you're like into conversations kind of thing. Storytelling you don't want to, you want to make a very conscious effort to mitigate these 

bad example. 

Yeah, interesting. for the listener, just so you have some context on this, this 

So an example that would be 

So an example of that would be, 

just say, 

you don't provide an example. So 

Is

I don't know what color is a robin's simple examples. So we might say a single example.

I

might

what kinda restaurant what kind of restaurant should I go eat 

food. 

on the next line, you could say Italian 

food. 

And then after that you could say, I'm hungry for lunch. What kind of restaurant should I go? and so it'll say, I have sort of this call and response of, you know, a question about where should I go eat?

And then a response of, here's a cuisine. You just asked me another question in that same vein. So let me give you another kind of cuisine. It's pretty smart at picking up patterns. Few shot or end shot, there's a bunch of different names for it, is when you provide more than one example did this with together, release that I put and so prompt that's gonna selling uh, this prompt that someone is selling this prompt someone is selling is job Searching. This type is, its type. Uh, entertainment and, you know, whatever the case may be. And then I could feed you a list and say, here are 40 more prompt titles.

Tell me the type for each one. And it would return that because the training that was done with this few shot prompting. 

It's amazing, like the amount of effort that would've with, you know, traditional machine learning versus here are three examples. You're fine. We'll figure it out. 

I'd have to look at the exact number that I did for that analysis, but it was 5,200 And 

sit down. 

And 

50, 

200 different 

50, different

mm-hmm. amazing From the The 

best cover letter

restaurant picker for deciding

restaurant picker for deciding where you're gonna take your spouse, you know, whatever they are. 5,000 of them. 

honestly 

ChatGPT did 

it. 

Honestly, 

I had 

to build an 

long 

I had to build an API call for it

mm-hmm

to do an like a single go in the chat. 

techniques that 

Are there other 

techniques 

that 

you all have used? 

I think we're doing something really unique with Quest G P T, and this is where, you know, Bruno being. Bian, he's doing this. If he wasn't doing this, he'd be building like Android bodies. Cuz I don't see a lot of use cases right now where, you're trying to use ChatGPT to.

Interact with it in a storytelling mode and not have it, almost not have it learned too much. It's very interesting. It goes against a lot of the other use cases. But I'm finding a tremendous amount of use around just speeding up traditional work for me, it's just in my regular life has been just incredibly powerful.

And just, even if it's you, Getting me 80% of the way there on my tasks versus, maybe it's not perfect right now. And if it's not perfect, the response, it's because my prompt's not perfect. But that's personally where I've been growing a lot lately. 

There's no distinction between work fun and like side hobby. Awesome. like everything I do, like for fun, I end up doing it for work basically, and vice versa. So for example I'm doing prompts where, so I have I usually rather than like work on individual prompts, I try to set up individual systems. For example, one I have is that I'm, I have a YouTube addiction problem and I end up like building playlists and playlists of specific things I wanna learn, but I sometimes just don't have enough time for that. So I'm like batch downloading the playlists' audio, transcribing it, and giving it to like the AI 

so I can just read it 

instead of just putting like the video at 1.5 times speed, I can just read it. There's no redundancy. It's much more efficient and I end up like learning some, like many things that I end up using here. I do that for to try to be up to date with the 

Yeah. 

prompting, 

techniques also.

And other, the same like for reading like papers, sometimes books.

Uh, but books harder because of The length. 

Interesting. Making sure I'm understanding you are. Pulling down the audio using a transcription. And then are you using G P T to do a summarization 

I'm pulling the audio. I'm transcribing it, then I'm like feeding it to the API 

Very cool. 

Usually I just I have the text, so I just copy to the web ui. I'm paying for the ChatGPT Plus thing. And usually I just wanna learn like very specific things, so I just write them there. 

some of these things, let's talk about the prompts that you, the prompts that you gave me. 

you know, it's this now classic use case of telling ChatGPT, like who they are. But I've been experimenting with giving them sort of two hats at once.

Cause I'm often wearing two, three hats. Why can't ChatGPT also be multiple people at once as well? So in this particular prompt, I'm telling them that they are a graphic designer and they're a social media manager generating images to go along with your Twitter campaign.

And then I had to explain what mid journey was. I. With the model I was using, it may not be so up to date, but it helped by saying, mid Journey is an app that can generate AI art from short prompts that describe simple images. I found it helpful this way. It was thinking like a social media manager.

It's gonna write me tweets like a social media manager, but it's also graphic designer, so it's thinking about how to describe images generate. Because my goal for this prompt was for, it's not only write tweets for me, but write prompts for mid journey that would correspond to the tweets. And it could be very basic, but.

If the tweet's about communities get together, give me a tweet, you don't, if the tweet's positive, gimme a positive image. You if the tweet is, angry, gimme an angry image, whatever it is just as a way to speed things along. And you'll notice in the bottom part of the tweet, I'm like 10 tweets based off of the following topic.

I just dumped a bunch of, this is a lazy prompt, but I just dumped a bunch of texts I had laying around that I had written for the website and other things that were attention grabbing, just a lot of the best all at once. And essentially just having a larger sort of topic to pull from.

It gives me a larger range of responses and so I'll often ask for 10 tweets. I'll often. Follow up this prompt with, Hey, make 70% of them educational, third percent of them humorous. And I'll also feed it my specific mid journey prompt image, my design language that I'm trying to figure out for branding purposes.

That's all

uh,

I'm like in love with pixel art right now and with Quest, g p t the Quest G p T Twitter page. We're writing with that pixel art right now.

So I'm essentially, You know, I did a lot of work to of a prompt that would put out things know, one outta four times at least gives me a very good image using Mid Journey. 

And so essentially it'll put out a long set of columns and then I'll copy and paste that into Google Sheets, and then kind of off to the races.

I could just copy and paste the one cell directly into Mid Journey. It has the prompt language, everything. And then once I'll I'll also follow up with, write me five more, but make it about how AI can be ethical or uh, good. Or write me five more about how. It could add to your bottom line or improve roi.

So it has its main mission. It's familiar with uh, the gist, but I can pepper in, specific and steer it and get some more stylized tweets out of it. So my goal is really to have this stuff laying around, use a ui. know, I can just go to it, type of my language in my ui.

Like, Okay, this week's tweets, we'll have a bent based off of building community or based off know, it will AI kill and then I just type that in and, know, the, my prompt will do the rest and it'll give me a good amount of tweets that I could just copy and paste and go on with my day essentially, and focus on more important things.

Focus on that human interaction. Just writing ad copy essentially.

That's awesome. 

Okay, so for the listeners who can. This, let me just read this off. 

Pretend you are a graphic designer and a social marketing media manager generating creative images to go with your Twitter campaign tweets using an AI app called Mid Journey, and then it goes on to describe Mid Journey and then.

Twitter tweets for the next five days based on the following topic. Quote, AI R P G game Masters similar to d and d. Dive into the world of custom AI R P G Game Masters for your Web three community. Engage your members like never before with tailored Adventures, and then some emoji in a hashtag, and then at the end is an actual mid journey prompt.

So slash Imagine prompt colon, digital dungeon 

The. 

sitting behind a table with a Web three community logo. Style of Owl boy pixel art, 8K, resolution 3000, et cetera, et cetera. I don't need to keep going. 

AK Pixel Art. I'm very proud of that one. Uh, 

pixel 

art. 

yep. What, what? Works, works. Don't ask questions. 

That's one of the things that I have been really interested in investigating is. Particularly with the, the, the role-playing technique, which, sorry, listeners by the way. That's what that's called. The pretend you are a graphic designer or a social media manager, or both. That's called role-playing. also called Act As. lots of different names for

things. ChatGPT

telling to pretend they are that person or to act as that kind of person. The community is found, you get massively better results. people are playing with, you know, nuances like a Euro graphic designer with 10 

years of experience about, layout and colors and whatever the case may be. And so people are trying to figure out like, what makes a difference? Why does it make a difference? How do we use this the yeah, It's so it's so interesting to me the tweaks you can try to get into the sort of psychology in air quotes of ChatGPT. 

Yeah, and I think there's also this regenerative aspect where. Sometimes it doesn't give me exactly what I want, but I'll try a couple times and then when it, where it's hitting my vibe, that's when I'll say, hit me with five more tweets. Hit me with five more tweets based off of this.

It figured it out at that moment. Awesome. Let's, Let's keep the party going. But yeah, it a certain amount of unpredictability uh, like you're saying, a small tweak even over how it's feeling that day. know, It's, you don't even have to change the It's it's interesting.

That actually brings up, uh, one of the parameters, and I'm 

curious if 

you all are using it: temperature and top p, which change how random and how creative an air quotes, um, Chachi PT is. Are those things you're using at all in Quest, G P T, or any of your back ends? Because obviously, you know, unless you're using playground, you can't actually use, change that value. 

yeah,

We're using all of them and even a bit more, so temperature is very important for us. So for example, we have different kind of kinds of products. One is QuestGPT, we also have this one where we, it's basically like an FAQ bot right now. So it's used inform people about certain things, right?

So for us, temperature is like one of the knobs we can tune to say, okay, like how engaging do we want it to be, but also like how correct and predictable do we want it to be. So if we want to inform people, we want to be like, predictable, maybe not as boring. So we won't go like temperature zero of course. But for, in the case of Quest, g p t, we want different temperatures for different things. For example, one of the things we do is we have this like character creation menu where we, dynamically generate descriptions of of your character based on some like personality test basically.

And, for that we wanted to be like 

funny and to be like, oh, So 

we want that with like higher temperature for that. when we are storytelling, course we want to be engaging, but if you like get it to be too hot, like it'll lose its purpose at some point. Like it'll forget what it's trying to do.

we want to not let that to be too hot. Then there's other kind of 

parameters, ones

For 

like how repetitive it is. Like how it maintains itself. you want it to be like, to imagine new things or do you want to just specifically stay on a course?

When we were first uh, working on quests, g p t at East Denver, know, it's, you want the, you want ChatGPT to the dungeon master hat on. But you them to have a certain amount of creativity in the storytelling. So it was like quite a balancing act as far as, know, Tell a wonderful story, like let this story go a little crazy, but don't ever forget who you are ever. You know? Cause it'll ruin the experience essentially. It was pretty funny. We we saw some wild things happening. I think that's one of the more fun parts of experimenting with this stuff uh, the temperature setting.

What what a lovely thing that they included of the SDK allow us to play with 

Yeah, definitely. And I'm curious, you mentioned that uh, quest, g p t, actually, even though it appears to have one agent, it's actually multiple. Is that part of the reason you have multiple agents? So one agent can have a high temperature, 

So it depends on what you define as an agent. we like to think about agents as the entity that does this specific thing, but also holds this specific set of. So if you're giving it a different set of memories and it has a different purpose, yeah, that's a different agent.

But if you're just like it's kind of part of the same thing, then it could add as well be called the same agent. But yeah. But but I feel like it could be like the same DM that's making up your description, your character flavor text,

when we are generating this completion, but for, as for the storytelling, it could, it will be called the same agent, but we are like having a different set of parameters for this. 

Yeah, I noticed that during the character creation it would be like strange quirks about the character, like the characters also afraid of good to go, like you're a strong know, you have all this stuff. Also you're afraid of spiders, and I love that's that's a lot of fun.

Yeah, I really enjoyed that nuance of it you were playing the, the sort of, you know, d and d experience, that was a lot 

of 

fun to see. 

Yeah, it's coming along. 

Uh, We're learning a lot. It's super fun. It was a problem when we kind of got it working cuz all we wanted to do was play with it. 

Yeah. 

it might be uh, mentioning the prompt engineering around the N f T generation,

Bruno, 

cause there was a lot of temperature there as well, or cajoling as far as. Getting the right prompts, then getting the right art style.

if you know about like, AI generation, for example, mid Journey is much better and it's much sim easier to talk to it. So you have to like, take into account if you're linking multiple ai, of course you need to change your prompt, but also change like your parameters. We, we kind of do is like, okay, we wanted to be this kind of description. We want to focus on this set of things, 

One other thing with I think it's one thing to get like predictable responses out of ChatGPT, but it's whole nother thing to get predictable art out uh, an AI art generator. And for us to consistently. Healthy, good, fun looking.

NFTs uh, I think was one of the most impressive things we during that time. 

By far I have played some of these and yeah. Two of the examples that I one of my play tests. I was running around with, of course, with, 

The Superman 

I was like, ok, fine. 

My character has heat 

vision 

and the N F T 

up 

and I think it 

had like red lines 

Awesome. That,

And then another one where 

somebody 

stabbed 

a 

with a 

a sword, and the

NFT actually show

that specific

action. in part of what amazed me was that was not, generate the N F T right after doing that. It I, I think it was like 15, 20 minutes later that then the generate n f T command was run. Was that? 

Yeah, we, 

the way we set it up, and I forget the exact language, but we wanted it to pull the coolest, most emotionally impactful moments. And that particular moment of, I said, uh, cause that was my n ft. I stabbed the dragon, like threw the ear into a brain killing it. I guess that was enough. Shay to probably a little too hardcore, but made a very cool N F T. I love it. Yeah.

Yeah, there's like different elements in that go into like, um, generating the NFT one is like recognizing the cool. One is like building the prompt for like the nft and one that's like

getting the, the actual image, 

I am curious, 

can 

you 

us 

more about how you did the find the most interesting part of the game prompt. 

kind of part of like a 

secret sauce things

we have like of asking for, certain things You know, like there, there's always a techniques coming out um, you can see like just week there, were like, two new, like great techniques out. like once the reflection that like you basically tell the, an AI to 

reflect on its own performance 

and then there's self-critique and improvement. You tell it what do you think you

did wrong? 

Like, how could you do it better? And it improves itself its own output. We're always like changing the prompts we use.

That's just the published ones, a lot of people are working on so many things that they're not publishing and they probably have like way better like, performance, what like the public stuff is. we ourselves have like, had better stuff than what's known, and we're, kind of, there's incentive, like not to share it.

We'll probably share it at some point, but yeah, like there's so

So 

many.

that not so many people that know some other ways of talking to it 

up a lot less. So it's, it's interesting, you know, everyone's learning 

coming from like a neuroscience background, can't see like a big pattern on, on whatever the everyone's trying to do with this techniques. So like one of the main limitations G P T models. It's not just like it's attention window, but it lacks things that we as human have. So like, like planning, they, they like these mental boys that, um, we normally have to plan and execute things and predict things and like tweak them before we even do them. So lot of these techniques are basically a way to compensate for that. For example, chain of thought is a way of telling it the ai, okay, do this, but like go step by. Um, don't just like come out with the output, like if you are out, like how you it step by. step and build up upon it's doing the job of like a mental voice if you think about it, of, of the planning and executing. And all of these are basically trying to achieve the same thing. They're working with this limitation. that's like a great approach or, or a gateway to thi to think. When you're building your prompts, think about its limitations, the things it lacks, and, uh, try to out like in what way is this affecting my output and how can compensate for that, telling it to do this other thing, uh, that can self preference from. 

Yeah, and just to really call that technique out, that 

basically 

say, 

know, do math or give me a or whatever the the thing you want is, and then you say, Think step by step or explain it to me step by step, for whatever reason, that brings in a much higher level of fidelity in both math problems and logic problems and all kinds of different stuff. Really 

uh, that's what we are asking the like come up with the, the answer right now. but sometimes like things are so complex that you need to break it, break them down, and that's what we do on our heads. uh, all the g t model has is it, is its attention. you have to put all the, all the, the mental like workflow into its attention window.

Yeah, that's one of the. hacks basically that I've seen for, making math problems actually get calculated correctly was teach it to do

Multiplication.

by doing each decimal place individually, and then add them 

all 

up. 

And 

apparently 

I 

have tested. 

but apparently that does work.

But you can get just as good of output if you just say, think step by step or calculate step by step 

Yep. 

What problems do you wish you could solve with generative ai? Could be text, could be images, whatever the case may be, but you haven't figured out a way to do that yet. 

Uh, That's a great question. I, I'm still using multiple tools. There's still an input I miss Clippy. You know, Clippy used to live on all the Microsoft things like Clippy needs to come back, Essent. I'm doing, 

like you're writing a letter. Would you like me to interrupt 

you? 

And that, 

what was it even trying to accomplish back then? But at least nowadays on all my apps, on all my systems and all my things, I need this thing everywhere. Right now it's driving me, not that I have to hop around. 

Yeah,

Sometimes it's more important to think like like problems and the que I feel like the big question a lot of people are missing is like, do this using AI in the first place? it looks like it, it's like the saying, like, when all you have is a hammer, everything 

Yep.

That that's something like we should keep in mind. like, a lot of people are trying to solve many things with ai. We don't need AI for a lot of problems. instead we should Like build interfaces with ai. so it can do the things using these other tools. 

So where can people follow you and, hear about Quest, G p t, and all the other awesome things that you're working on? 

The best place to follow us is our community. One Twitter. So at community you know, io. We also have a Quest G P T Twitter page as well. Uh, Both of those Twitter pages have links to our Discord. You can go to Quest gpt.io and play and also there's community one.io. I'm at Adam J. Boyle on Twitter. You could find me there. 

I, I just hide. I just lurk. So you don't need to look for me. 

Well, 

thank you both for being on the show! 

thank you.

It was nice being here.

thanks for coming to the prompt engineering podcast, the podcast dedicated to helping you be a better prompt engineer. Episodes are released every Wednesday. I also host weekly masterminds where you can collaborate with me and 50 other people live on zoom. To improve your prompts. Join us at PromptEngineeringMastermind.com for the schedule of the upcoming masterminds. 

Finally, please remember to like, and subscribe. If you're listening to the audio podcast, rate us five stars that helps us teach more people. And if you're listening to the podcast, you might want to join us on YouTube so you can actually see the prompts. You can do that by going to youtube.com/@PromptEngineerinPodcast. See you next week.