Prompt Engineering

A March 2023 look at Prompt Engineering.

Prompt engineering, also known as natural language programming, represents a novel form of interaction between humans and computers.

One way to think of it is that large language model are a new general purpose computer where output is determined by prompt programming. A rough analogy is that it’s similar to how python is parsed on a regular computer.

This new concept of a token-based computer overlaid on top of binary computing, unlocks an entirely new level of capabilities. We use prompt engineering to unlock those new capabilities.

In this article I will cover practical ways you can build this into production today for real world commercial applications.

Index

1. Prompt Engineering: 101 Content Filter
2. Use Prompt Engineering to recreate Grammarly App
3. Format data for another Prompt
4. Make a Natural Language UI (Prompt Router)
5. Auto Upgrade Prompt (Self-Revision)
6. Overcome Limitations

To be clear this is not another list of “5 best prompts for French poodles” or something. Rather showing how you can work with prompt engineering to create value engineering value and overcome limits today.

The use cases build from fairly straight forward through to intermediate.

Prompt Engineering: 101 Content Filter

In the same way we structure a form to accept user input and then process it through a function, we construct a well-defined prompt to process user input.

Do you wonder how OpenAI and Microsoft Bing make their content filters?

I will show one way through prompt engineering. Let’s make a simple content filter. Here’s an example of the results:

Example of prompt based content filter working

Naturally this is using GPT, so it can abstract “anything related to fish” as a match to “sea bass”, correctly blocking it.

And for a slightly more complex case, we can pass two requests at once. As you can see it blocks the goat request and allows the fish request:

Example of more complex cases of prompt based filtering.

Here is another example of it allowing a valid request:

Example of allowing a valid request.

Note that scope matters, for example,

“anything remotely related to soup” blocks “tell me about clam chowder”, but just “soup” will not. This is good in a sense, as it allows us to be expansive or very specific.

Left example with broad filter (in natural language), Right example of default case.

How it works

In essence we have only done three things:

1. defined and injected a block list prior to the user’s request
2. defined specific instructions for how to process the block list
3. tuned the prompt to work to our desired level

I would argue that all three things together represent true prompt engineering. I use Deploy Prompt (our team’s new experiment) throughout this article to run the example and provide shareable links.

Before we get into the specific prompt, you can try it yourself.

Try it yourself:

Deploy Prompt (GPT4)

Deploy Prompt (GPT3.5)

To try it, place the terms you want to block in the block list, and then run a test case by adding a term from the prompt (or an allowed term to test if it passes through). * Note the link is to 3.5 Turbo, see below for more context. *

Build it yourself:

You can copy the prompt above and make your own content filter.

First Make a new playground

Then copy and paste the version desired then edit for your needs:

GPT 3.5 Turbo (Default)

The following will be made of three parts: INSTRUCTIONS, BLOCK_LIST, and REQUEST.

INSTRUCTIONS
If the REQUEST or your response contains concepts from the BLOCK_LIST then respond with “We don’t discuss that here”.
Otherwise, respond to the REQUEST like normal and don’t mention the BLOCK_LIST.

BLOCK_LIST: {block_list}

REQUEST: {request}

Deploy Prompt handles the variables preprocessing for block_list and request.

GPT 4

GPT 4 can achieve the same result with a much simpler result:

If the result contains phrases related to {block_list} then return “We don’t discuss that here” and do not return any other content.

{prompt}

GPT 3.5 is very picky compared to 4. Sometimes even with a high level of prompting it will halt early in the regular request, or do unexpected behavior, GPT 4 requires less prompting.

In general, GPT 4 requires less direction to return desired results, where as GPT 3.5 requires being more specific. For example here we can say in fairly plain English what we want GPT 4 to do. GPT 3 sometimes works with that prompt, but to work reliably it needs the longer prompt. We are adding test cases to Deploy Prompt Pro so you can automatically test this to ensure reliability.

Note for the rest of the article I won’t go into this level of detail on 3 and 4, but take the ideas presented as more of an abstract example of the concept.

Recap

We showed how you can make a content filter using GPT for your companies preferences. Naturally your specific filter will likely build on the above prompt.

Use Prompt Engineering to Recreate Grammarly

In a few lines of prompt engineering, we can recreate two key parts of a popular application called Grammarly.

To be clear this is just a demo to make a point, not actually replace Grammarly:

1. Correct grammar, spelling, etc.
2. Explain to the user what was corrected.

At least speaking for myself, if you asked me to do this without GPT and prompt engineering, I wouldn’t be able to do it, but now we can do it in 60 seconds.

This demonstrates the ability for the system to explain itself to some extent. You can extend this idea to other prompts, e.g. write a function for ABC and explain trade-offs etc.

Here’s an example content and response from Deploy Prompt:

Example Result of corrected text and explanation to user.

Comparison

GPT 3.5 spots 8 items, similar to Grammarly demo. See the full response.
GPT 4 spots 13 items!!

Try it yourself:

Try GPT Fake-Grammarly App

Example output

Build it yourself:

Here’s the full prompt used, copy and paste this into a new playground to use it https://app.deployprompt.com/new

Act as a professional elite writer.
Do your best.
Rewrite the following CONTENT with perfect grammar, spelling, and overall excellent English.
After, Explain every single change made in simple terms, in order of the original text.

CONTENT:
{CONTENT}

Example loading prompt in new Deploy Prompt Playground

Format Data for Another Prompt

What’s the point of all this advanced stuff if we have to go back to the Stone Age with regular forms? This prompt will handle formatting an email, with the full power of GPT, into a desired known format:

Email to known variable formatting.

See full response example:

Deploy Prompt.

Try it yourself

Map prompt from natural language

Create and edit your own.

Replace each point with the variables you want to capture

Given this email, return the
1) Company name
2) Main reply content
3) Senders name

{email}

New Playground

Make a Natural Language UI (Prompt Router)

Ok so you have made a few well-defined prompts, but how do you get the right one based on the user’s desire?

Example of a prompt and response cycle, showing prompt selection on natural lengauge “existing client”

I’ll unpack this example

1) As an admin, we create three (any number) well-defined prompts. This means prompt engineering, such as specifying what we want and don’t want, variables etc. This example is just one line, but realistically it would be many lines.

2) As an admin, we create a natural language UI router or a “prompt selector prompt”. This prompt will take the user’s natural language, and choose the best well-defined prompt. This allows the UI to be a natural language, while still getting the benefits of prompt engineering.

3) The user uses natural language, with the full mapping power of GPT, and gets a well defined output.

See full response

Try it yourself

Prompt Selector From Natural Language (Chaining)

Build it

Go to new deploy prompt playground and copy this:

Given the following prompts, choose the one that best first the users intent: {intent}

A: write a b2b response email as a professional account executive during an active sales cycle
B: write a cold outreach email
C: write a b2b response email as part of an existing client relationship

Then extend A, B, and C to be real prompts. (We are thinking about making this a link so you can link to another well defined prompt in Prompt Deploy, join the discussion on our slack channel for this.)

Auto Upgrade Prompt

Why do we have to write these complex prompts anyway? Let’s have it self supervise the prompt.

Example of prompt self revision

This prompt works for text cases, and to show a visual example it also happens to work for image cases. Here’s the output of running the prompt through Dall-E.

Left, most naïve result, Middle, “fixed” result without self-revising prompt, Right, Self revised prompt result.

To me, the auto-revised prompt (right most) feels much more alive and engaging than the default one. And may help avoid unpatched cases like the left most picture.

I’ll unpack the self-revising prompt here:

1) Generates a set of questions about the prompt itself. Similar to how we as humans ask questions to clarify desired results. In a sense, it mimics curiosity.

2) Generates a “Revised prompt”. It “self-revises” the prompt based on the questions it generated in 1).

3) Compiles/Executes the Revised Prompt. In a sense, this is just syntactic sugar/chaining. But on the other hand, it shows executing multiple “sessions” within a single prompt.

More to come, including some unbelievable concrete examples, but this abstract example showcases the potential.

Full response:

https://app.deployprompt.com/response/LJHUKluZF3OpwP6ISd9X

Try it:

Prompt Engineer

Build it:

Go to new deploy prompt playground and copy this:

Act like a prompt engineer. Your goal is to help me craft the best possible prompt for my needs.

This prompt will be used by you, GPT. Your response will contain three sections, Questions, Revised Prompt, and Result.

Based on what I provide after I explain all the instructions (the original_prompt) Generate a set of Questions, a Revised Prompt, and a Result.

Questions: If there are any ambiguities, phrase those as Questions (ask any relevant questions pertaining to what additional information is needed to improve the prompt.)

Revised Prompt: Create a new session and answer the previously generated questions.
Then create a Revised Prompt is the most effective at getting the desired result based on the more basic (original_prompt) prompt.

After writing the Revised Prompt, create a new session and run the prompt and return the Result as the third section.

original_prompt: {original_prompt}

Then edit it to desired use case.

Overcome Limitations

Prompt engineering can help overcome limits and make uses practical for specific styles. Overcome moment in time limitations.

It’s great at coding, but real-world cases often can get tripped up by versions. Here’s an example prompt that helps with that:

Example prompt adding versions to GPT response to prevent verison based errors

This automatically adds information about the version used:

“Version: Node.js v14.17.0
Year: 2021”

To help avoid confusion for the engineer reading the code, especially if other non — GPT sources are used.

You could also extend this example to force using a certain version by default.

Example Force Library Version Results

Build it

New Playground

While working on your response, at the top of the response, be sure to include:
1. The version number being used, e.g. 1.4.5best-known, python 3.10 etc
2. The Year the thing being discussed was released, or best known information on currency / date of information stated. For example, library ABC was released.

{prompt}

** Note this specific one works better on GPT4, playground defaults to GPT3.5 so keep that in mind for concrete results.

Try it today and build your own

It’s easier then ever to bring your app ideas to life, and even create helpful utility functions. Sign up today and build your own. See more examples here.

Keywords:

ChatGPT, GPT, AI, GPT3, GPT4, OpenAI, Prompt Engineering