Crafting textual input to interact with Large Language Models (LLM) using Prompt Engineering

7 min readNov 11, 2023

Introduction

The Artificial Intelligence (AI) hype is all over the world and with the rise of new AI models, one would only see its adoption rate accelerate in coming years. These models are built on billions of parameters, trained on massive amounts of data, tweaked for various jobs and tailored to provide quick answers to domain specific queries, extremely beneficial to business and organizations.

Gartner Hype Cycle

According to the Gartner hype cycle, there are four types of AI development that would affect people and processes both inside and outside of an enterprise setting.

Data Centric — AI in this segment would enhance and enrich the data, which would be used to train the algorithms.
Model Centric — Highly on rise, the model centric AI would help add domain-knowledge to bootstrap causal AI models using domain specific datasets.
Application-Centric — AI Innovation here would add value to the existing application which includes decision intelligence AI, operational AI systems or using AI models such as natural language processing (NLP) to enrich an application.
Human Centric AI — This group of innovations includes AI Trust, Risk and Security Management (TRiSM), responsible AI, digital ethics, AI maker and teaching kits.

Courtesy: Gartner Hype cycle, gartner.com

In this article, we talk about Model centric AI such as “Generative AI models” and the approach to interact with them to get the best possible outputs. The approach is essentially the same, to obtain the best results from any particular AI model.

Generative AI

Generative AI is the technology that leverages AI models to generate data (content) such as text, images, audio, videos and even the program code. To put AI models to work, we as human need to interact with them. These interactions are crucial to get the best output, by knowing how to ask the right questions in the right way.

One could relate this to the Database world, where one needs to write queries (equivalent to prompt inputs) to fetch data from the database (equivalent to model). The only catch here is that the prompt input (human interaction) are in random order in plain English text, as compared to strict database-syntax.

The interaction part is where the “Prompt engineering” takes his place. Prompt engineering takes human inputs, which gets interpreted by Large Language Models (LLM) and returns with smart answers after processing through underneath AI models. These LLMs have defined a new paradigm of computing, hence LLMs need to be prompted effectively to produce the desired output.

Prompt Engineering

Prompt engineering focuses on crafting the optimal textual input to interact with the language models such as BERT, GPT-4 etc. This is done by selecting appropriate words, phrases, sentence structures, and punctuation. To better align human input to these models, one must comprehend how these models are trained and how they generate responses.

Next to the technical approach one must also give attention to the problem domain and indulge into problem formulation approach. Combining both approaches, as explained below would be effective to get the most out of Prompt engineering. In this article, we would only skim the surface of Prompt engineering, leaving room for a high level understanding of the specifics.

Designing Prompts

Approach 1: Understand and Interact with AI models

A key aspect of a prompt engineer’s role is the iterative process of testing and refining prompts based on the model’s responses. This often requires multiple cycles of adjustments to guide the AI system toward generating the best possible response. The best AI models are those, which are built on Human Feedback as well, hence the constant flow of feedback and enriching the Models and prompts is the significant process forward for better outcomes.

Models

Prompt engineering plays a vital role in interacting with the following models:

Language models like GPT-3, Jurassic models, GPT-J
Text to image models, like Dally, Mid-Journey, stable diffusion
BERT from google, which helps Google better understand the context around your searches.

In order for the AI models to understand the text input, it is necessary to arrange and use certain inputs, as explained below.

1. Textual input - Should have minimum of two or all of the following inputs, as mentioned below.

2. Prompt techniques - As of now, the Prompt engineering revolves around 3 best known techniques.

N-shot prompting (Zero-shot prompting, Few-shot prompting)
Chain-of-Thought (CoT) prompting
Generated knowledge prompting

The AI model would interact and understand the textual inputs at his best by using above techniques. One could streamline and construct prompt templates, which then be programmatically altered based on the context or dataset. Below are some of the tools, which uses above mentioned techniques, help developers and product teams to build high-performing applications on top of the Large Language Models (LLM).

Scale Spellbook
Humanloop
Promptable
Dust
Vellum

To build a powerful AI system, multiple LLMs are often chained together to deliver the most value.

Tip:
In Prompt engineering, the human inputs (words) are split into 
single word called token, which are processed more efficiently 
than words or sentences. The numeric representation of word is 
called tokenization. It is this small unit (token), which the 
AI model uses to understand user input. 

There are 2 types of tokens. The textual human input is 
the "prompt token", which one enters into the model using 
the prompt. The "sampled token" are the tokens, which the 
model generates in response to the human input.

Approach 2: Problem formulation

Another important aspect in this space is problem formulation. AI is a fast moving eco-system and according to Harvard Business Review (HBR), there comes a time when new AI language models itself like GPT4, would start rendering magic prompts, which could make developing “Prompts” obsolete.

According to the experts, the crux lies in a comprehensive understanding of the problem domain and ability to distil real-world issues. And it is obvious that without a well-formulated problem, even the most sophisticated prompts would fall short, as per HBR.

But once the problem is clearly defined, the linguistics nuances of a prompt become tangential to the solution (Approach 1). Hence, it is significant to have both approaches in place to get the most out of Generative AI models.

Experts have identified four key components for effective problem formulation:

Problem diagnosis — Problem diagnosis is about identifying the core problem for AI to solve.
Decomposition — Problem decomposition entails breaking down complex problems into smaller, manageable sub-problems. This is particularly important when you are tackling multifaceted problems, which are often too convoluted to generate useful solutions.
Reframing — Problem reframing involves changing the perspective from which a problem is viewed enabling alternative interpretations. By reframing a problem in various ways, you can guide AI to broaden the scope of potential solutions, which would help you find optimal solutions and overcome creative roadblocks.
Constraint design — Problem constraint design focuses on delineating the boundaries of a problem by defining input, process, and output restrictions of the solution search. You can use constraints to direct AI in generating solutions valuable for the task at hand.

Hands-on

Many AI research organizations, including OpenAI, provide APIs that allow you to interact with the language models. One could start crafting different prompts and observe how the model responds.

One could experiment with OpenAI.com examples, to try out prompts as shown below.

When working with Large language models like GPT-4, one typically interact with them via an API, which could be integrated into an application, as shown below.

In the API playground, one could play around with parameters as shown below, detailed in the OpenAI platform. These parameters influences the model output.

Tip: 
One could build exceptional AI-powered prompts using 
websites, as mentioned below:

 1. Open AI (https://Openai.com/api)
 2. AI21 Studio (https://www.ai21.com/studio/task-specific-models)

Conclusion

The wide range of AI innovations is expected to impact people and processes both inside and outside of an organization setting. From the business leaders to the enterprise engineering teams, AI advancements becomes an essential part, seeking to leverage the power of Large Language Models (LLM) through prompts, and their work would become essential to make sophisticated tools usable, effective, and accessible to variety of applications.

AI systems becomes truly powerful when it is combined with your own company data. Those, early adopters of these innovations can drive significant competitive advantage and business value.

Generative AI models together with prompt engineering are at their exciting moment to solve real-world problems, by discovering new and innovative ways to use LLMs. Finally, leaving you with this wonderful quote, in the context of AI …

“The hottest new programming language is English.” — A. Karpathy

References:

Gartner.com, HBR.org