The introduction of AI systems, such as ChatGPT, marked a significant milestone in computer technology. Although it may not currently surpass human performance in all tasks, it is progressing at an incredible rate. GPT, or “Generative Pre-trained Transformer,” has the ability to produce new content based on the input it receives.
“Generative” refers to its content generation capabilities, while “pre-trained” signifies that the model has already been trained on a massive dataset (commonly known as the “corpus”), which consists of diverse text sources like books, articles, and websites – equivalent to the content of 37.5 million textbooks. This pre-training allows GPT to gain a broad understanding of language and context before being fine-tuned for specific tasks. The term “transformer” pertains to the underlying neural network architecture used in GPT models to efficiently handle long-range dependencies and better comprehend relationships between words within a given context.
ChatGPT continuously refines its predictions through a feedback loop incorporating backpropagation, allowing the model to adjust and improve its predictions by learning from discrepancies between predicted and actual outcomes.
Unlike the human brain, which can learn a new subject like mathematics with just a few books and problem sets, pre-trained AI models require millions of pages of data to develop their predictive capabilities. Once the large language model (LLM) has accumulated a vast amount of knowledge equivalent to millions of books, it is equipped with an imprint that allows it to apply this extensive knowledge to perform specific tasks.
There are significant parallels between AI systems and the human brain regarding information processing and learning. To excel in a particular domain, an AI model must be furnished with relevant and up-to-date data, similar to an individual acquiring knowledge by studying a subject to perform specialized tasks. The burgeoning field of “prompt engineering” can be compared to guiding a person in specialized domains.
When hiring someone as a loan officer, it is expected that they possess a certain level of general knowledge gained through a college education, including language and math skills. Subsequently, through targeted training in specific knowledge over a brief period, the individual should achieve a satisfactory performance level and carry out the desired tasks within the organization.
Prompt engineering supplies the particular knowledge needed for an AI to develop an effective agent capable of performing a specific set of tasks. Prompts are employed to steer the AI model’s response, establish context, or determine the tone of the conversation. They act as a foundation for the AI system to generate pertinent and coherent output based on its pre-trained knowledge and comprehension of language.
The challenge in developing AI agents is to extract human expertise, developed over years of practice, in a short amount of time and translate it into effective prompts that result in the creation of efficient AI agents.
Cognitive Task Analysis (CTA) aims to extract and represent human expertise in a way that can be used to inform the design of training programs, user interfaces, decision support systems, or AI agents. By analyzing the thought processes and decision-making strategies of experts, CTA helps to identify the essential knowledge, skills, and cognitive strategies required for effective task performance.
Here’s how CTA extracts human expertise and builds AI agents:
- Utilize observations, think-aloud protocols, and interviews to gather information about the experts’ cognitive processes and decision-making strategies during task performance.
- Map the collected data to identify decision points, problem-solving strategies, and the essential knowledge and skills needed for successful task completion. This analysis entails coding and categorizing the data in terms of goals, subgoals, and methods within the human information processing flow. The process consists of steps such as detecting, perceiving, analyzing, deciding, executing, and obtaining feedback. These models aim to depict the experts’ thought processes throughout the information processing continuum.
- Ensure that the extracted knowledge accurately represents the experts’ cognitive processes and is relevant to the task at hand by discussing the findings with domain experts and other stakeholders.
- Generate a set of prompts that provide the specific knowledge needed for the AI to create an effective AI agent capable of performing a set of specific tasks.
- Test the AI agent and iterate on the prompts to refine them, ensuring that the agent’s performance reaches the desired level.