Generative AI Notes

Generative AI creates new high-quality content (text / images / sound / video) based on a query. Generative AI models are trained on vast amounts of raw data: texts, images, sounds, videos.

How LLMs generate text

• The model receives a text as input.
• The model tries to generate the most suitable next word.
• After that, the generated word is added to the initial text and fed back to the model. Again, the model tries to generate the most suitable next word. And so on. 

Algorithm for prompt engineering for solving applied problems using LLMs 

• Step 1: Formulate the initial prompt.
• Step 2: Run the prompt with a sample set of examples. 
• Step 3: Analyse errors.
• Step 4: Improve the prompt based on error analysis and repeat from step 2.

It is worth repeating the cycle and refining the prompt until the responses reach the necessary quality.

Basic recommendations for prompt engineering

Use clear and detailed instructions.

• Specify the role that the model plays when it needs to generate a response. For example, you can instruct it to act as a product manager or analyst who conducts research and analyses user feedback.
• Use separators between semantic blocks, such as "", ===, < >, <tag> </tag>. There are no formal recommendations on which type of separator to use for different use cases; the main consideration is that different semantic blocks should be explicitly separated.
• Specify the format of the response: JSON, HTML, table, or other.
• Provide examples of correct and incorrect answers for the given task.
• Ask the model to verify if all necessary conditions are met.

Give models 'time to think'

• List instructions for performing the task as a clear sequence of steps. Example: Step 1: do X, Step 2: do Y, ..., Step N: do Q. 
• Add the instruction 'Solve the problem step by step' to the prompt.
• Ask the model to provide explanations for its answers. This will help with ideas for improvements in the next iteration.

Always evaluate the quality of prompts

• We will use the train dataset to design prompts, analyze errors, and optimize the prompt’s quality.
• We will use the test dataset for the final evaluation of the prompt's quality. 
• A critical area of responsibility for an AI product manager is to decide what must be done based on business metric measurements, and where to make intuitive decisions based on quality metric measurements.

General approach to solving problems with LLMs

Understand the problem and design a pipeline.

For each element of the pipeline:

• Determine the quality metrics.
• Find or create the necessary data.
• Create prompts based on iterative improvement and error analysis to achieve good solution quality.
• Convert the output of the pipeline work into a useful format.
• Make sure that the AI system creates user and business value. 

AI systems

Creating an AI pipeline to solve your problem is just the first step towards developing a production solution. A full-fledged AI system contains many components to work predictably and reliably.

Aspects that are important to consider when implementing AI systems:

• Monitoring the quality of data and model responses.
• Monitoring cloud service changes.

Business value and costs of GenAI solutions

The AI product manager must address questions regarding the business value and costs of GenAI solutions. This includes:

• Evaluating the cost of using models.
• Choosing the optimal model for the project.
• Determining how to evaluate value for business.

In the case study, we showed how to quickly implement an AI pipeline for analyzing reviews, which was comparable to leading solutions in the industry. In the past, creating such a product required an expensive team of AI engineers. Modern technologies allow a small team or even one person without special knowledge to create the core of such a service in a few hours.

Even if you forget everything, remember this

Previously, solving AI problems required AI specialists with deep knowledge of machine learning, mathematics, and programming.

Modern generative AI tools allow solving many problems without such a team. However, it is still necessary to master the fundamentals of creating AI products, as well as consider the specifics of business problems.