Skip to content

How to finetune ChatGPT on your use case?

Picture of Pranay Kumar Jain

Frame 12367

Introduction

With the craze around ChatGPT the concept of LLM’s or Large Language models has been added in everyone’s new vocabulary. There isn’t a meeting or Zoom call where you don’t hear the words but what about ChatGPT and how can we use it. To use ChatGPT or similar technologies one of the most important aspects is to understand how to control and train these models on what works for your organization and a bit part of that is Fine tuning LLM’s. However, to fine-tune these models to suit specific use cases, we must understand how to approach this task effectively. In this article, we will discuss the importance of fine-tuning and how it is done in the context of OpenAI models in particular and LLM’s in general.

Understanding Large Language Models

What are Large Scale Language Models?

Large scale language models or LLM’s are NLP models that are trained on large datasets to improve their language processing capabilities. These models can process and analyze text data to perform a variety of tasks such as language translation, sentiment analysis, and content generation. The most notable examples of large language models include ChatGPT (OpenAI), Turing NLG (Microsoft), Gopher, Chichilla (Deepmind), T5, MT5 (Google), Ernie 3.0 (Baidu) which are all trained on billions of tokens.

Benefits of Large Language Models

The benefits of large language models are numerous, including the ability to generate high-quality text, improved language processing, and enhanced accuracy in predicting language-based outcomes. These models can also learn from and adapt to new data, improving their performance over time and powering technologies such as Chatbots, Conversational AI, Email bots, personalized Voice bots, SEO and even Natural language coding.

Fine-Tuning Pretrained Language Models

What is Fine-Tuning Pretrained Language Models?

Fine-tuning pretrained language models involves taking an existing language model that has already been trained on a large dataset and retraining it on a smaller, domain-specific dataset. This process fine-tunes the model to understand and produce language relevant to a specific domain or use case. Fine-tuning can

  1. It can significantly improve the accuracy of responses by having a better understanding of the task at hand and hence the accuracy of the responses.
  2. Change and match tone of the responses to your organizations requirements eg. professional and short or casual and friendly

As per OpenAI’s own findings Fine-tuning GPT-3 improves accuracy by 2 to 4x over what’s possible with prompt design.

How to Fine-Tune NLP Models

The process of fine-tuning NLP models involves selecting a pretrained model that is relevant to the domain and use case. Next, we train the model on a smaller, labeled dataset that reflects the specific domain or use case. As per OpenAI “It takes less than 100 examples to start seeing the benefits of fine-tuning GPT-3 and performance continues to improve as you add more data”. The model then learns to predict outcomes based on the new dataset, improving its accuracy and language processing capabilities for that domain or use case.

Is fine-tuning available for ChatGPT or gpt-3.5-turbo?

No. As of Mar 1, 2023, you can only fine-tune base GPT-3 models. See the fine-tuning guide for more details on how to use fine-tuned models. Presently the best option for finetuning ChatGPT is prompt engineering.

Fine-Tuning vs. Feature Extraction

What is the Difference between Fine-Tuning and Feature Extraction?

Fine-tuning and feature extraction are two approaches to reusing existing models for a specific task. Fine-tuning involves training the entire model on new data, while feature extraction only trains a new classifier on top of the existing model's pre-extracted features. Fine-tuning is typically more effective when the task is more complex or requires the model to learn more about the domain, while feature extraction can be more efficient for simpler tasks.

Why Do We Need Fine-Tuning?

Fine-tuning is essential because it allows us to reuse existing, powerful models for specific use cases. Instead of training an entirely new model from scratch, we can save time and resources by fine-tuning a pretrained model for our specific needs. This approach can also improve the accuracy and performance of the model, especially in situations where labeled training data is limited.

How to Fine-Tune a Large Language Model

  1. Preparing for Fine-Tuning Before fine-tuning LLM like ChatGPT, it is important to prepare the necessary data. This involves understanding the task at hand, choosing the right dataset, and formatting the dataset.
    1. Understanding the task at hand: Understanding the specific task you want LLM like CHATGPT to perform is crucial. For example, if you want to fine-tune LLM like CHATGPT for sentiment analysis, you need to understand what sentiment analysis involves, how many different sentiments do you want to classify the text into and how LLM like CHATGPT can be used for this task.
    2. Choosing the right dataset: Once you understand the task, you need to choose a dataset that is relevant to the task. The dataset should contain examples of the type of text that LLM like ChatGPT will be generating. For example, if you are fine-tuning LLM like ChatGPT for sentiment analysis, you need a dataset that contains examples of text with sentiment labels with all the sentiments you want the language model to understand.
    3. Formatting the dataset: The dataset needs to be properly formatted so that it can be used for fine-tuning. This involves separating the text into input and output sequences and ensuring that the labels are properly formatted.
  2. Fine-Tuning Techniques There are several fine-tuning techniques that can be used to optimize LLM like ChatGPT for specific tasks. These include few-shot learning, zero-shot learning, continual learning, and multi-task learning.
    1. Few-Shot Learning: Few-shot learning involves fine-tuning LLM like ChatGPT on a small amount of data. This technique is useful when you have limited labeled data for the task at hand.
    2. Zero-Shot Learning: Zero-shot learning involves fine-tuning LLM on a task that it has not been specifically trained for. This technique is useful when you want to use LLM for a task that is similar to the ones it has been trained on like content writing.
    3. Continual Learning: Continual learning involves fine-tuning LLM like ChatGPT over time as new data becomes available. This technique is useful when you want to keep LLM like ChatGPT up-to-date with new information and continue to improve its model based on new data. This is extremely useful to increase accuracy but if not implemented correctly it can also drop accuracy.
    4. Multi-Task Learning: Multi-task learning involves fine-tuning LLM like ChatGPT on multiple related tasks simultaneously. This technique is useful when you want to optimize LLM like ChatGPT for multiple tasks at once.

How to get started with finetuning?

  1. Setting up the environment: Before fine-tuning LLM like ChatGPT, you need to set up the environment by installing the necessary libraries and dependencies.
  2. Running the fine-tuning code: Once the environment is set up, you can begin running the fine-tuning code. This involves feeding the formatted dataset into the code and specifying the fine-tuning technique to use. The code will then fine-tune LLM like ChatGPT on the dataset and generate a fine-tuned model.
    This is how the dataset jsonl file would like.

{"prompt": "", "completion": "<ideal generated text>"}

{"prompt": "", "completion": "<ideal generated text>"}

{"prompt": "", "completion": "<ideal generated text>"}
....

You would have to take care to keep each record less than 2048 tokens.Once, you have the dataset ready, run it through the OpenAI command-line tool to validate it. Openai tools fine_tunes.prepare_data - <LOCAL_FILE>

You can also pass files in CSV, TSV, XLSX, JSON or JSONL format to this tool and it will help you convert it into a fine-tuning ready dataset.

  1. Train a new fine-tuned model
    Run the below command from the command line program to train your fine-tuned model.
    openai api fine_tunes.create -t <TRAIN_FILE_ID_OR_PATH> -m <BASE_MODEL>
    Replace the filename and choose a model name to base your model on

  2. Evaluating the fine-tuned model: After the fine-tuning process is complete, you need to evaluate the fine-tuned model to ensure that it is performing well on the task at hand. This involves using a separate dataset to test the performance of the fine-tuned model. If the model is not performing well, you may need to adjust the fine-tuning technique or hyperparameters and repeat the process.

Conclusion: The fine-tuning process can be iterative, and it may take several rounds of fine-tuning and evaluation to optimize LLM like ChatGPT for a specific task. However, with the right approach and techniques, fine-tuning LLM like ChatGPT can lead to impressive results and help unlock its full potential. if you want to see how this can be applied to your organization speak to us at technology@enterprisebot.ai or to our sales team at sales@enterprisebot.ai.