# Examples of running the convert\_checkpoint.py script

### <mark style="color:blue;">Examples of running the convert\_</mark><mark style="color:blue;">`checkpoint.py`</mark> <mark style="color:blue;"></mark><mark style="color:blue;">script</mark>

* The script provides three examples demonstrating different use cases for running the <mark style="color:yellow;">**`convert_heckpoint.py`**</mark> script.
* Each example shows the specific command to run <mark style="color:yellow;">**`convert_checkpoint.py`**</mark> with the corresponding arguments.
* The examples cover scenarios such as converting a checkpoint using a single GPU with FP16 or BF16 precision, and converting a checkpoint with specific model hyperparameters.

```bash
1. Convert a checkpoint using a single GPU and FP16:
python convert_checkpoint.py --model_dir ./path/to/model/directory \
                              --output_dir ./tllm_checkpoint_1gpu_fp16 \
                              --dtype float16

2. Convert a checkpoint using a single GPU and BF16:
python convert_checkpoint.py --model_dir ./path/to/model/directory \
                              --output_dir ./tllm_checkpoint_1gpu_bf16 \
                              --dtype bfloat16

3. Convert a checkpoint with specific model hyperparameters:
python convert_checkpoint.py --model_dir ./path/to/model/directory \
                              --output_dir ./path/to/output/directory \
                              --dtype float16 \
                              --n_layer 32 \
                              --n_head 32 \
                              --n_embd 4096
```

### <mark style="color:blue;">README: Converting Checkpoints with TensorRT-LLM</mark>

This guide explains how to use the `run_convert_checkpoint.py` script to run the `convert_checkpoint.py` script with transparent configurations using a YAML file.

First download the scripts from Github

```bash
git clone https://github.com/Continuum-Labs-HQ/tensorrt-continuum.git
```

### You will need to manually move the files into the right directory for now

### <mark style="color:blue;">Prerequisites</mark>

* Python 3.x
* TensorRT-LLM
* PyYAML (install with `pip install pyyaml`)

### <mark style="color:blue;">Configuration</mark>

1. Populate the `config.yaml` file <mark style="color:yellow;">**in the same directory**</mark> as the `run_convert_checkpoint.py` script. This file will contain the configurations for the `convert_checkpoint.py` script.
2. Open the `config.yaml` file and specify the desired configurations. The file should have the following structure:

```yaml
model:
  model_dir: ./path/to/model
  output_dir: ./path/to/output
  dtype: float16

checkpoint:
  tp_size: 1
  pp_size: 1
  vocab_size: 32000
  n_positions: 2048
  n_layer: 32
  n_head: 32
  n_embd: 4096
  inter_size: 11008
  # Additional checkpoint arguments
  # ...
```

3. Adjust the values in the <mark style="color:yellow;">**`config.yaml`**</mark> file according to your requirements. You can refer to the comments in the file for suggestions and available choices.

### <mark style="color:blue;">Running the Script</mark>

1. Open a terminal and navigate to the directory containing the <mark style="color:yellow;">**`run_convert_checkpoint.py`**</mark> script and the <mark style="color:yellow;">**`config.yaml`**</mark> file.
2. Run the following command:

```bash
python3 run_convert_checkpoint.py
```

3. The script will read the configurations from the `config.yaml` file and construct the corresponding command-line arguments for the `convert_checkpoint.py` script.
4. The `convert_checkpoint.py` script will be executed with the specified configurations.

### <mark style="color:blue;">How It Works</mark>

The <mark style="color:yellow;">**`run_convert_checkpoint.py`**</mark> script does the following:

1. It uses the `argparse` module to parse the command-line argument `--config`, which specifies the path to the YAML configuration file (default is `config.yaml`).
2. It loads the configurations from the specified YAML file using the `yaml.safe_load()` function.
3. It extracts the relevant configuration values from the loaded YAML data, such as `model_dir`, `output_dir`, and `dtype`.
4. It constructs the command-line arguments for the `convert_checkpoint.py` script based on the extracted configuration values.
5. It iterates over the checkpoint arguments in the YAML file and adds them to the command-line arguments list. Boolean values are handled separately, adding only the argument flag if the value is `True`.
6. Finally, it uses the `subprocess.run()` function to execute the `convert_checkpoint.py` script with the constructed command-line arguments.

By using the `run_convert_checkpoint.py` script and the `config.yaml` file, you can easily configure and run the `convert_checkpoint.py` script without explicitly specifying all the command-line arguments.&#x20;

This approach provides a more transparent and manageable way to set the configurations for the checkpoint conversion process.