# tensorrt\_llm.functional.rms\_norm

The `tensorrt_llm.functional.rms_norm` function in TensorRT-LLM applies Root Mean Square (RMS) normalization to a tensor. This operation is similar to layer normalization but with some key differences, particularly in how the normalization is computed. RMS normalization is often used in neural network architectures, including large language models, to help stabilize training and improve the learning process.

#### Function Purpose

* **RMS Normalization**: Normalizes the input tensor based on the root mean square of the elements along the specified axes. It's an alternative to the standard layer normalization, with the variance normalized separately and not jointly with the mean.

#### Parameters

1. **input (Tensor)**:
   * The input tensor to be normalized.
   * Typically, this would be the output of a neural network layer.
2. **normalized\_shape (int or Tuple\[int])**:
   * The dimensions over which normalization is performed.
   * In language models, it usually corresponds to the hidden dimension or feature dimension of the tensor.
3. **weight (Tensor, optional)**:
   * The scale coefficient (often denoted as gamma) for normalization, applied element-wise to the normalized tensor.
   * It should have the same shape as `normalized_shape`. If omitted, the tensor is normalized without scaling.
4. **eps (float)**:
   * A small constant (epsilon) added for numerical stability to avoid division by zero when computing the root mean square.
   * Typically a very small value like `1e-6`.

#### How to Use

* **Prepare Input Tensor**: Make sure your input tensor is ready and in the correct format (shape and data type).
* **Determine Normalization Shape**: Choose the `normalized_shape` based on the dimensions you want to normalize, typically the feature dimensions.
* **Optional Weight**: Provide a `weight` tensor for scaling if you have specific scaling parameters. If not provided, the operation defaults to RMS normalization without scaling.
* **Set Epsilon**: Choose an appropriate `eps` value to avoid division by zero issues.

#### Returns

* **Tensor**: The function returns a tensor that has been RMS normalized. It maintains the same shape as the input tensor.

#### Example Use Case

In a neural network layer, particularly in transformers used in large language models, RMS normalization can be applied to the output of layers (such as after a multi-head attention or feed-forward block). This normalization ensures that the scale of outputs is controlled, potentially leading to more stable and efficient training. RMS normalization is particularly useful when the variance of the features is of interest independently from the mean.

<br>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://tensorrt-llm.continuumlabs.ai/the-python-api/tensorrt_llm.functional.rms_norm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.