ONNX

As for ONNX, it stands for Open Neural Network Exchange.

It's an open-source project that provides a specification for the interoperability between machine learning models.

The project is backed by several major companies, including Microsoft, Facebook, and Amazon.

The goal of ONNX is to provide a model exchange format that enables AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.

Instead of being tied to a single framework or ecosystem, developers can choose the right tools for their project and not worry about compatibility.

For example, you might train a deep learning model using a framework like PyTorch, then use ONNX to convert the model to a format that can be run on a different system or framework, such as Microsoft's ONNX Runtime, for inference.

This could be particularly useful in situations where the model needs to be deployed in a different environment than it was trained in.In terms of its format, ONNX models are represented as graphs.

Each node in the graph corresponds to an operation, such as an addition, multiplication, or convolution, and the edges represent the tensors flowing between nodes.

It's worth noting that while ONNX aims to support a broad range of machine learning tasks, not all models can be represented in ONNX, and not all operations are supported by all ONNX runtimes. It's always important to check the current capabilities of the ONNX specification and the specific runtime you are using.

The ONNX (Open Neural Network Exchange) documentation provides a comprehensive overview of various aspects of ONNX. Here's a detailed summary:

Concept and Functionality

ONNX as a Specialised Language: ONNX is akin to a programming language focused on mathematical functions, particularly those necessary for machine learning model inference. It defines operations required to implement the inference function of a machine learning model.

Model Representation: Models in ONNX are often referred to as ONNX graphs. They can represent complex mathematical operations, like a linear regression, in a manner similar to Python coding but specifically using ONNX operators.

Structure of ONNX Models

Graphs and Nodes: An ONNX graph is built using ONNX Operators. It consists of nodes (operations like MatMul and Add) and connections representing data flow between nodes. Each node has a type (an operator) and inputs/outputs.

Inputs, Outputs, and Initializers: Models have inputs and outputs defined in a specific format. Constants or unchanging inputs can be encoded directly into the graph as 'initializers'.

Attributes: Fixed parameters of operators, like alpha or beta in the Gemm operator, which are unchangeable during runtime.

Serialization and Portability

Protobuf for Serialization: ONNX uses protobuf to serialize graphs into a single block, enhancing model portability and reducing size.

Additional Features

Metadata Storage: ONNX allows embedding metadata such as model version, author, training information, etc., directly into the model.

Operators and Domains: ONNX has a comprehensive list of operators covering standard matrix operations, image transformations, neural network layers, etc. It defines domains like ai.onnx and ai.onnx.ml, each containing a specific set of operators.

Supported Data Types

Primary Focus on Tensors: ONNX mainly supports numerical computations with tensors (multi-dimensional arrays). Tensors are characterized by type, shape, and a contiguous array of values.

Element Types: ONNX supports various data types, including different float and integer types. The list includes FLOAT, INT8, INT16, INT32, and others.

Sparse Tensors: ONNX also supports sparse tensors, primarily useful for arrays with many null coefficients.

Other Types: Besides tensors, ONNX handles sequences of tensors, maps of tensors, and sequences of maps of tensors. Serialization in ONNX can be explained in simpler terms as follows:

What is Serialization?

Serialization is the process of converting an ONNX machine learning model (or any other data structure in ONNX) into a format that can be easily saved, transferred, and later reloaded. This process turns the complex model into a simpler, more compact form that can be stored in a single file.

Saving a Model (Serialization)

1. How It's Done: You take your ONNX model and convert it to a string of bytes (a simple format).

2. Example: Let's say you have a model called onnx_model. You would use the command onnx_model.SerializeToString() to convert this model into a string format.

3. Saving to a File: After converting the model into a string, you can then save this string to a file (like "model.onnx") on your computer.

Loading a Model (Deserialization)

1. How It's Done: When you want to use the model again, you need to convert the string of bytes back into the original ONNX model format.

2. Example: Using the command onnx.load("model.onnx"), you can turn the string in the file "model.onnx" back into an ONNX model that you can work with.

Working with Different Data Structures

• NodeProto: This is just another type of data structure in ONNX, similar to a model but with different content. You can save and load it in the same way as you do with a model.

• TensorProto: This is a specific type of data structure for storing tensor data. There's a special command onnx.load_tensor_from_string() to load tensor data from a string.

Key Points

• SerializeToString: A method used to convert a model or data into a string of bytes for saving.

• ParseFromString: A method used to convert the saved string of bytes back into the original data structure.

• File Handling: Saving and loading involves reading from and writing to files, which requires handling these files correctly in your code.

In summary, serialization in ONNX is about converting complex data structures like models and tensors into a simpler, string-based format for easy storage and retrieval. This process is essential for saving models and later using them in different environments or applications.