Import a Model Registry using Triton Server

Import a Model Registry using Triton Server

Model Preparation

  • Since our AI Platform only accesses models from a Network Volume, you must first create a Network Volume. Pull your model from local file systems or cloud storage (AWS S3, Azure Blob, or GCS) into the Network Volume.
  • Ensure the model is in Triton-compatible format such as:
    • ONNX (.onnx)
    • TensorFlow (SavedModel or .pb)
    • PyTorch TorchScript (.pt)
    • TensorRT (.engine)
    • OpenVINO (.xml and .bin)
    • Ensemble Model (combining multiple models)

Step 1: Accessing the Model Registry

  • Log in to your GreenNode AI Platform account and navigate to the Model Registry Dashboard.
  • Find and click on the "Import a model registry" button.

Step 2: Import a Model Registry

  • Location & Model registry name: Select the location & a specific name for this model.
  • Container: Select the Pre-built container option to use as a supported framework.
  • Framework: Choose a model deployment framework & suitable version that meets your requirements. In this tutorial, we select Triton 22.12
  • Model source: Access to model stored on network volume: Select network volume as a data mount method for the training job.

    • Model repository: Specify the location where your model's registry is stored. To use Triton, we need to build a model repository. The structure of the repository as follows:

      network-volume
|
+-- model_repository
    |
    +-- resnet
        |
        +-- config.pbtxt
	+-- 1
            |
            +-- model.onnx

      It should be added to the "network-volume" section of the location path, e.g., "/network-volume/model_repository".

      Check the Triton documentation for compatibility guidelines and any necessary adjustments to your model format or configuration.

  • Click the “Import” button to complete the process.

    • Related Articles

    • Import a Model Registry using vLLM

      Step 1: Accessing the Model Registry Log in to your GreenNode AI Platform account and navigate to the Model Registry Dashboard. Find and click on the "Import a model registry" button. Step 2: Import a Model Registry Location & Model registry name: ...

    • Import a Model Registry using SGLang

      Step 1: Accessing the Model Registry Log in to your GreenNode AI Platform account and navigate to the Model Registry Dashboard. Find and click on the "Import a model registry" button. Step 2: Import a Model Registry Location & Model registry name: ...

    • Import a model registry with pre-built container

      Model Preparation Ensure your machine learning model is packaged and stored in a container image compatible with Triton Inference Server. Upload the container image containing your model to a storage location accessible by our AI Platform. The online ...

    • Import a Model Registry using NVIDIA NIM

      Step 1: Accessing the Model Registry Log in to your GreenNode AI Platform account and navigate to the Model Registry Dashboard Model Registry Dashboard at: . Find and click on the "Import a model registry" button. Step 2: Import a Model Registry ...

    • Import a model registry with custom container

      The model registry is a centralized repository for storing trained models, their metadata, versions, and associated artifacts. It allows for version control, management, and organization of models developed during the training phase. This enables ...