Import a model registry with pre-built container

Import a model registry with pre-built container

Model Preparation

  • Ensure your machine learning model is packaged and stored in a container image compatible with Triton Inference Server.

  • Upload the container image containing your model to a storage location accessible by our AI Platform.

  • The online prediction feature of our AI Platform currently only supports models built using Triton Inference Server. Ensure that your custom container image is compatible with Triton before deploying your model for online prediction.

Step 1: Accessing the Model Registry

Step 2: Import a Model Registry

  • Location & Model registry name: Select the location & a specific name for this model.

  • Container: Select the Pre-built container option to use as a supported framework.

  • Model framework & version: Choose a model training framework & suitable version that meets your requirements.

  • Access to training model stored on network volume: Select network volume as a data mount method for the training job.

    • Model repository: Specify the location where your model's registry is stored. It should be added to the "network-volume" section of the location path, e.g., "/network-volume/training-model".

    • Network volume: Select a network volume that you want to access from the training job. This network volume will be mounted at pre-defined folder.

  • Click the “Import” button to complete the process.

Important Notice Regarding Pre-Built Containers in Model Registry

To simplify model deployment, our Model Registry offers pre-built containers. However, please be aware of the following limitations when using the pre-built container option:

  • Triton Framework Only: The pre-built container exclusively supports the Triton Inference Server framework. If your model is not compatible with Triton, you will need to use a custom container.

  • Triton Version 22.12: The pre-built container is configured with Triton version 22.12. Your model must be compatible with this specific version to ensure successful deployment.

Conditions for Successful Deployment with Pre-Built Container

To deploy your model using the pre-built container, ensure your model registry meets the following criteria:

  1. Triton Compatibility: Your model must be compatible with the Triton Inference Server framework. This typically involves exporting your model in a format that Triton can understand (e.g., ONNX, TensorFlow SavedModel).

  2. Triton Version 22.12 Compatibility: Your model must be compatible with Triton version 22.12. Check the Triton documentation for compatibility guidelines and any necessary adjustments to your model format or configuration.

  3. Model Configuration: Provide a valid config.pbtxt file that describes your model's configuration for Triton. This file should include details like model name, input and output tensors, and backend settings.

  4. Dependencies: If your model has any specific dependencies (libraries, packages), ensure they are included in your model repository or can be installed within the pre-built container environment.

Recommendations:
  • Review Triton Documentation: Familiarize yourself with the Triton Inference Server documentation to understand how to prepare your model for deployment.
  • Test Thoroughly: Before deploying to the Model Registry, thoroughly test your model locally using Triton version 22.12 to ensure compatibility and identify any potential issues.
  • Consider Custom Containers: If your model is not compatible with Triton or requires a different Triton version, you can create and use a custom container.

We are actively working to expand framework and version support for pre-built containers. Your feedback is valuable in helping us prioritize these enhancements.

    • Related Articles

    • Import a Model Registry

      The model registry is a centralized repository for storing trained models, their metadata, versions, and associated artifacts. It allows for version control, management, and organization of models developed during the training phase. This enables ...

    • Import a Model Registry using Triton Server

      Model Preparation Since our AI Platform only accesses models from a Network Volume, you must first create a Network Volume. Pull your model from local file systems or cloud storage (AWS S3, Azure Blob, or GCS) into the Network Volume. Ensure the ...

    • Import a Model Registry using NVIDIA NIM

      Step 1: Accessing the Model Registry Log in to your GreenNode AI Platform account and navigate to the Model Registry Dashboard Model Registry Dashboard at: . Find and click on the "Import a model registry" button. Step 2: Import a Model Registry ...

    • Import a Model Registry using vLLM

      Step 1: Accessing the Model Registry Log in to your GreenNode AI Platform account and navigate to the Model Registry Dashboard. Find and click on the "Import a model registry" button. Step 2: Import a Model Registry Location & Model registry name: ...

    • Import a model registry with custom container

      The model registry is a centralized repository for storing trained models, their metadata, versions, and associated artifacts. It allows for version control, management, and organization of models developed during the training phase. This enables ...