Create a quantization job

Create a quantization job

To create a quantization job, you'll need to provide the following information:

Steps to Create a Quantization Tuning Job

  1. Access the Quantization Job Creation Interface: Use the provider's platform through the url: .
  2. Fill in the Input Parameters: Provide the required information for each parameter.
  3. Review and Submit: Carefully review your input parameters and submit the job.
  4. Monitor the Job: Track the progress of your tuning job through the platform's interface.

      Understanding the Input Parameters

  1. Quantization Job Name: A unique name for your quantiza job.
  2. Location: The geographic location where the tuning job will run (e.g., thailand bangkok).
  3. Model: Select a pre-trained base model from the provider's library (e.g., GPT-3, BERT). You can select from three sources:
    1. Network volume: Load model from your network-volume.
    2. GreenNode catalog: Choose from GreenNode's model catalog.
    3. Huggingface: Using model from Huggingface
  4. Quantization Method: 
    1. Weight-Only Quantization: Reduce model size and improve inference performance for latency sensitive applications with the latest research
      1. Supported Algorithms: GPTQ, AWQ.
  5. Instance Type: Choose the type of computing instances to use for quantization (e.g., g5-standard-16x250-1h100).
  6. Use LoRA during quantization: If merge LoRA weight with original model
    1. LoRA path: Path to directory containing the LoRA weight 
    2. Merged model path: Must not be the same as Output path (This is where your trained LoRA weight merged with original model weight is saved)
  7. Output Path: Provide the path where the quantized model will be saved.

    • Related Articles

    • Manage a model quantization job

      Model quantization provides you with a dedicated environment to develop and experiment with your AI models. After creating a model quantization job, follow these steps to seamlessly manage your model training job: Step 1: Accessing Model Quantization ...

    • Create a tuning job

      To create a supervised tuning job, you'll need to provide the following information: Steps to Create a Supervised Tuning Job Access the Tuning Job Creation Interface: Use the provider's platform through the url: . Fill in the Input Parameters: ...

    • Manage a model tuning job

      Model tuning, also known as hyperparameter optimization, is the process of adjusting the hyperparameters of a machine learning model to improve its performance. Hyperparameters are settings that determine the learning process of a model and are not ...

    • Create a network volume

      How to Create a Network Volume on Greennode AI Platform? This guide will walk you through the process of creating a new volume on your AI platform to store data, models, and other necessary files. Step 1: Access the Volume Creation Interface Log in: ...

    • Create a Network Volume

      How to Create a Network Volume on Greennode AI Platform? This guide will walk you through the process of creating a new volume on your AI platform to store data, models, and other necessary files. Step 1: Access the Volume Creation Interface Log in: ...