Managing Imported Models (New)

Before you import a model directly from Hugging Face:

• Decide which supported model from Hugging Face to import and note its recommended dedicated AI cluster unit size.
• To access and use some models, you need a Hugging Face token, especially the more recent and gated versions such as Llama 3 and Llama 3.1 For these models, generate an access token from your Hugging Face account settings under Access Tokens. Ensure it has the necessary permissions (at least "read" access).

Before you import a model from an Object Storage bucket:

• If you're not an OCI admin, ask one to give you IAM permission to manage Object Storage in your compartment:

  allow group <your-group-name> to manage object-family in compartment <compartment-with-bucket>
                              

• Decide on a model that works with the /v1/chat/completions endpoint—only these models are supported.
• Ensure the model supports only one of these capabilities:
  • TEXT_TO_TEXT: text in, text out
  • IMAGE_TEXT_TO_TEXT: image or text or both in, text out
  • EMBEDDING: text in, vector embeddings out
  • RERANK: query and candidate documents in, relevance scores and a reordered list out
• Save model artifacts in an Object Storage bucket.
• Important: The configuration file must be called config.json for a successful import, similar to most Hugging Face models.

To reach an imported model, you create an endpoint for that model on a dedicated AI cluster. Use the following table to request for dedicated AI cluster resources before you import a model.

After performing the prerequisites and importing a model, you can perform the following tasks on the imported models:

• List the imported models
• Get an imported model's details
• Update an imported model
• Move an imported model
• Delete an imported model