Oracle Cloud Infrastructure Documentation

For Multimodel Serving

Troubleshoot issues when using multimodel serving with model deployments.

Model loading fails due to invalid models

In the predict logs is the error: Failed to load model.

A faulty model exists in the model group.

  1. Fix the faulty model or remove it from the model group.
  2. Deploy the updated model group.

Live update request can't be processed

The model group size is larger than the maximum supported size.

Use a ZDT deployment instead.

Live update can't be fulfilled because of a change in the model group type

Live Update isn't supported when switching between different Model Group types, for example, from HOMOGENEOUSto STACKED.

Perform a ZDT update.

Deployment type can't be changed during an update

The deployment type can't be changed for an update operation.

Create a new model deployment with the required deploymentType (SINGLE_MODEL or MODEL_GROUP).

Live update fails if the update request isn't for modelGroupId

Live update is supported for updating modelGroupId only.

Perform the live update for updating modelGroupId only. A new ModelGroupId must have differences in member models only. The rest of the configuration must remain same as the deployed model group.

Model group creation fails because of invalid models

The models don't exist or are in a terminal state.

Use existing models not in a terminal state.