ML Applications

ML Applications is a self-contained representation of ML use cases in Data Science.

ML Applications is a new capability in Data Science that delivers a robust MLOps platform for AI/ML delivery. It standardizes the packaging and deployment of AI/ML functionality, enabling you to build, deploy, and operate machine learning as a service. With ML Applications, you can leverage Data Science to implement AI/ML use cases and provision them into production for your applications or customers. By shortening the development lifecycle from months to weeks, ML Applications quickens the time to market while reducing operational complexity and total cost of ownership. It provides an end-to-end platform for deploying, validating, and promoting ML solutions through every stage - from development and QA to preproduction and production.

ML Applications also supports a decoupled architecture by providing a unified integration layer between AI/ML capabilities and client applications. This allows independent development, testing, and evolution of ML solutions without requiring changes to the client application, enabling seamless integration and faster innovation.

ML Applications is ideal for SaaS providers, who need to provision and maintain ML functionality across a fleet of customers while ensuring strict data and workload isolation. It lets SaaS vendors deliver the same ML-powered capabilities to many tenants without compromising security or operational efficiency. Whether it’s embedding AI-driven insights, automating decision-making, or enabling predictive analytics, ML Applications ensures that each SaaS customer benefits from a fully managed and isolated ML deployment.

Beyond SaaS, ML Applications is also ideal for multiregion deployments and organizations looking to build an ML marketplace where providers can register, share, and monetize AI/ML solutions. Customers can seamlessly instantiate and integrate these ML capabilities into their workflows with minimal effort.

For example, an ML Application for a customer churn prediction use case could consist of:

a Pipeline with ingestion, transformation, and training steps that prepare training data, train a new model, and deploy it.
a Bucket that's used to store ingested and transformed data.
a Trigger that acts as an entry point and ensures secure run of pipeline runs in the context of a specific SaaS customer.
a Schedule that triggers periodic runs of the training pipeline to keep the model up-to-date.
a Model Deployment that serves prediction requests coming from customers.

ML Applications lets you represent the entire implementation as code and store and version it in a source code repository. The solution is packaged as an ML Application package that contains mostly metadata. The metadata includes versioning information, the provisioning contract, and declarations of environment dependencies, making the package region-independent and environment-independent. When built, the package can be deployed to any target environment without modifications. This helps to standardize the packaging and delivery of your ML functionality.

When an ML Application is deployed, it's represented by ML Application and ML Application Implementation resources. At this stage, it can be provisioned for use. Typically, a client application (such as a SaaS platform, enterprise system, or workflow automation tool) requests the ML Application service to provision a new instance of the ML Application for a specific customer or operational unit. Only at this point is the solution fully instantiated and ready for use.

To sum up, ML Applications provides a standardized way to build, package, and deliver ML functionality at scale, reducing complexity and speeding up time to production in various scenarios, including:

SaaS AI adoption, where SaaS platforms need to integrate ML capabilities for thousands of customers while ensuring security, scalability, and operational efficiency.
Multiregion deployments, where ML functionality must be consistently provisioned across different locations with minimal operational overhead.
Enterprise AI adoption, where organizations need to deploy isolated ML instances across teams, business units, or subsidiaries while maintaining governance and compliance.
ML marketplaces, where providers can package and distribute ML solutions, enabling customers to easily discover, deploy, and use them as a service.

Beyond SaaS, ML Applications can be used in various other scenarios. They're beneficial wherever a need to provision an ML solution many times exists, such as across different geographical locations. Also, ML Applications can be used to build a marketplace where providers can register their applications and offer them as a service to customers, who can then instantiate and use them.

The ML Applications capability itself is free. You're only charged for the underlying infrastructure (compute, storage, and networking) used, with no extra markup.

ML Applications Resources

OCI Resources delivered by ML Apps:

ML Application: A resource that represents an ML use case and serves as an umbrella for ML Application Implementations and Instances. It defines and represents an ML solution, letting providers deliver ML features to consumers.

ML Application Implementation: A resource that represents a specific solution for the ML use case defined by an ML Application. It contains all the implementation details that allow for the instantiation of the solution for consumers. An ML Application can have only one implementation.

ML Application Implementation Version: A read-only resource that represents a snapshot of an ML Application Implementation. The version is created automatically when an ML Application Implementation reaches a new consistent state.

ML Application Instance: A resource that represents a single isolated instance of an ML Application letting consumers configure and use the provided ML functionality. ML Application Instances play a crucial role in defining boundaries for the isolation of data, workloads, and models. This level of isolation is essential for SaaS organizations as it means they can ensure the segregation and security of their customers' resources.

ML Application Instance View: A read-only resource, which is an automatically managed copy of the ML Application Instance extended with extra details such as references to instance components. It lets providers track the consumption of their ML Applications. It means that providers can observe implementation details of instances, monitor and troubleshoot them. When consumers and providers work in different tenancies, ML Application Instance views are the only way for providers to gather information about the consumption of their applications.

Note

ML Application resources are read-only in the OCI Console. For management and authoring of the resources, you can use mlapp CLI that's part of the ML Application sample project, oci CLI or APIs.

ML Application Concepts

ML Applications introduce two key concepts:

ML Application package
ML Application trigger

While these aren't OCI resources, they're essential for understanding and operating ML Applications.

ML Application package: Allows for a standardized packaging of ML functionality that's environment and region-independent. It contains implementation details, such as Terraform components, descriptors, and configuration schema, and it's a portable solution that can be used in any tenancy, region, or environment. Infrastructure dependencies of the contained implementation (for example VCN and Log OCIDs) that are specific to a region or environment are provided as package arguments during the upload process.

ML Application trigger: Triggers let ML Applications providers specify the triggering mechanism for their ML jobs or pipelines, easing the implementation of fully automated MLOps. Triggers are the entry points for the runs of your ML workflows. They're defined by YAML files within the ML Applications package as instance components. Triggers are automatically created when a new ML Application Instance is created but only when all other instance components are created. So, when a trigger is created, it can refer to other instance components created before.

ML Application Roles

When working with ML Applications, it's important to distinguish between two key roles:

Providers
Consumers

Provider: Providers are OCI customers who build, deploy, and operate ML features. They package and deploy ML features as ML Applications and Implementations. They use ML Applications to deliver prediction services in an as-a-Service manner to consumers. They ensure that provided prediction services meet their agreed service level agreements (SLAs).

Consumer: Consumers are OCI customers who create ML Application Instances and use prediction services offered by these instances. Typically, consumers are SaaS applications such as Fusion. They use ML Applications to integrate ML functionality and deliver it to their customers.

Lifecycle Management

ML Applications cover the entire lifecycle of ML solutions.

This starts with early design stages, where teams can agree on contracts and can start working independently. It includes production deployment, fleet management, and the roll out of new versions.

Build

Representation as code: The whole solution, including all its components and workflows, is represented as code. This promotes software development best practices such as consistency and reproducibility.

Automation: With ML Applications, automation is straightforward. You can focus on automating the workflows within the solution using Data Science Scheduler, ML Pipelines and ML Application triggers. The automation of provisioning and configuration flows is managed by the ML Application service.

Standardized packaging: ML Applications provides environment-agnostic and region-agnostic packaging, including metadata for versioning, dependencies, and provisioning configurations.

Deploy

Service-managed deployment: You can deploy and manage your ML solutions as ML Application resources. When you create your ML Application Implementation resource, you can deploy your implementation packaged as an ML Application package. The ML Application Service orchestrates the deployment for you, validating your implementation and creating the corresponding OCI resources.

Environments: ML Applications lets providers deploy through various lifecycle environments such as development, QA, and preproduction, so giving a controlled roll out of ML Applications to production. In production, some organizations offer customers several environments, such as "staging" and "production." With ML Applications, new versions can be deployed, evaluated, and tested in "staging" before being promoted to "production." This gives customers great control over the uptake of new versions, including ML functionality.

Cross-region deployment: Deploy solutions across various regions, including noncommercial ones such as government regions.

Operate

As-a-service delivery: Providers deliver prediction services 'as-a-service,' handling all maintenance and operations. Customers consume the prediction services, without seeing implementation details.

Monitoring and Troubleshooting: Simplified monitoring, troubleshooting, and root cause analysis with detailed versioning, traceability, and contextual insights.

Evolvability: Help deliver quick iterations, upgrades, and patches with zero downtime, ensuring continuous improvement and adaptation to customer needs.

Key Features

Some key features of ML Applications are:

As-a-Service delivery: ML Applications lets teams create and deliver prediction services as SaaS (software-as-a-service). This means that providers can manage and evolve solutions without affecting customers. ML Applications serves as a meta-service, easing the creation of new prediction services that can be consumed and scaled as SaaS offerings.

OCI native: Seamless integration with OCI resources ensures consistency and simplifies deployment across environments and regions.

Standard packaging: Environment-independent and region-independent packaging standardizes implementation, making it easy to deploy ML Applications globally.

Tenant isolation: Ensures complete isolation of data and workloads for each customer, enhancing security and compliance.

Versioning: Supports evolving implementations with independent release processes, enabling quick updates and enhancements.

Zero-downtime upgrades: Automated instance upgrades ensure continuous service without interruptions.

Cross-tenancy provisioning: Supports cross-tenancy observability and marketplace consumption, expanding deployment possibilities.

ML Applications introduces the following resources:

ML Application
ML Application Implementation
ML Application Implementation Version
ML Application Instance
ML Application Instance View

Added Value

ML Applications serves as a platform-as-a-service (PaaS), providing organizations with the foundational framework and services necessary to build, deploy, and manage ML features at scale.

This equips teams with a pre-built platform for modeling and deploying ML use cases, eliminating the need to develop custom frameworks and tools. So, teams can dedicate their efforts to crafting innovative AI solutions, thereby reducing both time to market and the total cost of ownership for ML features.

ML Applications offers organizations APIs and abstractions to oversee the entire lifecycle of ML use cases. Implementations are represented as code, packaged into ML Application Packages, and deployed to ML Application Implementation resources. Historical versions of implementations are tracked as ML Application Implementation Version resources. ML Applications is the glue that establishes traceability among all components use by the implementations, versioning and environment information, and customer consumption data. Through ML Applications, organizations gain precise insights into which ML implementations are deployed across environments and regions, which customers use specific applications, and the overall health of the ML fleet. Moreover, ML Applications offers instance recovery capabilities, enabling easy recovery from infrastructure and configuration errors.

With well-defined boundaries, ML Applications helps the establishment of provisioning and prediction contracts, enabling teams to divide and conquer tasks more efficiently. For example, teams working on client application code (for example a SaaS platform or an enterprise system) can seamlessly collaborate with ML teams by agreeing on prediction contracts, letting both teams work independently. Similarly, ML teams can define provisioning contracts, enabling teams to handle provisioning and configuration tasks autonomously. This decoupling eliminates dependencies and speeds up delivery timelines.

ML Applications streamlines the evolution of ML solutions by automating the version upgrade process. Updates are orchestrated entirely by ML Applications, mitigating errors and enabling the scaling of solutions to many instances (customers). Implementers can promote and verify changes across a chain of lifecycle environments (for example, dev, QA, pre-prod, prod), maintaining control over the release of changes to customers.

Tenant isolation stands out as a key benefit of ML Applications, ensuring complete segregation of data and workloads for each customer, thereby enhancing security and compliance. Unlike traditional methods reliant on process-based isolation, ML Applications lets customers implement restrictions based on privileges, safeguarding tenant isolation even in the event of defects.

Designed with scalability in mind, ML Applications means that large organizations, especially SaaS organizations, can adopt a standardized platform for ML development, ensuring consistency across many teams. Hundreds of ML solutions can be developed without the whole ML initiative descending into chaos. Thanks to the automation spanning the implementation of ML pipelines, deployment of models, and provisioning, SaaS organizations can industrialize the development of ML features for SaaS and efficiently manage millions of pipelines and models running autonomously in production.

Time to Market

Standardized implementation: Removes the need for custom frameworks, letting teams focus on business use cases.

Automation: Enhances speed and reduces manual interventions, crucial for scaling SaaS solutions.

Separation of concerns: Clear boundaries, contracts, and APIs enable different teams to work independently, streamlining development and deployment.

Development and Operating Costs

Reduced development costs: Using prebuilt services reduces the need for foundational work.

Evolvability: Quick iterations and independent upgrades with zero downtime ensure continuous improvement.

Traceability: Detailed insights into environments, components, and code revisions aid in understanding and managing ML solutions.

Automated provisioning and observability: Simplifies the management and monitoring of ML solutions, reducing operational overhead.

Fleet management: Manage the full lifecycle of ML Application Instances at scale. ML Applications provides visibility into all provisioned instances, support fleet-wide updates and upgrades, and enable provisioning and deprovisioning as needed. Also, ML solutions built with ML Applications can be monitored using OCI Monitoring to track performance and health.

Security and Reliability

Data and workload isolation: Ensures each customer's data and workloads are securely isolated.

Reliability: Automated updates and robust monitoring eliminate errors and ensure stable operations.

No noisy neighbor problem: Ensures workloads don't interfere with each other, maintaining performance and stability.

Build and Deployment Workflow

The development of ML Applications resembles standard software development.

Engineers and data scientists manage their implementation in a source code repository, using ML Applications documentation and repositories for guidance. They typically start by cloning a sample project provided by the ML Applications team, which promotes best practices and helps SaaS teams quickly begin development while avoiding a steep learning curve.

By starting with a fully functional end-to-end example, teams can rapidly build, deploy, and provision the example, becoming familiar with ML Applications quickly. They can then change the code to add specific business logic, such as adding a preprocessing step to the training pipeline or custom training code. text

Development and testing can be conducted locally on their computers, but setting up Continuous Integration/Continuous Delivery (CI/CD) is essential for faster release cycles, enhanced collaboration, and consistency. Teams can use examples and tools provided by the ML Application team to implement CI/CD pipelines.

CI/CD pipelines deploy the ML Application to the required environment (for example, Development) and perform integration tests to ensure the correctness of the ML solution independently from the SaaS app. These tests can deploy the Application and its implementation, create test instances, trigger the training flow, and test the deployed model. End-to-end tests ensure proper integration with the SaaS application.

When your ML Application is deployed, it's represented as an ML Application resource along with an ML Application Implementation. The ML Application Implementation Version resource helps track information about the different versions that you have deployed. For example, version 1.3 is the latest deployed version, which updates the previous version 1.2.

The CI/CD pipelines need to promote the solution through lifecycle environments up to production. In production, it's often necessary to deploy the ML Application to several regions to align with the client application deployment. When customers have several production environments (for example, staging and production), the CI/CD pipelines need to promote the ML Application through all these environments.

You can use ML Pipelines as application components within your ML Applications to implement a multistep workflow. ML Pipelines can then orchestrate arbitrary steps or jobs that you need to implement.

Deployment Operation

The deployment operation involves several key steps to ensure that ML Applications is correctly implemented and updated across environments.

First, it's necessary to ensure that the ML Application and ML Application Implementation resources exist. If not, they're created. If they already exist, they're updated:

Create or update the ML Application resource. This represents the whole ML use case and all related resources and information.
Create or update the ML Application Implementation resource. This exists under the ML Application resources and represents the implementation, letting you manage it. You need the ML Application Implementation to deploy your implementation package.
Upload the ML Application Package. This package contains the implementation of your ML Application along with other metadata, packaged as a zip archive with a specific structure.

When a package is uploaded to an ML Application Implementation resource, the ML Application service validates it and then performs the deployment. Simplifying it, the key steps during the deployment of the package are:

The application components (multitenant components) are instantiated.
Any existing ML Application Instance resources already provisioned for SaaS customers are updated. The instance components of each instance are updated to match the newly uploaded definitions.

Provisioning Flow

Before ML Application Instances can be provisioned, the ML Application must be deployed and registered to the client application.

1. Customer subscription and ML Application check: The provisioning flow starts when a customer subscribes to the client application. The client application checks whether any ML Applications are provisioned for the customer.

2. Provisioning request: If ML Applications are needed, the client application contacts the ML Application service and requests to provision an instance, including tenant-specific configuration details.

3. Creating and configuring an ML Application Instance: The ML Application service creates the ML Application Instance resource with the specific configuration. This resource is managed by the client application, which can activate, deactivate, reconfigure, or delete the instance. Also, the ML Application service creates an ML Application Instance View resource for providers to inform them about the ML Application's consumption and instance components. The ML Application service also creates all the instance components defined in the implementation, such as triggers, buckets, pipeline runs, job runs, and model deployments.

Note

When a new ML Application package is deployed after the creation of instances, the ML Application service ensures that all instances are updated with the latest implementation definitions.

Runtime Flow

When your ML Application is provisioned for a customer, the entire implementation is fully instantiated.

Typically, the ML Application runs training pipelines to train models and serve prediction requests from the client application. The implementation components can be divided into two groups:

Multitenant components: Instantiated only one time within an ML Application, supporting all customers (for example, ML pipelines). These are defined as application components.

Single tenant components: Instantiated for each customer, running only within the context of a particular customer. These are defined as instance components.

Pipelines or jobs in your implementation are started by triggers. Triggers create pipeline and job runs in the context of each customer, passing the appropriate parameters (context) to the created runs. Also, triggers are essential to ensuring tenant isolation. They ensure that the resource principal (identity) of the ML Application Instance resources is inherited by all the workloads they start. This lets you create policies that establish a security sandbox for each provisioned instance.

Pipelines let you define a set of steps and orchestrate them. For example, a typical flow might involve ingesting data from the client application, transforming it into a format suitable for training, and then training and deploying new models.

Model Deployment is used as the prediction backend but isn't accessed directly by the client application. The client application, which can be either single-tenant (provisioned for each customer) or multitenant (serving many customers), gets predictions by contacting the ML Application Prediction Router. The Prediction Router finds the correct prediction backend based on the instance used and the prediction use case implemented by your application.

Fetal Risk Example

Let's look at the implementation details of a sample ML Application to understand how to implement one.

We use a fetal risk prediction application that leverages Cardiotocography (CTG) data to classify the health status of a fetus. This application aims to help healthcare professionals by predicting whether the fetal condition is Normal, Risky, or Abnormal based on CTG exam records.

Implementation Details

The implementation follows the standard online prediction pattern and consists of a pipeline with ingestion, transformation, and model training and deployment steps.

Ingestion: This step prepares the raw dataset and stores it in the instance bucket.
Transformation: Data preparation is performed using the ADS library. This involves:
- Upsampling: The dataset is skewed toward normal cases, so upsampling is done to balance the classes.
- Feature Scaling: Features are scaled using QuantileTransformer.
- The prepared dataset is then stored back in the instance bucket.
Model Training and Deployment: The XGBoost algorithm is used for training the models because of its superior accuracy compared to other algorithms (for example, Random Forest, KNN, SVM). The ADS library is used for this process, and the trained model is deployed to the instance model deployment.

The implementation consists of:
- Application components: The major component is the pipeline that orchestrates three jobs:
  1. Ingestion job
  2. Transformation job
  3. Training job
- Instance components: Four instance components are defined here:
  1. Instance bucket
  2. Default model
  3. Model deployment
  4. Pipeline trigger
- descriptor.yaml: Contains the metadata describing the implementation (the package).

The source code is available in this project.

Review the descriptor.yaml file:

descriptorSchemaVersion: 1.0
mlApplicationVersion: 1.0
implementationVersion: 1.2
 
# Package arguments allow you to resolve dependencies of your implementation that are environment-specific.
# Typically, OCIDs of infrastructure resources like subnets, data science projects, logs, etc., are passed as package arguments.
# For illustration purposes, only 2 package arguments are listed here.
packageArguments:
 
  # The implementation needs a subnet, and it is environment-specific. It is provided as a package argument.
  subnet_id:
    type: ocid
    mandatory: true
    description: "Subnet OCID for ML Job"
 
  # similarly for the ID of a data science project
  data_science_project_id:
    type: ocid
    mandatory: true
    description: "Project OCID for ML Job"
 
# Configuration schema allows you to define the schema for your instance configuration.
# It will be used during provisioning, and the initial configuration provided must conform to the schema.
configurationSchema:
   
  # The implementation needs to know the name of the external bucket (not managed by ML Apps) where the raw dataset is available.
  external_data_source:
    type: string
    mandatory: true
    description: "External Data Source (OCI Object Storage Service URI in form of <a target="_blank" href="oci://">oci://</a><bucket_name>@<namespace>/<object_name>"
    sampleValue: "<a target="_blank" href="oci://test_data_fetalrisk@mynamespace/test_data.csv">oci://test_data_fetalrisk@mynamespace/test_data.csv</a>"
 
# This application provides 1 prediction use case (1 prediction service).
onlinePredictionUseCases:
  - name: "fetalrisk"

All OCI resources used in your implementation are defined using Terraform. Terraform lets you declaratively specify the components needed for your implementation.

The pipeline and all the jobs are defined similarly using Terraform. An example injection job definition:

# For illustration purposes, only a partial definition is listed here.
resource oci_datascience_job ingestion_job {
  compartment_id = var.app_impl.compartment_id
  display_name = "Ingestion_ML_Job"
  delete_related_job_runs = true
  job_infrastructure_configuration_details {
    block_storage_size_in_gbs = local.ingestion_job_block_storage_size
    job_infrastructure_type   = "STANDALONE"
    shape_name                = local.ingestion_job_shape_name
    job_shape_config_details {
      memory_in_gbs = 16
      ocpus         = 4
    }
    subnet_id                 = var.app_impl.package_arguments.subnet_id
  }
  job_artifact = "./src/01-ingestion_job.py"
}

Terraform is also used for instance components. An example bucket definition:

resource "oci_objectstorage_bucket" "data_storage_bucket" {
  compartment_id = var.compartment_ocid
  namespace      = var.bucket_namespace
  name           = "ml-app-fetal-risk-bucket-${var.instance_id}"
  access_type    = "NoPublicAccess"
}

Instance components typically refer to instance-specific variables such as instance id. You can rely on implicit variables defined by the ML Application service. When you need, for example, the name of the application, you can refer to it with ${var.ml_app_name}.

In conclusion, implementing ML Applications requires a few key components: a package descriptor and a couple of Terraform definitions. By following the documentation provided, you can learn details of how to build your own ML Applications.

Oracle Cloud Infrastructure Documentation

ML Applications

ML Applications Resources 🔗

ML Application Concepts 🔗

ML Application Roles 🔗

Lifecycle Management 🔗

Build 🔗

Deploy 🔗

Operate 🔗

Key Features 🔗

Added Value 🔗

Time to Market 🔗

Development and Operating Costs 🔗

Security and Reliability 🔗

Build and Deployment Workflow 🔗

Deployment Operation 🔗

Provisioning Flow 🔗

Runtime Flow 🔗

Fetal Risk Example 🔗

Implementation Details 🔗