com.oracle.bmc.generativeaiinference

Interface GenerativeAiInference

All Superinterfaces:

AutoCloseable

All Known Implementing Classes:

GenerativeAiInferenceClient

@Generated(value="OracleSDKGenerator",
           comments="API Version: 20231130")
public interface GenerativeAiInference
extends AutoCloseable

OCI Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation, summarization, and text embeddings.

Use the Generative AI service inference API to access your custom model endpoints, or to try the out-of-the-box models to eNGenerative-ai-inferenceLatestChatResultChat, eNGenerative-ai-inferenceLatestGenerateTextResultGenerateText, eNGenerative-ai-inferenceLatestSummarizeTextResultSummarizeText, and eNGenerative-ai-inferenceLatestEmbedTextResultEmbedText.

To use a Generative AI custom model for inference, you must first create an endpoint for that model. Use the eNGenerative-aiLatest to eNGenerative-aiLatestModel by fine-tuning an out-of-the-box model, or a previous version of a custom model, using your own data. Fine-tune the custom model on a eNGenerative-aiLatestDedicatedAiCluster. Then, create a eNGenerative-aiLatestDedicatedAiCluster with an Endpoint to host your custom model. For resource management in the Generative AI service, use the eNGenerative-aiLatest.

To learn more about the service, see the [Generative AI documentation](https://docs.cloud.oracle.com/iaas/Content/generative-ai/home.htm).

This service client uses CircuitBreakerUtils.DEFAULT_CIRCUIT_BREAKER for all the operations by default if no circuit breaker configuration is defined by the user.

Method Summary

Modifier and Type	Method and Description
ChatResponse	chat(ChatRequest request) Creates a response for the given conversation.
EmbedTextResponse	embedText(EmbedTextRequest request) Produces embeddings for the inputs.
GenerateTextResponse	generateText(GenerateTextRequest request) Generates a text response based on the user prompt.
String	getEndpoint() Gets the set endpoint for REST call (ex, https://www.example.com)
void	refreshClient() Rebuilds the client from scratch.
void	setEndpoint(String endpoint) Sets the endpoint to call (ex, https://www.example.com).
void	setRegion(Region region) Sets the region to call (ex, Region.US_PHOENIX_1).
void	setRegion(String regionId) Sets the region to call (ex, ‘us-phoenix-1’).
SummarizeTextResponse	summarizeText(SummarizeTextRequest request) Summarizes the input text.
void	useRealmSpecificEndpointTemplate(boolean realmSpecificEndpointTemplateEnabled) Determines whether realm specific endpoint should be used or not.

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

refreshClient

void refreshClient()

Rebuilds the client from scratch. Useful to refresh certificates.

setEndpoint

void setEndpoint(String endpoint)

Sets the endpoint to call (ex, https://www.example.com).

Parameters:

endpoint - The endpoint of the service.

getEndpoint

String getEndpoint()

Gets the set endpoint for REST call (ex, https://www.example.com)

setRegion

void setRegion(Region region)

Sets the region to call (ex, Region.US_PHOENIX_1).

Note, this will call setEndpoint after resolving the endpoint. If the service is not available in this Region, however, an IllegalArgumentException will be raised.

Parameters:

region - The region of the service.

setRegion

void setRegion(String regionId)

Sets the region to call (ex, ‘us-phoenix-1’).

Note, this will first try to map the region ID to a known Region and call setRegion.

If no known Region could be determined, it will create an endpoint based on the default endpoint format (Region.formatDefaultRegionEndpoint(Service, String) and then call setEndpoint.

Parameters:

regionId - The public region ID.

useRealmSpecificEndpointTemplate

void useRealmSpecificEndpointTemplate(boolean realmSpecificEndpointTemplateEnabled)

Determines whether realm specific endpoint should be used or not. Set realmSpecificEndpointTemplateEnabled to “true” if the user wants to enable use of realm specific endpoint template, otherwise set it to “false”

Parameters:

realmSpecificEndpointTemplateEnabled - flag to enable the use of realm specific endpoint template

chat

ChatResponse chat(ChatRequest request)

Creates a response for the given conversation.

Parameters:

request - The request object containing the details to send

Returns:

A response object containing details about the completed operation

Throws:

BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

Example: Click here to see how to use Chat API.

embedText

EmbedTextResponse embedText(EmbedTextRequest request)

Produces embeddings for the inputs.

An embedding is numeric representation of a piece of text. This text can be a phrase, a sentence, or one or more paragraphs. The Generative AI embedding model transforms each phrase, sentence, or paragraph that you input, into an array with 1024 numbers. You can use these embeddings for finding similarity in your input text such as finding phrases that are similar in context or category. Embeddings are mostly used for semantic searches where the search function focuses on the meaning of the text that it's searching through rather than finding results based on keywords.

Parameters:

request - The request object containing the details to send

Returns:

A response object containing details about the completed operation

Throws:

BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

Example: Click here to see how to use EmbedText API.

generateText

GenerateTextResponse generateText(GenerateTextRequest request)

Generates a text response based on the user prompt.

Parameters:

request - The request object containing the details to send

Returns:

A response object containing details about the completed operation

Throws:

BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

Example: Click here to see how to use GenerateText API.

summarizeText

SummarizeTextResponse summarizeText(SummarizeTextRequest request)

Summarizes the input text.

Parameters:

request - The request object containing the details to send

Returns:

A response object containing details about the completed operation

Throws:

BmcException - when an error occurs. This operation uses RetryConfiguration.SDK_DEFAULT_RETRY_CONFIGURATION as default if no retry strategy is provided. The specifics of the default retry strategy are described here https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/javasdkconcepts.htm#javasdkconcepts_topic_Retries

Example: Click here to see how to use SummarizeText API.