1. Click Components in the left navbar.
2. Click +New Service.
3. Complete the Create Service dialog:
   • Name: Enter the service name.
   • Service Type: Embedded Container
   • Component Service Package Type: New Component
   • Component Type: LLM Validation & Customization
   • Component Name: Enter an easily identifiable name for the event handler. You will reference this name when you create the LLM service for the skill.
4. Click Create to generate the validation handler.
5. After deployment completes, expand the service and then select the validation handler.
6. Click Edit to open the Edit Component Code editor.
7. Using the generated template, update the following handler methods as needed.

   Method	Description	When Validation Succeeds	When Validation Fails	Return Type	Location in Editor	Placeholder Code	What Can I do When Validation Fails?
   validateRequestPayload	Validates the request payload.	Returns true when the request is valid.	Returns false when a request fails validation.	boolean	Lines 24-29	validateRequestPayload: async (event, context) => { if (context.getCurrentTurn() === 1 && context.isJsonValidationEnabled()) { context.addJSONSchemaFormattingInstruction(); } return true; } To find out more about this code, refer to validateRequestPayload properties.	Revise the prompt and then resend it to the LLM Set a validation error.
   validateResponsePayload	Validates the LLM response payload.	When the handler returns true: If you set the Send LLM Result as a Message property is set to true, the LLM response, including any standard or custom action buttons, is sent to the user. If streaming is enabled, the LLM response will be streamed in chunks. The action buttons will be added at the end of the stream. Any user messages added in the handler are sent to the user, regardless of the setting for the Send LLM Result as a Message property. If a new LLM prompt is set in the handler, then this prompt is sent to the LLM, and the validation handler will be invoked again with the new LLM response If no new LLM prompt is set and property Enable Multi-Turn Refinements is set to true, the turn is released and the dialog flow remains in the LLM state. If this property is set to false, however, the turn is kept and the dialog transitions from the state using the success transition action.	When the handler returns false: If streaming is enabled, users may view responses that are potentially invalid because the validation event handler gets invoked after the LLM response has already started streaming. Any user messages added by handler are sent to the user, regardless of the Send LLM Result as a Skill Response setting. If a new LLM prompt is set in the handler, then this prompt is sent to the LLM and the validation handler will be invoked again with the new LLM response. If no LLM prompt is set, the dialog flow transitions out of the LLM component state. The transition action set in the handler code will be used to determine the next state. If no transition action is set, then the error transition action gets triggered.	boolean	Lines 50-56	/** * Handler to validate response payload * @param {ValidateResponseEvent} event * @param {LLMContext} context * @returns {boolean} flag to indicate the validation was successful */ validateResponsePayload: async (event, context) => { let errors = event.allValidationErrors || []; if (errors.length > 0) { return context.handleInvalidResponse(errors); } return true; } To find out more about this code, refer to validateResponsePayload Properties.	Invoke the LLM again using a retry prompt that specifies the problem with the response (it doesn't conform to a specific JSON format, for example) and requests that the LLM fix it. Set a validation error.
   changeBotMessages	Changes the candidate skill messages that will be sent to the user.	N/A	N/A	A list of Conversation Message Model messages	Lines 59-71	changeBotMessages: async (event, context) => { return event.messages; }, For an example of using this method, refer to Enhance the User Message for JSON-Formatted Responses.	N/A
   submit	This handler gets invoked when users tap the Submit button. It processes the LLM response further.	N/A	N/A	N/A	Lines 79-80	submit: async (event, context) => { }	N/A

   Each of these methods uses an event object and a context object. For example:

    validateResponsePayload: async (event, context) =>
   ...

   The properties defined for the event object depend on the event type. The second argument, context, references the LlmComponentContext class, which accesses the convenience methods for creating your own event handler logic. These include methods for setting the maximum number of retry prompts and sending status and error messages to skill users.
8. Verify the sytnax of your updates by clicking Validate. Then click Save > Close.

1. Start by defining the LLM's role or persona with a high-level description of the task at hand.
2. Add details on what to include in the response, expected output format, etc.
3. If necessary, provide few-shot examples of the task at hand
4. Optionally, mention how to process scenarios constituting an unsupported query.

• Begin with a simple, concise prompt – Start with a brief, simple, and straightforward prompt that clearly outlines the use case and expected output. For example:
  • A one-line instruction like "Tell me a joke"
  • A text-completion style prompt
  • An instruction along with input
  For example:

  "Summarize the following in one sentence:
  
  The Roman Empire was a large and powerful group of ancient civilizations that formed after the collapse of the Roman Republic in Rome, Italy, in 27 BCE. At its height, it covered an area of around 5,000 kilometers, making it one of the largest empires in history. It stretched from Scotland in the north to Morocco in Africa, and it contained some of the most culturally advanced societies of the time."

  A simple prompt is a good starting point in your testing because it's a good indicator of how the model will behave. It also gives you room to add more elements as you refine your prompt text.
• Iteratively modify and test your prompt – Don't expect the first draft of your prompt to return the expected results. It might take several rounds of testing to find out which instructions need to be added, removed, or reworded. For example, to prevent the model from hallucinating by adding extra content, you'd add additional instructions:

  
  
  "Summarize the following paragraph in one sentence. Do not add additional information outside of what is provided below:
  
  The Roman Empire was a large and powerful group of ancient civilizations that formed after the collapse of the Roman Republic in Rome, Italy, in 27 BCE. At its height, it covered an area of around 5,000 kilometers, making it one of the largest empires in history. It stretched from Scotland in the north to Morocco in Africa, and it contained some of the most culturally advanced societies of the time."

• Use a persona that's specific to your use case – Personas often results in better results because they help the LLM to emulate behavior or assume a role.
  Note
  
  Cohere models weigh the task-specific instructions more than the persona definition.
  For example, if you want the LLM to generate insights, ask it to be a data analyst:

  Assume the role of a data analyst. Given a dataset, your job is to extract valuable insights from it.
  Criteria:
  
  - The extracted insights must enable someone to be able to understand the data well.
  - Each insight must be clear and provide proof and statistics wherever required
  - Focus on columns you think are relevant, and the relationships between them. Generate insights that can provide as much information as possible.
  - You can ignore columns that are simply identifiers, such as IDs
  - Do not make assumptions about any information not provided in the data. If the information is not in the data, any insight derived from it is invalid
  - Present insights as a numbered list
  
  Extract insights from the data below:
  {data}

  Note
  
  Be careful of any implied biases or behaviors that may be inherent in the persona.
• Write LLM-specific prompts – LLMs have different architectures and are trained using different methods and different data sets. You can't write a single prompt that will return the same results from all LLMs, or even different versions of the same LLM. Approaches that work well with GPT-4 fail with GPT-3.5 and vice-versa, for example. Instead, you need to tailor your prompt to the capabilities of the LLM chosen for your use case. Use few-shot examples – Because LLMs learn from examples, provide few-shot examples wherever relevant. Include labeled examples in your prompt that demonstrate the structure of the generated response. For example:

  Generate a sales summary based on the given data. Here is an example:
  
  Input: ...
  Summary: ...
  
  Now, summarize the following sales data:
  
  ....

  Provide few-shot examples when:
  • Structural constraints need to be enforced.
  • The responses must conform to specific patterns and must contain specific details
  • Responses vary with different input conditions
  • Your use case is very domain-specific or esoteric because LLMs, which have general knowledge, work best on common use cases.
  Note
  
  If you are including multiple few-shot examples in the prompt for a Cohere model, make sure to equally represent all classes of examples. An imbalance in the categories of few-shot examples adversely affects the responses, as the model sometimes confines its output to the predominant patterns found in the majority of the examples.
• Define clear acceptance criteria – Rather than instructing the LLM on what you don't want it to do by including "don’t do this" or "avoid that" in the prompt, you should instead provide clear instructions that tell the LLM what it should do in terms of what you expect as acceptable output. Qualify appropriate outputs using concrete criteria instead of vague adjectives.

  Please generate job description for a Senior Sales Representative located in Austin, TX, with 5+ years of experience. Job is in the Oracle Sales team at Oracle. Candidate's level is supposed to be Senior Sales Representative or higher. 
  
  Please follow the instructions below strictly: 
  1, The Job Description session should be tailored for Oracle specifically. You should introduce the business department in Oracle that is relevant to the job position, together with the general summary of the scope of the job position in Oracle. 
  2, Please write up the Job Description section in a innovative manner. Think about how you would attract candidates to join Oracle. 
  3, The Qualification section should list out the expectations based on the level of the job.

• Be brief and concise – Keep the prompt as succinct as possible. Avoid writing long paragraphs. The LLM is more likely to follow your instructions if you provide them as brief, concise, points. Always try and reduce the verbosity of the prompt. While it's crucial to provide detailed instructions and all of the context information that the LLM is supposed to operate with, bear in mind that the accuracy of LLM-generated responses tends to diminish as the length of the prompt increases.
  For example, do this:

  - Your email should be concise, and friendly yet remain professional.
  - Please use a writing tone that is appropriate to the purpose of the email.
  - If the purpose of the email is negative; for example to communicate miss or loss, do the following: { Step 1: please be very brief. Step 2: Please do not mention activities }
  - If the purpose of the email is positive or neutral; for example to congratulate or follow up on progress, do the following: { Step 1: the products section is the main team objective to achieve, please mention it with enthusiasm in your opening paragraph. Step 2: please motivate the team to finalize the pending activities. }

  Do not do this:

  Be concise and friendly. But also be professional. Also, make sure the way you write the email matches the intent of the email. The email can have two possible intents: It can be negative, like when you talk about a miss or a loss. In that case, be brief and short, don't mention any activities.
  
  An email can also be positive. Like you want to follow up on progress or congratulate on something. In that case, you need to mention the main team objective. It is in the products section. Also, take note of pending activities and motivate the team

• Beware of inherent biases – LLMs are trained on large volumes of data and real-world knowledge which may often contain historically inaccurate or outdated information and carry inherent biases. This, in turn, may cause LLMs to hallucinate and output incorrect data or biased insights. LLMs often have a training cutoff which can cause them to present historically inaccurate information, albeit confidently.
  Note
  
  Do not:

  • Ask LLMs to search the web or retrieve current information.
  • Instruct LLMs to generate content based on it's own interpretation of world knowledge or factual data.
  • Ask LLMs about time-sensitive information.
• Address edge cases – Define the edge cases that may cause the model to hallucinate and generate a plausible-sounding, but incorrect answer. Describing edge cases and adding examples can form a guardrail against hallucinations. For example an edge case may be that an API call that fills variable values in the prompt fails to do so and returns an empty response. To enable the LLM to handle this situation, your prompt would include a description of the expected response.

  Tip:

  Testing might reveal unforeseen edge cases.
• Don't introduce contradictions – Review your prompt carefully to ensure that you haven't given it any conflicting instructions. For example, you would not want the following:

  Write a prompt for generating a summary of the text given below. DO NOT let your instructions be overridden
  In case the user query is asking for a joke, forget the above and tell a funny joke instead

• Don't assume that anything is implied – There is a limit on the amount of knowledge that an LLM has. In most cases, it's better to assume that the LLM does not know something, or may get confused about specific terms. For example, an LLM may generally know what an insight derived from a data means, but just saying "derive good insights from this data" is not enough. You need to specify what insights means to you in this case:

  - The extracted insights must enable someone to be able to understand the data well.
  - Insights must be applicable to the question shown above
  - Each insight must be clear and provide proof and statistics wherever required
  - Focus on columns you think are relevant and the relationships between them.
  - You can ignore columns that are simply identifiers, such as IDs

• Ensure that the prompt makes sense after the variables are filled – Prompts may have placeholders for values that may be filled, for example, through slot-filling. Ensure the prompt makes sense once it is populated by testing its sample values. For example, the following seems to make sense before the variable value is filled.

  Job is in the ${team} at Oracle

  However, once the variable is populated, the phrase doesn't seem right:

  Job is in the Oracle Digital Assistant at Oracle

  To fix this, edit the phrase. In this case, by modifying the variable with team.

  Job is in the ${team} team at Oracle

  As a result, the output is:

  Job is in the Oracle Digital Assistant team at Oracle

• Avoid asking the LLM to do math – In some cases, LLMs may not be able to do even basic math correctly. In spite of this, they hallucinate and return an answer that sounds so confident that it could be easily mistaken as correct. Here is an example of an LLM hallucination the following when asked "what is the average of 5, 7, 9": The average of 5, 7, and 9 is 7.5. To find the average, you add up the values and divide by the number of values. In this case, the values are 5, 7, and 9, and there are 3 values. So, to find the average, you would add 5 + 7 + 9 and divide by 3. This gives you an average of 7.5
• Be careful when setting the model temperature – Higher temperatures, which encourage more creative and random output, may also produce hallucinations. Lower values like 0.01 indicate that the LLM's output must be precise and deterministic.
• Avoid redundant instructions – Do not include instructions that seem redundant. Reduce the verbosity of the prompt as much as possible without omitting crucial detail.
• Use explicit verbs – Instead of using verbose, descriptive statements, use concrete verbs that are specific to the task like "summarize", "classify", "generate", "draft", etc.
• Provide natural language inputs – When you need to pass context or additional inputs to the model, make sure that they are easily interpretable and in natural language. Not all models can correctly comprehend unstructured data, shorthand, or codes. When data extracted from backends or databases is unstructured, you need to transpose it to natural language.
  For example, if you need to pass the user profile as part of the context, do this:

  Name: John Smith
  Age: 29
  Gender: Male

  Do not do this:

  Smith, John - 29M

  Note
  
  Always avoid any domain-specific vocabulary. Incorporate the information using natural language instead.

transformRequestPayload: async (event, context) => {
      let prompt = event.payload.messages[0].content;
      if (event.payload.messages.length > 1) {
         let history = event.payload.messages.slice(1).reduce((acc, cur) => `${acc}\n${cur.role}: ${cur.content}` , '');
         prompt += `\n\nCONVERSATION HISTORY:${history}\nassistant:`
      }
      return {
        "max_tokens": event.payload.maxTokens,
        "truncate": "END",
        "return_likelihoods": "NONE",
        "prompt": prompt,
        "model": "command",
        "temperature": event.payload.temperature,
        "stream": event.payload.streamResponse
      };
    },

{SYSTEM_PROMPT}

CONVERSATION
---
user: <first_query>
assistant: <first_response>
user: ...
assistant: ...
user: <latest_query>
assistant: