RAG Tool Object Storage Guidelines for Generative AI Agents

Enhanced table understanding, a feature of RAG tools, aims to enhance the accuracy of responses to queries with answers embedded in PDF table data. It processes these tables to generate more precise and relevant responses aligned with the information they contain. In general, the RAG tools can read the tables. For the RAG tool to read the tables with enhanced table understanding, ensure that the tables have the following features:

• All cells of the table are separated with visible lines or object boundaries from other cells, including the header names in the first row.
• All columns including the first column have a header name.
• Each table has more than one column and more than one row, excluding the row with header names.

Use predefined metadata to apply filters during a chat. When filters are applied, an agent's searches in a chat session are limited to data files that are associated with the metadata, helping the model generate answers relevant to the content scope, thus enhancing the agent's response accuracy and relevance.

The following steps describe an overview of how to use the metadata filtering feature. After you understand the workflow overview, review the details for your use case in the sections provided after the overview steps.

1. In a text editor, create the metadata schema, which is required for the filters that you want to be made available. Write the schema in JSON format. Name the file _metadata_schema.json.

   Example:

   {
       "metadataSchema": [
           {
               "name": "publication_year",
               "type": "integer"
           },
           {
               "name": "title",
               "type": "string"
           }
   	]
   }

2. Upload the _metadata_schema.json file created in step 1 to the root level of the Object Storage bucket that contains the data files for a knowledge base.
3. Create JSON files to associate data files with the predefined metadata and provide the metadata values.

   Example:

   {
       "metadataAttributes": {
           "publication_year": 2020
       }
   } 

   You can associate one or more data files or all files in a bucket with the metadata. For details about the JSON file name conventions to use for the options you choose, see Metadata Filter Options (File Name and Location).

4. Upload the JSON files created in step 3 to the Object Storage bucket that contains the data files for a knowledge base. For each option, ensure that you save the file in the correct location in the hierarchy.
5. Create a knowledge base. Select Object Storage as the data store type, and the option to automatically start the ingestion job.

   When the data files are ingested, Generative AI Agents creates a list of the metadata names and the values that can be selected in a chat. To view the ingested metadata names and values, see Getting a Knowledge Base's Details in Generative AI Agents.

6. Create an agent with a RAG tool, selecting the knowledge base created in step 5. In the agent, select the option to automatically create an endpoint. If you need help, see Creating an Agent and Creating a RAG tool.
7. In a chat window, add one or more predefined metadata filters and select the values to apply. See Use Metadata Filters in a Chat.

This topic shows how to add or update the metadata object through OCI CLI.