Using Oracle Logging Analytics,
you can enrich event data by adding field-value combinations from lookups. Oracle Logging Analytics uses lookups to match
field-value combinations from events to an external lookup table, and if matched, Oracle Logging Analytics appends the field-value
combinations to the events.
Types of lookups:
Simple: This table would contain a simple listing of items where the rows
correspond to a single relation. For example, if you create a lookup table of
errors, then each row can have the information for a single error like error ID,
error message, cause, corrective action. See Create a Simple Lookup.
Dictionary: This is a smarter option to enlist the logical relations
between fields and their values using operators like
CONTAINS, CONTAINS IGNORE CASE MULTILINE
REGEX, and EQUAL. For an exhaustive list of
operators, examples, and steps to create this type of lookup, see Create a Dictionary Lookup.
Following are some of the Oracle-defined simple lookup tables currently available in Oracle Logging Analytics:
The size of a single lookup file must not exceed 10 MB. However, there is no
limit to the number of rows in the lookup.
Note
If you want to prevent a user from viewing the content of a lookup, even indirectly, then that user MUST NOT be given permission to:
Upload logs
View the logs that contain the fields enriched using such restricted lookups
How Do Lookups Enrich
Logs π
Often, the logs would not contain all the required information to consolidate
the analysis. In such cases, you can create a lookup table to pickup the information
that you have already collated to make more sense of the data in the logs. For example,
the Error ID field in log events doesnβt provide a description of the errors. You
can create a lookup that maps Error ID to descriptions, and then use the Field
Enrichment options to make the descriptions available to search in the log records. Some
example scenarios where you could create lookups:
Product inventory: Product ID, technical specifications, price,
available stock, rack number
You can also use simple lookups at query-time as shown below:
In the following lookup query, the error message is picked
up from the lookup table ORA Error Messages by mapping the Error ID
field, and displayed in a summary table along with other fields like log source and
entity:
Error ID: Error ID as mentioned in the logs
specified in the form ORA%
Log Source: The log source of the logs
Entity: The entity of the logs
Information from the Oracle-defined lookup table ORA Error
Messages:
errid: The error ID specified in the form
ORA%
errmsg: The error message for the error ID
The other details available in the ORA Error
Messages lookup table are errcause (Error Cause)
and erraction (Error Action) for each error ID. To view the
contents of the lookup table, use the searchlookup
command as
follows:
* | searchlookup table = 'ORA Error Messages'
View the Lookup Detail π
Open the navigation
menu and click Observability & Management. Under
Logging Analytics, click
Administration. The Administration
Overview page opens.
The administration resources are listed in the left hand navigation
pane under Resources. Click
Lookups.
Under Lookups, click on the lookup whose
details you want to view.
The Lookup Detail page displays the details that you had provided while creating it, and also the content of the lookup table. You can also perform the actions like Move resource, Add tags, and Delete. For details about these actions, see the following section.
To perform other actions on the lookup, in the Lookups listing page, select the Compartment, Type, and Creation Type of your lookup, and narrow down your search for your lookup in the Filters section. You can also include Tag filters to filter lookups by tags. After identifying your lookup, click Actions icon in the row of your lookup:
Reupload File: In case of user-created lookups, if you have a new version of the lookup table, or if a previous upload has failed, then you can re-upload it.
Delete: You may have to delete some of your old or unused
lookups. To delete the lookup, confirm in the Delete Lookup dialog box.
You can delete a lookup only if it is user-created.
Copy OCID: The OCID of the lookup resource is copied. You can use it to refer to this resource anywhere in Oracle Cloud Infrastructure.
Move resource: The Move Resource to a Different Compartment dialog box is displayed. From the options, select the compartment to which you want to move the lookup, and click Move Resource.
Update your CSV file with the field-value combinations.
Open the navigation
menu and click Observability & Management. Under
Logging Analytics, click
Administration. The Administration
Overview page opens.
The administration resources are listed in the left hand navigation
pane under Resources. Click
Lookups.
Under Lookups, click the Actions icon in the row that corresponds to the lookup that you want to update,
click Reupload File.
The Reupload File dialog box opens.
Select the updated lookup CSV file that you had created earlier, and
click Upload.
Create a Simple Lookup π
After creating the simple lookup, run a query with the searchlookup
command to list the values in the lookup, and use the lookup command in a query to
map the output fields to those values.
Create a lookup CSV file with the field-value combinations. For
example, to create a lookup that maps Error ID to descriptions:
errid,description
02323,Network Not Reachable
09912,User Activity
12322,Out of Memory
Note that the first row is the header with errid
and description titles for the values in the subsequent
rows.
Open the navigation
menu and click Observability & Management. Under
Logging Analytics, click
Administration. The Administration
Overview page opens.
The administration resources are listed in the left hand navigation
pane under Resources. Click
Lookups.
The lookups listing page is displayed. From the left hand navigation pane, select the compartment where the lookup must be created.
Under Lookups, click Create
Lookup.
In the Create Lookup page, enter the name of the lookup, such as server error code lookups and an optional description.
Under Type, select Simple.
Select the lookup CSV file that you had created earlier.
Optionally, expand the section Show Advanced Options, and add tags to the lookup. Click Create.
Create a Dictionary
Lookup π
After creating the Dictionary type lookup, use searchlookup
command to list the lookups. Use the lookup command to map to the fields with any
query only after using the link or cluster commands in the query.
Create a lookup CSV file with the field-value combinations.
For example:
Operator,Condition,Issue,Area
CONTAINS,message header or abbreviation processing failed,Processing Error,Messaging
CONTAINS,Failed to associate the transaction context with the response while marshalling,Marshalling Error,Response
CONTAINS,A RuntimeException was generated by the RMI server,Exception,RMI
Note that the first row is the header where
Operator and Condition are the
mandatory fields specified in the same order. The subsequent parameters are
listed in the header row after the mandatory parameters. The subsequent rows
are the values of the parameters listed in the header row in the same
order.
See the table below for the list of valid operators and examples
to use them.
Note:
If a field contains a comma, enclose the entire field in
double quotes.
If a field contains double quotes, escape the double
quote by using two double quotes.
Open the navigation
menu and click Observability & Management. Under
Logging Analytics, click
Administration. The Administration
Overview page opens.
The administration resources are listed in the left hand navigation pane
under Resources. Click Lookups.
The lookups listing page is displayed. From the left hand navigation pane, select the compartment where the lookup must be created.
Under Lookups, click the Create
Lookup link.
In the Create Lookup page, enter the name of
the lookup, such as cluster dictionary lookups and an
optional description.
Under Type, select Dictionary.
Select the lookup CSV file that you had created earlier.
Optionally, expand the section Show Advanced Options, and add tags to the lookup. Click Create.
CIDRMATCH Operator
The CIDRMATCH operator supports CIDR (Classless
Inter-Domain Routing) match operation rule inside a dictionary lookup. For example,
the following dictionary returns Network Name as Database Network if
the input IP Address falls in the range between 192.0.2.0 and 192.0.2.255:
Operator,Condition,Network Name
CIDRMATCH,192.0.2.10/24,Database Network
List of Valid Operators and Examples for Using
Them
Operator
Description
Example
CONTAINS
True if the value contains the string specified in
the Condition field. Case-sensitive
List of Numeric and Logical Operators and Examples
Note
Ensure that the numeric operators
are not mixed with the string operators in the same dictionary. If the operators are
mixed in the same dictionary and a string value is passed in the query, then an
error message is returned about the type while matching using the numerical
operators.
Operator
Description
Example
=
Numerical Equal To
=,1,Value is 1
!=
Numerical Not Equal To
!=,1,Value is Not 1
>
Above the given value
>,1,Value is above 1
<
Below the given value
<,1,Value is below 1
>=
Above or equal to the given value
>=,1,Value is equal or above
1
<=
Below or equal to the given value
<=,1,Value is equal or below
1
BETWEEN
Between the given two values, both inclusive
BETWEEN,1-10,Value is equal or above 1 and
equal or below 10
> AND <
Above N1 and Below N2
> AND <,1-10,Above 1 and below
10
>= AND <=
Same as Between. Above or equal to N1 and Below or
equal to N2
>= AND <=,1-10,Above or equal to 1 and
below or equal to 10
>= AND <
Above or equal to N1 and Below N2
>= AND <,1-10,Above or equal to 1 and
below 10
> AND <=
Above N1 and Below or equal to N2
> AND <=,1-10,Above 1 and below or
equal to 10
> OR <
Above N1 or Below N2
> OR <,1-10
Above 1 or below 10
>= OR <=
Above or equal to N1 or Below or equal to N2
>= OR <=,100-10,Above or equal to 100
or below or equal to 10
>= OR <
Above or equal to N1 or Below N2
>= OR <,10-1,Above or equal to 10 or
below 1
> OR <=
Above N1 or Below or equal to N2
> OR <=,100-10,Above 100 or below or
equal to 10
>= OR !=
Above or equal to N1 or not equal to N2
>= OR !=,10-1,Above or equal to 10 or not
equal to 1
<= OR !=
Below or equal to N1 or not equal to N2
<= OR !=,10-100,Below or equal to 10 or
not equal to 100
>= OR =
Above or equal to N1 or equal to N2
>= OR =,10-1,Above or equal to 1 or equal
to 1
<= OR =
Below or equal to N1 or equal to N2
<= OR =,10-100,Below or equal to 10 or
equal to 100
> AND !=
Above N1 and not equal to N2
> AND !=,10-100,Above 10 and not equal to
100
< AND !=
Below N1 and not equal to N2
< AND !=,10-1,Below 10 and not equal to
1
Use Comments While Defining Dictionary Lookups
Use # as the first field to add comments to a
dictionary lookup. Following is an example of a sample lookup with comments:
Operator,Condition,Label,Module
# ----------------------------------
# Startup/Shutdown and Terminations
# ----------------------------------
CONTAINS,Server started in RUNNING mode,Server Started,WebLogic Server
CONTAINS,A critical service failed. The server will shut itself down,Server Shutdown,WebLogic Server
CONTAINS,state changed to FAILED,Server Failed,
CONTAINS,Removing .* from cluster view due to PeerGone,Cluster Removed,WebLogic Server
# ----------------------
# Connection Error / Timeouts and Slowness
# ----------------------
CONTAINS,Unable to connect to WSM policy manager,WSM Policy Manager Connection Error,
CONTAINS REGEX,java.sql.SQLTimeoutException: \S+: user requested cancel of current operation,SQL Timeout,Database
CONTAINS,This member is running extremely slowly and may endanger the rest of the cluster,WebLogic Cluster Slowness,WebLogic Server
Use Dictionary Lookup in a
Query π
You can use the same dictionary in multiple queries. For example, the following
query applies the same dictionary to the Message field and shows the
rolled up results using the Pie Chart visualization:
'Log Source' = 'Linux Syslog Logs'
| lookup table = 'Linux Error Categories' select Issue, Area using Message
| stats count by Area
Allow Users to Perform All Operations on Lookups π
Individual resource-type: loganalytics-lookup
Part of aggregate resource-type:
loganalytics-features-family
Resource Policy Statement if Family Policy Not Defined:
Use Case
IAM Policies
Lookup can be in the tenancy
allow group <user_group> to READ loganalytics-lookup in tenancy
Restrict the permission for the lookup to a specific compartment.
allow group <user_group> to READ loganalytics-lookup in compartment <compartment_name>
Note that for the above policy statement to restrict the permission to the lookup resource with the required scope (tenancy or compartment) to work, there mustn't be any other policy statements that cover the resource-type family and a larger scope.
Note
If you want to prevent a user from viewing the content of a lookup, even indirectly, then that user MUST NOT be given permission to:
Upload logs
View the logs that contain the fields enriched using such restricted lookups
The following operations can be performed with each verb when you create IAM policy for loganalytics-lookup:
Inspect
Read
Use
Manage
NA
NA
Register a lookup
Manage has the same level of permissions and API operations as Use.
Typically, only administrators have USE and MANAGE permissions to create or delete lookup. If you want privileges to run lookup queries, then you require READ permission.
Aggregate Family Policy Statement Which Covers the Permission for the Lookup Resource:
Use Case
IAM Policies
Lookup can be in the tenancy
allow group <user_group> to read loganalytics-features-family in tenancy
Note
If you enabled Oracle Logging Analytics using the
onboarding UI which is available when you navigate to the service for the first time, then
some policies are already created. See Policies Created While Onboarding Logging Analytics.
Permissions for Using Lookup Resources
Typically, only administrators have USE and MANAGE permissions to create or delete lookup. If you want privileges to run lookup queries, then you require READ permission.
If you're already using Log Explorer, SDKs, or CLI to run the queries, then you already have the required permissions to run searchlookup or lookup query commands on the existing Oracle-defined and user-created lookups.
If you're using instance principal to run the queries, then provide the following permission if you have not already provided loganalytics-features-family access to the dynamic group:
Allow dynamic-group <dynamic-group-name> to {LOG_ANALYTICS_LOOKUP_READ} in tenancy
OR
Allow dynamic-group <dynamic-group-name> to {LOG_ANALYTICS_LOOKUP_READ} in compartment <compartment_name>