Creating SQL Endpoints
Follow these steps to create a Data Flow SQL endpoint.
Follow the steps to create a Metastore and set up IAM policies. - On the SQL endpoints page, select Create SQL endpoint. If you need help finding the SQL endpoints page, see Listing SQL Endpoints.
- Enter the name of the SQL endpoint in Name.
- (Optional) Description enter a description of the SQL Endpoint that you can use to help search for it.
- (Optional) Select Enable autoscaling to use autoscaling.
-
From the Driver shape list, select the shape you
want.
Select the number of OCPUs if you want a different number to the default value. The memory and block volume amounts change.
-
From the Executor shape list, select the shape you
want.
Select the number of OCPUs if you want a different number to the default value. The memory and block volume amounts change.
- Select the minimum number of executors.
- Select the maximum number of executors.
-
Select the metastore. If the metastore is in a different compartment, click
Change compartment to select its compartment.
Note
You must create a metastore in Data Catalog if one doesn't exist. - (Optional) The default managed table location is set automatically using the same default managed table location configured for the metastore.
-
Set the network configuration, by selecting either Allow secure
access from everywhere, or Virtual Cloud
Network.
- If you selected Allow secure access from
everywhere and selected Configure access
control rules:
- Select the IP notation type.
- Enter value.
- (Optional) Select +Another entry to add more access control rules.
- If you selected Virtual Cloud Network:
- Select the VCN. If the VCN is in a different compartment, select Change compartment to select its compartment.
- Select the subnet. If the subnet is in a different compartment, select Change compartment to select its compartment.
- (Optional) Enter a host name prefix.
- Select the network security group. If the network security group is in a different compartment, select Change compartment to select its compartment.
- (Optional) Select +Another Network security group to add more network security groups.
- If you selected Allow secure access from
everywhere and selected Configure access
control rules:
-
To set an advanced options select Show
advanced options.
The Spark configuration properties are shown. You can change the values for:
- spark.dynamicAllocation.executorIdleTimeout
- spark.dynamicAllocation.schedulerBacklogTimeout
- spark.dataflow.dynamicAllocation.quotaPolicy
- (Optional) To add tags to the application, select a tag namespace (for defined tags) and populate then specify a tag key and value. Add more tags as needed. For more information about tagging, see Overview of Tagging.
- Select Create SQL endpoint to create the SQL endpoint.
Use the create command and required parameters to create a SQL endpoint
oci data-flow sql-endpoint create [OPTIONS]
For a complete list of parameters and values for CLI commands, see the CLI Command Reference.
Run the CreateSqlEndpoint operation to create a SQL endpoint.