Data Flow now Supports Structured Spark Streaming

Streaming applications require continuous execution for a long period of time that often extends beyond 24 hours, and might be as long as weeks or even months.

Data Flow:

• Implements a graceful shutdown of the Spark clusters for Spark structured streaming to allow for regular maintenance.
• Uses Oracle Cloud Infrastructure Streaming.
• Uses Spark structured streaming check-pointing to restart from the point of failure without producing incorrect computational results.
• Provides access to the Spark UI and Spark History Server.

For more information see the Spark Streaming chapter in the Data Flow documentation.