CloudFlow coordinates distributed applications and microservices to build complex, multi-step, asynchronous tasks and long-running flows.
Transactional flow orchestration
Complex scenarios such as e-commerce, hotel booking, and flight reservations require applications to access multiple remote services with strict transactional semantics—all steps must either succeed or fail with no intermediate state. With small traffic and centralized storage, the ACID properties of relational databases can meet this requirement. In large-traffic scenarios, however, distributed microservices are typically used for high availability and scalability. Ensuring reliable multi-step transactions then requires additional queues and databases to store messages and track flow status, which increases development and O&M costs. CloudFlow ensures reliable distributed transaction processing in complex flows, letting you focus on business logic.
For more information about how to use CloudFlow to orchestrate transactional flows, see Reliably process distributed multi-step transactions.
Multimedia file processing
CloudFlow helps you orchestrate tasks such as transcoding, frame capture, face recognition, voice recognition, and review and upload into a complete multimedia processing flow. Use Function Compute to submit an Intelligent Media Management (IMM) task or a custom processor to produce the required output. Failed tasks can be reliably retried, significantly improving processing throughput.
Genetic data processing
CloudFlow orchestrates distributed batch computing jobs sequentially or in parallel and reliably handles large-scale, long-running, high-concurrency workloads. For example, in genetic data analysis, gene sequences are aligned, variation analysis runs on all chromosomes in parallel, and all chromosome data is then aggregated to produce results. Based on specified dependencies, CloudFlow submits batch computing jobs with varying CPU, memory, and bandwidth specifications, improving execution reliability and resource utilization while reducing costs.
Data pipelines
You can use CloudFlow to build highly available data pipelines. For example, measurement data from different sources is collected into Simple Log Service. A time-based trigger of Function Compute triggers CloudFlow at a specified time. CloudFlow uses Function Compute to process measurement data across multiple shards in parallel and write the results back to Simple Log Service. The data from all shards is then aggregated and written to Tablestore, and bills are generated. CloudFlow lets you retry failed steps to reduce failure probability. CloudFlow supports dynamic parallel task execution for highly scalable data processing.
Automated O&M
Automated O&M often involves cumbersome steps, unpredictable execution times, unreliable standalone scripts, complex dependencies, and lack of progress visibility. Combining CloudFlow and Function Compute addresses these challenges. For example, during automated software deployment, you build Docker containers, upload container images, start and track nodes, track images on all nodes, and start the containers with new images. Logs from each step are stored in Simple Log Service for querying. Compared with standalone O&M scripts, automation based on CloudFlow offers higher availability, built-in error handling, and visual progress tracking.