Amazon Managed Workflows for Apache Airflow (MWAA)
Amazon MWAA is a managed workflow automation service built on Apache Airflow, used to develop, schedule and monitor batch-oriented processing. Automating execution from within Amazon MWAA creates an additional island of automation for IT Operations to manage. Automation from Broadcom allows you to easily integrate directed acyclic graphs (Airflow DAGs) with your existing enterprise workload automation solution. You are instantly inheriting the advanced capabilities of your enterprise solution to deliver your digital transformation quicker and more successfully. By extending your existing enterprise automation to Amazon MWAA you maintain end-to-end visibility, regain centralized command and control, alerting, SLA management, reporting and auditing.
Benefits and Use Cases
- Trigger Airflow DAGs on a schedule or on demand to automate ETL processes, machine learning, and other data workflows to Control and manage your data pipelines at scale, seamlessly.
- Centralize oversight of DAG execution status and task progress with real time monitoring
- Access DAG run logs and failure details to support compliance and troubleshooting.
Example Use Case: Data Pipeline Orchestration and Monitoring
Execute scheduled or on-demand Airflow DAGs to orchestrate ETL workflows that extract, transform, and load data across systems. Monitor these DAG status and individual task progress in real time and automatically recover from failures by triggering retries or fallback DAGs based on scheduler logic.
Automic Features
- Schedule execution of Apache Airflow DAGs
- Monitor to completion Airflow DAGs
- Retrieve DAG logs for problem diagnosis
- Alert on failure of DAG processing
- Create dependencies between DAG executions
- Incorporate DAG processing in existing enterprise automation workflows including dependency enforcement