Airflow user guide Air ow Documentation Release Apache Air ow Jan C CContents Principles Beyond the Horizon Content Project History Committers Resources links Roadmap License Quick Start What ? s Next Installation Getting Air ow Extra Packages Initiating
Air ow Documentation Release Apache Air ow Jan C CContents Principles Beyond the Horizon Content Project History Committers Resources links Roadmap License Quick Start What ? s Next Installation Getting Air ow Extra Packages Initiating Air ow Database Tutorial Example Pipeline de ?nition It ? s a DAG de ?nition ?le Importing Modules Default Arguments Instantiate a DAG Tasks Templating with Jinja Setting up Dependencies Recap Testing Running the Script Command Line Metadata Validation Testing Back ?ll What ? s Next How-to Guides Setting Con ?guration Options Initializing a Database Backend Using Operators i C BashOperator PythonOperator Google Cloud Storage Operators Google Compute Engine Operators Google Cloud Bigtable Operators Google Cloud Functions Operators Google Cloud Spanner Operators Google Cloud Sql Operators Google Cloud Storage Operators Managing Connections Creating a Connection with the UI Editing a Connection with the UI Creating a Connection with Environment Variables Connection Types Securing Connections Writing Logs Writing Logs Locally Writing Logs to Amazon S Writing Logs to Azure Blob Storage Writing Logs to Google Cloud Storage Scaling Out with Celery Scaling Out with Dask Scaling Out with Mesos community contributed Tasks executed directly on mesos slaves Tasks executed in containers on mesos slaves Running Air ow with systemd Running Air ow with upstart Using the Test Mode Con ?guration Checking Air ow Health Status UI Screenshots DAGs View Tree View Graph View Variable View Gantt Chart Task Duration Code View Task Instance Context Menu Concepts Core Ideas DAGs Operators Tasks Task Instances Work ows Additional Functionality Hooks Pools Connections Queues XComs Variables Branching SubDAGs ii C SLAs Trigger Rules Latest Run Only Zombies Undeads Cluster Policy Documentation Notes Jinja Templating Packaged dags air owignore Data Pro ?ling Adhoc Queries Charts Chart Screenshot Chart Form Screenshot Command Line Interface Positional Arguments Sub- commands resetdb render variables deleteuser connections createuser pause syncperm taskfaileddeps version triggerdag initdb test unpause listdagruns dagstate run listtasks back ?ll listdags kerberos worker webserver ower scheduler taskstate pool servelogs clear listusers nextexecution upgradedb deletedag Scheduling Triggers DAG Runs Back ?ll and Catchup External Triggers iii C To Keep in Mind Plugins What for Why build on top of Air ow Interface Example Note on role based views Plugins as Python packages Security Web Authentication Password LDAP Roll your own Multi- tenancy Kerberos Limitations Enabling kerberos Using kerberos authentication OAuth Authentication GitHub Enterprise GHE Authentication Google Authentication SSL Impersonation Default Impersonation Flower Authentication Time zones Concepts Na? ve and aware datetime objects Interpretation of naive datetime objects Default time zone Time zone aware DAGs Templates Cron schedules Time deltas Experimental Rest API Endpoints CLI Authentication Integration Reverse Proxy Azure Microsoft Azure Azure Blob Storage Azure File Share Logging Azure CosmosDB Azure Data Lake AWS Amazon Web Services AWS EMR AWS S AWS EC Container Service AWS Batch Service AWS RedShift Amazon SageMaker Amazon SageMaker iv C Databricks DatabricksSubmitRunOperator GCP Google Cloud Platform Logging GoogleCloudBaseHook BigQuery Cloud Spanner Cloud SQL Cloud Bigtable Compute Engine Cloud Functions Cloud DataFlow Cloud DataProc Cloud Datastore Cloud ML Engine
Documents similaires










-
156
-
0
-
0
Licence et utilisation
Gratuit pour un usage personnel Aucune attribution requise- Détails
- Publié le Nov 20, 2021
- Catégorie Philosophy / Philo...
- Langue French
- Taille du fichier 1.5MB