|
- How to create a conditional task in Airflow - Stack Overflow
Airflow 1 x Airflow has a BranchPythonOperator that can be used to express the branching dependency more directly The docs describe its use: The BranchPythonOperator is much like the PythonOperator except that it expects a python_callable that returns a task_id The task_id returned is followed, and all of the other paths are skipped
- First time login to Apache Airflow asks for username and password, what . . .
user: airflow password: airflow If you want to create another account with docker use this (you have to be in the same folder of the docker-compose yaml file): docker-compose run airflow-worker airflow users create --role Admin --username admin --email admin --firstname admin --lastname admin --password admin
- How to use Dynamic Task Mapping with TaskGroups - airflow
In my actual DAG, I need to first get a list of IDs and then for each ID run a set of tasks I have used Dynamic Task Mapping to pass a list to a single task or operator to have it process the list
- python - How to run Airflow on Windows - Stack Overflow
As a workaround, use the [current folder]\build\scripts-2 7\airflow file, which is the python script for the airflow util Another solution is to append to the System PATH variable a link to a batch file that runs airflow (airflow bat):
- How to activate authentication in Apache Airflow
[webserver] authenticate = True auth_backend = airflow contrib auth backends password_auth And also remember to Restart Airflow Webserver, if it still doesn't work, run airflow initdb and restart the webserver Also, double-check in airflow cfg file that it does not contain multiple configurations for authenticate or auth_backend If there is
- Refreshing dags without web server restart apache airflow
In your airflow cfg, you've these two configurations to control this behavior: # after how much time a new DAGs should be picked up from the filesystem min_file_process_interval = 0 dag_dir_list_interval = 60
- Proper way to create dynamic workflows in Airflow
Problem Is there any way in Airflow to create a workflow such that the number of tasks B * is unknown until completion of Task A? I have looked at subdags but it looks like it can only work with a
- How to use apache airflow in a virtual environment?
Edit the airflow cfg file and set: load_examples = False dags_folder = path to your dag files If your airflow directory is not set to the default, you should set this env variable if it's annoying to change it every time, just set it in your pycharm project configuration or in your local OS (~ bashrc)
|
|
|