Viewing a single comment thread. View all comments

thundergolfer t1_j8b2g6w wrote

> If you're deploying models to production

Airflow is not a good tool for ML development. Leave Airflow back in 2018. Also Modal can do prod model deployment, model pipelines, and inference.

3

PHEEEEELLLLLEEEEP t1_j8bac1k wrote

Why is airflow bad and can you suggest alternatives? I'm using airflow now with no issues

3

thundergolfer t1_j8bjgpu wrote

If you don't have issues then definitely don't bother migrating! Something like Metaflow or Modal is much more built for purpose. Airflow was designed for the Hadoop era of data engineering; it's straining under changes that have happened in the Python, container, and ML ecosystems.

2