This is an interesting topic, in addition, we can start embracing data dependency driven coordination on top of Airflow/Prefect.
https://eric-sun.medium.com/data-dependency-driven-orchestration-d1bd8e1d695f
Sign up to discover human stories that deepen your understanding of the world.
Free
Distraction-free reading. No ads.
Organize your knowledge with lists and highlights.
Tell your story. Find your audience.
Membership
Read member-only stories
Support writers you read most
Earn money for your writing
Listen to audio narrations
Read offline with the Medium app
Written by Eric Sun
Advocate best practice of big data technologies. Challenge the conventional wisdom. Peel off the flashy promise in architecture and scalability.
