PinnedEric SuninThe StartupData Dependency Driven OrchestrationAirFlow and Prefect are probably the most popular schedulers in 2021. They are both more data-aware than the traditional orchestration…10 min read·Jan 11, 2021--2--2
PinnedEric SuninAnalytics VidhyaAre We Taking Only Half Of The Advantage Of Columnar File Format?Sorting the records in columnar data format is a critical design considerations that many of us have not paid attention. Let’s leverage it.·8 min read·Mar 16, 2020--2--2
Eric SuninThe StartupLego vs SoC, Apple M1 + MT8195, Microservices and Big Data ModelThis week (2020–11–10) was really big for System on a Chip: first Apple M1, and then followed by MediaTek MT8195/MT8192. But why on earth…6 min read·Nov 22, 2020--1--1
Eric SuninThe StartupReshape Data Lake: Delta, Iceberg, Hudi, or HiveThe super success of Spark in the ETL area also showed that many paradigms in the traditional data warehouse are indeed critical and useful8 min read·Mar 16, 2020--4--4