Eric Sun – Medium

Eric Sun

Pinned

Improve Ingest Latency and Query Efficiency of Data Lake — Partition and Index

Data Lake continues to offer better and better Cost Performance with the new features and integration patterns. With the summits of both…

Jul 3, 2024

Improve Ingest Latency and Query Efficiency of Data Lake — Partition and Index

Jul 3, 2024

Pinned

Data Dependency Driven Orchestration

AirFlow and Prefect are probably the most popular schedulers in 2021. They are both more data-aware than the traditional orchestration…

Jan 11, 2021

Data Dependency Driven Orchestration

Jan 11, 2021

Pinned

Are We Taking Only Half Of The Advantage Of Columnar File Format?

Sorting the records in columnar data format is a critical design considerations that many of us have not paid attention. Let’s leverage it.

Mar 16, 2020

Are We Taking Only Half Of The Advantage Of Columnar File Format?

Mar 16, 2020

Lego vs SoC, Apple M1 + MT8195, Microservices and Big Data Model

This week (2020–11–10) was really big for System on a Chip: first Apple M1, and then followed by MediaTek MT8195/MT8192. But why on earth…

Nov 22, 2020

Lego vs SoC, Apple M1 + MT8195, Microservices and Big Data Model

Nov 22, 2020

Reshape Data Lake: Delta, Iceberg, Hudi, or Hive

The super success of Spark in the ETL area also showed that many paradigms in the traditional data warehouse are indeed critical and useful

Mar 16, 2020

Databrick’s Delta Lake: Staging->Conformed->Aggregate/Feature Tier

Mar 16, 2020

Eric Sun

Eric Sun

Advocate best practice of big data technologies. Challenge the conventional wisdom. Peel off the flashy promise in architecture and scalability.

Following

Help

Status

About

Careers

Press

Blog

Privacy

Rules

Terms

Text to speech