Joined Hortonworks at Aug 2018, Cloudera and Hortonworks get merged at Jan 2019.
Apache YuniKorn Scheduler: Transform Big Data workloads from Hadoop to Kubernetes. Back in 2018, running Big Data workloads on Kubernetes posed significant challenges—especially around resource management and scheduling. To address this gap, Wangda Tan and I created YuniKorn, bringing the best of YARN’s capacity scheduler to Kubernetes. Today, it’s widely used in Data Engineering and Machine Learning infrastructure stacks.
Containerization effort: adopted Container Storage Interface in Apache Hadoop YARN. Umbrella JIRA: YARN-8811. Design doc: pdf download.