Weiwei Yang 杨巍威

ABOUT ME

Distributed system engineer, open-source leader — building large-scale infrastructure for AI/ML workloads.

• Foundation Model Infra @ Apple
• ASF Member — The Apache Software Foundation
• Ex-VP, Apache YuniKorn
• Ex-Chair, CNCF Batch Working Group
• Committer & PMC Member, Apache Hadoop
• Committer, Apache Ozone

📄 View my full portfolio → 🔗 Connect on LinkedIn →

My Talks/Posts:

• Ray at Scale: Apple’s Approach to Elastic GPU Management. RaySummit, San Francisco, 2024.
• Revolutionizing Kube Scalability Testing with KWOK. KubeConf North America, Chicago, 2023.
• Beyond Experimental: Spark on Kubernetes. KubeConf North America, Detroit. Oct, 2022.
• Next Level Spark on Kubernetes with Apache YuniKorn (Incubating). ApacheConf 2021, Oct, 2021.
• Efficient Spark Scheduling on K8s with Apache YuniKorn. ApacheConf 2020, Oct 2020.
• YARN to Kubernetes: How to tackle the challenages of resource management. Hadoop Meetup Shanghai, Sep 26, 2020. download pdf
• Cloud-Native Spark Scheduling with YuniKorn Scheduler. Spark & AI Summit, June 2020.
• Energize Multi-tenancy Flink on K8s with YuniKorn . Virtual Flink Forward Conference, San Francisco, Apr 24, 2020
• Next Generation Scheduling: In Hybrid Cloud Environment for High Performance and Optimized Execution of Mixed Workloads. Dataworks Summit, Washington D.C. May 23, 2019
• Open Hybrid Architecture: Running Stateful Containers on YARN. Hortonworks Tech Blog. Dec 17, 2018.
• Apache Hadoop YARN: State of the Union. ArchSummit, Beijing, China. slides download. Dec 07, 2018.
• Apache YARN 3.x in Alibaba. Dataworks Summit, San Jose, CA. Jun 21, 2018.
• Success at Apache: the Chance to Influence the World. ASF “Success at Apache” Blog Series. Jun 04, 2018.
• Apache Spark and DB2 with BLU Acceleration: Making ‘People Flow’ in Cities Measurable and Analyzable. IBM Insights Conference, Las Vegas. NV_ Oct 2015.
• Apache Hadoop Fundamentals. School of Computer Science and Technology. Aug 15, 2012.