Experience

Principal-level Research Scientist & Engineering Director

• Lead research and engineering strategy for AI compilers, model deployment, and distributed training and inference acceleration.
• Guide MLIR-based compiler initiatives (Triton-distributed), GPU/ASIC acceleration, and large-scale training and inference systems.
• Mentor cross-disciplinary projects in programming languages, computer architecture, and system acceleration.

Senior Staff-level Research Scientist & Engineering Manager

• Led an MLIR-based compiler team targeting CPU, GPU, edge, AI ASICs, and distributed systems; drove SW/HW co-design for multiple generations of in-house AI ASICs and DPUs. Directed ByteIR in collaboration with OpenXLA, Torch-MLIR, and ONNX-MLIR.
• Managed production deployment of 1,000s of models across 10,000s of in-house AI ASICs.
• Led a heterogeneous computing team optimizing training, inference, and deployment across commercial DLAs; oversaw benchmarking via ByteMLPerf.
• Directed GPU acceleration initiatives, including FLUX and an AMD GPU acceleration effort.
• Led Triton projects for in-house AI ASICs and distributed systems (Triton-distributed).
• Supervised distributed LLM training and inference acceleration, including veScale and ShadowKV.
• Initiated a large-scale model performance estimation program.
• Mentored projects in programming languages, computer architecture, and acceleration.

Staff-level Research Scientist

• Founded and led a model compilation team for an MLIR-based AI compiler across CPUs, GPUs, and AI ASICs, including SW/HW co-design for AI ASICs; launched ByteIR.
• Led a production team deploying numerous models on in-house AI ASICs.

Senior Software Engineer

• Led an AI compilation team building an LLVM/MLIR-based compiler for Maia 100, GPUs, and CPUs within ONNX Runtime. Led the Argo and Nuphar projects, delivering up to 3x cost reduction and 10x faster model release cycles for Azure Cognitive Services.
• Developed performance optimizations and a programming model for Maia 100.
• Co-designed pipeline parallelism for ONNX Runtime training.

Software Engineer

• Co-led the AI compiler and runtime for CNTK models, achieving 10x+ speedups over native CNTK.

Research Assistant

• Researched high-performance GPU computing, compiler optimization, and computer architecture.
• Built a performance-portable programming system for CPUs, GPUs, FPGAs, and clusters, spanning language, compiler, and runtime design.
• Contributed to benchmark suites for hardware characterization and optimization, including Parboil, SPEC ACCEL, and Chai.
• Developed GPU optimization techniques for dynamic parallelism, data transformations, reductions, and dense/sparse BLAS; contributed the first GPU pivoting tridiagonal solver for NVIDIA cuSPARSE and multi-dimensional EMD for signal processing.
• Analyzed cache sensitivity and implemented cache protection, bypassing, and thread throttling to improve throughput.

Intern

• Implemented real-time image inpainting optimizations for Tegra 3 to improve graphics performance.

Full-time Research Assistant & Engineer of a Stealth Mode Startup

• Designed and developed an embedded heterogeneous system for high-frequency real-time ultrasonic imaging using FPGA and GPU; the system was commercialized by a startup.

Undergrad. Research Assistant

• Conducted early research on the rolling shutter effect in CMOS cameras, developing analysis and compensation techniques.
• Explored light-field camera design and its visual effects.

Dan Vivoli Endowed Fellowship

Integrative Graduate Education and Research Traineeship (IGERT): Neuroengineering

Taiwan Merit Scholarship

Undergraduate Student Research Fellowship

Pan Wen-Yuan Scholarship

Presidential Awards

Gold Medal in Asian Pacific Mathematics Olympiad