主题：Sparse Kernel Optimization for Graph Neural Network System Acceleration

  嘉宾：戴国浩  清华大学

  时间：2021年6月11日 上午10：00~11：00

  地点：华中科技大学东五楼210学术报告厅

  报告摘要：

  As we are now in the Machine Learning (ML) and Big Data (BD) era, ML algorithms on graphs, especially the recently proposed Graph Neural Networks (GNNs), have achieved great success in various domains. There exists a huge gap between unstructured sparse graph data and structured parallel hardware like GPUs. Thus, accelerating GNNs on GPUs suffers from great challenges and is of great significance to improving the efficiency of understanding hidden information in the graph. In this talk, we target accelerating GNNs on GPUs from a novel kernel perspective. By extracting key sparse operators in GNNs, the problem can be broken down into sparse kernel acceleration on GPUs. Different from dense kernels like general-purpose matrix multiplication (GEMM) which can utilize the peak performance of GPU hardware, sparse kernels like sparse matrix-matrix multiplication (SpMM) achieves low FLOPs and the performance is closely related to the implementation. We optimize sparse kernels like SpMM on GPUs with several simple but effective methods, achieving up to 6.15x kernel speedup and 3.67x end-to-end speedup of GNN frameworks. Moreover, we are now building up an open ecology for GNN framework developers, sparse kernel developers, and hardware vendors, to improve the development efficiency of GNN frameworks by benefiting from the latest research achievement of sparse kernel developers and hardware vendors. We hope that researchers and developers in related domains could join this open ecology and contribute to the community.

  报告人简介：

  Dr. Guohao Dai is currently a postdoctoral and assistant researcher in the Department of Electronic Engineering, Tsinghua University, Beijing, China. He received the B.S. degree and Ph.D. degree (with honor) in Department of Electronic Engineering from Tsinghua University in 2014 and 2019. Dr. Dai’s research mainly focuses on large-scale graph computing, hardware virtualization, heterogeneous hardware computing, processing-in-memory, and etc. He has received Best Paper Award in ASPDAC 2019, and Best Paper Nomination in DATE 2018. He is also the recipient of the Outstanding Ph.D. Dissertation Award of Tsinghua University in 2019.