Machine learning algorithms, especially deep neural networks (DNNs), are rapidly emerging, where the automatic neural architecture search (NAS) methodologies are playing leading rules. Meanwhile, studies of efficient DNN deployment on hardware (GPU, FPGA, etc.) are also attracting broad interests especially for edge computing with limited hardware resources. In a typical top-down design flow, DNN models are first designed concentrating more on accuracy, expecting the deployment can achieve good performance through later hardware optimizations. Although this approach has been successful in many applications, it ignores the mutual impact of DNN designs and deployment optimizations, and thus misses a large opportunity of improving the solution quality of both DNN models and hardware implementation. An accuracy-oriented DNN design must be hardware-friendly to achieve the best implementation performance, and the hardware design must be efficient and flexible to accommodate different DNN models. Therefore, to achieve the highest accuracy and best hardware efficiency at the same time, it is best to apply neural architecture and hardware implementation co-search.
Facing this opportunity, we proposed Neural Architecture and Implementation Search (NAIS) methodology, to simultaneously pursue aggregated solutions of high accuracy DNN designs and efficient hardware deployments. To enable a comprehensive co-search framework, there are three indispensable components: 1) efficient FPGA accelerator design; 2) hardware-aware neural architecture search (NAS); 3) automatic design tools to quickly deploy DNNs to FPGAs. The three components need to be seamlessly integrated to work as a whole.
In this talk, I will introduce our efforts on these three components as well as our proposed full-stack NAIS methodology.
First, we have done an intensive exploration on efficient FPGA accelerator design and optimization techniques, such as accelerator architecture design, computation pipelining and parallelization, data quantization, memory access optimization, operation and kernel fusion, etc. I will summarize our achievements regarding FPGA accelerator designs and talk about their contributions to NAIS methodology.
Second, I will introduce our efforts on design automation tools, to enable fast and highly optimized DNN deployment on FPGA. By leveraging high level synthesis (HLS), we developed a higher level tool that automatically parses a DNN model and maps it to FPGA, which also performs design space exploration for implementation optimization.
On top of efficient FPGA accelerator design and with the help of design automation tools, I will introduce our full-stack neural architecture and implementation co-search methodology, NAIS. We first prototyped NAIS on the FPGA platform, and proposed a DNN/FPGA co-design method targeting embedded resource-limited FPGAs on the edge. Incorporating our expertise on FPGA accelerator design and automatic FPGA deployment tools, we further proposed hardware-aware NAS as the search engine of our co-design method. Then, we extend the DNN/FPGA co-design to a full NAIS methodology, which targets arbitrary hardware platforms and conducts simultaneous search for both DNN models and implementations.
In the end, I will briefly discuss my future directions and prospects for pushing AI development in practical applications.
