CONDOR: An automated framework to accelerate convolutional neural networks on FPGA
0 likes157 views
The document presents Condor, an automated framework designed to accelerate convolutional neural networks (CNNs) on Field-Programmable Gate Arrays (FPGAs). It discusses the framework's architecture, its methodology for hardware accelerator creation, and integration with cloud services, offering a solution for performance and power consumption trade-offs. Additionally, various architectural choices and improvements are considered to optimize CNN performance on FPGA platforms.
![Our first approach
!17
[1] Giuseppe Natale, Marco Bacis, Marco D. Santambrogio
“On how to design fpga-based accelerators for Convolutional Neural Networks”, ISVLSI 2017
DMA
in
CONV POOL CONV LINEAR
w/b w/b w/b
POOL
POOL
POOL
POOL](https://image.slidesharecdn.com/raspacondororacle1-180530171823/85/CONDOR-An-automated-framework-to-accelerate-convolutional-neural-networks-on-FPGA-17-320.jpg)
![Next Steps
!29
• Possibility to use URAMs as on chip storage (~33 MB)
• Higher number of DSPs (~2.3X)
• Efficient multiplication (8 bits fixed point -> 2 mul/dsp)
• Higher memory BW (4 DDR ports)
[2] Deep Learning with INT8 Optimization on Xilinx Devices](https://image.slidesharecdn.com/raspacondororacle1-180530171823/85/CONDOR-An-automated-framework-to-accelerate-convolutional-neural-networks-on-FPGA-29-320.jpg)
CONDOR: An automated framework to accelerate convolutional neural networks on FPGA
- 1. CONDOR AN AUTOMATED FRAMEWORK TO ACCELERATE CONVOLUTIONAL NEURAL NETWORKS ON FPGA Oracle Labs Redwood, CA May 30th, 2018 Niccolò Raspa, Marco Bacis, Giuseppe Natale, Marco D. Santambrogio
- 4. !4 CNN on siliconCNN on silicon GPU Fixed architecture High power consumption Adaptable FPGA Reconfigurable architecture Low Power Consumption Adaptable ASIC Fixed architecture Low Power Consumption Not adaptable
- 5. Manual Design !5 Extract the parameters and the weights Write the code Synthesis Evaluate DesignPackage IP Iterate
- 7. Framework Architecture !7 Parse structure of the CNN FRONTEND Creation of HW Accelerator CORE LOGIC Deployment BACKEND
- 8. Create DAG computation !8 PROTOTXT CAFFEMODEL { Input Data Convolution Pooling Fully Connected Convolution Pooling Fully Connected Input Dimension: (28, 28, 1) Output Dimension (24, 24, 20) Kernel: 5 Padding: 0 Stride: 1 Input dimension (28, 28, 1) Input Dimension: (24, 24, 20) Output Dimension (12, 12, 20) Kernel: 2 Padding: 0 Stride: 2
- 9. Map computation in hardware !9 Area Convolution Pooling Fully ConnectedConvolution Pooling
- 12. What if I don’t have an FPGA? !12 CONDOR
- 13. Features !13 Cloud Integration via Amazon F1 Instances Automatic creation of an hardware accelerator for FPGA Tune the tradeoff between performance and power consumption Support main deep learning libraries
- 14. Roadmap !14 Automated Framework Methodology for Acceleration of CNN Integration with Caffe Cloud Integration M. Bacis, G. Natale, E. Del Sozzo, and M. D. Santambrogio. “A Pipelined and Scalable Dataflow Implementation of Convolutional Neural Networks on FPGA” In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Giuseppe Natale, Marco Bacis and Marco Domenico Santambrogio. “On how to design dataflow FPGA-based accelerators for Convolutional Neural Networks” In: 2017 IEEE Computer Society Annual Symposium on VLSI (ISVLSI) * 2017 Support the new standard ONXX 2018 Open Source Release 2019? Extend Methodology XOHW Competition *
- 15. Architectural choices for the HW design !15 Parse structure of the CNN FRONTEND Creation of HW Accelerator CORE LOGIC Deployment BACKEND
- 16. FPGAs and CNNs !16 Dataflow Computation Data reusability Distributed Architecture
- 17. Our first approach !17 [1] Giuseppe Natale, Marco Bacis, Marco D. Santambrogio “On how to design fpga-based accelerators for Convolutional Neural Networks”, ISVLSI 2017 DMA in CONV POOL CONV LINEAR w/b w/b w/b POOL POOL POOL POOL
- 18. Bigger networks, bigger FPGAs… or not? !18 • Weights don’t fit on the on-chip BRAMs • Unrolling leads to the explosion of DSPs (multipliers) usage
- 19. Methodology improvements !19 • No complete unrolling - partial accumulations • Generic set of “one size fits all” blocks • Semi-dataflow architecture • More complex data movement
- 21. Customizable data flow !21 ReLU Pool ReLU ReLU Conv Conv Conv Datamover/Control Conv Pooling ReLU in w/b out Datamover/Control Conv Pooling ReLU in w/b out Datamover/Control Conv Pooling ReLU in w/b out Datamover/Control Conv Pooling ReLU in w/b out
- 22. MAC weights input result Dataflow Blocks !22 • Convolution, Pooling • Non-uniform memory partitioning • Streaming pattern • Optimal full buffering • Concurrent accesses
- 23. Partial accumulations approach !23 • Custom level of parallelism • Compute subset of both input/output feature maps • Accumulation done with a FIFO and/or from DDR
- 24. Memory control and data buffering !24 • Memory mapped to streaming and viceversa • Exploit the maximum transaction size and bursts
- 25. Weights Double Buffering !25 • Masks weights loading latency • Allows to not flush the MAC pipeline on each iteration ping pong
- 26. Input Caching !26 • Reduces memory accesses • Stores entire input for a layer • Used for small layers (avoid lots of small transactions) Datamover/ControlInput Cache in Datamover/ControlInput Cache in
- 27. Architecture Evaluation !27 •~4 MB BRAM •2880 DSPs (27x15 bits mult) •1 DDR port (512 bits wide) •115.2 GFLOPs max (100Mhz) Alphadata Virtex-7 Setup Results •30.6 GOPs, 56MB parameters •4 input, 4 output ports •27.2 GFLOPs estimated •14.4 GFLOPs reached VGG16 Network
- 28. Lessons Learned !28 • Floating point is dead, long live the fixed! • Off-chip memory vs On-chip memory
- 29. Next Steps !29 • Possibility to use URAMs as on chip storage (~33 MB) • Higher number of DSPs (~2.3X) • Efficient multiplication (8 bits fixed point -> 2 mul/dsp) • Higher memory BW (4 DDR ports) [2] Deep Learning with INT8 Optimization on Xilinx Devices
- 30. Next Steps MAC/ Window FSM Acc/ReLU Pooling 1024 out512 in 2-64 out Weights I/O Buffer
- 31. Marco Bacis [email protected] Niccolo’ Raspa [email protected] Giuseppe Natale [email protected] Marco D. Santambrogio [email protected] twitter.com/CondorAtNECST facebook.com/CondorAtNECST AN AUTOMATED FRAMEWORK TO ACCELERATE CONVOLUTIONAL NEURAL NETWORKS ON FPGA