“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” a Presentation from BrainChip
0 likes33 views
BrainChip Inc. presents Akida, a solution designed to enhance ultra-low power edge inference and on-device learning, addressing challenges like high training costs and manufacturing downtime. The technology leverages fully-digital, neuromorphic, event-based AI for efficient real-time computation while minimizing reliance on cloud services, improving performance, and reducing data privacy risks. Akida enables various AI applications such as anomaly detection and biometric recognition with reduced memory and processing demands, making it suitable for deployment across a wide range of edge devices.
“Enabling Ultra-low Power Edge Inference and On-device Learning with Akida,” a Presentation from BrainChip
- 1. brainchip Inc © Enabling Ultra-Low Power Edge Inference and On- Device Learning with Akida Nandan Nayampally CMO Brainchip Inc.
- 2. brainchip Inc © • The Challenge • The Approach • The Delivery • The Results • Akida in action Agenda 2
- 3. brainchip Inc © The Challenge
- 4. brainchip Inc © The Problem 4 $50B Annual losses in Manufacturing due to unplanned downtime2 1TB Data generated by 1 Car per day3 $6M Costs of training a single High-end model 1 $1.1T Healthcare cost and lost productivity due to preventable chronic disease4 4 Courtesy: fightchronicdisease.org 2 Courtesy: Deloitte.com 1 Courtesy: Spectrum.Ieee.org. “The cost of training, made retraining the model infeasible” 3 Courtesy: Forbes.com
- 5. brainchip Inc © The Opportunity 5 Image: Courtesy Pixabay (From Mckinsey AIoT 2030 forecast) $15.7T Global GDP Benefit from AI in 2030* * PWC analysis report $1.2T AIoT revenue in 2030** ** Forbes Business Insights 1T+ Edge devices in 2030**
- 6. brainchip Inc © The Challenge 6 Cost of cloud services Responsiveness & reduced latency Scalability & efficiency Privacy protection & security
- 7. brainchip Inc © The Challenge The Solution 7 Cost of cloud services Responsiveness & reduced latency Scalability & efficiency Privacy protection & security Reduce cloud inference cost Minimize cloud retraining Rapid computation at edge Real-time compute for critical tasks Efficiency within thermal and power budgets Reduced memory and system cost Prevent exposure of sensitive data Minimize raw data being sent to cloud
- 8. brainchip Inc © The Approach
- 9. brainchip Inc © Compelling high-performance Extreme efficiency Continuous learning Secure communication 9 Event-based processing Event-based communication Advanced spatial-temporal capability At-memory computation Event-based learning The Neuromorphic Advantage
- 10. brainchip Inc © 2nd Generation Fully-digital, neuromorphic, event-based AI Unique ability to learn on device without cloud dependency What’s New: High performance compute with extreme energy-efficiency on complex models Spatial-temporal convolutions that enable innovative handling of 3D and 1D data with Temporal Event-based Neural Nets (TENN) Low-power support for vision transformers in edge AIoT Improved accuracy with efficiency for production devices 10
- 11. brainchip Inc © Neuromorphic Principles in Real Solutions 11 Fully-digital event-based processing Easily deploys today’s models and networks metaTF Secure on-device learning and customization Complex models fully accelerated in hardware
- 12. brainchip Inc © Temporal Event Based Neural Nets 12 Extremely efficient 3D convolutions TENNs deliver the benefits of and are much more efficient to train than RNNs Easy to train and extremely data-efficient 3D data has spatial (frames) and temporal (time) components • TENN trains with backpropagation like a CNN • Extracts spatial (2D) + temporal (1D) kernel • Inference in recurrent mode 1D time series data focused on temporal components • Training 1D data with backpropagation • Extracts temporal kernels • Inference in recurrent mode
- 13. brainchip Inc © Vision Transformer (ViT) for Efficient Performance Boost 13 Vision transformer encoder block functionality fully supported in hardware • Optional configurations from 2 Node to 12 Nodes • Builds on at-memory compute benefits • Encoder block is fully self contained • Managed through DMA and runtime Delivers power and system efficient performance • 2 Nodes running at 800 MHz give 30 FPS performance (224x224x3) • No external memory bandwidth needed once layers are loaded • Size incremental to standard event-based node.
- 14. brainchip Inc © The Delivery
- 15. brainchip Inc © Sensor Agnostic AI Efficiency 15 Optimal for any network model Self-managed operation Reduced system Load Extreme efficiency Simplified development process
- 16. brainchip Inc © Akida Provides some compelling alternatives Enabling Compelling Edge AI (example) 16 Vibration Detection Anomaly Detection Keyword spotting Sensor Fusion Low-res Presence Detection Gesture detection Object Classifica -tion Gesture recogni- tion Biometric recogni- tion Advanced Speech Recogni- tion Object detection (classifica- tion + Localizatio n) Advanced Sequence Prediction Video Object Detection And tracking MCU (Not optimal but within performance range) MCU+ML (MCU carries a lot of computation) MPU+GPU+ML (Higher end, higher cost, higher power) Performance required (not to scale) Vision Transform er Networks Sensor Edge Network Edge -P With Mid-spec MCU or Mid-Spec MPU -S With Min-spec or Mid-Spec MCU -E Either Standalone or with Min-spec MCU How the market deals with these workloads today Akida products can implement any network
- 17. brainchip Inc © Simplifying Deployment with Akida 17 • Underlying AI operations offloaded to Akida IP • CPU and Akida IP running in parallel Akida IP benefits Data acquisition Data pre- processing Layer 1 operations Layer N operations Data post- processing CPU and memory pressure over time AkidaTM acceleration CPU acceleration Inference • Model’s parameters sitting in Akida IP local memory • No memory congestion during inference
- 18. brainchip Inc © BrainChip Akida Model Development 18 Generate Akida model that solves an application Using Akida Runtime Library, deploy Akida model on any Akida target device Development Platforms On Chip Deployment Akida model library Akida runtime library Run inference on any device that has Akida integrated Data Third Party applications Custom Expertise Development and Deployment With no-code requirement Build Tensorflow/Keras model Optimize using MetaTF
- 19. brainchip Inc © The Results
- 20. brainchip Inc © Network mAP Parameters (millions) MACs / sec (Billions) Sim CLR (Resnet50) 0.57 26 82 Akida TENN* + CenterNet 0.576 0.57 18 Equivalent Precision 50x fewer Parameters 5x fewer Operations KITTI 2D dataset Akida TENN matches the benchmark precision* Improved performance Substantially smaller model – less memory and system load Much greater efficiency SimCLR with a RESNET50 backbone is the benchmark in object detection Source: SiMCLR Review Reference: SimCLR overview * Can be further tweaked to improve mAP ** Estimates for Akida neural processing scaled from 28nm Benchmark Resolution:1352x512 < 75mW For 30FPS in 16nm** 20
- 21. brainchip Inc © • Akida Solution: Raw Audio directly fed to Network • No additional filtering or DSP hardware • Memory efficient, smaller models, fewer ops • Much faster and more power efficient Simplifying Raw Audio 21 • Today’s generic solution: MFCC + DSCNN • Hardware filtering, transforms and encoding. • Memory intensive • Heavier software load Disruptive solutions for consumer audio, hearing aids and more TENN Keyword Model Accuracy Parameters Total Memory (KB) MACs (M/sec) MFCC+DSCNN 92.43% 21 k 93.61 320 Model Accuracy Parameters Total Memory (KB) MACs (M/sec) Akida TENN 97.12% 52 k 26 19 Better accuracy Lower memory, BOM cost 16x fewer Ops DS-CNN Signal partitioning Fast Fourier Transform Filter bank Fast Fourier Transform Log Cepstral Coefficients Audio Signal Keyword <2µJ/inference in 28nm
- 22. brainchip Inc © Vital Signs Prediction: Heart Rate 22 Model Heart Rate Error (RMSE*) Number of Parameters (million) Billion MACs / sequence S4 (SOTA) 0.332 0.3 11.2 Akida TENN** 0.4721 0.063 0.021 ExpRNN 1.87 0.127 ~0.51 acceptable unacceptable S4 is a state of art algorithm that hasn’t yet made it to production. LSTM based models have been used, but not accurate enough for * Root Mean Square Error (lower is better) ** Accuracy can be further improved Source: 2206.11893.pdf (arxiv.org) RMSE=1.0 ~SoTA Accuracy 5x fewer Parameters 500x fewer Operations Beth Israel Deaconess Medical Center Dataset Akida TENN substantially more efficient than current State of The Art** and very close in accuracy. Works on raw data. No pre-processing required Better than any current industry standard models Enables compelling mobile/portable edge devices to be create
- 23. brainchip Inc © Akida in Action
- 24. brainchip Inc © Akida in Action FOMO – Faster Objects More Objects Nuts and Bolt Classification (with DVS Camera) 24
- 25. brainchip Inc © From Concept to Delivery 25 Ready to take your ideas to fruition Evaluate Models Zoo Design Develop Scale
- 26. brainchip Inc © Ecosystem & Partnerships 26 Early Adopters Licensees Partnerships Technology Enablement Integration
- 27. brainchip Inc © Ready When You Are! 27 Akida platform boosts performance and efficiency in disruptive edge AIoT solutions Ready for today’s complex models and future-proofed for the new ones Secure learning and intelligent customization at the edge without need for cloud retraining of model Easy to deploy with a fast-growing ecosystem We’re ready! Are you?
- 28. brainchip Inc © Resources 28 BrainChip main site: https://www.brainchip.com BrainChip white papers: https://www.brainchip.com/white-papers-case-studies BrainChip MetaTF: https://docs.brainchipinc.com Akida models zoo: https://doc.brainchipinc.com/user_guide/akida_models.html Edge Impulse Support: https://docs.edgeimpulse.com/docs/development-platforms/officially-supported- ai-accelerators/akd1000