LiteRT for Microcontrollers

LiteRT for Microcontrollers is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory. The core runtime just fits in 16 KB on an Arm Cortex M3 and can run many basic models. It doesn't require operating system support, any standard C or C++ libraries, or dynamic memory allocation.

Why microcontrollers are important

Microcontrollers are typically small, low-powered computing devices that are embedded within hardware that requires basic computation. By bringing machine learning to tiny microcontrollers, we can boost the intelligence of billions of devices that we use in our lives, including household appliances and Internet of Things devices, without relying on expensive hardware or reliable internet connections, which is often subject to bandwidth and power constraints and results in high latency. This can also help preserve privacy, since no data leaves the device. Imagine smart appliances that can adapt to your daily routine, intelligent industrial sensors that understand the difference between problems and normal operation, and magical toys that can help kids learn in fun and delightful ways.

Supported platforms

LiteRT for Microcontrollers is written in C++ 17 and requires a 32-bit platform. It has been tested extensively with many processors based on the Arm Cortex-M Series architecture, and has been ported to other architectures including ESP32. The framework is available as an Arduino library. It can also generate projects for development environments such as Mbed. It is open source and can be included in any C++ 17 project.

The following development boards are supported:

Explore the examples

Each example application is on GitHub and has a README.md file that explains how it can be deployed to its supported platforms. Some examples also have end-to-end tutorials using a specific platform, as given below:

Workflow

The following steps are required to deploy and run a TensorFlow model on a microcontroller:

  1. Train a model:
    • Generate a small TensorFlow model that can fit your target device and contains supported operations.
    • Convert to a LiteRT model using the LiteRT converter.
    • Convert to a C byte array using standard tools to store it in a read-only program memory on device.
  2. Run inference on device using the C++ library and process the results.

Limitations

LiteRT for Microcontrollers is designed for the specific constraints of microcontroller development. If you are working on more powerful devices (for example, an embedded Linux device like the Raspberry Pi), the standard LiteRT framework might be easier to integrate.

The following limitations should be considered:

  • Support for a limited subset of TensorFlow operations
  • Support for a limited set of devices
  • Low-level C++ API requiring manual memory management
  • On device training is not supported

Next steps