{"docs":[{"id":193,"title":"Running AI at the Edge: Microcontrollers, NPUs and Accelerators","heading":"Running AI at the Edge: Microcontrollers, NPUs and Accelerators","subheading":null,"category":"AI & Machine Learning","body":{"root":{"type":"root","children":[{"type":"heading","tag":"h3","version":1,"children":[{"type":"text","text":"Matching the model to the silicon","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Microcontrollers run small networks for sensor, audio and simple vision tasks, usually through runtimes such as TensorFlow Lite for Microcontrollers and optimised kernel libraries like CMSIS-NN on Arm Cortex-M. Memory, not arithmetic, is usually the first limit.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Application processors with an NPU run larger vision models under Linux through the vendor toolchain. Check early that every operator in your model is supported, because coverage and tooling vary considerably between vendors.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"tags":"AI & Machine Learning, Edge AI, Embedded Hardware","cardCta":{"label":null,"link":null},"pageCta":{"label":"Talk to AAVICO engineering","link":"/contact"},"excerpt":"Running a model on the device removes the network round trip and keeps data local. The hard part is matching the model to hardware that can actually run it.","featuredImage":null,"type":"Blog","readTime":"3 min read","publishedAt":"2026-11-15T14:36:00.000Z","heroImage":null,"introText":"Edge AI means running inference on the product itself rather than sending data to a server. It suits products that must respond quickly, work without a connection or keep sensor data private. The hardware ranges from a small microcontroller classifying vibration data to a module with a neural processing unit handling several camera streams. This article explains what each class of hardware suits and how to choose between them.","metrics":[{"id":"running-ai-at-the-edge-microcontrollers-npus-and-metric-1","value":"int8","label":"The usual numeric format for edge inference, a quarter of the size of float32 weights.","icon":null},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-metric-2","value":"NPU","label":"Dedicated hardware for the multiply-accumulate operations that dominate neural networks.","icon":null},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-metric-3","value":"Offline","label":"On-device inference keeps working when the network does not.","icon":null},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-metric-4","value":"Local data","label":"Raw images and audio can stay on the device instead of travelling to a server.","icon":null}],"gallery":[],"challenge":{"root":{"type":"root","children":[{"type":"paragraph","version":1,"children":[{"type":"text","text":"Teams often pick the processor first and the model second. A model developed on a desktop GPU can need far more memory and compute than the chosen part offers, and the gap is only found when the model is first ported.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Accelerators bring their own constraint. Each NPU supports a particular set of operators, and any layer outside that set falls back to the CPU, which can erase most of the expected speed-up.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"blockquote":{"quote":"Choose the model and the silicon together. Either one chosen alone is likely to be wrong.","author":"","role":"AAVICO Engineering"},"supportImage1":null,"deliverables":{"root":{"type":"root","children":[{"type":"paragraph","version":1,"children":[{"type":"text","text":"Define the task, the input rate and the latency the product needs, then estimate model size and compute from a candidate architecture.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Shortlist hardware whose memory and operator support fit that model, and confirm the vendor toolchain converts it.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Benchmark the converted model on a development kit with real input data before the board design is committed.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"results":[{"id":"running-ai-at-the-edge-microcontrollers-npus-and-result-1","title":"Fast Response","description":"Inference completes on the device, with no dependence on network latency."},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-result-2","title":"Privacy","description":"Sensitive data can be processed where it is captured."},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-result-3","title":"Right-Sized Hardware","description":"The processor fits the model, so the bill of materials carries no idle silicon."},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-result-4","title":"Fewer Surprises","description":"Operator support is checked before layout, not after."}],"lookingAhead":{"root":{"type":"root","children":[{"type":"paragraph","version":1,"children":[{"type":"text","text":"Plan how models will be updated in the field. A new model is a firmware update and needs the same signing and rollback protection.","version":1}]},{"type":"paragraph","version":1,"children":[{"type":"text","text":"Measure power with inference running. Duty cycle and wake strategy often matter more than the accelerator itself.","version":1}]}],"direction":"ltr","format":"","indent":0,"version":1}},"supportImage2":null,"testimonial":{"quote":null,"author":null,"role":null,"photo":null},"challengesSolved":[{"id":"running-ai-at-the-edge-microcontrollers-npus-and-challenge-1","title":"A model too large for the chosen part","description":null,"icon":null},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-challenge-2","title":"Layers that silently fall back to the CPU","description":null,"icon":null},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-challenge-3","title":"Hardware chosen before the model was sized","description":null,"icon":null},{"id":"running-ai-at-the-edge-microcontrollers-npus-and-challenge-4","title":"Power measured without inference running","description":null,"icon":null}],"relatedPosts":[],"contactUs":{"show":true},"updatedAt":"2026-11-15T17:32:32.682Z","createdAt":"2026-11-15T17:32:32.682Z"}],"hasNextPage":false,"hasPrevPage":false,"limit":10,"nextPage":null,"page":1,"pagingCounter":1,"prevPage":null,"totalDocs":1,"totalPages":1}