Edge AI on a Power Budget: Running Industrial Models on Microcontrollers and NPUs

A model that needs a data centre is not a factory model. Real deployments run on devices with a few hundred milliwatts to a few tens of watts, on machines where the network is unreliable and the site's power budget is fixed. This is a walk through how we take a model from a research notebook to a device next to the machine, and the decisions that matter most.

Start from the hardware, not the model

The single most common mistake is to train the best possible model and then look for hardware to run it. We do the reverse. The site constraints (power, connectivity, enclosure, cost per unit at 200 machines) pick the hardware class, and the hardware class picks the model family.

• Microcontrollers (Cortex-M class, tens to hundreds of milliwatts). Small CNNs and spectral features on audio and vibration. Ideal for battery or PoE sensor nodes covering many machines. Latency in milliseconds, at the lowest unit cost of the three classes.
• Application processors with NPUs (a few watts). Mid-sized models, multi-channel audio, low-resolution vision. The sweet spot for a sensor unit that fuses several modalities.
• Embedded GPUs (10 to 30 watts). Vision at full resolution, lidar, several models in parallel. One per cell or line, not one per machine.

The compression toolbox

Quantisation

Moving from 32-bit floats to 8-bit integers typically cuts the memory footprint to a quarter and reduces energy per inference, with little or no loss for sensing tasks, provided calibration is done on representative data. Quantisation-aware training closes the remaining gap on hard cases.

Pruning and structured sparsity

Removing channels and blocks that contribute little, then fine-tuning. Structured pruning is what actually speeds up inference on real hardware; unstructured sparsity mostly helps on paper.

Efficient architectures

Depthwise-separable convolutions, small attention blocks and learned front-ends replace the large backbones that were never necessary for a 16 kHz signal. Much of the published work our founder led is about representations that are both accurate and cheap.

Feature front-ends

A well-designed spectral front-end (STFT, mel or learned filterbanks) often does more for efficiency than any model trick. Computing 64 mel bands at the sensor and sending those instead of raw audio also helps privacy and bandwidth.

Profiling, then hardware-in-the-loop

Latency and memory are measured on the target device, not estimated. We build a test harness that replays recorded plant data through the deployed model on the actual board, compares outputs against the reference implementation and logs power draw. Only then does a device go to the line. This hardware-in-the-loop step is where surprises appear: a preprocessing library that allocates memory in a loop, a driver that drops samples under load, a thermal limit at 45 °C in a hot-season shift.

Running a fleet

One device is a prototype. Two hundred devices are a fleet, and a fleet needs MLOps: signed firmware and model updates, staged roll-outs, per-device health and drift monitoring, a rollback path, and a way to collect hard examples for retraining without hauling raw data home. We design this in from the start, because retrofitting it is painful and because it is the part the client's OT/IT team will run for years.

Energy-efficient edge AI is not a compromise for sites with weak infrastructure. It is what makes AI on every machine affordable anywhere.

If any of this matches a problem on your line, the fastest way to find out what is possible is a free discovery call followed, where it makes sense, by a feasibility study of two to ten days.