AI without the cloud

When people picture AI in a product, they usually picture a server somewhere doing the thinking. The product collects data, sends it off and waits for an answer. That works, and for some products it's the right design. But more and more models are small enough to run on the device itself, and that changes much more than where the computing happens.

Take one product and build it both ways. It's a small sensor clamped to an electric motor in a plant room, the kind that drives a pump or a fan. Its job is to notice that a bearing is starting to wear, weeks before it fails and takes the motor with it. It does that by listening to vibration, measured thousands of times a second.

In the first design, the sensor streams its measurements to a server, and the model runs there. In the second, the model runs on the sensor, and only its verdict leaves the device.

	Model on a server	Model on the sensor
What leaves the device	A constant stream of raw measurements	A short message when something changes, and a sign of life once a day
The radio	On most of the time	Off almost all the time
Without coverage	Blind until the connection is back	Keeps listening and warns as soon as it can
Where the data ends up	On someone's server	On the device
Time to react	Seconds, if the connection is good	A fraction of a second
Cost to run	Grows with every sensor installed	Paid once, when the sensor is made

None of the rows is a detail. Plant rooms are often in basements, where coverage is poor. Customers increasingly ask where their data goes, and "nowhere" is the easiest answer to give. And a server bill that grows with every sensor sold eats into the margin of a product that was meant to be cheap.

On a small, battery-powered device, sending data usually costs far more energy than processing it. A sensor that streams everything has to keep its radio busy, and its battery is sized for that. A sensor that thinks for itself can listen, decide that everything is fine and go back to sleep without saying a word.

That turns the usual trade-off around. Running a model costs energy, but sending the data it replaces usually costs more. Done well, the device that runs its own AI is the one that lasts longer.

The catch is that the device is small. Its memory is a tiny fraction of a phone's, and every extra calculation shows up in the battery. A model that works well on a powerful computer has to be made smaller and cheaper to run without losing what makes it useful, and then tested on the hardware it will actually run on.

That's why this kind of AI is as much a hardware question as a software one. The choice of chip, how often the sensor wakes up, how much it measures each time and how large the model can be are all decided together. Settled one at a time, they tend to undo each other. And because the model will live in products that are already installed, there has to be a safe way to update it when it improves.

Some things a single device can't do. A model that compares hundreds of motors to spot a pattern none of them shows on its own needs to see them together. Very large models don't fit on small hardware at all. And if the model changes every week, it's easier to change in one place than in thousands.

The strongest designs often split the work. The sensor decides on the spot whether its own motor sounds right, and now and then sends a short summary. The server looks across all of them, learns from the whole fleet and sends better models back out. Each part does what its position makes it good at.

The finished sensor spends nearly all of its life in silence. It listens, finds nothing wrong and sleeps. Once a day it sends a few bytes to say it's still there. Then one day a bearing starts to wear, and the message that comes through is the one the whole product exists to send. This motor needs a new bearing, and there are weeks left to replace it.

Where the model runs isn't a detail to settle once the hardware is done. It decides how big the battery is, what the product costs to run, where the data goes and what happens when the connection drops. Change it late and much of the product changes with it.

That's why we design the device, the model and the software around it together, from the first prototype. You find out early which design fits your product, and you only build it once.