Our Embedded World edge AI demo, and beating GPT-4o
Inspired by Thomas Wolf at Web Summit, we built a small AI that could fit in any appliance, using off-the-shelf models from Hugging Face. To hit the latency target, we use an encoder-only language model today.
The result is a set of bite-sized examples assembled into an app, using our Synaptics Python API for NPU acceleration of vision in a multi-modal implementation.
The journey was illuminating, and included pitting our SL1680 Linux board against ChatGPT-4o, and winning, thanks to the DeepSeek R1 1.5B distill.
I have both a hands-on workshop and a paper presentation at Embedded World in Nuremberg to share the journey, the code, and the models.