#llm
Posts tagged llm.
2025-03-10Our Embedded World edge AI demo, and beating GPT-4oFor Embedded World 2025 we built a small AI that fits in any appliance, using off-the-shelf Hugging Face models, and pitted it against ChatGPT-4o.
2025-02-11Embedded AI solves a problem ChatGPT-4o can'tRunning DeepSeek R1 distill fully on-device on a Synaptics Astra board, it solves an AIME problem that ChatGPT-4o gets wrong.
2025-01-27DeepSeek and reasoning models on the edgeDeepSeek's R1 distillations bring strong reasoning to small models. I got R1 Distill Qwen 1.5B running on a Synaptics Astra board.
2025-01-07Embeddings: the foundation of our on-device assistantEmbeddings power LLMs, RAG, and our on-device voice assistant at CES 2025. Here's how we prepared the data, with an interactive visualization.
2024-12-16Building a speech-to-speech assistant with only on-device AIHow we built a speech-to-speech assistant that runs entirely on-device, using a sentence transformer for semantic search instead of a full LLM.
2024-12-11A multi-modal on-device assistant that can seeA multi-modal AI assistant running 100% on-device, using an on-chip NPU to accelerate vision, with the agent accessing it via tool calling.
2024-12-10AI now fits in your hand: LLMs got about 10x smallerLLMs are roughly 10x smaller than a year ago. Here's why that matters for device makers building on-device AI.