In my previous post, What is Speculative Decoding and How Does a Draft Model Work?, I walked through how a small draft model guesses a few tokens ahead so a much larger target model can
Before getting into the setup, here are two short recordings of what this post builds. Both run entirely on my laptop through llama.cpp, with Laya making every decision locally. Link to video on Youtube: https://youtu.be/VTfF4eVHa0E
I was speaking to a colleague the other day about how LLMs process tokens and predict the next word (discussing the technical details of LLMs excites me), and when I brought up speculative decoding, which
Back in July I built the same login portal three times with Claude Code, once through a subagent relay, once with subagents dispatched in parallel, and once with an agent team whose members could message
