LightspeedLM
Infinite context, bounded cost.
A long-running agent should keep the thread of its own work: the corrections it was given, the reasoning behind decisions it made. LightspeedLM, from the research team at NexSpring, holds that continuity at the inference layer. The active footprint stays bounded as history grows, and nothing is summarized away or replayed to get it back: the agent works from the state it produced, not a paraphrase of it.
The continuity gap
Agents can search documents, retrieve facts, and summarize past conversations. A long-running one still loses the thread of its own work, because every way of handling a growing session forces a trade.
- Keep history active
- Memory, latency, and inference work grow with it.
- Replay old material
- The agent pays again to rebuild a state it already reached.
- Summarize or compact
- Details, rationale, and corrections go missing.
- Start a new session
- It comes back with records of the past and none of the continuity.
The result is familiar: repeated mistakes, lost decisions, stale instructions, and people rebuilding context before work can resume. Retrieval and explicit memory are still essential for searchable facts, and long-context models keep more within reach. Neither closes the continuity gap.
Continuity at the inference layer
What the model can see at any moment stays the same size however long the session runs. That is the active buffer, holding the key and value vectors the model has already computed for the tokens it is working from. Everything else moves to long-term storage, and KV splicing moves entries between the two in place, without re-running the model over them. Three moves do the work:
- Keep
- The cache persists between turns. A new turn appends only its new tokens, and nothing already computed is run again.
- Trim
- At the budget limit the engine evicts the least-attended entries rather than the oldest, and offloads them instead of discarding them.
- Re-pack
- Retained entries are re-positioned to stay inside the trained window, so a session can run past the model's native limit.
Drops into your harness
The private beta ships with an OpenAI-compatible API, so the agent framework you already run works against it without changes. Point your client at it and keep your prompts, tools, and orchestration as they are. It runs self-hosted or in your own cloud, so session state stays inside your deployment.
What it changes
- More consistent work
- Corrections, conventions, and task history carry across long workflows, so there are fewer repeated mistakes and less drift from decisions already made.
- Less replay
- Less repeated processing, and less time spent restoring context the agent already had.
- Fits your memory stack
- It sits alongside retrieval, files, and application memory. Searchable facts stay explicit; continuity covers what those systems miss.
Who it is for
Agent developers
Sessions measured in days.
Companions and coding agents that run for hours or days, keeping the character and decisions they built up.
Platform teams
Serving cost that stays flat.
Only new tokens are prefilled and the active buffer is fixed, so per-step work does not scale with lifetime history. Throughput against session length is what the beta will measure.
Status
LightspeedLM runs today as an experimental research prototype, and we are putting together a private beta for a small number of teams. It runs on a CUDA GPU against a Qwen2 or Qwen3 checkpoint. Tell us the framework you drive inference from and the workload you want to keep alive, and we will scope the beta around it.
On HotpotQA, in the distractor setting, an 8k active buffer scored 0.70 token-F1 against 0.65 for the same questions run with the full context. Eviction on its own, with nothing recalled after it left the buffer, scored 0.28. These are preliminary results on Qwen3-32B over 20 questions, and they have not been independently validated.
It is our first step toward agents that carry their experience forward without giving up speed or control.
Ask us about the upcoming private beta, and tell us what you are running and how long it needs to last.