← Explore papers

Paper · 2026

Context Language Models

Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh

Read the source ↗Open source PDF ↗arXiv 2609.37725

Watch and listen

Paper explainer

One shared explainer for this paper, with versions you can explore more deeply.

Needs review

Source version v1

Progress

84 saved steps

Stopped at: Applying correction (round 1, candidate 3)

Progress is saved while this explainer waits for review.

Requested
Last completed
Last status update
Activity log

Showing the latest 50 saved steps.

  1. Verifying a proposed correction (round 1)
  2. Verifying a proposed correction (round 1)
  3. Verifying a proposed correction (round 1)
  4. Verifying a proposed correction (round 1)
  5. Verifying a proposed correction (round 1)
  6. Verifying a proposed correction (round 1)
  7. Verifying a proposed correction (round 1)
  8. Verifying a proposed correction (round 1)
  9. Verifying a proposed correction (round 1)
  10. Verifying a proposed correction (round 1)
  11. Verifying a proposed correction (round 1)
  12. Verifying a proposed correction (round 1)
  13. Verifying a proposed correction (round 1)
  14. Verifying a proposed correction (round 1)
  15. Verifying a proposed correction (round 1)
  16. Verifying a proposed correction (round 1)
  17. Verifying a proposed correction (round 1)
  18. Verifying a proposed correction (round 1)
  19. Verifying a proposed correction (round 1)
  20. Verifying a proposed correction (round 1)
  21. Verifying a proposed correction (round 1)
  22. Verifying a proposed correction (round 1)
  23. Verifying a proposed correction (round 1)
  24. Verifying a proposed correction (round 1)
  25. Verifying a proposed correction (round 1)
  26. Verifying a proposed correction (round 1)
  27. Verifying a proposed correction (round 1)
  28. Verifying a proposed correction (round 1)
  29. Verifying a proposed correction (round 1)
  30. Verifying a proposed correction (round 1)
  31. Verifying a proposed correction (round 1)
  32. Verifying a proposed correction (round 1)
  33. Verifying a proposed correction (round 1)
  34. Verifying a proposed correction (round 1)
  35. Verifying a proposed correction (round 1)
  36. Verifying a proposed correction (round 1)
  37. Verifying a proposed correction (round 1)
  38. Checking claims
  39. Preparing spoken wording (round 1)
  40. Verifying spoken wording (round 1)
  41. Verifying spoken wording (round 1)
  42. Verifying spoken wording (round 1)
  43. Verifying spoken wording (round 1)
  44. Verifying spoken wording (round 1)
  45. Verifying spoken wording (round 1)
  46. Verifying spoken wording (round 1)
  47. Verifying spoken wording (round 1)
  48. Verifying spoken wording (round 1)
  49. Verifying spoken wording (round 1)
  50. Verifying spoken wording (round 1)
Needs review

Generation stopped before publication. Review the saved work before resuming.

Generation needs review before an explainer can be published.

Abstract

We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. Building CLMs zero-shot with existing models outperforms SOTA context management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task. Moreover, by shifting context management from external harness control to intrinsic model behavior, CLMs naturally enable both in-context and parametric learning of context-management strategies. We show that CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop, improving held-out accuracy by up to 35.9 points on a context-management task while reducing compute. We also introduce an online reinforcement learning method for CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Finally, we co-design Suffix Cache Reuse for CLM serving, further reducing server-side compute by 35% relative to standard SGLang at matched performance.

Source history

The shared explainer cites the source version used to generate it.

  • v1 · Source available for generation · 2026-10-05 UTC
  • v1 · Source available for generation · 2026-10-05 UTC