Does Giving AI More Context Always Lead to Better Answers?
More context does not guarantee better answers. Organize goals, evidence and task state, preserve uncertainty in summaries, and evaluate handoffs with controlled comparisons.
You have been working on a task for days, and the chat history keeps growing. Worried that the AI will forget something, you paste back every old plan, error log, change record, and reference, then add: “Please continue using all the context.”
That provides more information, but it does not guarantee the AI will have a clearer idea of what to do now. An old plan may have been abandoned, a log may predate a change, and an earlier claim that something was “done” may have been only a verbal assessment.
What you actually need to organize is: which facts this step should rely on, which constraints it must follow, and what result it should produce.
I previously discussed context engineering in “Before Switching Models to Make AI Smarter, Try This”. This article applies it to handing off and continuing long-running tasks. It draws on publicly available research and engineering materials, reviewed as of November 10, 2026. The comparison proposed here has not been carried out, and this article contains no measured data on improvements in results.
Being Able to Fit It In Does Not Mean the Model Can Use It Reliably
A context window describes how much information a model can receive at once. Its capacity alone does not show that the model will use every part of that information correctly.
The Lost in the Middle study found, in multi-document question-answering and key-value retrieval tasks, that the position of relevant information affected the performance of the models tested. Performance was particularly likely to decline when the information appeared in the middle of a long context.[1]
That study was published in 2023. It does not establish that all models today have the same degree of difficulty, much less imply a universal “best word count.” It reminds us to judge whether long context works by looking at a particular model’s results on a particular task, not just its advertised window size.
Anthropic’s article on context engineering likewise emphasizes selecting high-value information and maintaining the context as a task progresses. Its advice is not simply to make everything shorter, but to reduce redundancy while retaining enough information to specify the expected behavior.[2]
For example, keeping the full context may matter when analyzing clauses that refer to one another in a contract. When fixing a clearly identified button bug, however, pasting in the complete source code of unrelated modules may be less useful than providing the relevant files, reproduction steps, and test results.
Long Conversations Most Easily Blur “Then” and “Now”
Suppose we are preparing a blog post, and the record contains three statements:
First version: Do not publish it yet.
Second version: It can be published on the website.
Current instruction: Publish it on the website; save drafts only for the WeChat official account and Zhihu.
All three are part of the real history, but they do not constrain the current action in the same way. If the handoff summary says only “the user approved publication,” it loses the platform-specific limits. If it keeps only the first version, it stops work that has since been authorized.
Adding another instruction to “read carefully” will not fully solve this. The summary should state the currently effective instruction and note which later decision superseded the old one.
Similarly, “called the save API” and “read the draft back and confirmed it exists” should be kept separate. The first records an action; the second is evidence that can help determine whether submitting again would create a duplicate.
| Information | Suggested way to retain it | What to do before continuing |
|---|---|---|
| User’s current goal and scope | List them concisely, retaining essential original wording or sources | Check whether later messages changed the scope |
| Current files and versions | Record paths, versions, or commit identifiers | Read the actual files |
| Completed actions | Record object identifiers, receipts, and verification results | If the outcome is ambiguous, check it before doing the action again |
| Failures and unresolved issues | Record the specific failure point and methods already tried | Distinguish between not attempted, failed, and outcome unknown |
| Historical alternatives | Keep them in an archive, with reasons they were abandoned | Consult them when the history needs to be traced |
This table offers a way to organize tasks, not a guarantee of model capability.
Give a Status Summary First, Then Keep a Way to Check the Sources
A useful handoff package can consist of a short summary plus materials that can be consulted when needed.
The short summary answers “where do things stand now?” Files, logs, and links answer “what supports that assessment?” They do not all need to enter the model’s context at the same time.
Anthropic describes loading material on demand as a “just in time” strategy: retain lightweight identifiers such as paths, then read the content when needed. It also notes that exploring materials at runtime has a cost and can take more time; without a good way to retrieve information, the model may still miss crucial material.[2]
So you cannot simply give an AI a directory and assume it will find the answer. You can provide an explicit index:
Task: Publish article A on the website and save drafts on two platforms.
Scope: Public on the website; drafts only on WeChat and Zhihu.
Current version: Path to the article body and its corresponding cover file.
Completed: Website is live; public URL and body verification record are in the specified file.
Remaining: Read back the Zhihu draft.
Unknown: One save request timed out; whether it was written cannot be confirmed.
Next step: First search for the draft by title and original article link; decide whether to retry only after checking the result.
Acceptance criteria: Report the public page and draft statuses separately; do not describe everything as published.
This is an example template; paths and receipts must be filled in with real data. Its value is that it puts the goal, status, evidence, and next step in separate places, giving “continue” a precise meaning.
Do Not Compress Away Uncertainty
Long tasks need summaries, but summaries can lose important qualifications.
If “the submission API timed out; the outcome has not yet been found” becomes “submission failed,” the next round may create a duplicate. If “only the local preview was checked” becomes “verification passed,” the online check may be skipped.
Anthropic also cautions, in its discussion of context compression, that excessive compression can discard subtle but crucial information, especially conditions whose importance becomes apparent only later.[2]
A more reliable approach is to make the summary distinguish three kinds of content: observed facts, judgments based on those facts, and matters still unconfirmed. Keep sources with the summary; when resuming the task, recheck any status that will affect the next action.
Keep the original acceptance criteria, too. If successive handoff summaries shrink until only “make the page good” remains, the original requirements for mobile display, accessibility labels, and save behavior may disappear.
A Shorter Answer Alone Does Not Prove the Organization Worked
Whether organizing context improves results should be assessed by comparing similar tasks.
You could prepare two sets of materials: one that supplies the raw record directly, and another that supplies a current-status summary with sources available to read on demand. Keep the model, task instructions, tool permissions, and acceptance criteria the same for both groups, and record which materials were actually loaded.
At a minimum, compare whether requirements were missed, old states were cited, actions were repeated, how many rounds of rework were needed, and total time and usage cost. Do not select only one successful example; model outputs vary, so several runs will be more informative.
You can also design counterexamples: If the task requires comparisons across many chapters, has the summary removed necessary details? If the file index is incomplete, does on-demand retrieval work worse than providing the full text directly?
Good context organization should reduce irrelevant material while preserving the details needed to complete the task. When those goals conflict, prioritize completing the task in a verifiable way before discussing how many words were saved.
Sources
- Liu et al.: Lost in the Middle: How Language Models Use Long Contexts, 2023. This article cites the tasks and information-position effects described in its abstract; it does not generalize experiments on older models into a uniform conclusion about all current models.
- Anthropic: Effective context engineering for AI agents, 2025-09-29. Consulted for its engineering recommendations and caveats concerning context selection, on-demand retrieval, compression, and structured notes.
AI-generated concept illustration, not a test photograph or network topology diagram.
Discussion
Share a question or your experience. Comments appear after moderation. Your email is never displayed publicly. Comment privacy
Loading comments…