A document can fit in the prompt and still be missed.
You give an AI system a long report and ask whether its findings apply to your project. The answer sounds convincing, but overlooks a sentence saying the study covered a different group. The sentence was in the material you supplied. Including it did not ensure the model used it.
The material available to a model for a response is called its context. Its context window limits how much it can receive, usually measured in tokens—small units of text or other input. Lost in the Middle found that models in its tests often used relevant information less reliably when it appeared in the middle of a long input.[1] RULER tested more demanding tasks, including connecting and combining information, and also found performance falling as inputs grew.[2]
These studies give a reason to test the model and task you actually use. Instead of checking only whether all the documents fit, check whether the answer notices the details that could change the conclusion. Start by making those details easier to find, with enough surrounding material to interpret them correctly.
Check whether the answer used the important detail, not just whether the document fit.
Keep the project archive separate from the current task.
A project may contain interview transcripts, reports, earlier drafts, search results, and weeks of conversation. A request to compare two findings rarely needs all of that at once. Carrying everything forward also carries abandoned plans and old summaries that may no longer be correct.
Think of the current context as the material on your desk, with the rest of the project filed nearby. Include the question, the rules that still apply, the evidence needed now, and a note on where the work has reached. Systems such as MemGPT use a related separation between working context and stored information that can be brought back when needed.[6]
Keeping material outside the current prompt does not mean deleting it. Save it with a stable filename or identifier and make sure it can be retrieved. Keep permission limits and safety rules active throughout. For the remaining material, ask whether leaving it out could change this answer or prevent the next step from being checked.
Separate instructions, progress notes, and source material.
Label the different kinds of information you provide. Instructions say what the system should do. A progress note says what has already been checked and what remains open. Source passages provide evidence. Examples show the desired form of an answer. Keeping these distinct helps you spot when a previous AI summary is being treated as if it were an original source.
Update each part when its purpose changes. Keep ongoing rules available, and check that a revised instruction does not conflict with an earlier one. Refresh the progress note as work advances. Retrieve evidence for the question being answered. Use a few examples that show meaningful differences, and load tool descriptions when those tools could be needed.
Avoid copying the same claim into every part of the prompt. If a deadline changes, for example, an old conversation, a summary, and a task note may each show a different date. Keep a clear current record, with its source and update date, and mark the older date as replaced. Decide who maintains that record so the correction does not get lost.
Choose passages that answer this question.
A search tool may retrieve a passage because it uses similar words. That passage can still concern the wrong country, year, group, or product version. Before using it, state what the next step requires: a comparison, a quotation, a calculation, or a check of a claim.
For a comparison, find the same measure for both cases and check the periods match. For a quotation, include the surrounding sentences and any qualification. When checking a policy change, keep the effective dates and exceptions with the relevant versions. When interpreting interviews, include accounts that challenge the proposed theme as well as those that fit it.
You can keep links or file references in a progress note and open the fuller material when the next task needs it. Anthropic describes this as “just in time” retrieval.[4] It reduces the amount carried into every step, but depends on being able to find the right source again. Test that route before relying on a short reference in place of the document.
Choose evidence for the question you are answering now.
Leave out distractions, but keep inconvenient evidence.
More information can sometimes make an answer worse. In experiments using arithmetic word problems, Shi and colleagues found that unrelated details could distract language models.[3] In a research task, a subtler problem is a document that looks relevant but describes a different case. A report about last year’s product may not explain how the current version works.
Check why each item belongs in the task. Does it help answer the question? Can it support the kind of claim you are making? Does it cover the right people, dates, and definitions? A promotional page may help you find a technical report without establishing the product’s effectiveness. An older report may be useful for comparison if you clearly label its role.
Do not remove a source simply because it disagrees with the emerging answer. That disagreement may be exactly what the task needs. Remove material you have established is irrelevant, and check that important exceptions survive the selection. Instructions to ignore distractions helped in some of Shi and colleagues’ tests,[3] but a warning is not proof that the model will handle every confusing input correctly.
Use a summary to orient the work, and reopen sources for detail.
Shorter inputs can reduce cost and sometimes improve results. LongLLMLingua, for example, reported gains on its tested long-context tasks using prompt compression.[5] That does not mean any shorter summary is better. A summary preserves some details and leaves others out, often for a particular purpose.
Keep exact passages when wording or numbers matter. A dated list works well for decisions and unfinished tasks. A narrative summary can help someone understand the project, while a source index helps them locate detail. In each case, include a reference that leads back to the original material.
A meeting summary written to track actions may omit hesitation, disagreement, or the discussion that led to a decision. It should not later become the sole basis for interpreting participants’ attitudes. Label what the summary was made for and which source version it describes. Return to the original when a new question depends on detail the summary was not meant to retain.
A summary made for one purpose may leave out what the next question needs.
Refresh the context when the work changes.
Put related evidence together and keep a qualification beside the claim it limits. Make the current task easy to locate and distinguish source text from your interpretation. Because input order affected performance in Lost in the Middle,[1] try a different order on important test cases rather than assuming one arrangement works equally well for every task.
At the end of a search, save the useful sources, unresolved leads, and next question. After a calculation, save the output and how it was checked; carry forward only the log details needed to continue. If the research question changes, review the selected material again. Adding a new instruction beneath an outdated plan can leave both influencing the answer.
LongMemEval tests memory skills beyond recalling an earlier sentence, including handling updated facts, reasoning about time, and withholding an answer when information is missing.[7] A practical check is to correct a fact and see whether the next answer uses the correction. Date changing records, mark what they replace, and keep responsibility for updates clear.
Keep enough information to perform and check the next task, with a reliable way to reopen material left outside the prompt.
Test whether the model uses what matters.
Choose a few real tasks where you know what a correct answer must notice. Ask the system to retain a study’s key limitation, distinguish two dated policy versions, or resume work from a saved note. Include a case where a needed record is missing and the right response is to say what cannot be established.
Change one part of the input at a time. Try removing a summary, reordering documents, or adding a plausible but inapplicable source. Research on controllable working memory examines a related problem: whether models use relevant supplied facts and resist irrelevant context.[8] The practical tests proposed here let you check those concerns in your own workflow, without assuming the research results transfer unchanged.
Compare the answers against the original evidence. Keep a brief record of the context used, any summaries it relied on, and where the full sources remain available. Reduce unnecessary material when the checks support doing so. Keep or retrieve more when an omission changes the answer. The useful budget is the one that supports a correct, checkable result.