Sources

Redis published a clean side-by-side comparison: "RAG queries averaged a cost of $0.00008 per request, while full-context LLM queries averaged $0.10 per request, over 1,250 times more expensive." A well-designed RAG pipeline retrieves five to twenty chunks totaling two to ten thousand tokens; a long-context call sends the entire corpus on each invocation. The ratio is real and faithful to the workload that justified building RAG in the first place. It is also beside the point for the ephemeral case, where total spend in absolute dollars matters more than the ratio.

Atlas Workspace's comparison piece captures the actual product behavior: "Claude uses a hybrid approach. When your project files are small enough, everything gets loaded directly into context, meaning it has your entire project in memory at once." Students arriving at Week 5 with the Claude Projects mental model are not wrong about how the tool works. The course is what is missing the vocabulary. The same source contrasts NotebookLM (strictly grounded retrieval) with Claude Projects (context-augmented generation that can also draw on training knowledge), a distinction Week 5's current vocabulary cannot make.

Hemanand Rajendran's piece on the Google Cloud Community publication argues that "combining the long context window (2M tokens) with the Context caching feature, one can skip the lengthy process of building RAG pipelines in their applications." Important here because the same pattern (long window plus a context cache, no retrieval step) is documented for at least two of the three frontier providers. This is not a Claude-specific observation; it is the design pattern the major model APIs are converging on for ephemeral document workflows.

From Anthropic's documentation: "Cache write tokens at 1.25 times the base input tokens price for 5-minute caching, 2 times for 1-hour caching, and cache read tokens at 0.1 times the base input tokens price ... A cache hit costs 10% of the standard input price, meaning caching pays off after just one cache read for the 5-minute duration." Read that twice. The 5-minute TTL was effectively designed for ten-minute Q&A sessions over uploaded documents. Anthropic also reports up to 85% latency reductions for cached prompts, with a 100,000-token book example dropping from 11.5 seconds to 2.4 seconds.

MindStudio's analysis is the cleanest published version of the crossover: "For a 200,000-token knowledge base, traditional RAG (5 chunks): ~$0.012 per query. Long-context: ~$0.60 per query. Long-context with caching: ~$0.06 per query." Without caching, long-context costs fifty times more per query than RAG. With caching, it costs five times more. At five queries per session that is a difference of about 24 cents, a rounding error compared to the engineering cost of building a managed-cloud RAG pipeline you will use once. The same piece names the conditions where long-context wins: documents fit, low-to-medium query volume, frequently changing content, full-document coherence required.

Chan, Chen, Cheng, and Huang frame CAG as "particularly suited for applications with a constrained knowledge base where documents are of a limited and manageable size." That phrase is technically about benchmark suites; it is, in practice, an exact description of "a handful of PDFs." The paper claims CAG "eliminates retrieval latency and minimizes retrieval errors while maintaining context relevance." A peer-reviewed paper now exists with a name for the workflow Week 5 currently has no vocabulary for. Adopting the term, even just as a glossary entry, closes the gap between course vocabulary and student experience.

The published Anthropic privacy policy describes the conditions under which API inputs are logged, retained, and reviewed. The relevant practical fact for this article: API content is retained by default for safety and abuse review, and the boundaries on training use depend on which product surface and which contractual tier the customer is on. Citing Anthropic specifically because their published policy is the clearest of the three frontier providers; the same general pattern (default-on logging, opt-out training use under enterprise contract) holds for OpenAI and Google. For sensitive documents, this is the variable that overrides the cost analysis: cloud CAG is not a private channel, regardless of how cheap it has become.

Liu et al. write, in TACL: "Performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models." The "even for explicitly long-context models" qualifier is the part that should land in a Week 5 reading. Marketed window size is not the same as effective window size, and a CAG session that exceeds the model's effective window will silently lose accuracy on questions whose answers are buried mid-document.

Zilliz reports that "Gemini 1.5 maintains recall capabilities all the way to 1M tokens, but the average recall hovers around 60%. If you want to make sure the model is actually using the context you are sending it, you are best off curating it first." Zilliz is a vector-database vendor and the framing reflects that interest. The recall figure is striking nonetheless: even when the documents fit, the model uses about six in ten of the relevant facts on average. The article cites this as the empirical ceiling on "stuff the documents in" approaches, not as a vendor argument for vector databases.

Databricks Mosaic Research benchmarked twenty models and found that most show a U-curve. Specific thresholds from the blog: "GPT-4-0125-preview starts to decrease after 64k tokens. Llama-3.1-405b performance starts to decrease after 32k tokens. Mixtral-8x7b degradation begins at 4k tokens." The frontier models that maintain accuracy up to 100k tokens are a specific named set: o1, GPT-4o, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro. The same benchmark surfaced a non-obvious failure: at long contexts, Claude 3 Sonnet's copyright refusal rate climbed from 3.7% (16k) to 49.5% (64k), a categorical failure rather than gradual quality decay.

The arXiv paper underneath the Databricks blog. Provides the methodology and full numerical results behind the threshold claims, including the average retrieval recall plateau (0.788 at 8k tokens, 0.894 at 32k, 0.947 at 96k+) and the dataset-specific saturation behavior. Useful as the academic backing for the otherwise blog-only U-curve claim. Cited adjacent to ref 10 because the two are companion artifacts.

Yu et al. argue against the prevailing trend by showing that "extremely long context in LLMs suffers from diminished focus on relevant information and leads to potential degradation in answer quality." They propose Order-Preserve RAG (OP-RAG), which retrieves and concatenates chunks in original document order rather than relevance-ranked order. The relevance to this article: even when the documents fit in a long-context window, retrieval-curated context can still beat full-corpus context on accuracy. The strongest published counter-argument to the "long context replaces RAG" framing.

From the abstract: "When resourced sufficiently, LC consistently outperforms RAG in terms of average performance. However, RAG's significantly lower cost remains a distinct advantage." The paper introduces Self-Route, a hybrid mechanism in which the model itself decides per-query whether the retrieved chunks are sufficient or whether to fall back to long-context. The empirical efficiency result is the load-bearing line: "GPT-4O uses only 61% tokens while achieving comparable performance with LC, and Gemini-1.5-Pro uses 38.6% of the tokens." The actual cost-quality frontier, with peer review.

Elastic's own reference documentation describes Elasticsearch's native dense-vector field type, kNN retrieval, and hybrid scoring (BM25 plus vector similarity). Cited here for a single load-bearing claim: the engine the search community has been using for over twenty years now ships embedding-based ranking as a built-in feature. The "RAG vs traditional search" boundary is increasingly artificial. Most enterprise RAG stacks are, structurally, search engines that have grown an embedding layer; the engineering disciplines that distinguish good-enough Elasticsearch from production-grade Elasticsearch (query rewriting, freshness boosting, evaluation harnesses) transfer almost entirely to RAG.

Akita argues that for many ephemeral document workloads, the right substitute for vector RAG is not long context but a deterministic textual prefilter (grep, ripgrep) feeding a small context. The point that matters for the decision framework: once you stop assuming RAG is the default, the design space opens up beyond the RAG / CAG / hybrid trichotomy. A grep-prefilter pattern with no embedding model in the loop is genuinely competitive when the corpus has structure that text matching can exploit.

From LlamaIndex's piece: "While long-context LLMs will simplify certain parts of the RAG pipeline (e.g. chunking), there will need to be evolved RAG architectures to handle the new use cases that long-context LLMs bring along." LlamaIndex is a RAG-framework vendor and the framing reflects that interest. The argument is still useful: long context kills the chunking step but not the retrieval step, because in production the corpus is usually still bigger than any window. The article's three-pattern reframing is consistent with this view; it just names the bounded-corpus subcase explicitly.

Yu, Xu, and Akkiraju revisit the long-context vs RAG question on more current benchmarks. Their framing closes the loop on the academic literature: "Long Context generally outperforms RAG in question-answering benchmarks, especially for Wikipedia-based questions. Summarization-based retrieval performs comparably to LC, while chunk-based retrieval lags behind. RAG has advantages in dialogue-based and general question queries." The carve-out is the part Week 5 should hear: chunk-based RAG (the variant Week 5 currently teaches) is the worst-performing of the three retrieval forms. The vocabulary to describe the alternative now exists in peer-reviewed form.