Developers who build applications on top of large language models (LLMs) have long relied on a scaffolding layer—indexes, query engines, retrieval pipelines, and agent loops. But this layer is crumbling. Jerry Liu, co-founder and CEO of LlamaIndex, argues that this collapse is not a crisis but an evolution. In a recent interview, he explained that as models get smarter, the need for heavy frameworks disappears. What survives? Context, simplicity, and a new kind of modularity. Here are the key takeaways from his vision.
1. What does Jerry Liu mean by “the AI scaffolding layer is collapsing”?
Liu refers to the once-essential infrastructure developers needed to ship LLM applications: indexing layers, query engines, retrieval pipelines, and carefully orchestrated agent loops. These tools helped compose deterministic workflows. But as models gain the ability to reason over massive unstructured data, self-correct, and perform multi-step planning, that scaffolding becomes redundant. “There’s less of a need for frameworks to compose these workflows in a light and shallow manner,” he said. The collapse isn’t a failure—it’s a sign that models are maturing. Developers can now use simpler primitives to achieve what previously required complex orchestration. For example, coding agents can generate most of the code themselves, minimizing reliance on external libraries. The result: the layers between programmers and non-programmers are vanishing, making English the new programming language.
2. Why is context becoming the key differentiator?
When the scaffolding vanishes, what remains? According to Liu, context is the new moat. All LLMs need high-quality, well-structured data to produce accurate answers. The ability to extract the right information from diverse file formats—PDFs, images, spreadsheets—becomes critical. Liu explains that “whether you use OpenAI Codex or Claude Code doesn’t really matter. The thing that they all need is context.” LlamaIndex focuses on agentic document processing, using optical character recognition (OCR) to unlock data trapped in file containers. Higher accuracy and cheaper parsing give a competitive edge. As models improve in reasoning, the bottleneck shifts from orchestration to data quality. Companies that can deliver clean, relevant context will thrive, regardless of which model or framework they choose.
3. How are agent patterns consolidating?
Liu observes that agent patterns are moving toward a “managed agent diagram.” Instead of building custom orchestration for every workflow, developers now use a harness layer combined with tools, Model Context Protocol (MCP) connectors, and skills plug-ins. This shift is driven by models that can discover and use tools on their own. Claude Agent Skills plug-ins and MCP allow models to integrate capabilities without requiring a separate integration for each. The result is a simpler, more modular architecture. Developers no longer need to wire up complex retrieval pipelines manually; they can point an agent at a dataset and let it figure out the rest. This consolidation reduces code complexity and speeds up development, making it easier for non-experts to build advanced retrieval systems.
4. What role does AI-generated code play in this shift?
Liu reveals that about 95% of LlamaIndex’s code is now generated by AI. “Engineers are not actually writing real code,” he says. “They’re all typing in natural language.” This means the layers between programmers and non-programmers are collapsing. Instead of mastering APIs and document integration, developers can simply describe what they need in plain English—and let coding agents like Claude Code handle the implementation. Three years ago, such an approach would have been extremely inefficient or even broken the agent. Today, it’s the norm. AI-generated code accelerates prototyping and reduces the need for deep framework knowledge. However, Liu cautions that quality control remains essential. The focus shifts from writing code to curating context, verifying outputs, and ensuring the model has the right data to work with.
5. How does LlamaIndex position itself as frameworks become less relevant?
Even though Liu acknowledges that RAG frameworks like his own are becoming less necessary, LlamaIndex is pivoting to what it does best: agentic document processing. The company has invested heavily in OCR and data extraction to unlock context from locked-up file formats. Liu argues that this core set of data—PDFs, images, legacy documents—has been largely ignored. By providing high-accuracy parsing and context extraction, LlamaIndex adds value that no model can replicate on its own. The company’s tools help developers feed clean, structured context into any LLM. This positions LlamaIndex not as a framework but as a data preparation layer. As Liu puts it, “We’ve really identified that there’s a core set of data that has been locked up in all these file format containers.”
6. Should developers worry about vendor lock-in with Anthropic or OpenAI?
There is growing concern that builders like Anthropic (with Claude Code) or OpenAI (with Codex) might lock developers into their ecosystems. Liu addresses this by emphasizing modularity. While models improve rapidly, the tools for tool discovery (MCP, skills plug-ins) remain open and portable. Developers can swap models without rewriting their entire stack, as long as they invest in context extraction and a managed agent diagram. Liu advises keeping the stack modular: use generic connectors and avoid deep dependencies on a single provider. The real moat is the context pipeline, not the model API. By future-proofing data preparation and using standardized protocols, developers can avoid lock-in and take advantage of the best model for each task.
7. What is the future of RAG frameworks like LlamaIndex?
Liu predicts that traditional RAG frameworks will fade as models become capable of handling retrieval and reasoning directly. Instead, the industry will move toward lightweight harnesses that connect models to tools and context. LlamaIndex itself is evolving into a data context platform rather than a framework. Developers will use extremely simple primitives to build advanced retrieval. “It’s just way easier for people to build even relatively advanced retrieval with extremely simple primitives,” Liu notes. The future lies in context-rich, modular systems where the heavy lifting is done by the model, and the human role is to curate the data and define the goal. Frameworks won’t disappear entirely, but they will shrink to a thin layer—leaving context as the true competitive advantage.