Derivative reuse has always been one of the most practical—and one of the most dangerous—forms of content reuse. It begins with a sensible premise: use an existing content component wherever possible, but allow limited changes when the reuse context requires them. In Managing Enterprise Content, Ann Rockley and The Rockley Group distinguish derivative reuse from locked reuse by allowing reused content to change while retaining a relationship to the original content source. In principle, derivative reuse gives content teams the best of both worlds: reuse without forcing false sameness, and variation without losing traceability. [1]

In practice, derivative reuse can become one of the hardest reuse patterns to govern. The problem is not that derivatives exist. The problem is that derivatives often accumulate at the wrong level of granularity. When a sentence, word, phrase, or even a character varies by product, jurisdiction, market, customer, audience, or effective date, authors often respond by embedding more and more conditional logic directly inside the content. Over time, the source becomes less like an authored topic and more like a dense programming artifact. The content may still publish correctly, but it becomes increasingly difficult for authors, reviewers, subject matter experts, translators, and auditors to understand what the source actually says.

This article describes a field-tested approach for managing derivative reuse in high-variant structured content environments: the Core-Exception model for DITA content. The model was developed at NCCI (National Council for Compensation Insurance) to support a complex, state-specific publication with dozens of variants, frequent regulatory updates, strict traceability requirements, and a need to move from a manual exception-compendium model to state-specific publishing at build time. The approach was later presented publicly in the ConVEx conference session On the Road to Boca Raton, delivered in Tempe, Arizona, on May 4, 2022, where the architectural principles, publishing model, and implementation lessons were shared with the broader DITA community. [7][8]

The central idea is simple: stop placing most conditional logic inside the smallest pieces of prose. Instead, move the condition to a managed block of microcontent. One block represents the core rule that applies to most contexts. Zero or more additional blocks represent exceptions that apply to specific contexts. Publishing logic selects an applicable exception when one exists; otherwise, it falls back to the core.

The result is not the elimination of conditional content. It is the disciplined containment of conditional content.

Derivative reuse and the granularity trap

Rockley’s reuse model treats derivative reuse as a legitimate reuse option. A derivative is created when reused content is modified for a new context, while the relationship to the original is preserved. This is valuable when the substance of the content is shared but the expression must differ: spelling, tense, order, emphasis, regional needs, translation, or other contextual variation may justify a derivative. [1]

The difficulty begins when derivative reuse is implemented as a proliferation of tiny conditional fragments. DITA provides powerful mechanisms for profiling and conditional processing. Conditional attributes can be used to include, exclude, or flag content during processing, typically through DITAVAL profiles. [3] These mechanisms are essential for structured publishing. They make it possible to produce multiple deliverables from a common source.

But the same mechanism that works elegantly for a topic, section, paragraph, or reusable component can become fragile when applied repeatedly inside sentences. A phrase conditioned for five audiences may be manageable. A sentence containing conditions for dozens of jurisdictions, effective dates, business rules, and approval states is not. Multiply that pattern across hundreds or thousands of topics and the source becomes difficult to review, difficult to audit, and dangerous to modify.

This is the granularity trap: the content team chooses reuse at a level so small that the management cost begins to exceed the reuse benefit.

At that point, the content architecture needs a different pattern. The question is no longer “Can this word be reused?” The better question is “At what level should variation be governed?”

The NCCI challenge: many variants, high traceability, frequent updates

The NCCI use case involved a large policy-type publication of approximately 800 pages, chunked into numbered sections. The publication had historically been maintained through a manual workflow. A national version of the rules was distributed along with a compendium of state exceptions. Clients then had to use both the national content and the applicable exception material to determine how the rules applied in a particular state. [7]

The publication was not static. It required incremental updates four to six times per year. Some content applied only to certain states. Some content was excluded for certain states. Other content was substantively the same across states but differed in wording. During updates, impacted numbered sections were revised and sent to state regulators for review and approval. Regulators could approve, reject, or approve with changes, and each regulator worked on its own timeline. [7]

Traceability was not optional. The organization needed to track historical, current, approved future, and proposed changes. Some content could be approved but not take effect for six to eighteen months. Some historical versions remained relevant because of court cases, regulatory interpretation, or future product contracts. [7]

In this environment, ordinary conditional processing at the phrase level would have produced an authoring and governance problem. With 37 state clients at the time, and the possibility of more in the future, the content team needed a way to publish state-specific versions without forcing authors to manage dozens of inline conditions inside every affected topic.

As described in On the Road to Boca Raton, the challenge was not simply one of publishing multiple outputs. It was a challenge of balancing reuse, regulatory variation, workflow, version control, and long-term maintainability. The solution was to combine DITA, microcontent, structured writing, and conditional publishing into a more governable architecture. [8]

From topics to microcontent blocks

The first architectural move was to break each numbered section into a block of microcontent. Because workflow, approval, versioning, and traceability needed to be managed in the component content management system, each microcontent block was managed as a topic. [7]

This had an important consequence: the “topic” no longer represented a traditional, full-bodied unit of exposition. Instead, it became a managed block of structured information. The team applied strict writing and titling rules aligned with Precision Content practices and typed each block according to its information purpose: concept, task, process, rule, or reference. [7]

This decision also aligned with the broader logic of information typing in DITA. DITA uses information types such as concept, task, and reference to distinguish different kinds of information and to help authors keep content focused, modular, searchable, navigable, and reusable. [5] Precision Content similarly emphasizes modular, reusable, typed content, organized through repeatable patterns and governed standards. [6]

The key architectural insight was that the standard topic body structure could be too heavy for this use case. Instead of authoring a large task topic with context, prerequisites, steps, results, and related supporting information inside one body, the team decomposed those structures into separately managed microcontent topics. The body was removed from the topic design, and the key semantic structures were moved into a specialization of the topic abstract for each information type. [7]

As discussed in the ConVEx presentation, this shift represented a move away from thinking of topics as documents and toward thinking of topics as governed information assets. The architecture emphasized management, reuse, and traceability at the microcontent level rather than at the publication level. [8]

That move made the block—not the paragraph, phrase, or character—the primary unit of governance.

The Core-Exception model

Once the content was managed as typed microcontent blocks, the team could address derivative reuse at the correct level. Instead of embedding state-specific conditions inside a single block of prose, the team organized each affected section around a Core-Exception pattern. [7]

The topic model has three main components:

1. A typed container topic that holds the title and prolog for the topic blocks.

2. A Core Block: an embedded microcontent topic block representing the content that applies to the majority of states.

3. Zero or more Exception Blocks: embedded microcontent topic blocks representing content that applies to one or more specific states.

Each Exception Block carries the conditions for the states to which it applies. The publishing logic looks for exceptions first. If an exception matches the state being published, that exception is included and the Core Block is suppressed. If no exception matches, the Core Block is included. Non-applicable Exception Blocks are excluded. [7]

This changes the authoring model dramatically. Without the Core-Exception model, a topic serving 37 states might require many conditional fragments inside the same source topic. With the Core-Exception model, the topic contains one core expression and only the exceptions that actually exist. If 31 states use the core wording and six states require changes, the author manages one Core Block and six Exception Blocks—not 37 parallel variants. [7]

An important characteristic of the model is that most content requires only a Core Block. Exception Blocks are created only when a meaningful variation exists. As a result, the majority of topics can remain simple and free of conditional complexity, while still supporting extensive variation where necessary.

The ConVEx presentation emphasized that the architecture intentionally treats exceptions as managed content objects rather than conditional fragments. This distinction is critical because it preserves readability, reviewability, and governance while still leveraging DITA’s conditional processing capabilities. [8]

The pattern also scales more gracefully. When a new state is added, the team does not need to touch every topic. Only the sections that differ for that state require new exception blocks. [7]

Exception-first publishing logic

The Core-Exception model depends on clear publishing logic. The processor must evaluate exception applicability before selecting the core. Conceptually, the rule is:

• If a matching exception exists, publish the exception, and

  • Exclude the core and non-matching exceptions.

• Else if no matching exception exists, publish the core, and

  • Exclude all exceptions.

• Else if the matching block is marked as not applicable, exclude the topic, and

  • Exclude its dependent child topics in the map.

This final point is important. The model was extended beyond individual topics into the publication map. If a topic was excluded according to the Core-Exception rules, subsequent child topics in the map under that excluded topic were also excluded. [7]

DITA already supports conditional processing through profiling attributes and DITAVAL, and DITA branch filtering allows filtering conditions to be applied to specific branches of a map rather than only globally. [3][4] The Core-Exception model can be understood as a business architecture and authoring governance layer built on top of such conditional publishing concepts. It gives authors a predictable pattern for where variation belongs and gives publishing logic a deterministic way to resolve that variation.

As demonstrated in On the Road to Boca Raton, the publishing framework effectively creates a controlled inheritance model in which the core serves as the default expression and exceptions override that default only when applicable. This exception-first approach significantly reduces the amount of conditional markup required in source content. [8]

Why the model works

The Core-Exception model works because it separates three concerns that are often tangled together:

• the stable identity of the content unit

• the default expression of the content, and

• the variant expressions required for specific conditions.

The typed container topic preserves the identity of the numbered section. The Core Block represents the default content. Exception Blocks represent controlled deviations from the default. Conditions are attached to the exception blocks rather than scattered throughout the prose.

This makes the content easier to reason about. Reviewers can compare the core and the exception directly. Authors can see whether a state has a true exception or simply inherits the core. Information architects can govern the number and type of exceptions. Publishing engineers can implement a clear fallback rule. Auditors can trace which block appeared in which state-specific output and why.

The model also protects readability in the source. Authors are no longer forced to read a sentence interrupted by multiple conditional fragments. Instead, they read complete, coherent blocks. This is especially important when reviewers are regulators, legal experts, or business stakeholders who must approve the meaning of the content, not the cleverness of the markup.

Relationship to profiling and conditional content

It would be a mistake to present the Core-Exception model as an alternative to DITA profiling. It is better understood as a disciplined use of profiling.

DITA conditional processing is designed to filter or flag content based on processing-time criteria. It allows attributes such as audience, platform, delivery target, props, and specialized props to determine whether elements are included, excluded, or flagged during processing. DITAVAL defines the conditional-processing profile. [3]

The Core-Exception model uses that general capability but changes where the conditions live. Instead of conditioning many small fragments inside a topic, it conditions whole microcontent blocks. This creates a more governable relationship between reuse and variation.

In simple publications, inline profiling may be entirely appropriate. If a short phrase differs for two products, an inline condition may be the simplest solution. But as the number of variants rises, the burden shifts. The question becomes not “Can the tool support this?” but “Can people safely understand, review, and maintain this over time?”

The ConVEx presentation explicitly positioned the Core-Exception model as a response to the limitations of highly granular conditional content. Rather than replacing profiling, the model establishes architectural boundaries around where profiling should be applied and how variation should be represented. [8]

The Core-Exception model is most useful when:

• variants are numerous

• exceptions are sparse relative to the full variant set

• reviewers need to approve complete statements

• traceability and effective dating are important

• the publication has a stable logical structure

• new variants may be added over time, and

• differences are meaningful enough to deserve block-level governance.

Change tracking and version baselines

The final major challenge in the NCCI implementation was change tracking. Each state could potentially be associated with different versions of the same section. Some states might have no change between releases. Others might have substantial changes. Some approved changes might not yet be effective. [7]

The Core-Exception model supports this complexity because each microcontent block is managed as a discrete topic in the CCMS. That means the system can version, review, approve, and baseline the Core Block and each Exception Block independently. Rather than treating a large topic as a monolithic unit with hidden internal variation, the architecture exposes the meaningful units of change.

This has several governance benefits. First, it allows the organization to know exactly which block was active for which state at which time. Second, it enables review workflows to focus only on affected blocks. Third, it reduces the risk that a change intended for one state will inadvertently affect another. Fourth, it gives auditors a clearer record of the relationship between source content, approval status, effective date, and published output.

For regulatory and policy content, this is often more important than authoring efficiency. Faster publishing matters, but trustworthy publishing matters more.

Design principles for adopting the Core-Exception model

Organizations considering this pattern should treat it as a content architecture decision, not merely a publishing customization. The following principles are essential.

1. Establish the correct unit of variation

Do not begin by asking how small a reusable element can be. Begin by asking what unit reviewers, approvers, and auditors need to understand. In many high-variant environments, the right unit is not the word or phrase. It is the rule, requirement, procedure, condition, or statement.

2. Make the core explicit

The core is not “everything that remains after exceptions are removed.” It is an authored block that represents the majority case. It should be complete, reviewable, and valid on its own.

3. Treat exceptions as first-class content

An exception should not be a hidden fragment. It should be a managed content block with metadata, ownership, review status, version history, effective dates, and applicability conditions.

4. Use exception-first resolution

Publishing logic must look for applicable exceptions before falling back to the core. Without this rule, exceptions become unpredictable and authors lose confidence in the model.

5. Avoid overusing null exceptions

The NCCI model included triggers to indicate that a block was not applicable as a core or as a state exception. This is powerful, but it should be governed carefully. “Not applicable” is a business statement, not just a publishing trick.

6. Extend the rule to the map when needed

If excluding a parent topic leaves child topics without context, exclusion logic must cascade through the map. Otherwise, the publication may contain orphaned content.

7. Keep authors out of unnecessary logic

The authoring experience should make the model visible without making authors think like build engineers. Authors should understand core, exception, applicability, and effective date. They should not need to mentally simulate the publishing pipeline.

Conclusion: reuse needs architecture

Derivative reuse is not a mistake. It is an inevitable requirement in enterprise content. Real organizations serve multiple products, jurisdictions, channels, audiences, and timeframes. Content must vary.

The danger is unmanaged variation. When derivatives are handled through copy-and-paste, they drift. When they are handled through excessive inline conditioning, they become unreadable. When they are handled as governed, typed, block-level exceptions, they can remain reusable, reviewable, traceable, and publishable.

The Core-Exception model demonstrates that the solution to derivative reuse at scale is not simply “more reuse” or “more conditions.” The solution is better content architecture. By moving variation to the block level, preserving a clear core, treating exceptions as managed microcontent, and resolving applicability at publishing time, organizations can support complex derivative reuse without sacrificing authoring clarity or governance control.

The experience documented by NCCI and shared publicly in On the Road to Boca Raton reinforces a broader lesson for structured content practitioners: successful reuse is not merely a technical capability. It is an architectural discipline that requires thoughtful decisions about granularity, governance, workflow, and publishing behavior. [8]

For high-variant DITA publications, especially in regulated or policy-driven environments, the Core-Exception model offers a practical path forward: write once where content is truly common, vary only where variation is real, and make every exception visible enough to govern.

References

[1] Ann Rockley and The Rockley Group, “Fundamental Concepts of Reuse,” excerpt from Managing Enterprise Content: A Unified Content Strategy.

[2] IBM, “Content Reuse,” technical content practices documentation.

[3] OASIS, DITA 1.3 Architectural Specification, “Conditional processing (profiling).”

[4] OASIS, DITA 1.3 Architectural Specification, “Branch filtering.”

[5] DITA language specification, “Information typing.”

[6] Precision Content, “The Precision Content Method.”

[7] Internal NCCI source document, “Derivative Content Reuse: The Core-Exception Model.”

[8] NCCI, “On the Road to Boca Raton,” presentation delivered at ConVEx Conference, Tempe, Arizona, May 4, 2022.

For more information

Visit our site at www.precisioncontent.com to download our ConVEx presentation and white paper.

Yet beneath the excitement lies a hard truth: AI does not fix content problems—it exposes them.

AI systems can only be as effective as the content they rely on. When content is ambiguous, bloated, inconsistent, or poorly structured, AI simply delivers confusion faster. When content is precise, modular, and semantically clear, AI becomes transformative.

This is where the Five Moments of Need® and structured authoring intersect—not as complementary ideas, but as mutually dependent enablers of intelligent delivery.

The 5 Moments of Need®: Context is the missing piece

The most important contribution of the 5 Moments of Need® to adult learning is not simply instructional—it is contextual.[1]

By distinguishing between learning moments and doing moments, the framework made explicit something AI systems still struggle with today.

The same information is not equally useful in every situation.

• Someone learning for the first time needs explanation, structure, and progression

• Someone applying knowledge needs speed, clarity, and confirmation

• Someone troubleshooting needs diagnostic guidance, not background theory

• Someone adapting to change needs contrast—what’s different, and why

The 5 Moments of Need® provides a human performance map that defines why information is needed in a given moment—not just what information exists.

For AI systems, this is critical. Without contextual signals, AI can retrieve content—but it cannot reliably deliver the right content, in the right way, at the right time.

Structured authoring: Giving AI something it can actually work with

If the 5 Moments of Need® supplies context, structured authoring supplies precision.

AI systems do not reason well over:

• Long-form narrative documents

• Implicit meaning

• Mixed intent content

• Inconsistent terminology

They excel when content is:

• Modular and atomic

• Explicitly typed by purpose

• Predictably structured

• Semantically labeled

Structured authoring turns content into intelligent building blocks—microcontent that answers one question, supports one task, or explains one concept. This is the level at which both humans and machines perform best.

Instead of asking AI to “figure it out,” structured content removes ambiguity before delivery ever happens.

Precision + Context: How AI delivers the right answer, not just an answer

In an AI-enabled environment, structured content and the 5 Moments of Need® play distinct but inseparable roles:

• Structured content ensures accuracy, clarity, and reuse, while

• The 5 Moments of Need® informs delivery logic and situational relevance.

Together, they allow AI systems to

• distinguish between learning support and performance support

• adjust tone, depth, and format based on the moment

• assemble just-enough guidance dynamically, and

• deliver answers that feel intentional rather than generic.

This is the difference between an AI system that retrieves information and one that supports performance.

Without structured content, AI lacks reliable input.

Without the 5 Moments of Need®, AI lacks meaningful context.

From content assets to AI-ready knowledge infrastructure

When organizations adopt structured authoring aligned to the 5 Moments of Need®, content stops behaving like static artifacts and starts functioning as knowledge infrastructure.

The same content components can:

• Power formal learning experiences[2]

• Support in-workflow performance guidance

• Feed chatbots and conversational assistants

• Adapt to new tools, processes, and policies

• Scale across channels without rewriting

This is especially important in AI environments, where content must be:

• Continuously updated

• Instantly reusable

• Trustworthy in every context

Well-structured content also reduces risk. Changes propagate cleanly. Contradictions are minimized. Outdated guidance is easier to identify and remove—an essential capability when AI systems are amplifying delivery at scale.

An AI-ready strategy that pays off today

Perhaps the most compelling aspect of this approach is that it delivers value immediately.

Organizations that can align structured authoring with the 5 Moments of Need® will

• improve findability and usability for today’s workforce

• reduce dependence on formal training for performance issues

• increase confidence and speed at the point of work, and

• strengthen trust in content.

At the same time, they are quietly building an AI-ready foundation—content that is already modular, contextualized, and semantically clear.

When AI capabilities mature or expand, these organizations are not scrambling to retrofit content. They are ready.

Preparing for the future by designing for reality

AI will not replace learning content. It will reshape how learning and performance support are delivered. Organizations that succeed will be those that recognize a simple principle:

Intelligence in delivery depends on discipline in content.

The 5 Moments of Need® gives us a proven model for understanding human performance. Structured authoring gives us a proven method for creating content machines—and people—can trust.

Together, they form a strategy that improves work today and enables intelligent systems tomorrow.

That is not just content modernization.

It is intelligent, organizational AI-enablement.[3]

[1] For more information, visit https://www.5momentsofneed.com

[2] Microcontent and the 5 Moments of Need®, Rob Hanna, ConVex Virtual Conference, September 2020

[3] Precision Content® is a registered trademark of Precision Content Authoring Solutions, inc. The 5 Moments of Need® is a registered trademark of APPLY Synergies.

In practice, many RAG pilots succeed for the wrong reason. A small repository, one product version, and a narrow set of documents can make hallucination rates appear low. As the repository grows to include multiple product versions, regional variants, archived manuals, overlapping procedures, duplicate topics, and inconsistent terminology, the same system becomes harder to trust. The problem is not simply model quality. It is content supply-chain quality.

Lean RAG is the discipline of reducing unnecessary content volume, semantic duplication, and retrieval ambiguity before content reaches the vector store, search index, graph, or prompt. It combines structured source content, governed metadata, semantic chunking, reuse-aware publishing, provenance, and lifecycle control. For technical documentation teams, the most practical path to Lean RAG is often already present in their existing investments: Rich XML for structured source content, a component content management system for governance and lifecycle management, globally-standardized metadata for exchange and discovery, and a publishing pipeline that transforms authoritative source into AI-ready retrieval units.

The goal is not to put less useful information into the system. The goal is to put fewer ambiguous copies of consistently chunked information into the system, with enough context, metadata, and lineage for the retrieval layer to select the right evidence at the right time.

Why RAG became the default pattern

RAG emerged as a response to a basic limitation of large language models: even when models contain broad factual knowledge in their parameters, they do not reliably know the current, proprietary, or highly specific facts that enterprise applications require. The original RAG research framed retrieval as a way to combine parametric memory, the model’s learned knowledge, with non-parametric memory, an external index of source documents. This matters especially for knowledge-intensive tasks, where source access, specificity, and provenance are central to trust (Lewis et al., 2020).

For product and service organizations, the appeal is obvious. Product documentation changes constantly. Software features are released, procedures are corrected, warnings are updated, parts are replaced, supported configurations shift, and regulatory language evolves. Training or fine-tuning a model every time these facts change is not a viable operating model. A RAG architecture lets the enterprise keep the model stable while updating the knowledge source underneath it.

This is why RAG is surfacing as the default strategy for chatbots, voice assistants, support copilots, field-service tools, and intelligent delivery portals. It promises current answers without repeated model retraining. It also promises traceability: an answer can be grounded in retrieved source material rather than generated solely from model memory.

But RAG does not solve the content problem by itself. Retrieval only works as well as the evidence available to retrieve.

The hidden failure mode: RAG looks best before it scales

Many RAG projects begin with a small proof of concept. A team loads one manual, one policy set, or one product documentation collection into a vector database. The early results look impressive. The assistant answers recognizable questions. It quotes relevant passages. It appears to avoid hallucinations. Stakeholders conclude that the system is nearly production-ready.

This early success can be misleading.

With only one version of a documentation suite in the repository, the retrieval problem is artificially simple. There are fewer near-duplicates, fewer conflicting procedures, fewer obsolete variants, and fewer semantically similar passages competing for selection. As more versions and variations accumulate, the system must distinguish between the current procedure and an outdated one, the domestic version and the European version, the standard product and the premium model, the installation guide and the maintenance guide, the warning that applies to one configuration and the warning that applies to another.

Long context windows do not eliminate this problem. Research on long-context language models shows that models may perform worse when the relevant evidence is buried in the middle of a long input, even when the input technically fits within the context window (Liu et al., 2024). In other words, simply giving the model more content is not the same as giving it the right content.

This is the central argument for Lean RAG: a production RAG system must control ambiguity before retrieval and prompting, not merely compensate for ambiguity afterward.

Lean RAG defined

Lean RAG applies lean process principles to the content supply chain: it removes semantic waste, reduces retrieval variation, preserves context, and builds quality into AI-ready content before the model is asked to generate an answer.

A Lean RAG system is not thin. It is disciplined. It contains the right content at the right granularity, with the right metadata, linked to the right source, governed by the right lifecycle controls.

A lean repository has six characteristics:

1. Authoritative source: retrieved units originate from approved source content, not arbitrary file dumps.

2. Semantic boundaries: meaningfully labelled chunks defining expressions of intent such as tasks, warnings, error codes, prerequisites, and reference facts.

3. Context preservation: small units retain enough surrounding context to remain intelligible outside the page where they were authored.

4. Metadata control: retrieval can filter or rank by product, version, audience, lifecycle phase, component, information type, and safety relevance.

5. Lineage and provenance: every answerable unit can be traced back to its source, version, publication, and approval state.

6. Incremental update: changed content can be reprocessed without duplicating every unchanged unit across every publication or product release.

Lean RAG is therefore not only an AI architecture. It is a content operations model.

Start with the source content

The two most important factors in RAG quality are the source content and the system used to manage that content. Models matter. Embeddings matter. Vector databases, re-rankers, graph retrieval, and prompt patterns all matter. But none of them can reliably compensate for content that is duplicated, inconsistent, stale, context-poor, or unmanaged.

For best results, enterprise content needs to be lean and structured before it enters the RAG pipeline. It should be written in focused units, governed by standards, enriched with contextual metadata, and maintained as reusable components rather than uncontrolled files. Word documents, PDFs, and unmanaged Markdown pages can be useful delivery formats, but they are weak source formats for large-scale RAG governance. They often hide structure, flatten metadata, obscure reuse, and make it difficult to maintain referential integrity across publications and variants.

The CCMS as the control plane for RAG

A component content management system (CCMS) is not merely a repository for documentation files. In a Lean RAG architecture, the CCMS becomes the control plane for the content supply chain.

A CCMS can manage reusable topics and fragments, workflow states, approvals, version history, product applicability, publication maps, conditional processing, metadata inheritance, and relationships among content units. These are not peripheral features. They are the exact controls that a RAG system needs in order to avoid retrieving the wrong information.

Systems that treat documents as binary objects cannot easily see inside the content to manage metadata, reuse, lineage, or semantic relationships. A code repository or document management system may provide version control at the file level, but file-level versioning is not the same as content-unit governance. Code repositories lack the ability to preserve referential integrity of link and dependencies. A RAG system needs to know whether a procedure is approved, which product variant it applies to, whether it replaced an older procedure, which warnings must travel with it, and which publication contexts contain it.

The CCMS houses the single source of truth for product and service information. The RAG repository should be a governed publication target, not a shadow content store. RAG output becomes another delivery target along with any number of other channels.

Why DITA is a strong source model for Lean RAG

DITA was created for topic-based, information-typed, reusable technical content. Its core building blocks are topics and maps, and its architecture supports metadata, content reuse, conditional processing, and structured relationships. The OASIS specification describes topics and maps as the basic building blocks of DITA, with metadata available on maps, topics, and elements to support reuse and conditional publishing (OASIS DITA Technical Committee, 2010; OASIS DITA 1.3, 2015/2018).

This makes DITA unusually well aligned with the needs of RAG, even though DITA predates modern generative AI. DITA’s value comes from several properties that map directly to AI retrieval needs:

• Topics define meaningful boundaries. A topic is already a candidate retrieval unit because it is designed to be coherent and reusable.

• Information typing expresses intent. Task, concept, reference, troubleshooting, and other specialized structures help distinguish what a unit is meant to do.

• Maps preserve publication context. A topic can be reused in multiple outputs while the map supplies structure, sequence, and product scope.

• Metadata supports filtering and applicability. Audience, product, platform, lifecycle phase, and other attributes can guide retrieval.

• Reuse reduces duplication. A shared topic can remain one authoritative object even when it appears in many publications.

• Specialization supports enterprise semantics. Organizations can define structures and constraints that reflect their products and domains.

This is why content that was originally designed for human cognition can become valuable for machine cognition. DITA’s topic boundaries, information types, and metadata are not just publishing conveniences. They are retrieval signals.

XML ingestion challenges

DITA XML is an excellent source format, but it is not usually the best format to hand directly to a language model as prompt evidence. DITA was designed for structured authoring, reuse, specialization, and multichannel publishing. A RAG pipeline needs something different: readable, context-rich, deduplicated retrieval units that can be ranked, filtered, cited, and assembled into reliable answers.

Content reuse is one of DITA’s greatest strengths, but it creates ingestion challenges. Topics, fragments, and phrases may appear in many outputs through conrefs, keyrefs, maps, and conditional processing. If every rendered instance is indexed as independent evidence, the retrieval system may treat repeated text as more important than it really is. The problem is not reuse itself; the problem is unmanaged reuse in the retrieval layer.

Preparing DITA XML for Lean RAG

A Lean RAG pipeline should rationalize reuse before indexing. Topic-level and chunk-level reuse should be deduplicated in the repository, while metadata records each location, product, variant, or deliverable where the chunk is reused. This preserves the value of reuse without allowing repetition to distort retrieval ranking or answer generation.

Phrase-level reuse requires a different approach. Product names, feature names, variant labels, and other key-based terms may resolve differently across maps or versions. These resolved values should be preserved in the rendered Markdown and added to the search thesaurus or lexical index as aliases, deprecated terms, or variant-specific terms. This helps users retrieve the right chunk even when product terminology changes.

Much of DITA’s semantic markup should be stripped away. Relevant markup should be transformed into explicit retrieval metadata where it improves filtering, ranking, or answer assembly. Topic type, task structure, product applicability, lifecycle phase, audience, component, variant, and safety information can all become useful metadata fields. The readable content itself should be rendered as clean Markdown, preserving headings, lists, procedures, tables, and warnings where formatting carries meaning.

Transform source DITA into AI-ready retrieval units

The goal is to transform DITA into AI-ready evidence: deduplicated where reuse could mislead retrieval, enriched where semantic context improves ranking, rendered where structure helps the model read, and preserved through metadata where DITA’s original semantics matter.

A Lean RAG pipeline should treat RAG delivery as a publishing output. Just as the same DITA source can be transformed into HTML, PDF, help systems, or training materials, it can also be transformed into AI-ready retrieval packages. There is no need to connect to content repositories via APIs when it can be optimized by the publishing system.

The DITA Open Toolkit already provides an extensible transformation architecture for converting DITA maps and topics into deliverable formats. In a Lean RAG model, a DITA-OT plug-in can generate semantically chunked, metadata-enriched output for search indexes, vector stores, or graph databases.

The transformation should not simply strip tags and split text by token count. It should use the source structure to determine the retrieval unit:

• A task topic may become one topic-level retrieval unit, with steps preserved in sequence.

• A warning may become a required child fragment that must travel with the relevant task.

• An error-code table may become row-level reference units, each linked back to the parent table and product context.

• A prerequisite may become a reusable fragment attached to multiple procedures.

• A troubleshooting topic may become a set of symptom/cause/remedy units, while retaining the parent topic context.

The result should be a canonical chunk object with a stable identifier, source lineage, lifecycle state, applicability metadata, relationships, and a model-facing body. JSON is usually a strong internal representation for this object because it supports validation, schema control, stable IDs, and metadata fields. Markdown is usually better for the human-readable body that the model receives, because headings, lists, steps, and tables remain easy for both humans and models to interpret. Current prompt engineering guidance from OpenAI also supports Markdown for hierarchy and XML-style tags or attributes for clear boundaries and metadata in prompts (OpenAI, 2026).

The practical pattern is therefore hybrid:

• DITA XML as governed source

• JSON as the canonical chunk and metadata record

• Markdown as the model-facing content body

• RDF or JSON-LD when standards-based semantic exchange is required

Semantic chunking beats blind chunking

Chunking is one of the most consequential design decisions in any RAG system. Chunks that are too large may contain irrelevant information and increase cost. Chunks that are too small may lose context and become misleading. Microsoft’s RAG guidance emphasizes that effective chunking and search strategies are necessary to optimize relevance, reduce false positives and false negatives, and avoid poor outcomes from chunks that lack sufficient context (Microsoft Azure Architecture Center, 2025).

Blind chunking divides content by character count, token count, paragraph count, page breaks, or generic document layout. Semantic chunking divides content by meaning and function. For structured technical content, semantic chunking can use the authoring model itself: topic type, element role, heading hierarchy, step sequence, warning placement, table structure, relationship metadata, and map context.

Internal semantic chunking tests at Precision Content point in the same direction. In a comparison of four chunking approaches, semantically enriched subtopic chunks were directionally preferred over non-semantic markup-based chunks, topic-level chunks, and non-semantic PDF-based chunking. The study supports the practical hypothesis that semantic enrichment is a promising path for improving RAG for technical documentation.

The key lesson is not that the smallest chunk always wins. It is that chunks need semantic integrity. A procedure step without the procedure title, actor, prerequisites, result, or warning may be too thin. A whole chapter may be too broad. A Lean RAG pipeline should choose the smallest unit that remains safe, coherent, and answerable.

Content standards matter

How the content is written has a material impact on the quality of the response. To create content that can apply relevant chunking consistently at scale, standards need to be in place that govern how information is typed to align with intent or the function of information. Precision Content’s structured authoring methods are constructed to provide guidance on writing for intent to produce viable microcontent for better human cognition and machine processing.

Precision Content defines microcontent as concise blocks of information written, structured, and labeled based on the intended user response for the information. Microcontent must have

• Focus – based on a single idea or concept

• Function – based on user intent according to defined information types

• Structure – use repeatable patterns and language according to intent, and

• Context – metadata that can draw microcontent back together.

These principles tie into Lean RAG to eliminate content overlap, redundancy, and consistency for chunking.

Context is not optional

Context loss is one of the most common reasons RAG systems fail. A retrieved fragment may be textually relevant but semantically incomplete. For example, the sentence “Remove the cover” is not useful unless the system knows which product, which procedure, which cover, which conditions, and which safety precautions apply.

Anthropic’s Contextual Retrieval work addresses this issue by prepending a short chunk-specific context summary before embedding and indexing. Anthropic reports that contextual embeddings and contextual BM25 reduced failed retrievals substantially in their evaluation, and that additional reranking improved results further (Anthropic, 2024). The principle is directly applicable to structured technical content: a chunk should carry concise inherited context from the source topic, map, publication, and metadata.

In DITA terms, this means that a retrieval unit may need context from multiple layers including

• element or block

• topic metadata

• map metadata

• the publication or product context

• system-level metadata

• conditional processing values

• related warnings and prerequisites, and

• version and approval metadata.

This inherited context should not necessarily be dumped into the prompt as a large metadata blob. Too much metadata can become noise. Instead, the pipeline should distinguish between metadata used for filtering and metadata rendered for the model. Filterable metadata can stay in the search index. Prompt-visible metadata should be concise: product, variant, component, information type, audience or role, lifecycle phase, safety relevance, and source date or version when needed.

iiRDS and the role of standardized metadata

DITA provides structure at the source level. iiRDS (Intelligent Information Retrieval and Delivery Standard) provides a standardized metadata vocabulary and exchange model for intelligent information delivery. iiRDS specifies a package format and metadata vocabulary for technical documentation, with RDF as a core expression of metadata and, as of version 1.3, optional JSON-LD serialization (iiRDS Consortium, 2025).

For Lean RAG, iiRDS is valuable because it describes the kinds of contextual distinctions that retrieval systems need to make: document, topic, fragment, product, component, event, lifecycle phase, information type, role or qualification, and relationships among information units. These distinctions can reduce retrieval ambiguity when a repository contains many similar units.

However, metadata should be purposeful. The iiRDS guidance emphasizes that annotation requires significant concept knowledge and should be driven by use cases. A Lean RAG implementation should not “tag everything” simply because fields exist. It should prioritize metadata that changes retrieval behavior:

• product and product variant

• component or feature

• lifecycle phase

• information type

• audience, role, or qualification

• geography or regulatory scope

• source publication

• effective date and version

• safety or compliance relevance, and

• relationships such as prerequisite, replacement, version-of, and related procedure.

This metadata can support hybrid retrieval, where vector search handles semantic similarity, keyword search handles exact terms such as part numbers and error codes, and filters enforce applicability.

Provenance and trust

A generated answer is only as trustworthy as the organization’s ability to show where it came from. Provenance is therefore not an afterthought. It is a core requirement for production RAG.

The W3C PROV model defines provenance as information about entities, activities, and agents involved in producing a piece of data or thing, which can be used to assess quality, reliability, or trustworthiness (W3C, 2013). Applied to Lean RAG, provenance means every retrieval unit should be traceable to its source topic, map, publication, version, transformation event, approval state, and publication date.

A strong RAG chunk should answer questions such as:

• What source object produced this chunk?

• Which publication or product release included it?

• Which version replaced it or was replaced by it?

• Was it approved, draft, deprecated, or superseded?

• Which transformation generated it?

• Which metadata was inherited, and from where?

• Which warnings, prerequisites, or related fragments must accompany it?

This lineage is essential for audit, troubleshooting, and user trust. It also supports incremental updates: when a source topic changes, the pipeline can reprocess only affected retrieval units and update their metadata without duplicating all unchanged content.

Why Lean RAG reduces cost and saves energy

RAG cost is not only the cost of model tokens. It includes indexing, storage, embedding, reranking, prompt size, evaluation, governance, support escalation, and ongoing maintenance. A bloated repository increases these costs in several ways.

Duplicate and near-duplicate content increases embedding volume and retrieval competition. Overly large chunks increase prompt tokens and may include irrelevant evidence. Overly small chunks increase retrieval calls and require more context reconstruction. Poor metadata increases false positives, which then require reranking, longer prompts, or model reasoning to resolve. Stale content increases risk, which increases the need for human review and escalation.

Lean RAG reduces these costs by keeping one authoritative copy of reusable content, carrying applicability through metadata, and publishing only the retrieval units needed by the target system. The same chunk of content can be valid across multiple publications and product releases without being physically duplicated in the RAG repository. The repository can store one approved unit with metadata that links it to the applicable products, versions, publications, and contexts.

This is the same logic that made structured content and reuse valuable long before generative AI. RAG simply makes the cost of poor content operations more visible.

Governance: The missing layer in many RAG projects

AI governance is often discussed in terms of model behavior, privacy, bias, security, and human oversight. Those concerns are real, but content governance deserves equal attention in enterprise RAG. The NIST AI Risk Management Framework organizes AI risk work around governance, mapping, measurement, and management. Lean RAG applies those same principles to the knowledge supply chain that feeds the model (NIST, 2023).

A governed Lean RAG program should include:

• Content standards for topic types, microcontent blocks, terminology, warnings, titles, and metadata.

• Source governance through workflows, approvals, review states, and release controls.

• Retrieval-unit standards defining when to publish topics, fragments, table rows, examples, troubleshooting units, and safety notes.

• Metadata governance defining required fields, inherited fields, controlled vocabularies, and validation rules.

• Evaluation practices using test questions, reference answers, pairwise judgments, regression tests, and retrieval diagnostics.

• Lifecycle rules for replacing, deprecating, archiving, and removing retrieval units.

• Operational ownership across documentation, product, support, engineering, compliance, localization, and AI teams.

A RAG system is not production-ready just because it can answer a test question. It is production-ready when the organization can keep the source content accurate, the retrieval layer current, the metadata trustworthy, and the answer behavior measurable over time.

A practical Lean RAG pipeline

A practical Lean RAG pipeline for technical documentation can be implemented in seven stages.

1. Authors create and maintain structured source content in DITA or another semantically rich authoring model. They write using clear information types and focused blocks that correspond to user intent.

2. The CCMS manages reuse, lifecycle state, product applicability, variant conditions, relationships, and publication context.

3. The publishing pipeline resolves keys, conditions, reusable components, and map-level context so the output reflects the correct publication and product scope.

4. A semantic chunking Open Toolkit transformation generates retrieval units at the appropriate granularity: topics where the topic is the best answer unit, fragments where reusable microcontent is safe and useful, and row-level units where tables contain independent facts.

5. The pipeline enriches each unit with inherited metadata, relationship links, provenance, and a concise context summary.

6. The retrieval layer indexes the units for hybrid retrieval, using vector embeddings, keyword search, metadata filters, and, where useful, graph relationships.

7. The prompt renderer gives the model compact evidence in Markdown, with clear boundaries and only the metadata needed to answer accurately.

This pipeline allows DITA XML to remain the governed source while each downstream AI system receives the representation it needs.

Conclusion: The future of RAG is a content supply chain issue

RAG is often presented as a model architecture. For enterprise technical content, it is better understood as a content supply-chain problem.

A RAG system retrieves what the organization gives it. If the repository contains uncontrolled documents, duplicate versions, conflicting variants, stale procedures, weak metadata, and context-poor chunks, the model will inherit those weaknesses. If the repository contains structured, approved, semantically coherent, metadata-rich, provenance-aware units, the model has a far better chance of producing accurate, current, and trustworthy answers.

Lean RAG is the bridge between technical communication discipline and generative AI performance. It does not abandon structured authoring principles. It extends them into AI delivery. DITA supplies structure. The CCMS supplies lifecycle control. iiRDS-style metadata supplies machine-readable context. Semantic intent-based chunking supplies answerable units. Provenance supplies trust. Governance supplies durability.

The organizations that succeed with RAG will not be the ones that simply load the most documents into the largest vector database. They will be the ones that treat AI delivery as another output of a governed, lean, intelligent content supply chain.

References

Anthropic. (2024). Introducing Contextual Retrieval. Anthropic Engineering.

DITA Open Toolkit. (2026). DITA Open Toolkit architecture documentation.

iiRDS Consortium. (2025). iiRDS Version 1.3 materials and Guide for the Standardized Use of iiRDS.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems.

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics.

Microsoft Azure Architecture Center. (2025). Develop a RAG solution: Chunking phase. Microsoft Learn.

National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0).

OASIS DITA Technical Committee. (2010). Darwin Information Typing Architecture (DITA) Version 1.2: Architectural Specification. OASIS.

OASIS DITA Technical Committee. (2015/2018). Darwin Information Typing Architecture (DITA) Version 1.3. OASIS.

OpenAI. (2026). Prompt engineering, retrieval, and structured output guidance. OpenAI Developers.

World Wide Web Consortium. (2013). PROV-DM: The PROV Data Model. W3C Recommendation.