Home Recent Research People Papers Contact

DARMA

A research platform for measuring semantic change in versioned knowledge, developing and testing drift metrics, and tracing changed meaning into RAG behavior.

Retrieval-augmented generation solves one reliability problem by grounding a model in external documents. It leaves another largely unmeasured: those documents are revised. A passage can remain highly relevant to a query while no longer carrying the same meaning it carried in an earlier version.

DARMA is a research platform for studying that change. It provides a repeatable experimental path from versioned documents to aligned passages, semantic geometry, controlled drift measurement, and downstream RAG testing. At its center is a four-metric drift suite developed to characterize distinct forms of semantic movement rather than collapsing revision into a single similarity score.

Relevance tells you what a passage is about. DARMA asks whether its meaning moved.

Standard retrieval is optimized to find material that is similar to a query. It does not characterize how the semantic content of that material changed across versions. DARMA turns version-to-version meaning change into an observable research variable and provides an experimental platform for developing, comparing, and validating ways to measure it.

Measure versioned meaning

Compare semantically corresponding passages across revisions rather than relying on timestamps, lexical diffs, or document-level similarity alone.

Measure change on four semantic axes

Use DARMA's developed metric suite to distinguish translational movement, neighbourhood rewiring, distributional change, and changes in semantic concentration instead of treating drift as one undifferentiated score.

Build falsifiable controls

Use null, style-only, and substantive-scope conditions so a positive signal has to survive tests against both measurement noise and superficial rewording.

Trace consequences into RAG

Run matched questions against different document versions and quantify how revised knowledge changes the resulting answers.

Quadrant framework: endpoint translation (Cosine Drift) on the horizontal axis against neighbourhood rewiring (Jensen-Shannon Divergence) on the vertical axis, dividing the space into four labelled modes of semantic shift.
Semantic geometryTranslation and neighbourhood rewiring define distinct modes of change. DARMA preserves those modes instead of collapsing every revision into one generic drift score.

From document versions to aligned passages to semantic geometry

DARMA treats semantic drift as both a measurement problem and an experimental systems problem. The platform is built around a reproducible chain from versioned text to aligned semantic geometry, metric evaluation, controlled calibration, and downstream response analysis.

01

Version the source

Start with document revisions whose relationship is known well enough to construct controlled comparisons.

02

Chunk and align

Split each version into passages, then match corresponding chunks so the comparison is between semantic counterparts rather than arbitrary text windows.

03

Embed and measure

Represent each aligned passage in embedding space and compute complementary drift metrics describing displacement and local structural change.

04

Calibrate against controls

Use the null condition to estimate the measurement noise floor and the style-only condition as a negative control for superficial rewriting.

05

Probe the RAG consequence

Load different source versions into the retrieval pipeline, ask matched questions, and compare the resulting answers with lexical and semantic divergence measures.

Four developed drift metrics make semantic movement measurable from different directions

The metric suite is a central scientific contribution of DARMA. Rather than treating semantic drift as one distance value, the platform operationalizes four complementary measurements over aligned passage embeddings, each exposing a different geometric property of revision.

Cosine Drifttranslation

Measures how far an aligned passage moves from its earlier direction in embedding space.

kNN Changerewiring

Measures whether the passage's nearest semantic neighbours change across versions.

Jensen-Shannon Divergencedistribution

Compares the passage's neighbourhood-attraction profile between versions.

Entropy Differenceconcentration

Measures whether the local semantic relationship becomes more concentrated or more diffuse.

The platform is built to distinguish meaning change from the things that only look like it

The first DARMA validation study uses three conditions: a null control for the noise floor, a style-only revision that changes wording without intended scope change, and a substantive scope revision. The style condition is especially important because it gives the platform a falsification test: if a candidate drift measure fires on rewriting alone, it is not yet measuring what it claims to measure.

Scatter of scope-change and style-change passages in Cosine Drift versus Jensen-Shannon Divergence space, with dashed detection thresholds.
Scope versus styleSubstantively revised passages occupy regions of the joint metric space that style-only revisions do not, providing a controlled test that the instrument is responding to meaning rather than wording alone.
Histogram of Cosine Drift under the null control condition, heavily concentrated near zero, mean 0.0038, with a 95th-percentile noise floor marked at 0.0258.
Empirical noise floorAt the selected 720/28 chunking configuration, null-control Cosine Drift is concentrated near zero (mean 0.0038). Its 95th percentile, 0.0258, provides the study-specific 5% false-positive boundary used in the subsequent analysis.

The signal is measurable, structured, and sensitive to how the knowledge base is represented

0.758
Cosine Drift AUROC

The strongest of the four document-level detectors for substantive scope change at the selected 720/28 configuration.

81
Chunking configurations

The design was swept across nine chunk sizes and nine overlaps rather than assuming one segmentation scheme was neutral.

75 / 81
Scope signature

Across most configurations, substantively modified chunks enriched the high-translation, low-rewiring mode described in the discussion.

ROC curves for the four drift metrics: Cosine Drift AUROC 0.758, kNN Change 0.742, Jensen-Shannon Divergence 0.694, Entropy Difference 0.663, all above the random-classifier diagonal.
Detection performanceAll four drift metrics carry signal above chance at the selected configuration. Cosine Drift leads with AUROC 0.758, followed by kNN Change at 0.742.
Grid of heatmaps showing quadrant proportions across 81 chunking configurations for scope and style conditions.
Configuration sensitivityThe style-control signature remains stable across the 81 chunking configurations, while the strength of the substantive-change signal varies. Chunking is therefore part of the measurement design, not a neutral preprocessing choice.

Document-level drift matters because the generated answer can change with it

When questions target revised sections, paired answers generated from different document versions show lower similarity and greater divergence than answers for unchanged sections. In the x1-grounded analysis, all five answer-level measures were statistically significant; in the x6-grounded analysis, the directional pattern remained but only ROUGE-L and BERTScore F1 retained significance.

The answer metrics also agree with one another. Cosine Divergence is negatively correlated with BLEU, ROUGE-L, and BERTScore F1, with the strongest relationship against BERTScore F1 at r = −0.84. That correlation is among answer-level measurements; it is evidence of measurement coherence, not a direct source-drift-to-answer correlation.

Per-question answer-level scores across five metrics for unchanged versus changed document sections.
Downstream RAG effectQuestions targeting revised sections produce lower paired-answer similarity and higher divergence than questions targeting unchanged sections. The effect is strongest and most consistently significant in the x1-grounded analysis.
Correlation matrix of answer-level metrics; Cosine Divergence correlates negatively with all similarity measures, most strongly with BERTScore F1 at r = -0.84.
Answer-metric agreementAt the answer level, Cosine Divergence varies inversely with the similarity measures; its strongest relationship is with BERTScore F1 (r = −0.84). This is evidence that the answer-level measures are capturing a coherent divergence signal.

A reusable platform for semantic-drift experiments, metric development, and RAG impact testing

DARMA is more than the four scores produced in one study and more than a single operating threshold. It is reusable experimental infrastructure for turning document revision into aligned, testable semantic measurements, evaluating how those measurements behave, and tracing meaningful change into retrieval-grounded model behavior.

Metric development and benchmarking

Use a common controlled framework to compare DARMA's four drift metrics, examine their complementary behavior, and evaluate additional candidate measures against the same semantic-change conditions.

Version-to-version semantic experiments

Construct controlled comparisons across document revisions and measure change at passage level rather than relying on whole-document summaries.

Multi-axis drift characterization

Study semantic change as displacement, neighbourhood rewiring, distributional divergence, and concentration change, preserving structure that a single aggregate score can hide.

Chunking sensitivity studies

Treat chunk size and overlap as experimental variables, exposing when a drift result depends on segmentation architecture.

Noise-floor calibration

Estimate an empirical baseline under a null condition and derive configuration-specific operating points rather than importing an arbitrary global threshold.

Style-versus-scope discrimination

Use a style-only negative control to test whether a detector is responding to semantic change rather than surface reformulation.

Paired RAG impact testing

Ask the same questions against different source versions and measure how changed knowledge propagates into changed model responses.

Current scope. The study establishes the first validated DARMA configuration on a controlled versioned-document corpus and a specific RAG pipeline. Its thresholds are configuration-dependent, and the paper's broader threshold-calibration phase remains unfinished. DARMA is therefore best understood at this stage as a research platform for controlled semantic-drift measurement and experimentation, not as a universal production alarm with one fixed cutoff.

Knowledge bases are not static objects. They are moving semantic systems.

RAG reliability depends not only on retrieving the right source, but on knowing what happened to that source as it changed. DARMA provides a research platform for studying that problem experimentally: develop and test drift measurements, separate substantive change from rewriting, characterize the geometry of revision, test the sensitivity of the measurement system itself, and observe what reaches the generated answer.

DARMA turns changing knowledge into an experimental object: measure how meaning moved, test how well the metrics detect it, and observe what changes downstream.

DARMA is a reusable research platform within the Computational Meaning Dynamics Research Group's broader work on making semantic change observable, measurable, and experimentally tractable in AI systems.