> For the complete documentation index, see [llms.txt](https://docs.etiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.etiq.ai/use-cases.md).

# Use Cases

## **Testing**

Agent-generated code can pass a final-output check while still containing incorrect intermediate logic. A data or analytics agent may use the wrong input, apply an incorrect filter, introduce leakage, or perform the wrong join or aggregation - and still produce a plausible result.

Etiq creates testable boundaries throughout the execution by linking interim lineage objects to the functions that produced them. Tests and verification checks can therefore be applied to the relevant function-and-artifact pairs, rather than only to the final output or the complete generated script.

Use Etiq to:

* test intermediate datasets, models, agent states, and generated artifacts
* verify the inputs and outputs of individual pipeline stages
* target tests at the parts of the execution affected by an agent’s changes
* compare expected and observed behaviour across repeated runs
* add deterministic checks without relying on the agent to instrument or describe its own code

**Benefit**: More granular testing of agent-created code, with failures tied to the execution step and state that produced them rather than reported only at the end of the workflow. Etiq’s function-and-artifact lineage is specifically intended to support test harnesses and checks on interim outputs that conventional agent traces do not expose.

## **Debugging**

Logs and agent traces can show that a script ran, a tool was called, or a task failed. They do not always reveal which transformation first made the underlying result incorrect.

Etiq traces outputs backwards through the functions and interim lineage objects on which they depend. Developers and agents can begin with a suspect result, inspect its producer function, and follow the relevant dependency path upstream until they identify the earliest meaningful divergence.

Use Etiq to:

* identify the function that produced a wrong interim result
* distinguish a bad input from a function that transformed a valid input incorrectly
* inspect captured inputs and outputs at the point of failure
* expand into nested functions only where additional detail is required
* limit reruns and repairs to the affected branch of the workflow

**Benefit**: Faster and more targeted root-cause analysis. Instead of searching the entire repository, transcript, or run history, the reviewer can focus on the execution path that contributed to the failed output. The graph can point to the producer of an incorrect interim artifact, while nested-function drill-down allows inspection to start at a meaningful stage and deepen only along the suspect branch.

## **Observability and Governance**

Agent orchestration records what an agent planned and which tools it invoked. Runtime telemetry records events around the execution, such as model calls, subprocesses, timings, logs, and exit codes. These layers remain important, but they do not necessarily explain what happened inside the generated code.

Etiq adds a separate code-execution-lineage layer. It records the functions that ran, the lineage objects they consumed and produced, and the dependencies connecting those states to the final result.

Use Etiq to:

* trace a generated output back through its underlying code and data lineage
* inspect the interim data, model, or agent states associated with a run
* establish which function produced or changed a governed artifact
* preserve deterministic evidence independently of the agent’s own explanation
* connect review decisions, tests, and approvals to the execution evidence they used
* provide reviewers with a navigable record of how an output was produced

**Benefit**: Stronger auditability and governance for agent-created code. Teams can retain evidence of the transformations behind an output, rather than relying only on the agent’s plan, a final artifact, or platform-level telemetry. This complements orchestration and OpenTelemetry rather than replacing them: those layers describe the agent and surrounding runtime, while Etiq provides granular visibility into the code execution itself.

## **Long-Horizon Agents**

Long-horizon coding agents work across extended sequences of planning, code generation, execution, review, delegation, retries, and resumption. As the task grows, its state becomes distributed across functions, artifacts, agent calls, retries, and points in time. It can no longer be reconstructed reliably from the latest message or a summary of the conversation.

Etiq provides an execution-grounded state and memory layer for these workflows. It records what actually ran, which inputs and outputs were observed, what was produced, and which later results depend on earlier execution. The etiq graph can also support more reliable context selection. Instead of repeatedly passing the full conversation, complete execution history, or every available artifact to the agent, a workflow can retrieve the graph region relevant to the current decision.

While semantic state records the objective, plan, constraints, decisions, and rationale, the execution state supplied by the etiq graph records the functions that ran, the inputs they consumed, the outputs they produced, and their dependencies.

The etiq execution graph can help constitute a form of shared state and memory for these more complex agents. The complete history of a long-running workflow may contain more detail than the agent needs for any one decision. Etiq can help a context builder select a bounded graph region based on the current operation.

Below example agent operations on the etiq graph:

<table data-header-hidden data-search="false"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td><strong>Operation</strong></td><td><strong>Purpose</strong></td></tr><tr><td>Planning</td><td>Retrieve trusted upstream outputs, relevant prior evidence, and known gaps before deciding what to do next.</td></tr><tr><td>Review</td><td>Compare observed execution with the task objective and acceptance criteria.</td></tr><tr><td>Retrace</td><td>Follow a failed or suspect result backwards to the earliest meaningful divergence.</td></tr><tr><td>Repair</td><td>Change or rerun the affected branch from the last trusted upstream boundary.</td></tr><tr><td>Resume</td><td>Continue from the latest reviewed state without repeating trusted work.</td></tr><tr><td>Memory</td><td>Retrieve comparable failures and previously reviewed repairs with their supporting evidence.</td></tr></tbody></table>

Each operation can request a different bounded view of the graph. Review may need a stage and its artifacts, while retrace may need a deeper dependency path and resume may need only the latest trusted boundary and unresolved downstream work.

**Benefits:**

* More reliable context selection by retrieving only the graph region relevant to the current decision
* Targeted failure recovery without replaying the full conversation
* Smaller rerun scope by preserving trusted upstream and unrelated work
* Reduced memory drift by retaining executed state rather than repeatedly summarising it
* Inspectable hand-offs between agents or workers through graph-linked artifacts
* Checkpoint and resume based on reviewed execution rather than conversational position
* Independent verification of what the code actually did, rather than relying on the agent’s own description

The graph can therefore operate as more than an audit record. It can provide the shared evidence and bounded retrieval layer from which the agent decides what it can trust, which context it needs, and where it should continue.

We will publish our example long horizon harness using the etiq graph shortly.

<figure><img src="https://518012520-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEPbKmhwsGKY7PeVOfaDr%2Fuploads%2FeeRj5UEbJfQXZmWldLSi%2Fimage.png?alt=media&amp;token=2d0dc706-3cff-43a7-89ca-1b966dd78d02" alt=""><figcaption></figcaption></figure>

## IntentLineage

This is the experiment write-up for our latest approach. Public repo [here](https://github.com/ETIQ-AI/intentlineage_public).&#x20;

### Abstract

Agent workflows pass data between functions and tools, creating intermediate execution states where user requirements can be violated. Checking these states requires identifying which parts of the user's intent and which domain constraints apply, then assessing them against the recorded data. We investigate whether this intermediate-state verification reveals faults missed by reviewing only the initial and final states.

IntentLineage links user intent, ontology concepts and constraints to captured code-execution states. An LLM reviewer selects the relevant elements, checks recorded values against supplied rules and cites the observations supporting its assessment. These assessments remain associated with the states they describe, allowing violations to be tracked even when later steps change the data.

We evaluate the prototype retrospectively on six related variants of a synthetic payment workflow, each assessed once under five review conditions. The variants include two nonfault controls and four injected faults: amount and bank-account changes that either persisted or were repaired before completion. All five conditions detected both persistent faults. Reviews restricted to initial and final states missed both repaired faults, even when repeated. High-level intermediate review detected three of the four faults, with an invalid reviewer response in the remaining case. Adaptive intermediate review, which allowed requests for additional execution evidence, detected all four. No condition raised a scored alarm on either nonfault control.

All adaptive detections occurred during the initial function reviews, before additional evidence was supplied. In this pilot, linking requirements to intermediate execution states enabled the reviewer to detect violations that remained undetected through repeated endpoint review; deeper inspection provided no additional detections.

### 1. Introduction

Some requirements concern the final output; others govern what may happen during execution. If a user requires an amount to remain unchanged, restoring it before completion does not erase an earlier unauthorized mutation. To check whether the requirement was met, we need to examine the data and producer function recorded at each relevant workflow step against the user's requirements.

These checks become harder to follow when a request passes through functions, tools, transformations and different agents. Execution provenance records what happened during a computation: which functions ran, what inputs they used, and what outputs they produced. This helps explain how a result was obtained [\[1\]](https://braganholo.github.io/papers/murta2014-ipaw.pdf). Representing requirements at runtime means keeping them explicitly available within a running system so they can be inspected and reasoned about during execution [\[2\]](https://publications.aston.ac.uk/id/eprint/19494/1/Requirements_as_runtime_entities.pdf). Runtime verification checks observed execution against specified requirements [\[3\]](https://christian.schallhart.net/publications/2009--jlap--a-brief-account-of-runtime-verification.pdf).

IntentLineage combines these approaches, linking user intent, ontology concepts and constraints to intermediate execution states. IntentLineage turns the user's goals, domain concepts and requirements into structured records, then stores the LLM's assessment of how each record relates to the execution. Our pilot compares what LLM reviewers detect when they receive endpoints alone versus when they also receive intermediate execution evidence.

RQ1: Does checking intermediate execution states against user intent, ontology concepts and applicable constraints reveal violations missed by initial- and final-state checks? RQ2: What information must the reviewer LLM see to determine which parts of the user's intent, ontology and constraints apply at each execution step, and which should continue to apply downstream? We evaluate RQ1 through fault detection and explore RQ2 through the reviewer's use of the initially supplied evidence and its requests for further inspection.

### 2. Approach: from requirements to execution evidence

The system receives the user's request and a domain ontology. User intent expresses objectives and qualifications; ontology concepts describe domain types and relationships; constraints state obligations. The extractor identifies goals, parameters and constraints in the request. The compiler assigns stable identifiers and source references, then derives rule records using authored templates and a selected ontology axiom. Rules specify fields, reference values, applicable stages and comparison policies. These semantic elements form the reviewer's inventory; a requirement can appear in both an intent record and a rule record.

Etiq captures graphs linking functions and operations to input/output data states, recording how running code used and produced values. An authored domain adapter maps ordinary records to selected ontology relationships. Projection code prepares bounded summaries, decoded receipt fields and local graph views. Field-name and function-term matches supply conservative applicability hints; unmatched elements remain eligible for review.

The LLM receives the request, inventory and disclosed evidence, selects applicable elements and applies their supplied criteria. It returns supported, contradicted or insufficient\_evidence. The LLM explains why a requirement applies, notes any uncertainty and points to the recorded states, fields and values supporting its assessment. It is told how to compare values and when each rule applies, but must assess whether the execution meets the requirement.

Before review, programmed checks confirm that the test cases behaved as intended, including whether faults were introduced and later repaired (§3.2). After review, further checks confirm that the cited values match the recorded evidence. These checks verify the test setup and the accuracy of the citations; they do not establish whether the LLM’s interpretation is correct.

The system checks that the reviewer’s response has the required structure, accounts for the supplied semantic elements, cites recorded values accurately and requests only permitted additional evidence. These checks confirm that the response follows the review rules; they do not establish whether its conclusions are correct.

Each assessment is stored alongside the semantic element and the part of the execution it concerns. For example, the record can show that an amount was initially correct, changed in violation of a requirement, and was later restored. Keeping these assessments separately ensures that the later correction does not erase the recorded violation.

### 3. Controlled pilot

#### 3.1 Worked example: a repaired amount

Let’s take an example where the initial user request specifies GBP 100 and states “Keep the amount unchanged.” Compilation supplies an exact-decimal preservation criterion against initial request values, applicable from intake onward. The adapter maps payment records and receipts to the ontology's payment–amount relationship; preservation remains a separate user constraint.

In the repaired-amount case, approve\_payment receives GBP 100 and returns GBP 10,000. The reviewer identifies the change as a violation of amount preservation and cites the captured input and output values. Later, execute\_payment restores GBP 100, which remains in the final output and receipt (Figure 1).

<img src="https://518012520-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEPbKmhwsGKY7PeVOfaDr%2Fuploads%2FrWG1OySU7xncveSqMFoJ%2Funknown.png?alt=media&amp;token=7c78c997-5240-469b-89a8-d3e4f492cdf3" alt="" height="188" width="602">

&#x20;*Figure 1. Schematic excerpt of case 004 using actual captured values and identifiers. Solid arrows show captured function input/output bindings; red dashed arrows link the requirement and captured values to the assessment; blue dotted arrows abbreviate execution paths with intervening nodes omitted. Approval is native group 28 and execution group 1. The diagram is not the complete execution graph. Selected reviewer record. Shortened fields from the actual request and reply; artifact references are abbreviated to state identifiers.*

> ***Request:** “Keep the amount unchanged”; `operator=preserve`, `criterion=initial_value_preservation`, `comparison=exact_decimal`; applicable stages include `approved`. Observations: `state:000018` amount `"100"` and `state:000022` amount `"10000.0"`.*
>
> ***Reply:*** *`assessment="contradicted"`; relevance: “Initial amount preservation is due through approval.” Citations identify those two states at `/sample/0/amount`, with the exact observed values above.*

The preservation rule applies at approval, so the mutation violates it despite later repair. In a live workflow, a temporary wrong amount could affect approval or a payment instruction; a temporary wrong account could direct funds to another recipient. Restoring the final field would not itself reverse those effects. Correction before any consequential use could avoid harm. Here, mutation and repair are programmed, no real payment occurs, and the reviewer neither causes nor prevents either change.

#### 3.2 Workflow, evidence conditions and scoring

The synthetic workflow was developed with an LLM coding assistant and includes supplier validation, order matching, approval, execution and reconciliation. The reviewer LLM assesses its captured execution against the user’s intent, ontology concepts and constraints. Before the experiment, LangExtract used local Qwen2.5:1.5b to extract 12 records: one goal, six parameters, four constraints and one domain concept. Compilation added ontology mapping and rules, producing the shared inventory of 11 intent records, two ontology concepts and 11 rules. Supplement S3 explains the extraction settings and record validation.

Six dependent variants covered clean, decimal-equivalent benign, persistent/repaired amount mutations and persistent/repaired account substitutions. Deterministic checks of captured input, approval, execution and output values confirmed mutation activation, matched inputs and endpoint restoration. Their fault labels and comparison results were withheld from the reviewer.

Endpoint review (E) received initial/final job states. High-level intermediate review (H) additionally reviewed seven function scopes, one per call. Adaptive intermediate review (A) started with the same scopes and allowed up to two further disclosures and nine calls. Repeated endpoint controls (R-H and R-A) matched their paired treatment's actual call counts, including rejected replies. We scored their final endpoint assessment rather than an any-positive vote. These controls match calls, not tokens or aggregation procedures.

Every call contained the request, inventory and endpoints. Intermediate calls added function inputs/outputs, producer identities, local graph views and applicability hints. An opening supplied further evidence for a fresh review. All 172 experimental reviewer calls used client-effective GPT-5.5 (gpt-5.5, low reasoning effort) in isolated sessions, without source code or earlier/sibling answers. GPT-5.5 selected relevant elements, assessed execution evidence and, in A, requested further inspection; Qwen performed only the earlier extraction. One evaluation per case/condition produced 30 outcomes.

Detection required a valid contradiction of a requirement targeted by the injected fault in a retained scope assessment. For nonfault cases, any retained inventoried contradiction counted as an alarm. Invalid responses were separate outcomes. Intermediate assessments were retained by scope; repeated controls used their final reply. Fixture checks did not establish semantic ground truth for every assessment.

### 4. Results

All five conditions detected both persistent faults, whose changes remained visible at endpoints (Table 1). Endpoint review missed both repaired faults; high-level and adaptive review detected the repaired amount, and adaptive review also detected the repaired account.

Table 1. Descriptive case outcomes. There are two persistent faults, two repaired faults and two nonfault controls per condition.

| Review condition            | Persistent faults | Repaired amount | Repaired account | Nonfault alarms |
| --------------------------- | ----------------- | --------------- | ---------------- | --------------- |
| Endpoint (E)                | 2/2 detected      | Missed          | Missed           | 0/2             |
| High-level intermediate (H) | 2/2 detected      | Detected        | Invalid          | 0/2             |
| Adaptive intermediate (A)   | 2/2 detected      | Detected        | Detected         | 0/2             |
| Repeated endpoint (R-H)     | 2/2 detected      | Missed          | Missed           | 0/2             |
| Repeated endpoint (R-A)     | 2/2 detected      | Missed          | Missed           | 0/2             |

High-level review of the repaired-account case stopped when a genuine citation exceeded the response contract's length cap: three detections among four scheduled faults, with one invalid outcome. This format failure establishes neither a semantic miss nor adaptive superiority.

Both repeated endpoint controls missed both repaired faults; unchanged endpoint evidence did not expose the earlier mutations. All conditions had zero scored alarms on the two nonfault cases.

All four adaptive detections preceded additional disclosure, using initial function evidence plus the shared inventory and endpoints. Three subsequent disclosures included one nested-child review and added no first detection.

All six adaptive reviews retained incomplete software coverage, including gaps and cross-scope support/contradiction flags. Opposing labels can correctly describe the repaired amount's state change; these flags were not independently adjudicated as semantic errors.

### 5. Discussion and limitations

The repaired cases show why retaining assessments at intermediate states matters: earlier violations remain identifiable after values are restored. Endpoint reviewers lacked distinguishing evidence of those mutations, even when repeated. This demonstrates the prototype's requirement-to-state association for the inspected faults.

Holding the inventory fixed leaves the separate benefits of semantic representation, ontology, graph structure and context selection unmeasured. RQ2 has only indirect findings: initial function evidence plus shared context supported detections, but minimum context and state-only sufficiency were not tested. Applicability and propagation were not independently scored. Context/component ablations and graph-versus-flat comparisons with equivalent evidence are needed.

Six variants share one workflow, with one evaluation per case/condition. We report descriptive counts without inferential significance tests or independence-based confidence intervals: neither 172 calls nor overlapping elements are independent samples. Zero alarms on two nonfault cases cannot establish a general false-positive rate. Broader workflows, realistic mutations, benign transients and repeated evaluations are needed.

Target contradictions were inspected; other assessments lack exhaustive semantic annotation. Citation validation and coverage do not measure independent semantic accuracy, and first-detection call does not measure first-fault localization. Further inspection might help other faults or explanations despite adding no detections here.

Legitimate intent changes, learning from verified trajectories, pre-action Defenders and autonomous multi-agent workflows remain future evaluations.

### 6. Related work

Execution provenance supplies the history needed to relate an output to the computation that produced it. noWorkflow captures function activations, arguments, return values and dependencies, and supports graph visualization, comparison between executions and queries [\[1\]](https://braganholo.github.io/papers/murta2014-ipaw.pdf). Its discussion of capture granularity also identifies a practical tension: coarse records can omit relevant transformations, while fine records can overwhelm analysis. IntentLineage uses Etiq's function/data evidence and adds records linking user requirements to assessments at particular execution scopes. The present pilot does not isolate the contribution of graph structure from access to intermediate values.

Requirements research provides a complementary account of what an execution should achieve. Goal-oriented requirements engineering connects objectives, refinements, responsibility assignments and conditions on operations [\[4\]](https://webperso.info.ucl.ac.be/~avl/files/RE01.pdf). Requirements reflection proposes preserving requirements explicitly within running systems so they can be reconsidered as circumstances change [\[2\]](https://publications.aston.ac.uk/id/eprint/19494/1/Requirements_as_runtime_entities.pdf). Runtime verification checks observed executions against specified conditions, including conditions over sequences of states or actions [\[3\]](https://christian.schallhart.net/publications/2009--jlap--a-brief-account-of-runtime-verification.pdf). These foundations motivate retaining both a requirement and its execution evidence. Our prototype uses authored checking criteria and LLM assessments; it does not establish the soundness or completeness associated with a formally specified monitor.

Agent guardrails also connect decisions to requirements and supporting context. ProvenanceGuard checks whether a proposed tool, its parameters and the interpretation of a request are justified by the user query, tool documentation and interaction history before execution [\[5\]](https://arxiv.org/abs/2607.01236v2). ShieldAgent extracts policy rules, retrieves relevant rule circuits for an action trajectory and uses tools and executable code to check compliance [\[6\]](https://proceedings.mlr.press/v267/chen25ae.html). These approaches make structured justification central to agent checking. IntentLineage's evaluated setting is retrospective: it associates supplied semantic elements with captured program states and retains their assessments through later changes. This pilot does not evaluate an intervention that blocks unsafe actions or compare directly against either guardrail.

Execution feedback is also used to improve agent plans. Data Interpreter represents tasks and actions in hierarchical graphs and refines generated code using execution and verification feedback [\[7\]](https://arxiv.org/abs/2402.18679v4). DS-STAR's verifier considers the user query, cumulative plan, solution code and execution result when deciding whether further work is needed [\[8\]](https://arxiv.org/abs/2509.21825v1). MAST classifies failures involving system design, inter-agent alignment and task verification, using annotated execution traces [\[9\]](https://arxiv.org/abs/2503.13657v3). These studies motivate checking workflow progress and verification decisions. Our narrower unit of analysis is an individual requirement at a captured execution scope; the programmed workflow does not test autonomous multi-agent coordination.

Process-supervision research distinguishes the correctness of intermediate reasoning from final-answer correctness. Lightman et al. compare reward models trained with step-level and outcome feedback [\[10\]](https://arxiv.org/abs/2305.20050v1). Uesato et al. separately assess reasoning errors among solutions with correct final answers [\[11\]](https://arxiv.org/abs/2211.14275). This distinction motivates examining what an endpoint can conceal. IntentLineage assesses externally captured program states, rather than the model's reasoning steps, and the pilot performs no reward-model training or evaluation of learning gains.

Context studies inform RQ2. Lost in the Middle finds sensitivity to the position of relevant information [\[12\]](https://arxiv.org/abs/2307.03172v3), while RULER tests retrieval, multi-hop tracing and aggregation as context length and complexity vary [\[13\]](https://arxiv.org/abs/2404.06654v3). Together, they motivate testing which evidence reviewers receive and can use, rather than assuming that a larger context is sufficient. Neither establishes a benefit for our disclosure policy. Finally, Zheng et al. identify position, verbosity and self-enhancement biases in LLM judging [\[14\]](https://arxiv.org/abs/2306.05685). Their findings motivate independent semantic annotation: exact citations and valid response structure alone cannot establish that a reviewer's applicability or compliance assessment is correct.

### 7. Conclusion

IntentLineage links user intent, ontology concepts and constraints to execution states and retains assessments with exact citations. Intermediate review detected repaired violations missed by repeated endpoint review. Further work must identify which representations and context improve assessments under equivalent evidence.

### References

\[1] Leonardo Murta, Vanessa Braganholo, Fernando Chirigati, David Koop, Juliana Freire (2014). [noWorkflow: Capturing and Analyzing Provenance of Scripts](https://braganholo.github.io/papers/murta2014-ipaw.pdf). DOI: 10.1007/978-3-319-16462-5\_6

\[2] Nelly Bencomo, Jon Whittle, Pete Sawyer, Anthony Finkelstein, Emmanuel Letier (2010). [Requirements Reflection: Requirements as Runtime Entities](https://publications.aston.ac.uk/id/eprint/19494/1/Requirements_as_runtime_entities.pdf). DOI: 10.1145/1810295.1810329

\[3] Martin Leucker, Christian Schallhart (2009). [A Brief Account of Runtime Verification](https://christian.schallhart.net/publications/2009--jlap--a-brief-account-of-runtime-verification.pdf). DOI: 10.1016/j.jlap.2008.08.004

\[4] Axel van Lamsweerde (2001). [Goal-Oriented Requirements Engineering: A Guided Tour](https://webperso.info.ucl.ac.be/~avl/files/RE01.pdf). DOI: 10.1109/ISRE.2001.948567

\[5] Yining She, Yiliang Liang, Eunsuk Kang (2026). [Safeguarding LLM Agents from Misalignment through Provenance Analysis](https://arxiv.org/abs/2607.01236v2). arXiv:2607.01236v2

\[6] Zhaorun Chen, Mintong Kang, Bo Li (2025). [ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning](https://proceedings.mlr.press/v267/chen25ae.html).

\[7] Sirui Hong et al. (2024). [Data Interpreter: An LLM Agent For Data Science](https://arxiv.org/abs/2402.18679v4). arXiv:2402.18679v4

\[8] Jaehyun Nam, Jinsung Yoon, Jiefeng Chen, Jinwoo Shin, Tomas Pfister (2025). [DS-STAR: Data Science Agent via Iterative Planning and Verification](https://arxiv.org/abs/2509.21825v1). arXiv:2509.21825v1

\[9] Mert Cemri et al. (2025). [Why Do Multi-Agent LLM Systems Fail?](https://arxiv.org/abs/2503.13657v3). arXiv:2503.13657v3

\[10] Hunter Lightman et al. (2023). [Let's Verify Step by Step](https://arxiv.org/abs/2305.20050v1). arXiv:2305.20050v1

\[11] Jonathan Uesato et al. (2022). [Solving math word problems with process- and outcome-based feedback](https://arxiv.org/abs/2211.14275). arXiv:2211.14275

\[12] Nelson F. Liu et al. (2023). [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172v3). arXiv:2307.03172v3

\[13] Cheng-Ping Hsieh et al. (2024). [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://arxiv.org/abs/2404.06654v3). arXiv:2404.06654v3

\[14] Lianmin Zheng et al. (2023). [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/abs/2306.05685). arXiv:2306.05685

### Supplementary material

#### S1. Detailed outcomes and measurement rules

This supplement reports methods and accounting for the six-case, five-arm study. Exact definitions and evidence locations are recorded in measurement-definitions.md, ground-audit-rows.csv, ground-audit-20261014.json and claim-evidence-table.md.

Table S1. Every scheduled case/arm outcome; parentheses are actual reviewer-call counts.<br>

| Case | Variant                   | E            | H            | A            | R-H          | R-A          |
| ---- | ------------------------- | ------------ | ------------ | ------------ | ------------ | ------------ |
| 001  | Clean                     | No alarm (1) | No alarm (7) | No alarm (7) | No alarm (7) | No alarm (7) |
| 002  | Decimal-equivalent benign | No alarm (1) | No alarm (7) | No alarm (7) | No alarm (7) | No alarm (7) |
| 003  | Persistent amount         | Detected (1) | Detected (7) | Detected (8) | Detected (7) | Detected (8) |
| 004  | Repaired amount           | Missed (1)   | Detected (7) | Detected (7) | Missed (7)   | Missed (7)   |
| 005  | Persistent account        | Detected (1) | Detected (7) | Detected (7) | Detected (7) | Detected (7) |
| 006  | Repaired account          | Missed (1)   | Invalid (3)  | Detected (9) | Missed (3)   | Missed (9)   |

H has three detected fault cases and one invalid: scheduled detection yield 3/4, definitive detection rate 3/3 conditional on excluding the invalid result. A has four definitive fault pairs against E or R-A; H has three against E or R-H. The nonfault alarm predicate includes any effective inventoried contradiction; fault detection uses designated injection targets. Repeated controls score the final endpoint assessment, not an any-positive vote. Historical outcomes are unchanged.

#### S2. Calls, response validity and resource accounting

There are 30 scheduled outcomes, 172 completed reviewer calls and 171 structurally valid replies. The rejected H reply in repaired-account case 006 cited a genuine 145-byte receipt at /sample/0/execution\_receipt/decoded, exceeding the frozen 64-byte permitted-value cap. The arm stopped after two valid calls and that invalid call; four later initial scopes were unattempted. Including the rejected scope, five lacked validated review. R-H matched its three actual calls. No historical answer is repaired or retried.

Table S2. Reported usage, including the rejected call.

| Arm   | Charges | Input tokens | Output tokens | Cached input subset | Total tokens |
| ----- | ------- | ------------ | ------------- | ------------------- | ------------ |
| E     | 6       | 76,315       | 24,370        | 6,144               | 100,685      |
| H     | 38      | 645,269      | 167,453       | 52,224              | 812,722      |
| A     | 45      | 776,674      | 188,042       | 67,584              | 964,716      |
| R-H   | 38      | 483,350      | 155,892       | 53,760              | 639,242      |
| R-A   | 45      | 572,653      | 184,750       | 67,584              | 757,403      |
| Total | 172     | 2,554,261    | 720,507       | 247,296             | 3,274,768    |

<br>

Cached input is already included in input. The invalid call contributes 15,682 input plus 5,045 output tokens, 20,727 total. All usage fields are reported for all 172 calls. Runtime is 9,029.585516 seconds (approximately 2 h 30 min 30 s); summed per-call elapsed time is 8,715.68 seconds. Monetary billing and physical backend identity are unknown. All 12 case-level treatment/control charge matches hold. There were no uncharged transport failures or reasoning retries; 26 of the 198 allowed calls were unused. Startup E/H observations are counted once, with zero separate qualification calls. “Charge” is the harness budget category, not an observed monetary bill.

#### S3. Capture, extraction and disclosure details

Input preparation used LangExtract 1.7.0 with the local Ollama Qwen2.5:1.5b model, once before the experimental reviews. The archived extraction records one pass, no automatic retries, temperature 0 and seed 17. Its configured context window was 8,192 tokens and maximum output was 2,048 tokens; these are limits, not observed token counts. Twelve records passed the extraction pipeline's validation: one goal, six parameters, four constraints and one domain concept. “Accepted” does not establish independently measured extraction accuracy or completeness.

Compilation retained 11 intent records and the Invoice concept, added the authored Payment mapping, and produced 11 rule records, yielding 24 elements. Requirements overlap across representations; these are not 24 independent obligations. The frozen inventory was shared by all conditions. The ontology-derived existential uses the authored PaymentRevision/receipt adapter, not an OWL entailment proof; exact values, preservation and approval remain separate user constraints. Source and adapter bindings are in the claim ledger.

GPT-5.5, with low reasoning effort, performed all 172 experimental execution reviews, selecting relevant elements, assessing the disclosed evidence and optionally requesting further inspection in A. Qwen did not perform those reviews; its local preparation is separate from the reviewer-call and token totals in S2.

Table S3. Original arm-policy table.

| Arm | Evidence and policy                                                                                | Charged review calls per case                    |
| --- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| E   | Whole-job inputs/outputs                                                                           | One                                              |
| H   | Endpoints plus seven initial high-level function scopes, one function per call; opening prohibited | Seven unless invalid content terminates the arm  |
| A   | Same seven initial scopes as H, plus bounded selected native disclosure and follow-up              | At most nine                                     |
| R-H | Unchanged endpoint evidence, fresh neutral repetitions                                             | Actual paired H charges, including invalid calls |
| R-A | Unchanged endpoint evidence, fresh neutral repetitions                                             | Actual paired A charges, including invalid calls |

A permits two disclosures and at most 100,000 added evidence bytes, while still reviewing H’s seven initial scopes. First adaptive target contradictions occur in case 003/call 1, case 005/call 1, case 004/call 2 and case 006/call 4. Subsequent disclosures are one in case 003 and two in case 006, adding 14,677 bytes. One actual nested child was reviewed; a deeper request was call-budget denied. No live artifact inspection occurred. First detection is not an independently scored localization metric.

Coverage labels are A: six incomplete; H: three complete/three incomplete; E/R-H/R-A: six complete each under their endpoint policies. Repaired-amount H/A have four cross-scope conflict flags despite empty remaining-insufficiency lists. Those flags were not independently semantically adjudicated and can arise from correctly observing a change across states. No full semantic accuracy or equal-scope completeness comparison is inferred.

#### S4. Worked-example evidence and figure provenance

worked-example-evidence.json selects actual case-004 captured values; figures/repaired-amount.svg uses their state IDs and native groups. Rule rule:constraint:amount:1, element pe-54145b5fad2d10d3422f855a, comes from “Keep the amount unchanged,” characters 91–116 of source m3-ordinary-payment.

Adaptive call 2 cites state:000018/state:000022, pointer /sample/0/amount, values 100/10000.0, and labels the rule contradicted. Approval group 28 binds those states; execution group 1 binds state:000023/state:000027, amounts 10000.0/100. Final state:000031 and its decoded receipt have amount 100. Native handoffs are 45→117→116 and 22→123→122, the latter reaching reconciliation. Blue dotted paths omit intervening nodes; red dashed links represent semantic associations.

#### S5. Reviewer example and division of work

reviewer-example-excerpt.json selects the case-004 adaptive approval request/reply. The main example abbreviates artifact references. Supplied criteria include preservation, exact-decimal comparison, applicable stages and a field-match hint. The reply supplies the label, explanation and citations; the relevance sentence is verbatim.

Compilation and the adapter specify criteria and mappings. Projection prepares evidence and relevance hints without computing satisfaction verdicts. Separate deterministic checks qualify fixtures and validate cited values. implementation-clarification-20261018.md identifies the retained functions; this pilot does not compare against a deterministic intermediate-state checker.

\ <br>

<br>
