Methodology coverage¶
Temporary planning document; planning only. It answers one question: with SyRF's existing capabilities, does this plan add up to a very high quality systematic review facility for preclinical reviews, and what is added, proposed or deliberately left out. It resolves the round-2 methodology findings (review SR in full; V2-01..04 and V2-10..14; PH-10, PH-12, PH-23, PH-24; review AC-04, AC-20, AC-21; MS-11; DD-08; NS-06). The text it proposes for the package files has been merged into them: PRISMA amendments (preamble, summary, G, H, J, K, L and the new M, N and O), contracts C3, C12 and C14, the integrated plan (R3b, R5b and R5c scope; the P1, P2, O1 and X1 lane rows), acceptance criteria (the release criteria cited below and the §7 PRISMA fixtures) and open questions (E88–E93, A-39, A-40, Q-37); this page explains and justifies.
Labels follow the package: OWNER, RECOVERED, PROPOSAL, OPEN (Q-xx), ASSUMPTION
(A-xx), CODE-MAIN, CODE-PR, DOC-APPROVED, DOC-DRAFT. Anything that needs Chris cites a
Batch D question ID from the resolution brief; this page mints no question IDs. Methodology
sources are named inline: PRISMA 2020 (Page et al. 2021), PRISMA-S (Rethlefsen et al. 2021),
Cochrane Handbook v6 (chapter 4 selection, chapter 5 data collection, chapter 6 effect measures,
chapter 7 risk of bias), SYRCLE's RoB tool (Hooijmans et al. 2014), the CAMARADES quality
checklist (Macleod et al. 2004), ARRIVE 2.0 (Percie du Sert et al. 2020), ASySD (Hair et al.
2023), Vesterinen et al. 2014, and the inter-rater reliability literature (Cohen 1960, Fleiss
1971, Krippendorff 2004, Byrt et al. 1993 for PABAK, Gwet 2008 for AC1). Claims about main
were checked against /home/chris/workspace/syrf/main at de3e98c59 (3 October 2026); anything
not checked is marked UNVERIFIED.
1. Purpose and status¶
- Verdict. The package already makes the evidence core sound: immutable revisions, snapshot gold, no fabricated history, as-of reproduction, per-profile collective outcomes as the only PRISMA authority (PR1, amendments A and H). What it lacked, and what review SR showed, is the layer a journal or a funder sees: a defined inter-rater reliability basis, a human full-text retrieval workflow, protocol and search documentation, risk-of-bias templates, extraction provenance and validators, PRISMA arithmetic, report-to-study linkage, analysis-ready exports and calibration. With the additions below the plan does add up to a very high quality facility. Each addition is small relative to the engine work; most are data fields, rules and templates on aggregates the plan already creates.
- What this page decides. Nothing. It restates confirmed owner decisions, adopts review
improvements as
PROPOSALs, and routes product choices to Batch D (D4-01..D4-21, plus D3-11, D3-12, D3-25, D2-12 and D2-15 where they bear on methodology). - Status of the PRISMA amendments. A, C, D, G, H, I and J are approved (Q-06a); B, E and F are open (Q-06b); K and L were requested by Chris with rules pending (Q-37). This page adds M (full-text retrieval), N (Citation to Publication link) and O (report-to-study linkage, conditional on D4-08), and restates G, H, J, K and L. The amendment text is in prisma-amendments.md. Owner session, 4–5 October 2026: B, E and F are approved (Q-06b, package E1); K is approved (Q-37); L is approved with rule 4's alias merge replaced by the consolidated Study model (D2-12 replaced); M is approved (D4-07, E2); O is replaced by prepared multi-source links, with report grouping deferred (D4-08). §1.1 lists the rest.
- Relation to the other round-2 documents. The IRR markers it proposes live in C3 (contract
owner: the consistency drafter writes C18/C19; this page supplies the C3 rows). Version
compatibility guards for
autoUpdate(SR-16) are decided inversioning-model.md(§1.9 of the brief); this page adds only the export columns that make them visible. Deletion versus history is D3-12; this page records the methodological consequence only.
1.1 Owner-session integration (5 October 2026)¶
Chris recorded decisions on the methodology questions in the owner session of 4–5 October 2026
(consolidation §4, §5
and §7; session register;
condensed packages S1 to S5
and E1 to E4). The consolidation wins over older text on this page. Earlier recommendations are
kept below and marked where they are superseded. These are planning approvals only: brief
approval and implementation authorisation are separate (D1-04) and on hold since 5 October 2026,
no work has started and no gate has passed. The detailed rules are in the
reconciliation and screening (RS),
training and inference (TI) and
reporting, imports and AI screening
(RI) specifications. Amendment IDs (OS-A…) are those of the
owner-session integration.
Status of the decisions this page routes (vocabulary: Decided · Decided-amended · Replaced · Removed · Deferred (beyond MVP) · Brief item · Specialist input · Open owner decision):
| ID | Topic | Status | Where it is now specified |
|---|---|---|---|
| D4-01 | Unsure at title and abstract | Decided (S1), with the bounded-handling amendment (OS-A16) | §3.2; RS §5.10 |
| D4-02 | Discussion route | Decided (S2) | §3.3; RS §5.12 |
| D4-03 | Extract and verify | Decided-amended (R1 final, OS-A28): one candidate plus required human reconciliation; no Verified authority | §5.8; RS §5.1 |
| D4-04 | Calibration and training | Decided-amended (S4 expansion, OS-A18) | §3.4; TI spec |
| D4-05 | Protocol, registration and search documentation | Decided (E2) | §7; RI §3.6, §3.7 |
| D4-06 | Risk-of-bias and reporting-quality templates | Specialist input T-SI-03 (catalogue governance settled by D2-15) |
§6; RI-R58 |
| D4-07 | Full-text retrieval actions | Decided (E2) | §8; RI §3.8 |
| D4-08 | Report-to-study linkage | Brief item (carry-forward alignment): prepared multi-source links; grouping deferred | §10.1; RI §3.1 |
| D4-09 | Lane X1 analysis-ready exports | Decided (E3) | §12.1; RI §3.9 |
| D4-10 | Graph-estimated values | Decided (E3) | §5.7; RI-R28 |
| D4-11 | Box 1 from previous-review counts | Decided (E1) | §11.4; RI §3.4 |
| D4-12 | Agreement observation basis and methods | Specialist input T-SI-02 |
§4; RI §3.16 |
| D4-13 | Primary-reason hierarchy | Decided-amended (S3 clarification, OS-A17): template guidance, never a question-count limit | §3.5; RS §5.13 |
| D4-14 | FEAT-004 annotation import | Decided-amended (E3 with the AI expansion, OS-A20) | §13.1; RI §3.10 |
| D4-15 | Routing by answer values | Deferred (beyond MVP) | §13.2 |
| D4-16 | Early screening-profile adoption | Decided-amended (R4: universal baseline conversion, OS-A15) | §13.3; BC spec |
| D4-17 | Stale-answer acknowledgement | Decided-amended (O3): configurable warn (default) or block | §13.4; RS §5.8 |
| D4-18 | Funder mapping and WCAG audit | Answered on 3 October (outside the session register); its recorded reading awaits Chris's confirmation as G0-D1 in the G0 dossier | §15 |
| D4-19 | Blinding and random serving | Decided-amended (O4 with reviewer-pool browsing, OS-A27) | §3.8 |
| D4-20 | Disabled members' work | Decided-amended (O1, OS-A24) | §13.5; ACD spec §3.2 |
| D4-21 | Deduplication parity meaning | Specialist input T-SI-04; thresholds proposed, not approved |
§9.3; RI §3.15 |
| Q-06b | Amendments B, E, F | Decided (E1) | §1; RI §3.1, §3.2 |
| Q-16 | Agreement denominators and formulas | Specialist input T-SI-02 |
§4.3 |
| Q-17 | Event-count fields and "variation" | Specialist input T-SI-01 |
§5.3; RI-R57 |
| Q-22 | Reason coverage and distinct excluded Studies | Decided (S3) | §3.5; RS §5.13 |
| Q-23 | Reporting units | Brief item (carry-forward alignment) | §10; RI §3.1 |
| Q-33 | Withdrawn searches | Decided | §11.5; RI §3.5 |
| Q-37 | External steps and deduplication controls | Decided-amended (alias rule replaced) | §9, §11.2; RI §3.4, §3.15 |
| D2-12 | Duplicate merge | Replaced (consolidated Study, reversible unmerge, OS-A29) | §9.1; DM spec |
| D2-15 | Template ownership | Decided-amended (one CAMARADES catalogue) | §6; ACD spec §3.4 |
| D3-11 | Agreement store | Brief item | §4.7; RI §3.16 |
| D3-12 | Deletion versus history | Decided-amended (reversible project deletion; erasure unapproved) | §11.5; ACD spec §3.3 |
| D3-25 | Conversations as audit record | Decided (O2) | §3.3, §4.1; ACD spec §3.8 |
Amendments outside the register count that this page carries: tie adjudication with member or group assignment (OS-A13, §3.9); bounded Unsure (OS-A16, §3.2); exclusion-reason guidance (OS-A17, §3.5); training (OS-A18, §3.4); the inference beta (OS-A19, §5.9); external and AI-model-generated screening decisions (OS-A20, §3.10, §11.7); one candidate plus human reconciliation (OS-A28, §5.8); the consolidated merge (OS-A29, §9.1); historical pool coverage with actual review as the reporting priority (OS-A09, §11.6).
Specialist inputs (tracker rows; none is complete).
| Row | Input | Owner | Blocks |
|---|---|---|---|
T-SI-01 |
Field-level event-count schema and the meaning of "variation" (Q-17) | Chris or CAMARADES methodologists | The event-count outcome schema (RI-R57); catalogue infrastructure and the legacy-compatible schema may come first |
T-SI-02 |
Denominators, percent agreement, prevalence, initial independent observations, per-profile screening agreement, statistical review of multi-rater formulas, and how Unsure, adopted hints, discussion exposure and machine sources are treated (Q-16, D4-12) | A statistician with CAMARADES methodologists | κ, Fleiss, Krippendorff or other multi-rater figures in R5c; until then percent agreement with counts |
T-SI-03 |
Verification of the SYRCLE (with outcome-specific items), CAMARADES and ARRIVE Essential 10 templates and their applicability (D4-06) | CAMARADES methodologists | Publishing those templates to the catalogue (RI-R58) |
T-SI-04 |
ASySD parity and benchmark method (D4-21) | Specialist with the P2 owner | The P2 release decision; thresholds stay proposals |
T-SI-05 |
PRISMA box mapping for stage measures, pool history versus review through the stage, Unsure, pending adjudication and AI sole-screener exclusions | Methodologists with the RI owner | R5b's final box mapping (before F6b) |
Still not approved (no work may assume them): every proposed numeric threshold on this page
(Unsure bounds beyond the owner's example, the 0.5 percentage-point and F1 ≥ 0.99 parity figures,
80,000 citations in under an hour, the 5% QC share, the default 30-study training sample, discussion
time limits, assignment expiry durations); automatic acceptance of multiple agreeing candidates
(T-POL-03); permanent physical erasure (T-POL-01); an identity-erasure process (T-POL-02).
Exact PRISMA mapping, statistical methods and scientific event definitions need the specialist
inputs above.
2. Capability map¶
Columns: what main does today (verified unless marked), what the package already plans, what
this page proposes (with its Batch D ID where Chris decides), and what is deliberately out of
scope for this programme.
| Area | Today on main | This plan (already in the package) | Proposed here (Batch D) | Out of scope |
|---|---|---|---|---|
| Protocol and registration | A free-text Protocol Url on the project (user-guide/projects/settings.md:22; Project.cs:37,71 protocolUrl, AddProtocolLink) and free eligibility text. No registration record, no amendment log. CODE-MAIN |
Profile versions with publication impact (Q-26) are the de facto criteria history. Amendment F (open, Q-06b) says corrections and protocol amendments append. | A project "Protocol and registration" record: registry, ID, URL, date, protocol document link and version, append-only amendments log; publishing a profile version that changes eligibility requires an amendment entry (D4-05). Methods summary export (§14). Owner session: D4-05 decided (E2); a source-policy change is also proposed to need an amendment entry (RI-R22, PROPOSAL). |
Registry integrations (PROSPERO or OSF lookups); protocol authoring inside SyRF. |
| Search documentation and rounds | SystematicSearch holds name, description, file type, living-search link and a file-derived study count (SystematicSearch.cs:42-66). No source type, platform, date searched, strategy or limits (FEAT-011 gap G1/G2, prisma-flow-diagram-mapping.md:242-243). Living search exists behind livingSearchConfigurable (env-mapping.yaml:1165-1169); funder status "deferred" (docs/funding/index.md:66). CODE-MAIN |
P1 adds sourceType and sourceName (FEAT-011), amendment K external step records, amendment C earliest-source column, withdrawal (J). |
PRISMA items 6 and 7 fields on SystematicSearch (date searched, platform, strategy text or file, limits and filters, date range) and a minimal searchRound with updateOf (SR-04, SR-24; D4-05). Owner session: D4-05 decided; the documentation is versioned, each save creating an immutable version that reports pin (RI-R19). |
Automated search execution, multi-database retrieval, living-search automation (funder "future development"). |
| Deduplication | None in code: "No deduplication tracking at all" (prisma-flow-diagram-mapping.md:244); no pmPublication, no citations[], no lifecycleStatus (no hits in SyRF.ProjectManagement.Core/Model). FEAT-012 is an Approved specification only. DOC-APPROVED |
P2 implements FEAT-012 natively (amendment L): two-stage ASySD, tiers, review queue, merge wizard, audit log, retroactive dedup, parity suite. | T-SI-04 with thresholds proposed, not approved); Publication privacy and enrichment visibility (V2-03); extended pool exclusion list (V2-12). |
Cross-project review-data sharing; tuning the 25 classification rules per project (FEAT-012 §4.2 defers it). |
| Screening | Binary decisions (ScreeningDecision.cs:3-7: Included=1, Excluded=0); random serving (user-guide/stages/screening.md:28); two independent screeners plus a third by default, single screening configurable and "not recommended" (:73-75); skip via Next (:30); no structured reasons (gap G8, mapping :249); CSV import of decisions mapped to investigators and a stage (user-guide/studies/upload-search.md:38-54). CODE-MAIN |
R3a canonical decisions and steps; R3b profiles, derived decisions (DP3), reasons (DP5), own-Exclude correction (DP2), collective outcomes per profile; R4p adjudication; amendments A and H. | Template defaults for new projects (PROPOSAL, §3.1); Unsure at title/abstract (D4-01); discussion route (D4-02); calibration step kind (D4-04); primary-reason hierarchy (D4-13); imported decisions as authority = Imported (SR-03); bibliographic blinding option (SR-25); blinding and random serving as core behaviours (D4-19). Owner session: Unsure decided with bounded handling (OS-A16); discussion decided; explicit tie policy with a separate adjudication step and member or group assignment (OS-A13, §3.9); calibration becomes training (OS-A18); primary reason is template guidance (OS-A17); Imported stays a candidate provenance kind for mapped human imports (§3.6); external and AI-model-generated screening decisions count under the profile's ScreeningSourcePolicy (OS-A20, §3.10); blinding owned by form or profile (§3.8). |
excluded_automatic; no lane).T-SI-05. Machine-learning prioritisation inside SyRF stays out of scope. |
| Full-text retrieval | PDFs can be linked or bulk-uploaded; nothing records whether full text was sought or obtained (gap G7, mapping :248). Bulk PDF (FEAT-021) is a separate programme, flag off. CODE-MAIN |
P1 "retrieval status recorded with the PDF acquisition and processing programmes" (plan :689); AC-P1-03 events only. |
Human actions Sought, Retrieved (how), Not retrieved (reason, author-contact date); PDF attachment only suggests Retrieved; amendment M makes retrieval fullTextStatus only and supersedes FEAT-011's FullTextNotRetrieved lifecycle precedence (SR-02, SR-15; D4-07). Owner session: D4-07 decided (E2). |
Automated PDF retrieval (funder "future development"); open-access resolvers. |
| Data extraction | Unit hierarchy (experiment, cohort, disease model, treatment, outcome); average mean/median (export also mode); error SD/SEM/IQR; units as free text; GreaterIsWorse; cohort n (user-guide/data-extraction.md:72-92; data-dictionary/quantitative.md:158-180). Graph digitiser is dead: graph2data flag exists (env-mapping.yaml:1501-1505) but the library import is commented out (src/services/web/src/polyfills.ts:55). CODE-MAIN |
O1 schemas (legacy-compatible, event-count, custom) with roles, types, validators and cardinality (C14); reviewer-created measures with one direction (ODIR1); O2 migration; R4c outcome reconciliation. | Extraction-method provenance; unit vocabulary and same-measure unit validator; dispersion catalogue; domain validators; sample-size rule; graph-estimated default when a region is linked (SR-06; D4-10); T-SI-01; experimental-group inference is an opt-in beta (OS-A19, §5.9). |
Graph digitiser lane (decided after O1 pilots, D4-10); machine-assisted extraction; effect-size computation. |
| Risk of bias and reporting quality | Ad hoc annotation questions (seed category "Risk of Bias" with randomisation and blinding items, docs/architecture/seed-data-quality-analysis.md:311); the AI RoB tool is mothballed (docs/roadmap/product-features-roadmap.md:39; docs/features/calculate-rob-authorization.md:66-67); no user-guide page (grep of user-guide/ for SYRCLE or risk of bias: none). CODE-MAIN |
R1a question templates; TC1 feature-required entity types; R4a reconciliation gives two-assessor independence for any form. | SYRCLE RoB (per-outcome items bound to Outcome Assessment), CAMARADES checklist, ARRIVE Essential 10 as curated versioned templates with semantic roles, and a domain × study export matrix (SR-05; D4-06, D2-15). Owner session: catalogue governance decided (one CAMARADES catalogue, D2-15 decided-amended); template content is specialist input T-SI-03 (D4-06). |
Reviving the AI RoB tool; automated RoB judgements. |
| Reconciliation and agreement | No reconciler form on main (plan R4a); manual reconciliation via the help desk (screening.md:77); legacy AgreementMeasure getters only; agreement and kappa excluded from FEAT-024 (materialized-project-statistics/README.md:223). CODE-MAIN |
R4a form reconciliation and gold (GS1, RE2, VS2 exposure), R4p profile adjudication, R5c agreement view (AG1–AG3; Q-04, Q-16). | Observation basis in C3 (initial independent submission; collective exposure at correction; questioned in reconciliation, NS-06; imported authority); screening IRR per profile; methods for fixed and rotating raters; prevalence shown; drift over time; reconciler-override QC (SR-01, SR-18; D4-12, D3-11). Owner session: methods are specialist input T-SI-02; D3-11 is a brief item; hint adoption and discussion exposure are recorded and excluded from independent observations; external human and machine sources are separate classes (RI-R46). |
Automated adjudication; agreement as a FEAT-024 family (D3-11 keeps it in its own store). Owner session: a human adjudication step resolves AI sole-screener Unsure (RI-R39); SyRF adjudicates nothing automatically. |
| Export for synthesis | Long and wide CSV; one row per timepoint per cohort; blinding level on investigator columns; Reconciled flag (quantitative.md:14,30-38). No RIS export, no comparison export, no codebook. CODE-MAIN |
R5a current, previous and as-of exports with manifests (C11); export disclosure (C10). | Lane X1: comparison-level export, machine-readable codebook, RIS export (SR-08; D4-09); extraction exports default to collectively Included studies with surplus labelling (SR-17); metaAnalysisIncluded owner and capability (SR-20); goldDiffersFromAllCandidates column (SR-18); version columns (SR-16). Owner session: D4-09 decided (E3); machine-source columns and model configuration entries in the codebook (RI §3.9). |
Effect sizes (SMD, NMD), meta-analysis and plots inside SyRF (recipe in the user guide only). |
| PRISMA reporting | None (no generator; 11 gaps in mapping §5). DOC-APPROVED |
P1, P2, R3a/R3b write shapes, R4p, R5b frozen snapshots with manifests; amendments A–L (A–P after the owner session). | Published arithmetic identities per column with remainders (SR-09); entry-phase rule and per-box combination for external steps (V2-04); box 1 from K via D4-11; snapshots from authoritative records only (MS-11); amendments M, N, O; withdrawn searches per D3-12. Owner session: M approved, O replaced by prepared links (D4-08); withdrawn searches per Q-33 (decided); stage measures with actual review through the stage as the reporting priority (OS-A09); report identity coverage and the machine-assisted share in every snapshot; exact box mapping is T-SI-05. |
Full updated-review support beyond reported box 1 counts; PRISMA-S checklist authoring beyond the methods summary. |
| Living and updated reviews | Living search feature flagged and funder-deferred; new arrivals re-enter stages (R3c reopening). | R3c automatic reopening for new arrivals; "box 1 stays deferred". | Minimal search rounds (searchRound, updateOf) on searches and external records; per-round identification in R5b; box 1 as reported counts (D4-11). Owner session: D4-11 decided (E1). |
Automatic re-screening workflows for update searches; previous-review import with PreviouslyIncluded (reserved, FEAT-011 prisma-flow-diagram-mapping.md:264). |
| Audit and as-of reproducibility | Questions locked after first answer, no versioning (product-features-roadmap.md:37); timestamps settable (DateTimeCreated, C11). |
Immutable revisions, versions and snapshots; HLC ordering (§1.2); as-of exports with coverage labels (EX2); frozen PRISMA snapshots; append-only lifecycle, pool-entry and external records. | PRISMA snapshots computed from authoritative records at a watermark, never FEAT-024 rows (MS-11); conversations as audit record only (D3-25); dedup transparency in the manifest. Owner session: D3-25 decided (O2); structured, queryable history events (OS-A11, C20). | Selective per-project restore (D2-13 rules it out). Owner session: D2-13 is now a brief item (isolated point-in-time recovery with rehearsal; BC spec §8.4). |
| Blinding and independence | Reviewer identities blinded in exports by a level setting (quantitative.md:38); candidates never see each other's screening decisions; random serving. |
VS1 candidate isolation, |
PROPOSAL classification); random order with no chronology clues; random serving default and explicit assignment an audited exception (D4-19, decided-amended with optional reviewer-pool browsing, OS-A27); bibliographic blinding per profile (SR-25); discussion exposure recorded (D4-02). |
Unblinded "open" reconciliation modes. Owner session: a names-visible setting may exist on the form or profile (RS-R30, PROPOSAL); stages never set blinding. |
| Calibration and training | None. | None (QM v2 "training rounds" brief unused, PH-33). | Calibration step kind: a fixed sample offered to every reviewer, records with purpose = calibration that never vote, qualify, enter PRISMA or default IRR; live per-criterion agreement; |
|
| Quality control | None beyond reconciliation. | R4b queries; R3c readiness; FEAT-024 counts. | Dedup QC sample and reviewer duplicate flag (SR-19); reconciler-override count and column (SR-18); extraction QC view (graph-estimated share, unit mismatches, SD/SEM flips, missing n); methods caveat label for single screening or target-1 extraction (SR improvement 10); near-miss excluded list (PRISMA item 16b). Owner session: the caveat also covers Single annotator accepted results and AI sole-screener outcomes (§14.3). | Statistical outlier detection on extracted values. |
| External and AI-model-generated screening decisions (owner session) | CSV screening columns mapped to investigators and a stage (upload-search.md:38-54). |
None. | E3 AI expansion (OS-A20): versioned AIScreeningModelConfiguration; profile ScreeningSourcePolicy (ContributingVote or SoleScreener); ExternalScreeningRun and ExternalScreeningDecision provenance; reruns as versions; model Unsure to human adjudication under a sole-screener policy; training-set outputs never validation; machine-assisted outcomes distinguishable; source classes kept apart in agreement (RI §3.11 to §3.14; §3.10 here). |
Training or hosting AI models inside SyRF; prioritisation; any default score threshold (SyRF proposes none). |
| Experimental-group inference (owner session) | None. | Lanes C1 (explicit classification) and C2 (inferred cohorts). | Q-18 and Q-19 decided; beta, off by default, explicit project-designer opt-in (OS-A19); four operators only; inputs are the reviewer's own snapshot or a pinned accepted result; reported answers never changed and no animal counts invented (TI spec; §5.9 here). | Bounds reasoning, operators beyond the four, routing work by inference (D4-15 deferred). |
3. Screening methodology¶
3.1 Dual independent screening defaults for new templates¶
PROPOSAL (SR-21). R3a reproduces the legacy maths for the default profile (inclusion ratio
threshold, third vote decides) so adopted projects keep their meaning. New projects deserve an
explicit, defensible default, authored as template content by CAMARADES methodologists through
the R1a template mechanism (D2-15 for ownership) and recorded as PROPOSAL thresholds at F5:
| Template | Target | Decision rule | Conflict route | Reasons | Notes |
|---|---|---|---|---|---|
| Title and abstract | 2 independent screeners | Unanimity; Unsure allowed (D4-01); |
Blinded third screener (extra vote, D1/D2 Allow). Owner session: the tie policy is chosen explicitly in the template (ExtraReview with a bound of one, then adjudication; or Adjudication), never inferred (RS-R50) |
Optional | Cochrane Handbook ch. 4: "when in doubt, include" at this phase |
| Full text | 2 independent screeners | Unanimity | Adjudication (R4p); discussion off (D4-02). Owner session: explicit tie policy Adjudication, adjudicator a member or group (OS-A13) |
Required, DP5 On, hierarchy on (D4-13). Owner session: the primary reason is template guidance; more reason questions stay allowed (OS-A17) | PRISMA 2020 item 16b needs a reason per excluded report |
| Single-screener mode | 1 | Reviewer's decision stands | None. Owner session: with Unsure enabled, a lone Unsure follows the profile's tie policy (RS §5.10 item 4, recommendation) | As configured | Kept (student projects, screening.md:75); sets the methods caveat (§14.3) |
| Extraction | 2 plus reconciliation, or |
RE2 final submission or Verified goldHumanReconciled accepted result; no Verified authority |
R4a | n/a | AcceptedResultVersion.authority: "Single annotator" under AutoAccept, "Human reconciled" after one-candidate reconciliation |
| Risk of bias | 2 plus reconciliation | RE2 | R4a | n/a | PRISMA 2020 item 11 |
A project may change any of these; the methods summary (§14.1) reports what was actually used.
Owner session: template content (including these defaults and the exclusion-reason guidance) is
written by CAMARADES methodologists (RS §12; T-SI-03 for the risk-of-bias templates); the
thresholds above stay PROPOSALs until the F5 brief.
3.2 Unsure at title and abstract (D4-01)¶
Recommended per profile, default on in the title/abstract template and off in the full-text
template. Rules if approved: Unsure routes like Include for downstream availability; the
profile's collective rule says how Unsure combines (default: Unsure + Unsure and Include +
Unsure need a third vote; Exclude + Unsure is a conflict); a collective Unsure is never Excluded,
so PRISMA counts the study as not excluded at title/abstract and it proceeds to retrieval;
agreement statistics report Unsure as its own category (three-category κ) and collapsed into
Include. DP3 derived decisions may derive Unsure when a criterion answer is "unclear". Today a
reviewer who is unsure must Include (contaminating agreement) or skip indefinitely
(screening.md:30).
Owner session (D4-01 decided through S1; bounded handling OS-A16, 4 October 2026). Unsure is a
profile-version setting, on in the proposed title/abstract template. It counts as not excluded for
availability and never counts toward a definite collective Include (RS-R46 to RS-R49). The default
combination rule in the paragraph above is superseded. For the owner's example configuration
(two initial decisions, two agreeing definite decisions required, extra review bounded to one more
decision, then adjudication): Include + Include and Exclude + Exclude are sufficient without a
third; Include + Unsure, like any pair without a sufficient definite outcome, requests a third
decision; Include + Unsure + Include gives Include; Exclude + Unsure + Exclude gives Exclude;
Include + Unsure + Exclude and Include + Unsure + Unsure go to adjudication instead of more
reviewers. Exclude + Unsure also requests one more decision under extra review (a derived row
to confirm in the brief). The profile chooses extra review with its bound or earlier adjudication; SyRF imposes no common
threshold. The full order-independent transition table is RS §5.10. Still to specify and test in
the screening brief: all-Unsure outcomes, in-flight decisions, four or more decisions, other
thresholds and larger bounds (RS §5.10 items 1 to 6; brief item under T-RS-00). Whether a
reviewer's own Unsure lets them progress like an own Include is one owner-visible confirmation
(RS §12). The agreement treatment of Unsure (three-category or collapsed) is specialist input
T-SI-02, and its PRISMA treatment with pending adjudication is T-SI-05.
3.3 Discussion route (D4-02)¶
Recommended per profile, default off. After a conflict both candidates may see each other's
decision and reasons; the exposure is recorded in C3 as a collective-exposure event with kind
discussion; either may correct through DP2 (a new immutable submission); the profile rules
re-run; if the conflict persists the configured route (extra vote or adjudication) applies.
IRR uses initial independent observations (§4), so the route never inflates agreement. VS1 is
not broken: the exposure is explicit and labelled. Discussion text, where captured through
3944's conversations, is audit record only (D3-25).¶
Owner session (D4-02 decided through S2; D3-25 decided through O2). Discussion is off by default
per profile and starts only after submitted independent decisions reveal a conflict. Initial
observations and exposure are recorded; corrections create new decision versions. Independent
agreement uses the pre-exposure initial observations, and the current outcome uses the applicable
current decisions. A disabled or unresolved discussion falls back to the profile's configured
extra-review or adjudication route (RS-R60 to RS-R64). The end conditions (a participant declines
or marks "cannot agree", an admin closes it, an optional admin-set time limit with no default) are
PROPOSALs (RS-R63). Discussion text is a permissioned audit record outside candidate-answer
exports and agreement statistics.
3.4 Calibration rounds (D4-04)¶
Owner session (D4-04 decided-amended through the S4 expansion, OS-A18): "R3c (or after GA with the C6 admission hook reserved now)" below is superseded. Training is in delivery scope in the training lane TR1 of the rollout plan, after R3a steps, R2a forms and R3b profiles; reserving a hook while deferring the rest is not sufficient without a separate owner scope agreement. The lane adds versioned training reference answers, scoring rubrics and pass criteria, manual assessment that can pass or fail an attempt, feedback under the policy, retries under a versioned policy, and optional automatic admission to a project group whose existing permissions allow live review (TI spec §3 and §5). The "default 30 studies" sample below is proposed, not approved. Promotion stays an explicit admin action, as the paragraph says.
Recommended as a step kind, R3c (or after GA with the C6 admission hook reserved now). A fixed
sample (admin-chosen or random N; PROPOSAL default 30 studies) is offered to every reviewer
regardless of target. Records carry purpose = calibration: they never vote, never qualify a
contribution, never create pool-entry or screening events for PRISMA (C12 rule), and are
excluded from default IRR, with their own calibration agreement report per criterion. "Promote
calibration decisions to live" is an explicit admin action, off by default, that creates new
live submissions with provenance; it never relabels. The drift view (§4.6) is the feedback loop
Cochrane Handbook ch. 4 asks for when criteria are piloted.
3.5 Primary-reason hierarchy (D4-13)¶
Recommended: the profile's configured criteria order is the reason hierarchy; a profile rule
"primary reason = first failing criterion in configured order" is on by default and can be
turned off to let the reviewer choose. The derived decision (DP3) records the primary reason
automatically; DP5 reconciliation compares the full set of failing criteria (the R4p view shows
each candidate's failing-criteria vector, SR improvement 4); PRISMA reports the primary.
Reasons stay countable categories, never free text alone (FEAT-011
prisma-constraint-annotations.md:260-264); when the rule is off and no primary was chosen,
the outcome's reason coverage says so (amendment E). This makes box 9 reproducible.
Owner session (D4-13 and Q-22 through the S3 clarification, OS-A17). The primary exclusion reason is template and reporting guidance, never a platform limit. Profiles support several screening annotation questions and recorded reasons with explicit must-agree answers, and additional answers are never discarded (RS-R65). A template may nominate "primary reason = first failing criterion in configured order" for a simple mutually exclusive breakdown, with a per-profile option for reviewer choice (RS-R66). Reports count distinct excluded Studies separately from per-reason counts, label overlapping reason categories as such, never sum them into a Study total, and disclose reason coverage (RS-R67; RI owns the labels). Exact template text and diagram-compatible configuration advice come from CAMARADES methodologists in the screening brief.
3.6 Imported screening decisions¶
PROPOSAL (SR-03). Decisions imported from CSV columns (upload-search.md:38-54) or through
FEAT-004 (§13.1) are canonical ScreeningDecision records with authority = Imported and
provenance: source system, import job, mapped investigator, and independence (unknown, or
declared-independent with the declaring admin and time). The import wizard for canonical
projects requires the declaration. Declared-independent decisions count toward profile
sufficiency and appear in a separately labelled IRR view, never in the default view. Unknown or
not-independent decisions are recorded and shown but never count toward sufficiency or IRR.
For PRISMA every imported decision counts as "screened in SyRF (imported record)", so amendment
K's external counts for the same phase are refused for those records (no double counting).
Amendment H's authority list gains Imported. Imported decisions never become gold or
adjudicated outcomes automatically.
Owner session (E3 with the AI expansion, OS-A20; RI §3.13). Imported stays a candidate
provenance kind for mapped human imports, with the independence declaration above. It is not an
outcome value or an accepted-result authority. "Amendment H's authority list gains Imported" is
superseded: a screening outcome now records finalSource (ProfileRule, Adjudicated,
MergeResolved) plus composition fields (machineContribution, externalHumanContribution),
separate from accepted-result authority. Amendment K's "included elsewhere" stays. Configured
external sources (AI screening models, external human reviewers, non-AI tools) count by the
profile's ScreeningSourcePolicy (§3.10), which supersedes the requirement that every
target-counted contribution maps to a SyRF reviewer for those sources. Refusing amendment K counts
for records with accepted external decisions is proposed (RI-R15, PROPOSAL).
3.7 Bibliographic blinding¶
PROPOSAL (SR-25). A per-profile presentation option, default off, hides authors, journal and
year during screening; the exposure provenance records that metadata was hidden. Cheap, and
some protocols require it to reduce prestige bias.
3.8 Blinding and random serving as core behaviours (D4-19)¶
The SSI RSMF expression of interest states that blinding of reviewer identities and random study
serving are core platform behaviours, not optional settings (docs/funding/ssi-rsmf.md:100).
Recommended consequence: candidates are always blinded in reconciliation; BL1 becomes the
choice of alias scheme (stable per-project aliases or per-task aliases), never an "off" switch;
unmasking goes only through the audited export disclosure contract (C10, U26). Random serving
stays the default; explicit assignment (R4a assignment, RA5 requests, allocation plans) is an
audited exception. Presence disclosure follows D3-20.
Owner session (D4-19 decided-amended through O4; blinding and ordering amendment OS-A02, OS-A03).
The recommendation above is superseded in part. Reconciliation identity blinding is owned by
the annotation form (for its tasks) and by the screening profile (for adjudication and
screening-annotation reconciliation). It is blinded by default, and stages and steps have no
blinding setting. A names-visible choice on the form or profile is a PROPOSAL classification
(RS-R30). Aliases are drawn at random per reconciliation context and independently for every
Study, so no alias carries across Studies. Candidate order is a random permutation, and blinded
screens carry no submission times or other chronology clues (RS-R24 to RS-R29). Random serving
stays the default. Optional, dynamic reviewer-pool browsing within a stage may be enabled; it shows
only the reviewer's own pool and never other reviewers' decisions, answers or identities (OS-A27;
SP spec §3.6). Presence disclosure (D3-20) is a brief item: counts for everyone, names only with
the Monitor capability, never across reconciliation blinding.
3.9 Tie adjudication and adjudicator assignment (owner session, OS-A13)¶
Every screening profile version records its tie policy explicitly: ExtraReview with a bound, or
Adjudication. SyRF infers no default; templates preselect one visibly, and baseline conversion maps
legacy behaviour (RS-R50). Extra review requests one more decision at a time up to the bound and then
goes to adjudication (RS-R51). Decision ties, unresolved Unsure, reason disagreement and AI
sole-screener Unsure are separate triggers with separately configured handling; extra review never
resolves a reason disagreement (RS-R52). Adjudication runs in a system adjudication step (RS-R59;
SP §3.4). It may be assigned to an authorised member or an eligible project group; any eligible group
member may claim, and the individual resolver is recorded (RS-R53, RS-R54). The adjudicator sees the
exact tied decision versions under the profile's blinding and writes a new immutable adjudication
version; a rationale is required only where the profile asks for it (Q-32, RS-R56). Pending
adjudication is not a definite outcome (RS-R57). Methodologically this matches Cochrane Handbook
ch. 4's third-reviewer or consensus resolution while keeping each original decision and its exposure.
The agreement and PRISMA treatment of adjudicated outcomes needs T-SI-02 and T-SI-05.
3.10 External and AI-model-generated screening decisions (owner session, OS-A20)¶
Chris explicitly approved importing externally produced screening decisions, including AI-model-generated screening decisions from an AI screening model (the required terminology; "model decision" is superseded wording). Methodological rules (RI §3.11 to §3.14, RI-R32 to RI-R47):
- The machine source has its own identity, separate from the importing user; it has no login and no permissions.
- The project keeps versioned
AIScreeningModelConfigurationrecords (model, provider, version or artifact, intended use, label mapping, thresholds the project supplies, training-set context, documentation, and explicit "not supplied" markers). A profile version pins the exact configuration version in itsScreeningSourcePolicy. - A source counts only under its declared role,
ContributingVoteorSoleScreener, within its declared scope. Reruns and corrections are new versions of the same source's decision, never extra independent screeners. - Under a sole-screener policy a model Unsure goes to human adjudication; not-excluded availability never makes it a collective Include. As a contributing vote it follows the profile's sufficiency, conflict and Unsure rules (§3.2).
- Outputs on the model's training or evaluation inputs are flagged; they are never independent
validation evidence and never add the underlying human decisions again. By default they do not
count toward sufficiency or agreement (RI-R44,
PROPOSAL). - Imported decisions change outcomes and pools only after validated acceptance under the declared policy.
- Reports, exports and the methods summary keep machine-assisted outcomes distinguishable from
exclusively human ones; agreement keeps human independent, human informed, external human and
machine classes apart until
T-SI-02defines any combined statistic. - SyRF proposes no default score threshold. Where a sole-screener AI Exclude belongs in the PRISMA
diagram (box 3 automation or box 5 screening) is RI ambiguity A1, decided under
T-SI-05with both computations kept in the manifest (§11.7).
4. Agreement and reliability¶
4.1 Observation basis¶
R5c computes agreement "from canonical revisions" but never said which revision per reviewer is
the observation (SR-01). Under DP2 a reviewer can correct an Exclude while review is possible,
extra votes resolve conflicts, a reviewer can infer the collective state from availability
messages, and a reconciler's question may paraphrase other candidates' answers (NS-06). A
correction made after any of that is not an independent observation. C3 today records exposure
only for accepted gold (contracts.md:163).
PROPOSAL (adopted per brief §1.17 and NS-06): C3 gains four markers, frozen at F1a once D4-12's F1a part is answered:
| Marker | Meaning | Written when |
|---|---|---|
| Initial independent submission | The first effective Complete (form) or decision (screening) by a reviewer for a (study, form) or (study, profile) context, made before any collective outcome, accepted answer or adjudicator output for that study was visible to that reviewer | Derived at commit from the commit order and the visibility events; stored on the session version |
| Collective exposure at correction | For DP2 corrections, extra votes and the discussion route: whether the collective outcome (Pending, Conflict, Included, Excluded, Unsure) or any reconciler or adjudicator output was visible to the actor, and through which route (availability, discussion, monitor) |
On the correcting submission |
| Questioned in reconciliation (NS-06) | A reconciler questioned this reviewer's session on this study × form (thread, time); every later version of that reviewer's session on that study × form is informed | By #3965 when a conversation is created, looked up by session; consumed by R5c, C11 manifests and R6 adoption mapping |
| Imported authority | authority = Imported with independence (§3.6) |
At import |
Lost or missing markers fail safe: "available but unrecorded" is treated as informed or unknown,
never as independent (C3's existing rule). Calibration records (purpose = calibration) are
excluded by construction. Conversations themselves are never inputs to agreement; they are
audit record, exportable only behind an audit capability with aliases (D3-25).
Owner session additions. The "imported authority" marker becomes imported provenance with an
independence declaration, restricted to mapped human imports (§3.6). Two exposure kinds join C3:
reconciledHintShown and reconciledHintAdopted (OS-A04; RS §3.6). An adopted hint still counts
toward qualification and progress but is never an independent observation (RS-R35). A live session
on a Study whose training reference the reviewer saw is marked informed (TI-AE12); training
attempts replace calibration records and are excluded by construction. External screening sources
form their own observation classes (human external, machine; RI-R46). The formulas that consume
these markers wait for T-SI-02.
4.2 Views¶
Default IRR view = initial independent observations. A current-decision view is available and labelled "current decisions (includes corrections)". Corrections after collective visibility are labelled "informed (collective)"; sessions after a reconciler's question are "informed (questioned)"; declared-independent imported decisions appear in their own view (§3.6). Fixture: a DP2 correction after a visible conflict never changes the initial-observation κ; a questioned reviewer's later version is classified as informed (NS-06). Owner session: external human and machine-source decisions appear in their own labelled views, never mixed into the human independent view without an approved statistical definition (RI-R46); answers adopted from hints are labelled "informed (hint adopted)" (RS-R35).
4.3 Screening IRR per profile¶
Screening-level agreement is an explicit R5c deliverable (AG1–AG3 and AC-R5c-01 are annotation-answer rules today). Per profile and per phase: percent agreement with explicit denominators (always shown), the prevalence of Include (always shown, because κ is depressed at low inclusion rates typical of title/abstract screening), and:
| Rater design | Statistic | Source |
|---|---|---|
| Two fixed raters | Cohen's κ (with 95% CI) | Cohen 1960 |
| Three or more fixed raters | Fleiss' κ | Fleiss 1971 |
| Rotating pairs (the SyRF norm: random serving) | Pooled pairwise κ over all rater pairs with ≥ n shared studies, and Krippendorff's α over the incomplete rater × study matrix | Krippendorff 2004 |
| Low prevalence | PABAK and Gwet's AC1 as supplementary, labelled | Byrt et al. 1993; Gwet 2008 |
| Three categories (Unsure) | Three-category κ plus the collapsed binary | §3.2 |
Per-criterion agreement (which eligibility criterion the raters disagreed on) uses the DP3
failing-criteria vectors. All of this is PROPOSAL pending a statistician's review of
denominators (D4-12; Q-16). Until then R5c ships percent agreement with counts, as Q-16 already
says. Owner session: Q-16 and D4-12 are one specialist input, T-SI-02; the statistics table
above is the input to that review, not an approved method. The register's one fixed point stands:
a reconciled or accepted result is never an extra independent observation (RI-R50).
4.4 Form and entity agreement¶
Unchanged: AG2 (identical multi-select sets), AG3 (N/A rules, compatible-version flags), Q-04
missing-state contract. Verified gold (D4-03) has no IRR; exports label it "single extraction,
verified". Superseded (D4-03 decided-amended, OS-A28): there is no Verified authority. A
target-one form either accepts automatically as "Single annotator" or requires one-candidate human
reconciliation ("Human reconciled"); neither has IRR, and exports label results by
AcceptedResultVersion.authority. Q-04 is decided: a blank candidate comparison is "not assessed",
never disagreement, and Unknown and Not reported are answers (RS-R13 to RS-R16).
4.5 Reconciler-override QC¶
PROPOSAL (SR-18). RE1 keeps explanations optional. The R5c view (or R4a's pool page) shows a
"reconciler overrides" count: gold differs from every candidate with no explanation, per form
and question, exportable; the non-blocking reminder is on by default; exports gain
goldDiffersFromAllCandidates.
4.6 Drift over time¶
PROPOSAL (SR improvement 3). Agreement per reviewer pair and per criterion by screening order
(first 100, next 100, …) so drift is visible early; it is the feedback loop for calibration.
4.7 Store¶
Agreement statistics live in their own rebuildable store, not FEAT-024 (D3-11; MS-20); FEAT-024
excludes kappa (materialized-project-statistics/README.md:223). Owner session: D3-11 is a
brief item. The store (AgreementResult, RI §3.16) is keyed by project, form or profile version,
method version and study set, with a source watermark and a measured budget (proposed, not
approved: full recompute for RV-DS-03 within 10 minutes; view p95 within 2 seconds).
5. Data extraction quality¶
5.1 Extraction-method provenance¶
PROPOSAL (SR-06a; C14, E12). Every observation carries an extractionMethod role: reported
(text or table), graph-estimated, calculated (by the reviewer; the formula noted), author-
supplied, unknown. A series-level dataSource note holds the location (table, figure, page).
When a PDF graph region is linked, graph-estimated is the default. Cochrane Handbook ch. 5 and
Vesterinen et al. 2014 require graph-derived data to be flagged; today the graph link is only a
region assignment and the digitiser is dead (polyfills.ts:55).
5.2 Unit vocabulary and the same-measure validator¶
PROPOSAL (SR-06b). A project unit vocabulary: a controlled list with SI-aware labels and
free-text fallback, seeded from a CAMARADES list. A validator "same measure, different unit"
warns at Save and blocks binding at reconciliation (R4c) until the reconciler maps or confirms.
Today "mm3" and "mm³" are two strings (data-extraction.md:78).
5.3 Dispersion catalogue and the legacy-compatible fields¶
PROPOSAL (SR-06c), to be fixed before Q-17 closes (E12): average {mean, median, other};
dispersion {SD, SEM, 95% CI lower and upper, IQR Q1 and Q3, range min and max, none reported};
n at observation (default "same as cohort n", with provenance); events and total for
dichotomous outcomes (event-count schema); time with unit. Dispersion is never converted on
export (the analyst converts; the recipe is in the user guide). This gives "variation" (OC1,
Q-17) a definition: the dispersion role and its catalogue value. Legacy IQR and export
mode map to catalogue values with a recorded alias (O2 mapping contract, outcome-data
migration proposal :95). Owner session: Q-17 is specialist input T-SI-01; the event-count
schema is built only after a field-level scientific specification is recorded, while catalogue
infrastructure and the legacy-compatible schema may come first (RI-R57).
5.4 Domain validators¶
PROPOSAL (SR-06d), enforced on Save (AC-O1-02, AC-O1-11): SD ≥ 0 and SEM ≥ 0; n an integer > 0; events
≤ total; time monotone within a series; CI lower ≤ average ≤ CI upper; Q1 ≤ median ≤ Q3; an
SEM/SD plausibility warning when a series mixes types (SEM × √n ≈ SD); a direction never
derived from values (OC2). Warnings never block Save; blocking rules block Complete.
5.5 Sample-size rule¶
PROPOSAL (SR-06e). The analysis n is the observation-level n when recorded, else the cohort n;
exports carry both and the rule applied (nSource). O2 never overwrites a cohort count with a
series count (migration proposal :96).
5.6 Extraction QC view¶
PROPOSAL (SR improvement 5), O1 or R4c: per form, the count of graph-estimated observations,
unit-mismatch warnings, SD/SEM corrections made at reconciliation and observations with missing
n. These are the questions referees ask.
5.7 Graph digitisation (D4-10)¶
Recommended: ship the provenance flag in O1 now; decide a digitiser lane after O1 pilots report how often a graph is the only source. Until then "graph-estimated" values are entered by hand from a linked region. Owner session: D4-10 decided as recommended (E3, 4 October 2026); RI-R28.
5.8 Extract and verify (D4-03)¶
Superseded (R1 final owner clarification, 4 October 2026; OS-A28). Second-person checking uses
the same reconciliation mechanism: a target of one qualifying candidate session and required
human reconciliation. The form version's ReconciliationPolicy.targetOneHandling chooses
AutoAccept (an immutable "Single annotator" accepted result with system-rule provenance, which is
not human reconciliation or independent verification) or RequireHumanReconciliation (a
one-candidate reconciliation task; the reconciler confirms or corrects, with resolver attribution).
No separate Verification step kind, session type or Verified authority exists. Changing the
handling publishes a new form version with impact. Automatic acceptance of several agreeing
candidates is not approved (T-POL-03). The default value of targetOneHandling is an
owner-visible brief item (RS §12; recommendation AutoAccept, visibly chosen in templates). The
original recommendation follows for history.
Recommended: a "Verification" step kind on target-1 forms. A second reviewer with the verify
grant sees the single candidate's answers (exposure recorded; labelled informed), confirms or
edits, and the result becomes an attributed gold snapshot with authority = Verified, distinct
from Reconciled and from Q-29's accept-as-gold. No IRR for verified forms; exports and the
methods summary say "single extraction, verified". Cochrane Handbook ch. 5 accepts one-extracts-
one-checks with caveats; without this step teams fake it with target-1 plus informal review and
no provenance of the check (SR-12).
5.9 Inference beta for experimental groups (owner session, OS-A19)¶
Q-18 and Q-19 are decided (S5). The first reasoner supports only conjunction, containment, disjointness and exhaustiveness. Design holders publish project rules; reviewers confirm per-paper applicability through mapping answers; ordinary reconciliation resolves disagreement. The capability starts as a beta, off by default, enabled only by an explicit project-designer opt-in, never by baseline conversion; ordinary annotation and reconciliation work the same without it (TI §3.7, §5.5). Methodological guarantees (TI §5.6): candidate inferences use only the current reviewer's own authorised snapshot and are shown only to that reviewer; collective inferences use a pinned accepted result; unreconciled answers from different reviewers are never combined; reported answers are never changed and no animal count is invented; when inputs or rules change, results are recomputed and the old ones kept in history as superseded. Bounds reasoning and wider collective authority wait for a later approved scope. GA promotion is a separate decision.
6. Risk of bias and reporting quality templates¶
Owner session: template ownership is decided (D2-15 decided-amended: one application-wide
CAMARADES catalogue run by users with a system permission; versioned copies into projects with
provenance; publication requests from other users; ACD spec §3.4). The scientific content is
specialist input T-SI-03 (D4-06): methodologists verify SYRCLE (with its outcome-specific items),
the CAMARADES checklist and ARRIVE Essential 10 and their applicability before any of them is
published to the catalogue (RI-R58). The table and rules below are the input to that verification.
PROPOSAL, with content ownership and scope put to Chris (D4-06; template ownership D2-15):
| Template | Level | Items | Binding |
|---|---|---|---|
| SYRCLE RoB (Hooijmans et al. 2014) | Study, with items 6 (random outcome assessment), 7 (blinding of outcome assessors) and 8 (incomplete outcome data) per outcome | 10 items, judgement yes / no / unclear → low / high / unclear risk | Per-outcome items bound to the Outcome Assessment entity (TC1 feature-required type), so judgements are per outcome and cannot be lost |
| CAMARADES quality checklist (Macleod et al. 2004) | Study | 10 items, yes / no | Study level |
| ARRIVE 2.0 Essential 10 (Percie du Sert et al. 2020) | Study (reporting quality) | 10 items with sub-items | Study level |
Template rules: items carry semantic roles (rob.domain, rob.judgement, rob.support) so
exports produce a domain × study matrix (and domain × outcome where items are per outcome);
templates are versioned with a review date and an owner; copies never change when the template
does (R1a rule); per-outcome items must be answered per outcome entity. Independence of
assessors comes from the ordinary target-2 form plus R4a reconciliation; PRISMA 2020 item 11
(tool, process, number of assessors) is reported by the methods summary. SR-05 cites the
little-DOMS investigation as evidence that per-outcome judgements have been lost in ad hoc
forms; not re-verified here. Acceptance: AC-R1a-09 and AC-R1a-11 (importing the SYRCLE template
yields per-outcome items bound to Outcome Assessment) and an export fixture producing the matrix.
7. Protocol, registration, search documentation and search rounds¶
7.1 What PRISMA asks for¶
PRISMA 2020 item 6 (information sources: name, platform, date last searched), item 7 (full search strategies with limits and filters), item 24 (registration details, where the protocol can be found, amendments with reasons); PRISMA-S (Rethlefsen et al. 2021) adds dates of searches, update searches and the deduplication method and counts (item 16). SyRF holds a protocol URL and a search name and file (§2). Amendment F says protocol amendments append, but there is no protocol entity to amend, and Q-26 (profile re-publication) is not tied to an amendment record although a mid-review criteria change is a protocol amendment (SR-04).
7.2 Proposed (D4-05)¶
- P1, on
SystematicSearch(nullable, N-1 rule):searchDate,platform,strategyTextor an attached strategy file,limitsAndFilters,dateRange,searchRoundandupdateOf(SR-24), alongside FEAT-011'ssourceTypeandsourceName; exposed in the upload wizard and the admin source-classification tool (AC-P1-04). - R3d, or an earlier small release: a project "Protocol and registration" record: registry (PROSPERO, OSF, other; whether PROSPERO accepts the project's animal-review scope is UNVERIFIED), registration ID, URL and date, protocol document link and version, and an append-only amendments log (date, what changed, reason, which profile or form version it corresponds to).
- F5 binding: publishing a profile version whose eligibility rules changed requires an amendment entry ("why it changed" is optional for questions; required for profiles).
- R5b: the PRISMA manifest and the methods summary (§14.1) export all of it.
Owner session (D4-05 decided through E2, 4 October 2026). Search documentation is versioned: each
save creates a new immutable SearchDocumentationVersion with actor and time, and reports and the
methods summary pin the version they used (RI-R19). The protocol and registration record has an
append-only amendment log tied to the profile, form or filter version each entry corresponds to
(RI-R20). Publishing an eligibility-changing profile version needs an amendment entry in the same
publication; SyRF suggests whether eligibility changed and the publisher confirms or overrides
with a reason (RI-R21; detection PROPOSAL). A change to a profile's source policy is proposed to
count as an eligibility and selection-method change (RI-R22, PROPOSAL). Whether PROSPERO accepts
the project's animal-review scope stays UNVERIFIED.
7.3 Search rounds (minimal now)¶
searchRound on SystematicSearch and on ExternalStepRecord; report snapshots filterable by
round; R5b shows identification per round. Full updated-review support stays deferred; C12
reserves the computation of box 1 from PreviouslyIncluded plus round for later
(prisma-flow-diagram-mapping.md:262-268). Box 1 as reported counts is D4-11 (§11.4).
8. Full-text retrieval workflow and amendment M¶
8.1 The defect¶
FEAT-011 is internally inconsistent. The taxonomy makes FullTextSought (3) and
FullTextNotRetrieved (4) lifecycle states (study-lifecycle-and-source-taxonomy.md:100-102,
148-166), with transitions T4, T12, T13 (:241, :249-250), calls FullTextNotRetrieved
terminal (:258), derives box 6 from lifecycle ∧ TA Included (:453) and box 7 from lifecycle
(:461), and gives the lifecycle precedence over an Included outcome (:597-601). The mapping
document already derives boxes 6, 7, 8, 12, 13 and 14 from Study.fullTextStatus
(prisma-flow-diagram-mapping.md:126, 132, 138, 152, 158, 164), and the three-level model
defines FullTextStatus {Pending, Sought, Retrieved, NotRetrieved} (three-level-data-model.md:
211-219) while listing both derivations for box 7 (:368). The precedence rule contradicts the
plan's "lifecycle = pipeline position" rule and amendment H's per-profile outcomes (SR-15), and
nothing in the plan gave a human the action that populates the boxes (SR-02): box 7 would always
be 0 and box 8 ≠ box 6.
8.2 Amendment M (new)¶
Full text: prisma-amendments.md §M.
- Retrieval is
fullTextStatusonly. Lifecycle never changes for retrieval; T4, T12 and T13 are removed; ordinals 3 and 4 stay reserved and are never written (enum ordinals are appended, never reordered); the precedence rule at:597-601is deleted; the Included transition (taxonomy rule 6) is unaffected by retrieval. - Actions (D4-07): Sought (date), Retrieved (how: PDF in SyRF, read externally), Not
retrieved (reason from a small controlled list plus free text; author-contact date). Reasons
PROPOSAL: not available from any source; paywalled and not obtainable; author contacted, no response; wrong document supplied; language or format not usable; other. - Actors: project administrators and reviewers with a stage grant (capability placeholder
per A-03). Each action is an append-only
StudyLifecycleEventwith actor and time (P1 domain model). - Defaults: a title/abstract collective Include sets Pending → Sought automatically (system
actor, recorded). Attaching a PDF (manual link, bulk PDF, study-source upload) only suggests
Retrieved; a human confirms. Reading the full text outside SyRF is Retrieved with
how = external. - Admission: full-text steps admit only Retrieved studies (admin override, audited); a Not retrieved study receives no full-text outcome.
- Boxes: 6 and 12 = TA-Included with
fullTextStatus ∈ {Sought, Retrieved, NotRetrieved}; 7 and 13 = TA-Included withNotRetrieved; 8 and 14 = Retrieved ∧ entered the full-text pool; each by source column (amendment C). Adopted legacy projects without retrieval history show "retrieval not recorded" coverage, never an inferred Retrieved. - Releases: P1 (events and actions), R3a/R3b (admission), R5b (boxes). Acceptance: AC-P1-11 and AC-R5b-18 (a study TA-included, marked Not retrieved and never FT-screened appears in boxes 6 and 7 and not in box 8, with the reason exported); FX-PRISMA-05b.
- Owner session: D4-07 is decided (E2) and amendment M approved. The automatic Pending → Sought
on title/abstract collective Include stays a
PROPOSAL(RI-R26), and the Not retrieved reason list stays proposed, not approved. - Ambiguity found in the owner-session integration (no owner answer assumed). The
specifications do not say where amendment M's "full-text steps admit only Retrieved studies"
lives under the stage filter model, whose clauses read screening-profile outcomes and
reconciled answers only. Options: (a) add a retrieval-status clause type to stage study filters,
so Not retrieved Studies leave the full-text pool with their own departure reason; (b) keep it as
a step admission rule in the full-text stage, so Not retrieved Studies stay in the pool but are
never offered; © a project-wide admission rule outside stages. Recommendation: (b), because
it matches amendment M's wording, keeps retrieval out of review evidence and leaves pool history
for filter matches only; the reason "not retrieved" joins the admission decision's closed reason
set. To settle in the R3a and P1 briefs (
T-SP-00,T-RI-00).
9. Deduplication¶
9.1 Merge as an alias (amendment L restated)¶
Superseded (D2-12 replaced in the owner session, 4–5 October 2026; OS-A29). A confirmed duplicate merge produces one consolidated current Study with enough history for a reversible unmerge, specified in the duplicate merge specification and contract C21. The Study parent carries a current-version pointer and a Current or Tombstoned state; the input Studies become tombstoned history and are never allocated, listed or counted in current totals. Conflicts (same-reviewer sessions or decisions, conflicting accepted results) are resolved before an atomic commit in a reconciliation-like view, by the merging user or by the original reviewer as a delegated task, with the actual resolver recorded. A reviewer is counted once and the merger confirms the resulting effective target. Unmerge is a new immutable action with per-item carry-forward choices. References never move: the consolidated Study links them, and Study-level bibliographic fields carry per-field source or override provenance. Distinct reports or investigations are never merged under this feature. Reports count a consolidated Study once and never count tombstoned originals. The alias text below is kept for history only.
Brief §1.11, DD-08, V2-02, D2-12. A merge never re-keys immutable records (every natural key
carries studyId; C1 forbids editing revisions). The secondary Study gets mergedInto; the
primary gets a StudyAlias set. Reads, reconciliation candidate selection, statistics and
PRISMA resolve aliases; ContributionQualificationPolicy counts a reviewer once across aliased
studies (SF2). When one reviewer reviewed both duplicates, resolution is per reviewer: the
current session is chosen (admin choice in the wizard, default the later Complete), the other is
superseded with provenance, and the reviewer is counted once. The primary's gold and outcome
histories continue; the secondary's gold and outcomes become candidates with lineage, never
promoted automatically. Merges and splits run as ADR-020 operations that write both Study
documents and refuse busy studies; split removes the alias and re-derives. FEAT-012's "canonical
Study" is renamed "primary Study" in amendment L (the plan's "canonical" means the engine).
FEAT-012 scenarios restated by form and profile instead of stage (service-specification.md:
431-437): scenario 2 (one reviewed) is admin-reviewed under amendment D with the reviewed Study
as primary (V2-12); scenario 3 (both have evidence on at least one shared form or profile) is an
alias merge with candidate joining and per-reviewer resolution; scenario 4 (evidence only on
disjoint forms and profiles) is also an alias merge, because sessions belong to forms, not
stages, so no "same stage" test exists; PRISMA then counts one study and all its Citations. The
FEAT-012 "link only via Publication" outcome is kept for the case where the admin judges the two
records to be different studies of one publication (then they are reports, §10).
9.2 QC and reviewer flags¶
PROPOSAL (SR-19). AutoConfirmed merges are applied before screening; ASySD's specificity above
0.999 (Hair et al. 2023; service-specification.md:54-57) still means some false merges at
scale, removing a record from screening silently. P2 adds: an admin QC sample of AutoConfirmed
groups shown in the review queue (configurable share; PROPOSAL default 5% with a minimum of
20 groups); a reviewer action "Flag as possible duplicate of…" that creates a
DuplicateReviewItem; and manifest fields for the ASySD algorithm version (AlgorithmVersion,
:341), tier rules version, the auto-confirmed versus reviewed share, reversals and the QC
sample result (PRISMA-S item 16).
9.3 Parity metric (D4-21)¶
Chris's question states the meaning: pinned R outputs, identical AutoConfirmed groups,
ProbableDuplicate pair-set F1 ≥ 0.99, published sensitivity and specificity, 80k citations in
under an hour on Bramble. SR-22's concern is folded in as the test design: the pinned fixture
includes the normalisation table (case, punctuation, Unicode, DOI prefix) so "identical groups"
is testable; every divergent pair is listed for review; sensitivity and specificity on the
labelled datasets published with Hair et al. 2023 (names and licences UNVERIFIED; pinned by
commit in the fixture) each within 0.5 percentage points of the R package (PROPOSAL); pair
agreement is the secondary indicator. AC-P2-01r replaces the retired AC-P2-01 accordingly (AC-21).
Owner session: D4-21 is specialist input T-SI-04. Every threshold in this section (identical
AutoConfirmed groups, F1 ≥ 0.99, the 0.5 percentage-point tolerance, 80,000 citations in under an
hour on Bramble) is proposed, not approved, and not achieved (RI-R49). The release decision
needs the specialist parity method.
9.4 Privacy of Publication and cross-project enrichment (V2-03)¶
Publication is not "bibliographic data only": FEAT-011 gives it linkedProjectIds[] and
per-field provenance with sourceProjectId and sourceCitationId
(three-level-data-model.md:97-109), and FEAT-012 enriches it automatically across projects
(service-specification.md:415-423, overwriting previous provenance at :422). Rule: reading a
Publication never exposes project or citation IDs from projects the caller cannot access;
linkedProjectIds and provenance are internal fields served only to platform administrators.
A project sees "metadata enriched from another SyRF project (not identified)". Enrichment is a
recorded event with an HLC stamp (§1.2), and each as-of export states the Publication metadata
version it used; Citations stay the raw, immutable source so an export is reproducible without
the Publication. AC-P2 gains a privacy criterion; fixture 8's cross-project part moves to P2 as FX-PRISMA-08a.
9.5 Pool exclusion list (FEAT-012 §12)¶
Admission (C6) and pool filters exclude lifecycleStatus ∈ {Duplicate, Merged,
PendingDuplicateReview, PendingDedupCheck, RemovedByAutomation, RemovedOther}
(service-specification.md:646-651); only Active enters pools (taxonomy rule 1, :254). This
extends L.5 and AC-P2-06r (V2-12). Withdrawn-search studies and Not retrieved studies at
full-text steps are excluded by admission rules, not by lifecycle (amendments J and M).
9.6 External deduplication (K) and box 3¶
Box 3 = SyRF-detected duplicates (FEAT-012 §11.1) plus reported external duplicates (K); the manifest keeps both parts; FEAT-012 §11.2's count-consistency equation holds over SyRF-held Citations only (V2-13).
10. Reports versus studies and amendments O and N¶
10.1 Amendment O, report-to-study linkage (D4-08)¶
Owner session (D4-08 and Q-23 carry-forward alignment). Amendment O is deferred. This rollout
prepares links to several source documents and keeps distinct-report grouping for later (RI-R03):
each StudyVersion carries reference links with an optional sourceDocumentKey and the basis it
was established on, and work stays owned by the Study. A conference abstract and a journal article
may remain separate Studies under the protocol. Reports label imported references, source
documents and Study items, carry a report identity coverage value (Established, Partial or Not
established), and say "one report per Study assumed" where identity is not verified (amendment B;
Q-06b decided). The StudyLink grouping below is a future design, not a deliverable.
Boxes 10 and 16 need "studies" and "reports"; amendment B (open) fixes report identity but
nothing groups several papers into one study (SR-07); Cochrane Handbook ch. 4 requires collating
reports of the same study, and multiple papers from one experiment are common in preclinical
work. Recommended: a "Link reports to one study" action for administrators and reconcilers that
creates an append-only StudyLink group (members, reason, provenance, actor; dissolution is an
appended event), surfaced in the study view and exports. Linking never merges screening or
extraction evidence: each report keeps its sessions, outcomes and gold; extraction stays per
report with a group key (later: linked reports' PDFs side by side, follow-up). Counting: a group
counts once as a study when at least one member is Included; included members count as reports;
an excluded member stays in box 9 with its reason. The duplicate review queue's pair view is
reused with a different outcome, "same study, different report" (SR improvement 7). Fixture 9:
two reports linked → 1 study, 2 reports in box 10. Until B is approved, exports label totals as
records (A-11).
10.2 Amendment N, Citation to Publication link (V2-01)¶
P1 writes immutable Citations before any Publication exists (P2), yet FEAT-011 makes
Citation.publicationId required (three-level-data-model.md:147) and forbids changing a
Citation (:167). Recommended: the link lives in an append-only CitationPublicationLink record
(citation, publication, how linked: DOI, PMID, fuzzy group, admin; time); Citation.publicationId
becomes optional and write-once at creation when the identifier is known; linking never rewrites
a Citation (AC-P2 criterion); Study.publicationId stays a mutable pointer. Whether P1 also
creates Publications for exact DOI/PMID matches (FEAT-012 Stage 1 brought forward) is an
engineering choice at F-P (E92); a link record is needed either way for Stage 2 results.
11. PRISMA accuracy¶
11.1 Published arithmetic identities (SR-09)¶
Box-by-box tests are not enough. R5b publishes the identities every snapshot must satisfy, per source column (Database/Register, Other, Unclassified) with explicit remainders that the diagram footnotes and the manifest show (the PRISMA2020 R template assumes equality, which only holds when nothing is pending):
| # | Identity | Remainder shown |
|---|---|---|
| I1 | dbr_total_identified (#31) = database_results (#3) + register_results (#5) |
none |
| I2 | other_total_identified (#32) = other_results (#29) = website + organisation + citations + other-source records |
none; corrects FEAT-011, whose #32 omits Other records (prisma-flow-diagram-mapping.md:201 versus :211) |
| I3 | total_identified (#33) = #31 + #32 |
none |
| I4 | records_after_removal (#34) = #33 − duplicates (#7) − excluded_automatic (#8) − excluded_other (#9) |
pending dedup review and pending dedup check (FEAT-012 §11.2) |
| I5 | per column: records_after_removal = records_screened + not yet entered screening |
"not yet screened" (early stop, batches, unreleased pool) |
| I6 | per column: records_screened = records_excluded + sought_reports + unresolved at title/abstract |
pending, conflict, collective Unsure awaiting a vote |
| I7 | per column: sought = not_retrieved + assessed + awaiting retrieval or assessment |
Sought and Retrieved not yet in the full-text pool |
| I8 | per column: assessed = excluded_with_reasons + included + unresolved at full text |
pending, conflict |
| I9 | Σ reasons = excluded_with_reasons − reason not recorded |
reason coverage (amendment E) |
| I10 | new_studies = Σ columns included (resolving StudyLink groups once); new_reports ≥ new_studies |
none |
| I11 | total_studies = new_studies + previous_studies; same for reports (D4-11) |
none |
| I12 | total_studies_ma ≤ total_studies |
none |
| I13 | reported external counts (K) reconcile with the same identities per field, and identified at source − reported removals before import = imported records per search |
K's mismatch warning |
A mismatch blocks freezing a snapshot unless an administrator records an explanation (as K
already does); the explanation is part of the manifest. Acceptance: AC-R5b-09 (field 32 equals
field 29: AC-R5b-25); FX-PRISMA-08b (early-stopped, batched review). Owner session: while
report grouping is deferred (§10.1), I10 resolves no StudyLink groups and "reports" equals
included Studies under the "one report per Study assumed" label; I5's "not yet screened" remainder
reads WorkFirstReleased Studies with no counted decision (RI §3.3, proposed reading for
T-SI-05).
11.2 External steps, the entry-phase rule and per-box combination (V2-04)¶
K lets a project report title/abstract screening, retrieval, assessment and previous-review counts done outside SyRF, but FEAT-011's later boxes depend on SyRF's own outcomes, so a review that screened outside SyRF gets box 6 = 0 and an overstated box 10. Rules (amendment K extension, under Q-37):
- Entry phase per search or import:
identified,after deduplication,after title/abstract screening,after retrieval,after full-text assessment (included elsewhere). Records imported after an outside step count as having passed that step: they are not in SyRF's pool-entry or outcome counts for that phase, and the external record supplies the phase's counts with coverage "reported externally". - Included elsewhere: the import sets
lifecycleStatus = Includedthrough admission with the required profiles' outcomes resolved throughProfileRulefrom the imported decisions under their source policy, withexternalHumanContributionset, coverage "external", so box 10 counts them without inventing SyRF decisions. - Per-box combination: boxes 2–9 and 11–15 = computed (SyRF) + reported (external) for the same field, both parts in the manifest and the diagram marking "includes n reported outside SyRF"; derived fields #31–#34 are never reported, always computed from their components (K.2 and K.3 agree: identification uses the reported "identified at source" count where one exists, otherwise the imported count, labelled "as imported; processing before import not reported", V2-13); box 1 only from the "previous review version" step type (D4-11); boxes 10, 16 and 17 are computed only.
- No double counting: external screening counts are refused for records that have SyRF decisions at that phase, including imported decisions (§3.6).
- Entry of the other step types (title/abstract screening, retrieval, assessment, previous review) is assigned to R5b's records UI (P1 covers identification and deduplication only).
11.3 Snapshots from authoritative records only (MS-11)¶
A PRISMA snapshot is computed from Citations, ExternalStepRecords, ScreeningOutcomes,
StudyLifecycleEvents, StudyEnteredPool entries (FEAT-011's pool-entry events), StudyLink groups, alias sets and the
PrismaPhaseMapping version at the report watermark (an HLC stamp, §1.2), and stored frozen.
FEAT-024 rows are never a report input: a source-type dimension there is a catalogue change and
retained checkpoints never gain it, and "regenerating a frozen report gives identical numbers"
cannot rest on a disposable projection. AC-P1-07 is reworded
(acceptance criteria §4.23). Statistics screens
may still show FEAT-024 counts; reports do not. Owner session (terminology and inputs):
StudyEnteredPool is renamed WorkFirstReleased (first release of work to anyone, with a
release kind) so that it no longer clashes with stage pool entry; stage pool membership is recorded
separately as StagePoolBaselineMember, StagePoolEntered and StagePoolDeparted history events
(C20). "Alias sets" become merge lineage (§9.1). StudyLink groups are deferred. Snapshot inputs
also include accepted ExternalScreeningDecisions, review-start events, search documentation
versions and the protocol record version (RI §3.2).
11.4 Box 1 (D4-11)¶
R5b said "box 1 stays deferred" while K's step types include "studies from a previous review
version" (SR-14). Recommended: K populates box 1 as reported counts (previous_studies,
previous_reports); when such a record exists the diagram switches to the updated-review
template variant and box 16 = new + previous (mapping :264-268); full updated-review support
(importing the previous review's included studies with PreviouslyIncluded) stays deferred.
Owner session: D4-11 decided as recommended (E1). Previous-review counts are only ever supplied
values; SyRF never derives them from its own data, a reference list or an earlier snapshot
(RI §3.4). Frozen reports stay immutable and missing information is disclosed.
11.5 Withdrawn searches and deletion versus history (D3-12)¶
Withdrawing a search hides its Studies from pools and from current reports ("excluded from this
report: n records from withdrawn search X") but keeps Citations and canonical evidence; its
external step records are withdrawn with it; frozen reports never change (amendment J, Q-33).
Deleting a whole project follows ADR-014 (24-hour grace, then physical removal with a minimal
tombstone, ADR-014-reversible-deletion-and-permanent-tombstones.md:59-74, 142-146, 220-250);
PRISMA snapshots do not survive their project, so the user guide tells administrators to export
reports before deletion. The canonical collections' place in ADR-014's deletion scope is X-DEL
(programme integration), not this page.
Owner session (Q-33 decided; D3-12 decided-amended through O1). The withdrawal rule above stands.
The project-deletion sentence is superseded: ordinary whole-project deletion is reversible. The
project is hidden, review and editing are blocked, and drafts, evidence, history and frozen reports
are kept for restoration from a restricted "Deleted projects" view. Permanent physical erasure is a
separate policy that is not approved (T-POL-01), so PRISMA snapshots survive an ordinary deletion
(ACD spec §3.3). The uncommitted deletion design labelled ADR-014 needs a new number, because
ADR-014 is already used on main (source-status inventory §1).
11.6 Calibration and Unsure in C12¶
Calibration records are not pool-entry or screening events. A collective Unsure counts as "not excluded" at title/abstract (box 5 excludes only collective Excluded).
Owner session. Training records replace calibration records here and are likewise never pool,
release or screening events (TI-R04, TI-AE01); promoted training evidence counts only from its
promotion onwards, like any live contribution. A collective Unsure still counts as not excluded;
a pending adjudication is not a definite outcome. Reporting priority (OS-A09). Flow reporting
prioritises Studies actually reviewed through a stage (with the eligibility justification recorded
at review start). Ever-in-pool membership is separate audit data and never the reviewed count. A
filter departure is never reported as a screening exclusion, evidence collected through another
route is shown as "satisfied elsewhere", re-entry never adds a Study, and pre-tracking history is
never fabricated: an eligible Study with no review shows "no review recorded" (RI-R04 to RI-R09).
The mapping of these stage measures onto exact PRISMA boxes is specialist input T-SI-05.
11.7 Machine-assisted outcomes in PRISMA (owner session, OS-A20)¶
Every snapshot carries the machine-assisted share per phase from the outcome composition fields
(RI §3.2, §3.13). A screening outcome resting on an accepted AI-model-generated decision is counted
as a screening outcome of the phase-mapped profile, never as a human decision, and the manifest
keeps the machine-only breakdown. Whether a sole-screener AI Exclude also belongs in box 3
(excluded_automatic, "records marked ineligible by automation tools") or only in box 5 is RI
ambiguity A1: the recommendation is to let T-SI-05 decide before F6b while the manifest holds both
computations. External screening counts (amendment K) are proposed to be refused for records with
accepted external decisions at that phase (RI-R15, PROPOSAL).
12. Exports for synthesis¶
12.1 Lane X1, analysis-ready exports (D4-09)¶
After O1 and R4c: (a) a comparison-level export (gold by default, candidates optional), one
row per comparison × timepoint, with the pairing rule derived from Experiment membership and the
control flags (a control cohort is one whose treatment units are all flagged control and whose
disease-model units match the treatment cohort's; one control serving several treatment cohorts
is flagged sharedControl), columns in metafor's escalc() shape (m1i, sd1i, n1i, m2i,
sd2i, n2i, dispersion type carried, never converted; direction; units; extractionMethod;
experiment, study, report, group key); (b) a machine-readable codebook per export (question
identity, version, wording, options, semantic role, entity scope, requiredness; per answer
answeredUnderVersion and qualificationPolicy, SR-16) so versioned data (AG3) is
interpretable; © a RIS export of any study set (included; excluded with reason; duplicates;
not retrieved) from the Citation raw fields. SMD and NMD computation stays outside SyRF; the
user guide documents the recipe. Owner session: D4-09 decided as recommended (E3). The codebook
also lists, for screening columns, the decision source types and the AI model configuration
versions used. Screening exports gain decisionSourceType, aiModelConfigurationVersion,
externalRunId, confidence, thresholdApplied, onTrainingInput and, on outcomes,
machineContribution (RI §3.9). Estimated-from-graph provenance ships with O1 (D4-10).
12.2 Extraction export defaults (SR-17)¶
PROPOSAL (C11, F6a). Under DP6 and EW1 extraction evidence exists for studies that are
collectively Excluded or Pending (owner session: under own-Include progression, Q-01, and the
saved-work and continuation rules, Q-28 and OS-A05, the same holds). Extraction exports default to studies whose required profiles
(phase mapping) are collectively Included; an explicit option includes others, with per-row
collectiveOutcome, surplusAssessment and profileVersion columns. The same default applies
to the X1 comparison export. Whether today's annotation export filters by screening outcome is
UNVERIFIED (SR-17).
12.3 Synthesis inclusion and metaAnalysisIncluded (SR-20)¶
AC-R5b-06 names a UI that no release delivers. PROPOSAL: a per-study "Synthesis inclusion"
attribute (included, excluded with reason such as no usable data or outcome not reported, not
applicable) under a capability placeholder Record synthesis inclusion (A-03), owned by L12 in
R5b, exported in R5a and X1 and used by box 17; never derived from extraction completion (C12
rule already).
12.4 Other export columns¶
goldDiffersFromAllCandidates (SR-18); extractionMethod, nSource (§5); authority and
independence on screening exports (§3.6); the near-miss preset (§14.2). Owner session: the
machine-source columns of §12.1; finalSource and the composition fields in place of a single
outcome authority (§3.6); accepted-result authority on extraction exports; an "excluded from
current use" label where an exporter includes excluded contributions (OS-A24, PROPOSAL).
13. Imported decisions, routing, early adoption and related inputs¶
13.1 FEAT-004 annotation import (D4-14)¶
FEAT-004 (docs/features/annotation-import/brief.md:22-26, Draft, marked urgent) imports
answers from Rayyan, Covidence or spreadsheets through a five-step wizard (:63-82); its open
question 2 asks whether imported answers are gold or candidates (:131). Recommended: a lane
after R2a; imported answers get provenance kind Imported (source system, import job, mapped
reviewer, declaration as in §3.6); they count toward the target only when mapped to a SyRF
reviewer and declared independent; they are excluded from default independence statistics; they
never become gold automatically; they pin the current question version (:126). The migration
writer list's "annotation import" label refers to question-template import (#2781, #3934) and
should be corrected by the orchestrator (PH-10). Owner session: D4-14 is decided-amended (E3
with the AI expansion). Annotation-answer imports land in a later lane after the first engine
release (XA1, proposed; RI §3.10) and keep the reviewer-mapping rule above. That rule is
superseded for configured external screening sources, which count by the profile's
ScreeningSourcePolicy (§3.10).
13.2 Routing studies by answer values (D4-15)¶
A lane after R4a, not GA: a step-dependency rule on gold values ("only rat studies go to step B"). Methodological rule: routing reads gold or collective outcomes only, never a single candidate's answers, so routing cannot leak one reviewer's decision to another. Owner session (E4): optional accepted-answer branching between steps is deferred beyond the MVP, with no delivery commitment until a future brief is approved. Reconciled-answer clauses in stage study filters stay in scope, and they obey the same methodological rule. No further stage-entry gate is introduced.
13.3 Early screening-profile adoption (D4-16)¶
FEAT-007's just-in-time adoption (screening-profiles/README.md:172-185) is dropped by the plan
(PH-23). Recommended: admin-initiated adoption of screening-only, unreconciled stages after R3b,
through a generated manifest, reversible until the first canonical write, with Q-21's
compatibility-profile labelling. Owner session (D4-16 decided-amended through R4; OS-A14,
OS-A15): the recommendation is superseded by universal faithful baseline conversion: opt-in trials
in staging, then production pilots, then conversion of every remaining project, with explicit
legacy-gap states and no fabricated history. A pilot may convert a complete screening scope only
when its readers and writers are canonical; the first-canonical-write boundary applies to every
conversion (BC spec).
13.4 Stale-answer acknowledgement (D4-17)¶
FEAT-001 D54 and D55 are replaced by RE2's non-blocking warning; enforcement levels are dropped.
Consequence: stale answers are surfaced and exported with answeredUnderVersion, never
blocked. Owner session (D4-17 decided-amended through O3): handling is configurable. An authorised
project designer or admin chooses warn and allow Complete (the default) or block completion until the
flagged answers are addressed. Neither mode bypasses validation, applicability, permissions or
mandatory publication and re-review treatment; drafts are kept, and each completed version records
the mode and evidence versions it used. The setting is part of the form version (RS-R39 to RS-R42).
"Never blocked" above is superseded.
13.5 Disabled members' work (D4-20)¶
Completed work keeps counting and stays in reconciliation because evidence is never erased. An
audited admin action can exclude a reviewer's contributions from a form; it is recorded as a
withdrawal with reason, never a deletion, and IRR excludes withdrawn contributions. Owner session
(D4-20 decided-amended through O1; OS-A24): exclusion may cover an annotation form, a screening
profile, a stage (selected by recorded route provenance) or the whole project, including from the
membership-disable workflow. It needs an impact preview, a reason, the actor and the time, and it is
recorded as a ContributionExclusion; a reviewer's own withdrawal is a separate action (ACD-R05).
Submitted versions keep their named attribution
in history. Excluded contributions leave current qualification, readiness, candidate outcomes,
current statistics and current agreement; as-of views before the exclusion keep them. Dependent
results stay intact and flagged, their authors are told, and new result versions need an explicit
decision (ACD spec §3.2).
13.6 FEAT-007 and FEAT-009 as inputs¶
FEAT-007's success metrics (screening-profiles/README.md:212-217: 80% fewer multi-project
workarounds; ≤ 5 minutes to configure a two-stage pipeline; select-next p95 < 400 ms) become
R3a/R3b acceptance inputs (PH-23). FEAT-009's reconciliation pool settings
(screening-annotations/README.md:348-432: default "reconcile when annotations exist", bypass
criteria by question set, all-studies option, and truncation of disagreed sub-reasons) are
inputs to the profile reconciliation settings and to amendment E and Q-22: truncation becomes a
reason coverage value, "primary agreed; sub-reason not agreed" (PH-24).
14. Transparency outputs¶
14.1 Methods-summary generator (R5b)¶
PROPOSAL (SR improvement 1). From data the plan already captures, R5b emits a structured
"Methods" block (JSON and prose) covering PRISMA 2020 items 5 (eligibility criteria: profile
versions), 6 and 7 (information sources and strategies: §7), 8 (selection process: reviewers,
independence, Unsure, conflict route, discussion, calibration, IRR with basis), 9 (data
collection: targets, verify or reconcile), 10 (data items: form versions and schemas), 11 (RoB
tool and process), 16 (results of selection, with the near-miss list), 24 (registration,
protocol and amendments), plus the deduplication method and counts (PRISMA-S item 16), external
steps and retrieval failures. It turns the audit trail into publication text. Owner session: the
summary reports automation tools (RI-AE36): each AI screening model configuration version used,
its source policy role and scope, the run identities and the machine-assisted share per phase.
"Calibration" in item 8 becomes training (separate from live review), and "verify or reconcile" in
item 9 becomes the accepted-result authority actually used (Single annotator or Human reconciled).
14.2 Near-miss excluded list¶
PROPOSAL (SR improvement 2; PRISMA 2020 item 16b). An export preset "full-text excluded
studies with primary reason and reviewer or reconciler provenance" in R5a or R5b; the data
exists once R3b and R4p ship. Acceptance criterion: AC-R5b-21
(acceptance criteria §4.22).
14.3 Methods caveat label¶
PROPOSAL (SR improvement 10). Where a project uses single screening, target-1 extraction
without verification, or unverified imported decisions, the project overview and the PRISMA
manifest carry a persistent "methods caveat" label; the user guide already warns
(screening.md:75). Owner session: "target-1 extraction without verification" now means
accepted results with "Single annotator" authority (AutoAccept); AI sole-screener outcomes also
carry the caveat (PROPOSAL).
15. Funder alignment¶
Status: the contract and grant positions are UNVERIFIED beyond docs/funding/ as of 14 March
2026 (D4-18 asks Chris to confirm with the funders). The mapping below is provisional (A-40).
| Funder item | Source | Release that satisfies it | Status |
|---|---|---|---|
| NC3Rs Contract 1 Period 4: question editing (annotation questions design interface) | docs/funding/nc3rs.md:172 |
R1a (templates and shared editor), R2a (versioned forms) | UNVERIFIED whether still expected |
| NC3Rs Period 4: screening types and study filtering | :173 |
R3a (steps and routing), R3b (profiles) | UNVERIFIED |
| NC3Rs Period 4: in-app reconciliation (qualitative) | :174 |
R4a (form reconciliation and gold); PH Q4's recommendation not to build a reconciler on the legacy model stands | UNVERIFIED |
| NC3Rs Period 4: customisable project groups (enhanced) | :175 |
R1c | UNVERIFIED |
| NC3Rs Period 4: bulk upload of PDFs | :176 |
FEAT-021 bulk PDF programme (separate; flag off) | UNVERIFIED |
| NC3Rs "future development": de-duplication; common question templates | :191, :199 |
P2 (amendment L); R1a | Brought back into scope by this plan |
| NC3Rs "future development": PDF retrieval automation, machine-assisted extraction, multi-database retrieval, living search, Zotero | :192-198 |
Out of scope here; living search stays flagged and deferred | Unchanged |
| SSI RSMF objective 1: annotation question versioning with full audit trails, months 4–8 from a 1 October 2026 start | docs/funding/ssi-rsmf.md:53, 71-73, 44 |
R2a–R2d | Grant outcome UNVERIFIED (document shows EoI submitted, decision expected April 2026) |
| SSI RSMF objective 3: WCAG 2.1 AA with an independent audit | :55, :80 |
GA (D4-18): AC-ALL-07 and AC-GA-08 (WCAG 2.1 AA with an audit step); the accessibility harness in ux-strategy | UNVERIFIED |
| SSI RSMF objective 5: Community Steering Group and public roadmap | :57, :76 |
Tester panel (D1-06) and the delivery operating model's public STATUS ledger | UNVERIFIED |
| SSI RSMF claim: blinding and random serving are core behaviours | :100 |
D4-19 (§3.8) |
16. Decisions needed, engineering items and assumptions¶
16.1 Decisions (Batch D)¶
Methodology: D4-01 Unsure; D4-02 discussion route; D4-03 extract and verify; D4-04 calibration; D4-05 protocol, registration and search documentation; D4-06 RoB and reporting-quality templates; D4-07 retrieval actions; D4-08 report linkage (amendment O); D4-09 lane X1; D4-10 graph digitisation; D4-11 box 1; D4-12 IRR basis and methods; D4-13 primary-reason hierarchy; D4-14 FEAT-004; D4-15 routing by answer values; D4-16 early profile adoption; D4-17 D54/D55; D4-18 funder mapping and WCAG audit; D4-19 blinding and random serving; D4-20 disabled members; D4-21 parity meaning. Cross-cutting: D2-12 alias merge; D2-15 template ownership; D3-11 agreement store; D3-12 deletion versus history; D3-25 conversations as audit record. Open questions this page depends on: Q-06b (B, E, F), Q-16, Q-17, Q-22, Q-23, Q-33, Q-37.
Owner session, 5 October 2026: the list above is history. Current statuses are in the §1.1
table. In summary: decided or decided-amended are D4-01, D4-02, D4-03, D4-04, D4-05, D4-07, D4-09,
D4-10, D4-11, D4-13, D4-14, D4-16, D4-17, D4-19, D4-20, Q-06b, Q-22, Q-33, Q-37, D2-15, D3-12 and
D3-25; D2-12 is replaced; D4-15 is deferred beyond the MVP; D4-08 and Q-23 are carry-forward brief
items; D3-11 is a brief item; D4-06, D4-12, D4-21, Q-16 and Q-17 are specialist inputs
(T-SI-01 to T-SI-04, with box mapping as T-SI-05); D4-18 was answered on 3 October and its
reading awaits G0-D1. None of these is an open owner decision; the repository's one open owner
decision is D2-09, outside this page.
16.2 Engineering items E88 to E93¶
| ID | Contract | Lane / contract | Gate |
|---|---|---|---|
| E88 | Observation-basis markers and the agreement store: initial-independent-submission marker derived at commit; collective-exposure record at correction (route kinds); "questioned in reconciliation" exposure looked up by session (#3965); imported authority and independence; calibration purpose; computation from canonical revisions into the rebuildable agreement store (D3-11) under the method contract (E9). Owner session: imported provenance (not authority); hint shown and adopted exposure kinds; training exposure in place of calibration purpose; external human and machine source classes; methods from T-SI-02 |
L1, L11 / C3, C11 | F1a (markers), R5c (store) |
| E89 | Full-text retrieval event model: StudyLifecycleEvent kinds for Sought, Retrieved (how) and Not retrieved (reason list, author-contact date) with actor; automatic Pending → Sought on title/abstract collective Include; the "suggest Retrieved" hook from the PDF programmes; full-text admission on fullTextStatus; box derivations per amendment M |
L12, L4 / C12, C6 | F-P (P1), F3 (admission) |
| E90 | Extraction provenance and validators: extractionMethod and dataSource roles; unit vocabulary with SI-aware labels and the same-measure validator; dispersion catalogue; domain validators; nSource rule; graph-estimated default on region link; QC view queries |
L10 / C14 | F-O (O1) |
| E91 | PRISMA arithmetic and authoritative snapshots: the identity checker (I1–I13) with remainders and explanations; entry-phase and per-box combination for external records; computation from authoritative records at an HLC watermark; template-variant switch for box 1; withdrawn-search exclusion | L12 / C12 | F6b (R5b); K and L parts at F-P |
| E92 | Link records: CitationPublicationLink (amendment N; whether P1 also creates Publications for exact DOI/PMID matches) and StudyLink groups (amendment O) with alias and group resolution in counting and exports. Owner session: StudyLink deferred (D4-08); alias resolution becomes merge lineage (C21) |
L12, L1 / C12, C11 | F-P (N), P2 (O) |
| E93 | Analysis-ready exports and transparency outputs: comparison pairing rules, codebook schema, RIS tag mapping, Record synthesis inclusion capability and attribute, near-miss preset, methods-summary schema, domain × study RoB matrix |
L11, L12 / C11, C10 | X1, R5b |
16.3 Assumptions¶
| ID | Assumption | Basis | Cost if wrong |
|---|---|---|---|
| A-39 | For canonical projects the initial-independent-submission marker can be derived at commit from the commit order and the visibility events already planned (availability messages, VS1 exposure, conversations); for adopted legacy projects it is unknown, and every legacy decision is labelled "basis unknown" in agreement statistics | C3 three-state exposure; EX2 no fabricated history | R5c shows no IRR for adopted projects, only percent agreement labelled "basis unknown"; or an explicit marker must be written by every submit path |
| A-40 | The funder mapping in §15 reflects docs/funding/ as of 14 March 2026; neither the NC3Rs contract position nor the SSI RSMF outcome has been confirmed since |
docs/funding/nc3rs.md, docs/funding/ssi-rsmf.md |
Release order could change (for example R4a earlier for NC3Rs); WCAG audit timing could move |
16.4 Follow-up backlog (not in this scope)¶
A graph digitiser lane (after O1 pilots, D4-10); full updated-review support with
PreviouslyIncluded; linked reports' PDFs side by side in extraction; registry lookups
(PROSPERO, OSF); direct connectors to Covidence or Rayyan (FEAT-004 brief rules them out);
scored training against gold as an admission prerequisite (hook reserved by D4-04); statistical
outlier checks on extracted values. Owner session: scored training with optional group admission
moved into the plan (lane TR1, OS-A18). Added to the backlog: distinct-report grouping
(amendment O, D4-08); accepted-answer branching between steps (D4-15); inference bounds and further
operators (S5); match-based training scoring for repeatable entities (TI §12).
Resolution record¶
| Finding | Category | Where | Note |
|---|---|---|---|
| SR-01 | Adopted | §4.1–§4.3, §16.2 E88; contracts C3, plan R5c, AC-R5c-06, 10, 11 and 12 | IRR observation basis in C3 (PROPOSAL); screening IRR per profile in R5c; methods under D4-12 |
| SR-02 | Adopted | §8; prisma-amendments M, plan P1 row, AC-P1-11, AC-R5b-18 | Human retrieval workflow; actors and PDF suggestion are D4-07 |
| SR-03 | Adopted | §3.6; contracts C3, prisma-amendments H authority list | authority = Imported, independence declaration; K no-double-count |
| SR-04 | Question | §7; D4-05 | Search fields in P1 and the protocol record adopted as PROPOSAL pending D4-05 |
| SR-05 | Question | §6; D4-06, D2-15 | Template rules and AC-R1a-09 and AC-R1a-11 proposed |
| SR-06 | Adopted | §5.1–§5.5; contracts C14, plan O1 row, AC-O1-02, AC-O1-11 and AC-O1-12 | Graph digitiser decision is D4-10 |
| SR-07 | Question | §10.1; prisma-amendments O | D4-08; FX-PRISMA-09 |
| SR-08 | Question | §12.1; plan lane X1 row, AC-X1 | D4-09 |
| SR-09 | Adopted | §11.1; contracts C12, AC-R5b-09 and AC-R5b-25 | Also Corrected: FEAT-011 #32 omits Other records (I2) |
| SR-10 | Question | §3.5; AC-R3b-13 (failing-criteria view: AC-R4p-09) | D4-13 |
| SR-11 | Question | §3.4 | D4-04; C12 rule and C6 hook |
| SR-12 | Question | §5.8 | D4-03 |
| SR-13 | Question | §3.2 | D4-01 |
| SR-14 | Corrected | §11.4; plan R5b scope, prisma-amendments K, AC-R5b-07 | R5b and K were inconsistent; box 1 from K per D4-11 |
| SR-15 | Corrected | §8.1–§8.2; prisma-amendments M and summary table | Lifecycle precedence superseded; recorded for register §2 |
| SR-16 | Adopted | §12.1 (codebook columns) | Compatibility guards themselves are decided in versioning-model.md (brief §1.9) |
| SR-17 | Adopted | §12.2; AC-R5a-10 | Default by collective outcome; surplus labelling |
| SR-18 | Adopted | §4.5, §12.4 | Override count and export column |
| SR-19 | Adopted | §9.2; prisma-amendments L, AC-P2-17 | QC sample, reviewer flag, manifest fields |
| SR-20 | Adopted | §12.3; plan R5b scope | Owner L12, R5b; capability placeholder |
| SR-21 | Adopted | §3.1 | PROPOSAL template defaults; depends on D4-01, D4-02, D4-13 |
| SR-22 | Question | §9.3; AC-P2-01r | D4-21 as Chris stated, with divergent-pair listing and the 0.5 pp tolerance as test design |
| SR-23 | Question | §3.3 | D4-02 |
| SR-24 | Adopted | §7.3; plan P1 row | Minimal search rounds now |
| SR-25 | Adopted | §3.7 | Per-profile option, default off |
| SR improvement 1 | Adopted | §14.1; plan R5b scope | Methods-summary generator |
| SR improvement 2 | Adopted | §14.2; AC-R5b-21 | Near-miss list |
| SR improvement 3 | Adopted | §4.6 | Drift view |
| SR improvement 4 | Adopted | §3.5 | Failing-criteria vectors in R4p |
| SR improvement 5 | Adopted | §5.6 | Extraction QC view |
| SR improvement 6 | Adopted | §9.2; prisma-amendments L | Dedup transparency in the manifest |
| SR improvement 7 | Adopted | §10.1 | Conditional on D4-08; side-by-side PDFs are follow-up |
| SR improvement 8 | Adopted | §12.1 | Codebook in X1 |
| SR improvement 9 | Adopted | §3.1, §6 | Content ownership by CAMARADES methodologists (D2-15) |
| SR improvement 10 | Adopted | §14.3 | Methods caveat label |
| SR Q-1 | Question | §3.2 | D4-01 |
| SR Q-2 | Question | §3.3 | D4-02 |
| SR Q-3 | Question | §5.8 | D4-03 |
| SR Q-4 | Question | §3.4 | D4-04 |
| SR Q-5 | Question | §7 | D4-05 |
| SR Q-6 | Question | §6 | D4-06 |
| SR Q-7 | Question | §8.2 | D4-07 |
| SR Q-8 | Question | §10.1 | D4-08 |
| SR Q-9 | Question | §12.1 | D4-09 |
| SR Q-10 | Question | §5.7 | D4-10 |
| SR Q-11 | Question | §11.4 | D4-11 |
| SR Q-12 | Question | §4.3 | D4-12 |
| SR Q-13 | Question | §3.5 | D4-13 |
| V2-01 | Adopted | §10.2; prisma-amendments N, AC-P1-12 | Link record; Citation never rewritten |
| V2-02 | Corrected | §9.1; prisma-amendments L | Alias merge; scenarios by form and profile; E33 restated |
| V2-03 | Corrected | §9.4; prisma-amendments L rule 7, AC-P2-15, FX-PRISMA-08a | Privacy rule; enrichment events; as-of basis |
| V2-04 | Corrected | §11.2; prisma-amendments K, AC-R5b-07 | Entry-phase rule; per-box rules; box 1 via D4-11; under Q-37 |
| V2-10 | Corrected | prisma-amendments G | Lines 279–280; MIG-13 and MIG-14 covered |
| V2-11 | Corrected | prisma-amendments H | All three placeholders amended; Imported added |
| V2-12 | Corrected | §9.1, §9.5; prisma-amendments L, AC-P2-04, AC-P2-06r, AC-P2-01r | Scenario 2 change listed; exclusion list extended; criteria fixed |
| V2-13 | Corrected | §9.6, §11.2; prisma-amendments K | K.2 and K.3 agree; #31–#34 never reported; FEAT-012 §11 in Amends; §11.2 over SyRF-held Citations; other step types assigned; withdrawn searches |
| V2-14 | Corrected | prisma-amendments preamble | Freeze timing and Q-37 cited |
| PH-10 | Question | §13.1 | D4-14; inventory label correction for the orchestrator |
| PH-12 | Question | §15, §3.8 | D4-18 and D4-19; traceability table provided; AC-ALL-07 and AC-GA-08 wording in acceptance criteria |
| PH-23 | Question | §13.3, §13.6 | D4-16; metrics as acceptance inputs |
| PH-24 | Adopted | §13.6 | FEAT-009 settings and truncation as inputs to E and Q-22 |
| PH question 4 | Question | §15 | D4-18 |
| PH question 5 | Question | §3.8 | D4-19 |
| PH question 6 | Question | §13.1 | D4-14 |
| PH question 9 | Question | §13.3 | D4-16 |
| review AC-04 | Adopted | acceptance criteria §7 fixtures FX-PRISMA-01..09 | Versioned data with per-release evidence assertions; splits by release |
| review AC-20 | Adopted | AC-P1-09.., AC-P2-10.., AC-R3a-18 and 19, AC-R5b-08.., AC-C1-06 | FEAT-011 MUSTs as criteria; validation procedures reused |
| review AC-21 | Adopted | §9.3; AC-P2-01r, AC-P2-06r and AC-P2-11 to 14 | Metric per D4-21; datasets and licences UNVERIFIED |
| MS-11 | Corrected | §11.3; contracts C12, AC-P1-07 | Authoritative-only snapshots; AC-P1-07 reworded |
| DD-08 | Corrected | §9.1; prisma-amendments L | Alias merge; "primary Study" |
| NS-06 | Adopted | §4.1; contracts C3, AC-R5c-07 | "Questioned in reconciliation" exposure kind; R5c, C11 and R6 consume it; D3-25 |
Owner session, 5 October 2026: the "Question" rows above routed findings to Batch D. Their current statuses are in §1.1; the round-2 categories are kept as history.