Specification: reporting, methods, imports and AI-model screening¶
Planning specification, written from the owner session of 4–5 October 2026. The owner
decisions recorded here are planning approval only. Brief approval and implementation
authorisation are separate steps, given per freeze gate (D1-04), and both remain on hold: Chris
placed feature implementation on hold on 5 October 2026. Nothing in this document authorises
implementation, migrations, imports of any kind, production activation, statistics changes in
production or notification delivery. No work has started under it and no gate has passed.
Storage choices are PROPOSALs. Every numeric threshold below is proposed, not approved. Exact
PRISMA box mapping, statistical methods and scientific event definitions need specialist input
before the dependent build.
Sources. The consolidation §5 "Reporting, methods and imports", §3 "Structured historical justification" and §7; the session register entries Q-06b, Q-16, Q-17, Q-23, Q-33, Q-37, D3-10, D3-11, D4-05 to D4-12, D4-14 and D4-21 and its appended sections (historical pool coverage, the reporting-priority clarification, structured events, the E3 AI expansion, AI-model metadata ownership, source identity and terminology); the condensed packages E1 to E3 and the brief table; the stage filter and step model. Package context: PRISMA amendments, methodology coverage, contracts C3 and C11 to C14, programme integration §7 and acceptance criteria §4.20 to §4.24 and §4.36.
Identifiers. Rules are RI-R01 onwards; acceptance evidence is RI-AE01 onwards. Amendments
the owner session agreed outside the register count are cited by short name (for example "E3 AI
expansion"); the integration document assigns their OS-A IDs. Related specifications in this
folder: review domain and versioning (RD),
duplicate merge (DM), stage pools, steps and history
(SP), reconciliation and screening (RS),
baseline conversion (BC), training and inference
(TI), UX, devices and work discovery (UX) and
access, communications and deletion (AC).
1. Summary in plain English¶
SyRF reports must say exactly what they count. A project holds three different things that older plans sometimes blurred. An imported reference is one row from one import file. A source document is a real paper, abstract or preprint. A Study is the item the project reviews. Two imports of one article are two references, one document and, after duplicate merging, one Study. Reports label each count with its unit and say when a document's identity is not known.
For flow diagrams, the owner's priority is actual review through a stage, together with the
eligibility justification recorded when the review started. SyRF also keeps a full history of
which Studies entered and left each stage's pool, but that history is audit data. A Study that
sat in a pool and was never reviewed is not "screened". A Study that left a pool because a filter
clause stopped matching was not "excluded" by screening. How these measures map onto exact
PRISMA boxes is a specialist decision (tracker row T-SI-05).
Reports that have been frozen never change. Corrections and protocol amendments append and show up only in later reports. An updated review's "previous studies" box uses counts that someone explicitly typed in; SyRF never guesses them. Withdrawing a search keeps its references and review history, removes its Studies from current work and current reports, and says so in the report. Counts for steps done outside SyRF (for example deduplication in another tool) live in a ledger, kept apart from the counts SyRF computes, and the two are never double counted.
The project records its methods: versioned search documentation (date, platform, strategy, limits, round) and a protocol and registration record with an append-only amendment log. Publishing a screening-profile version that changes eligibility needs an amendment entry. Full-text retrieval is recorded by explicit actions (Sought, Retrieved, Not retrieved, with who, when and why); attaching a PDF only suggests Retrieved.
Exports gain a comparison-level analysis-ready file for meta-analysis tools, a machine-readable codebook and an RIS file for any chosen set of Studies. Values read off a graph carry an "estimated from graph" flag. Graph digitisation and imports of answers from other tools wait for later, separately approved lanes.
SyRF will accept AI-model-generated screening decisions produced outside SyRF. The AI screening model is recorded as a machine source with its own versioned description in the project. It never pretends to be a person and it never gets a login. A screening profile declares whether the model is one contributing vote or the sole screener. Rerunning the model produces new versions of its decisions, not extra voters. A model Unsure is never an Include: under a sole-screener policy it goes to a human adjudicator, and as a contributing vote it follows the profile's Unsure rules (RS §5.10). Model outputs on the studies it was trained on are never treated as independent validation. Reports, exports and agreement statistics keep machine-assisted results distinguishable from purely human ones.
Two statistics rules finish the picture. Live statistics used at a publication boundary must be a pinned read whose source identity is recorded, and production statistics readiness stays provisional (FEAT-024 gate (b) failed on latency on 3 October 2026, #3510). Reviewer-agreement statistics live in their own rebuildable store, separate from progress statistics.
2. Decisions covered¶
| ID | Decision in one line | Status | Section |
|---|---|---|---|
| Q-06b | Keep imported-reference, source-document and Study counts distinct; disclose reason and history coverage; separate accepted answers from reporting status; preserve time, protocol amendments and frozen reports; honest labels where report identity is missing (amendments B, E, F) | Decided (E1, 4 Oct) | §3.1, §3.2, RI-R01–R03, RI-R10–R12 |
| D4-11 | Previous-review box from explicitly supplied counts with the updated-review template; full updated-review workflow deferred | Decided (E1) | §3.4, RI-R13 |
| D4-05 | Versioned search strategy, date, platform, limits and round; protocol and registration record with append-only amendments; eligibility-changing profile publication needs an amendment entry | Decided (E2) | §3.6, §3.7, RI-R19–R22 |
| D4-07 | Explicit Sought, Retrieved and Not retrieved actions with actor, time and reasons; a PDF suggests and never confirms; retrieval separate from lifecycle | Decided (E2) | §3.8, RI-R23–R26 |
| D4-09 | Comparison-level analysis-ready export, machine-readable codebook and RIS for a selected set; no effect sizes inside SyRF | Decided (E3) | §3.9, RI-R27 |
| D4-10 | Estimated-from-graph provenance in the first outcome-data release; digitisation decided after pilots | Decided (E3) | §3.9, RI-R28 |
| D4-14 | Answer imports in a later lane with provenance and no automatic gold or independence credit; target credit only when mapped to a SyRF reviewer for annotation imports; the mapping rule is superseded for configured external screening sources | Decided-amended (E3 and the E3 AI expansion) | §3.10, §3.12, RI-R30–R31 |
| E3 AI expansion (owner amendment, outside the register count) | AI-model-generated screening decisions: machine source separate from importer; versioned AIScreeningModelConfiguration; profile ScreeningSourcePolicy (ContributingVote or SoleScreener); runs and decision provenance; reruns as versions; model Unsure to human adjudication; training-set outputs never validation; machine-assisted outcomes distinguishable; agreement keeps source classes apart |
Decided (4 Oct, with metadata ownership, source identity and terminology clarifications) | §3.11–§3.14, RI-R32–R47 |
| Q-33 | A withdrawn search keeps its references and appends a withdrawal event; current reports exclude it and explain; frozen reports never change | Decided | §3.5, RI-R17–R18 |
| Q-37 (counts and QC part) | External-step ledger with per-box combination, reported versus computed, no double counting, permissions, warnings and withdrawal; deduplication keeps imports, excludes non-current records, protects privacy, samples for QC, lets reviewers flag duplicates and validates parity; the alias merge rule is replaced | Decided-amended | §3.4, §3.15, RI-R14–R16, RI-R48 |
| Q-23 | Apply the reporting-unit distinction; full report or investigation grouping stays deferred | Brief item (carry-forward alignment) | §3.1, RI-R01–R03 |
| D4-08 | Prepared multi-source links, Study-owned work, deferred distinct-report grouping | Brief item (carry-forward alignment) | §3.1, RI-R03 |
| Reporting priority (owner clarification, outside the register count) | Flow reporting prioritises actual review through a stage with its contemporary eligibility justification; ever-in-pool history is separate audit data; filter failure is not a screening exclusion | Decided, with specialist input T-SI-05 for box mapping |
§3.3, RI-R04–R09 |
| Q-17 | Field-level event-count schema and the meaning of "variation" | Brief item (specialist input T-SI-01) |
§12, RI-R57 |
| Q-16 | Denominators, percent agreement, statistical review before multi-rater formulas | Brief item (specialist input T-SI-02) |
§3.16, §12, RI-R50 |
| D4-12 | Initial independent observations, per-profile screening agreement, statistical review of pooled pairwise κ or Krippendorff's α | Brief item (specialist input T-SI-02) |
§3.16, §12, RI-R50 |
| D4-21 | ASySD parity and performance method; F1 ≥ 0.99 and 80,000 citations in under an hour are proposals, not approved or achieved | Brief item (specialist input T-SI-04) |
§3.15, §12, RI-R49 |
| D3-10 | Source-pinned statistics at the publication boundary; protected pilot; profile granularity; preview exemption; production readiness provisional | Brief item | §3.17, RI-R52–R56 |
| D3-11 | Separate rebuildable agreement store with a source watermark and measured budget | Brief item | §3.16, RI-R51 |
| D4-06 | Methodologist curation of SYRCLE, CAMARADES and ARRIVE Essential 10 templates and their applicability | Brief item (specialist input T-SI-03) |
§12, RI-R58 |
Referenced, owned elsewhere. Q-22 and D4-13 (the reason model and the primary-reason rule; RS spec; this spec owns the count of distinct excluded Studies, RI-R05a), D4-01 and the S1 Unsure amendment (RS spec), D3-12 and reversible project deletion (ACD spec), D2-12 and the consolidated merge (DM spec), structured history events and pool events (SP spec, contract C20), D2-13 recovery (BC spec), S4 training steps (TI spec; a different thing from an AI model's training set).
Scope of the counts. Of the 25 alignment, brief and validation entries in the repository universe of 89, this specification owns nine: Q-23, D4-08, Q-17, Q-16, D4-12, D4-21, D3-10, D3-11 and D4-06.
3. Concepts, entities and storage¶
Every storage line below is a PROPOSAL. A distinct domain record does not by itself justify a
new collection; the brief chooses physical storage against measured volume and query needs.
3.1 Reporting units¶
| Unit | Plain English | Identity | Versions and mutability | Proposed storage (PROPOSAL) |
What it is not |
|---|---|---|---|---|---|
Imported reference (Citation, shown as "record") |
One row of one import file, with its raw fields, source type, search and import time | Citation ID, minted at import | Immutable; never edited; linking writes a separate record (amendment N) | Immutable records on Study.citations[] as the domain model proposes, or their own collection if size requires it, decided before P1 |
Not a source document and not a Study |
| Source document (shown as "report") | A distinct document: journal article, conference abstract, preprint, thesis | Established from a Publication match (DOI or PMID, P2), from a duplicate merge that asserts "same document", or from an administrator's confirmation | The link set is part of the immutable StudyVersion (DM spec); a new link set is a new StudyVersion |
A referenceLinks[] entry on StudyVersion carries an optional sourceDocumentKey and the basis it was established on; files of that report (PDFs, supplements) sit in RD's sourceDocumentLinks[] and may carry the same key (RD §3.3) |
Not an investigation and not a grouping of distinct papers |
| Study (shown as "Study") | The project's reviewable item; forms, sessions and targets attach here | Study ID; current content in StudyVersion |
Mutable parent with currentVersionId and state (Current or Tombstoned); immutable versions (DM spec) |
Existing pmStudy plus StudyVersion (DM spec) |
Not a citation; not automatically one document when report identity is unknown |
| Publication | System-wide bibliographic identity used for enrichment | Publication ID; DOI and PMID unique | Enrichment events append; privacy rule of amendment L.7 | pmPublication (P2) |
Never a carrier of review data; never exposes other projects' identities |
| Investigation grouping (deferred) | Several distinct reports of one experiment, counted once as a study | Not built in this rollout | None | None now; amendment O's StudyLink stays a future design |
Not delivered by prepared links |
Report identity coverage. Every report snapshot and export carries a reportIdentityCoverage
value per counted population (PROPOSAL name): Established (every counted Study has a
confirmed source-document key), Partial (some do; the count of unconfirmed Studies is shown)
or Not established. While report grouping is deferred, each Study has at most one distinct
source document in practice, so "reports of included studies" equals "included Studies". The
report says this in a footnote ("Report grouping across Studies was not performed; each Study is
counted as one report"). Where identity is not established, the report labels the figure as
records or as "one report per Study assumed (identity not verified)", never as verified reports
(amendment B; Q-06b).
Worked example. Search A (PubMed) and search B (Embase) both return the 2019 article on
minocycline in rat stroke. SyRF holds two Citations. ASySD flags them; Priya confirms the merge.
The consolidated Study S-101 has one StudyVersion with two reference links that share one
sourceDocumentKey (basis: DOI match). The PRISMA report counts 2 records identified, 1 duplicate
removed, 1 report and 1 Study. The 2018 conference abstract of the same work was imported as
Study S-117. It stays a separate Study and a separate report in this rollout; the methods summary
notes that report grouping was not performed.
3.2 Frozen report (PrismaFlowSnapshot) and its manifest¶
- What it is. A PRISMA flow report computed at one watermark (a hybrid-logical-clock stamp) from authoritative records and then frozen. It holds every box value, the computed and reported parts of each box, the definitions and versions used, the coverage values and the content digests.
- Identity. Snapshot ID plus a per-project sequence. A later snapshot may name an earlier one as the one it supersedes.
- Mutability. Immutable once frozen. Regenerating it from its manifest gives identical numbers. Corrections, amendments, withdrawals and merges after the watermark appear only in later snapshots.
- Inputs. Citations,
ExternalStepLedgerentries,ScreeningOutcomes, acceptedExternalScreeningDecisions,StudyLifecycleLedgerentries,HistoryEventrecords of the pool and review-start types (C20),WorkFirstReleasedentries, Study and merge lineage, search documentation versions, the protocol record version and thePrismaPhaseMappingversion, all at the watermark. FEAT-024 rows and FEAT-024 history are never inputs (MS-11). - Coverage values carried. Report identity coverage (§3.1); reason coverage (amendment E; Q-22 in the RS spec); retrieval-not-recorded coverage; pool-tracking coverage (the date pool history began and whether the baseline is complete); reported external parts per box; machine-assisted share per phase (§3.13); legacy coverage labels from baseline conversion (BC spec).
- Storage.
pmPrismaFlowSnapshotwith the manifest embedded (domain model). - What it is not. A live view. A cached copy of statistics. A place where an old figure is corrected.
3.3 Stage reporting measures and the PRISMA priority¶
The SP spec defines the events. This section defines what reporting reads from them. All measures count distinct current Studies (re-entry never adds a Study; a tombstoned original of a merge is never an extra current Study).
| Measure | Plain English | Read from | Used for |
|---|---|---|---|
| M1 Ever in the pool | Studies that matched the stage study filter at any time in the period | StagePoolBaselineMember, StagePoolEntered |
Audit; the stage history view; coverage disclosure |
| M2 Work first released | Studies whose review work was first offered to anyone (shared batch opening, personal grant, explicit assignment, first availability with remaining work in an unbatched stage, or inheritance through a merge; SP §3.7 release kinds) | WorkFirstReleased (renamed from the round-2 StudyEnteredPool) |
Amendment A's "made available to screeners"; the "not yet screened" remainder |
| M3 Reviewed through this stage | Studies with at least one review start (ReviewStartEligibility) and one submitted decision or session version whose route provenance names this stage |
Review-start and submission events with route provenance | The reporting priority for stage flow views, with the eligibility justification attached |
| M4 Satisfied by evidence from elsewhere | Studies in the pool whose activities were already sufficient from project-wide evidence collected through another route | Current evidence and its route provenance | Shown separately; never counted as reviewed in this stage |
| M5 Departed | Studies that left the pool, by reason category | StagePoolDeparted with reason codes |
Audit; explanation of departures |
| Phase outcomes | Per PRISMA phase, the collective outcome of the phase-mapped profile | ScreeningOutcome (amendments A and H), accepted external decisions under their source policy |
PRISMA boxes for screening and full-text assessment |
Departure reason categories (codes owned by SP §3.5.3, all PROPOSAL): screening Exclude
outcome under a filter clause (SCREENING_OUTCOME_EXCLUDED); another outcome or accepted-answer
clause stopped matching or could not be evaluated (SCREENING_OUTCOME_CHANGED,
ACCEPTED_ANSWER_NOT_MATCHING, ACCEPTED_ANSWER_ABSENT, CLAUSE_UNKNOWN); filter reconfigured
(FILTER_RECONFIGURED); search withdrawn (HIDDEN_BY_SEARCH_WITHDRAWAL); merged or unmerged
(STUDY_NOT_CURRENT_MERGED, STUDY_NOT_CURRENT_UNMERGED); lifecycle no longer Active
(LIFECYCLE_NOT_ACTIVE). Project deletion writes no per-Study departure (SP §4.4; ACD §3.3). Reports
never label a departure a screening exclusion unless its reason is a screening Exclude outcome.
Excluded Studies and their reasons (Q-22, OS-A17; RS-R67 hands the counting here). Each
exclusion figure has two parts that are never added together: the number of distinct excluded
Studies, each counted once whatever its number of reasons, and the per-reason counts, which can
overlap because one Study may fail several criteria. Per-reason counts follow RS-R66: the primary
reason where the profile defines one, otherwise every reason answer. When any Study contributes to
more than one reason, the per-reason counts are labelled as overlapping, and their sum is never
shown as an excluded total. Label text is a PROPOSAL.
Proposed reading for the specialist (T-SI-05, not approved). Per PRISMA phase, "records
screened" means Studies with at least one counted decision under the phase-mapped profile
(human, or an accepted external decision under its source policy). M2 Studies with no counted
decision form the "not yet screened" remainder of identity I5. M1 Studies that were never released
appear only in audit views. Stage-level M3 drives SyRF's stage flow view and the eligibility
explanation. Until the specialist input is recorded, reports present the measures with their
labels and do not silently pick a different mapping.
Coverage. If pool tracking began after the project started, the report states the baseline date and that earlier membership is not reconstructed. A Study that was eligible and never reviewed shows "no review recorded"; SyRF never invents a reason for the absence.
3.4 External step ledger and previous-review counts¶
- What it is. The existing
ExternalStepLedgerof amendment K: counts for steps done outside SyRF (identification at source, deduplication, automation removal, other removal, title/abstract screening, retrieval, full-text assessment, studies from a previous review version), each with PRISMA fields, a non-negative count, timing,searchRound, tool, evidence note and author. - Previous-review counts (D4-11). The "studies from a previous review version" step type holds
explicitly supplied
previous_studiesandprevious_reports. When one exists, the diagram uses the updated-review template and box 16 adds new and previous. A field nobody supplied shows "not supplied". SyRF never derives previous-review counts from its own data, from a reference list or from an earlier snapshot. - Identity, versions, mutability. Entry ID; append-only; a correction supersedes and keeps the earlier entry. Frozen snapshots pin the entry versions they used.
- Storage.
pmExternalStepLedger(domain model). - What it is not. A source of derived fields #31 to #34; a way to report a count for records SyRF already screened at that phase.
3.5 Search withdrawal¶
- What it is. An appended
SearchWithdrawnevent on the search, with actor, time and reason, and a withdrawal state onSystematicSearch. Reinstating appendsSearchReinstated(PROPOSALname). - Effect on Studies. A Study identified only by withdrawn searches leaves current pools
through a
StagePoolDepartedevent with reason "search withdrawn" (HIDDEN_BY_SEARCH_WITHDRAWAL, SP §3.5.3) and leaves current reports. A Study also identified by a current search stays, and its identification counts come from the current search's records only. Review evidence on every Study stays intact and visible in history. External ledger entries tied to the search are withdrawn with it. - Storage.
StudyLifecycleLedgerentry plus the search's withdrawal state (domain model). - What it is not. Project deletion (ACD spec); a deletion of Citations or evidence.
3.6 Search documentation versions¶
- What it is. The PRISMA item 6 and 7 record of a search: database or source name, platform,
date searched, strategy text or attached strategy file, limits and filters, date range,
searchRoundandupdateOf. - Versions. Each save of the documentation creates a new immutable version with actor and time; the search points to its current version. Reports and the methods summary pin the version they used.
- Storage. An append-only
documentationVersions[]list onSystematicSearch(PROPOSAL; nameSearchDocumentationVersionsettled at the F1a naming ADR). Nullable fields follow the N-1 rule of the existing search document. - What it is not. Automated search execution or registry lookup (out of scope).
3.7 Protocol and registration record¶
- What it is. One project record holding registry (PROSPERO, OSF, other), registration ID, URL and date, the protocol document link and version, and an append-only amendment log.
- Amendment entry. Date, what changed, reason, the profile, form or filter version it corresponds to, actor and time.
- Link to publication. Publishing a screening-profile version that changes eligibility requires
an amendment entry in the same publication (D4-05). SyRF proposes "eligibility changed" when the
profile's eligibility questions, options, derived-decision rules, collective rule, Unsure
handling or source policy changed; the publisher confirms or overrides the suggestion with a
reason (
PROPOSALdetection). - Storage. A
ProtocolRegistrationdocument per project with embeddedProtocolAmendmententries (PROPOSALnames, settled at F1a). - What it is not. A protocol authoring tool; a registry integration.
3.8 Full-text retrieval¶
- What it is. Append-only
StudyLifecycleLedgereventsSought(date),Retrieved(how: PDF in SyRF or read externally) andNotRetrieved(reason from a controlled list plus free text, optional author-contact date), each with actor and time;Study.fullTextStatusis the current projection (amendment M). - Suggestion. Attaching a PDF (manual link, bulk PDF, study-source upload) raises a suggestion "PDF attached: mark as Retrieved?" that a human confirms. The suggestion is not an event.
- Storage.
pmStudyLifecycleLedgerand the projection on Study (domain model). - What it is not. A lifecycle state; FEAT-011's
FullTextNotRetrievedprecedence rule stays superseded.
3.9 Exports and estimated values¶
| Export | Plain English | Basis |
|---|---|---|
| Comparison-level analysis-ready export | One row per comparison and timepoint in the shape meta-analysis tools expect, with pairing from Experiment membership and control flags, dispersion type carried and never converted, extractionMethod, Study, report and group keys |
X1 (D4-09); gold by default, candidates optional; collectively Included Studies by default |
| Machine-readable codebook | For every exported column, the question identity, version, wording, options, semantic role, entity scope and requiredness, plus per-answer answeredUnderVersion and qualificationPolicy; for screening columns, the decision source types and AI model configuration versions used |
D4-09; E3 metadata ownership |
| RIS for a selected set | RIS built from Citation raw fields for any chosen set (included, excluded with reason, duplicates, not retrieved) | D4-09 |
| Screening export columns (added) | decisionSourceType, aiModelConfigurationVersion, externalRunId, confidence, thresholdApplied, onTrainingInput, and on outcomes machineContribution |
E3 AI expansion |
- Estimated-from-graph. Every observation carries
extractionMethod;graph-estimatedis the default when a PDF graph region is linked (C14, O1). Graph digitisation is a later decision after O1 pilots show how often a graph is the only source (D4-10). - Storage. Exports are generated under the C11 manifest rules; whether files are stored or regenerated is the C11 ADR's choice.
- What it is not. Effect-size computation, meta-analysis or plots inside SyRF.
3.10 Annotation-answer imports (later lane)¶
- What it is. FEAT-004's import of answers from other tools or spreadsheets into annotation sessions, in a lane after the first engine release.
- Rules carried from D4-14. Provenance kind
Importedwith source system, import job and mapped reviewer; target credit only when mapped to a SyRF reviewer; excluded from default independence statistics; never automatic gold; pinned to the current question version. - What it is not. The external screening path of §3.12 to §3.14, which counts by the profile's source policy instead of reviewer mapping.
3.11 AI screening model configuration (AIScreeningModelConfiguration)¶
- What it is. The project's own description of an AI screening model whose decisions it may import. One project can describe several models.
- Contents of a version. Model name; provider; model version or artifact reference; intended use; decision-label vocabulary of the model and the mapping of each label to Include, Exclude or Unsure; threshold configuration where scores are mapped to labels (the values the project supplies; SyRF proposes none); the training-set context (§3.14); default run context; links to documentation; and an explicit list of metadata items marked "not supplied".
- Identity and versions. A configuration ID with immutable versions and a per-configuration sequence; the head points to the current version for new references only. Every change creates a new attributable version. Profile versions, runs and decisions reference an exact version.
- Mutability. Versions are immutable. The head pointer moves. A retired configuration stays readable.
- Proposed storage.
pmAIScreeningModelConfigurationholding immutable version documents keyed by{ProjectId, ConfigurationId, Seq}(PROPOSAL). - What it is not. A SyRF user, project member, login or permission holder. A model-training or model-hosting service. A SyRF domain model (the terminology rule exists to avoid that confusion).
3.12 Screening source policy (ScreeningSourcePolicy)¶
- What it is. Part of an immutable
ScreeningProfileVersion. It declares each external source allowed to contribute screening decisions to that profile, and how. - Fields. Source type (
AIScreeningModel,ExternalHumanReviewer,ExternalNonAITool); for an AI source, the exactAIScreeningModelConfigurationversion; for other sources, a source descriptor (name, version, documentation) held in the policy (PROPOSAL); role (ContributingVoteorSoleScreener); scope (the Studies the role applies to, for example all Studies in the profile's applicable population, or one named import population); the label mapping in force; Unsure routing (to the profile's adjudication step). - Versions. Changing a policy is a new profile version published through the Q-26 impact process (RD and RS specs). Earlier decisions keep the profile version they were accepted under.
- What it is not. A reviewer. A target. A hard-coded extra vote.
3.13 External screening runs and decisions¶
ExternalScreeningRun. One delivery of outputs from one source.
| Field group | Contents |
|---|---|
| Source | Source type, policy source entry, exact model configuration version (AI) |
| Run identity | The source's run ID; SyRF run ID; supersedesRunId for a rerun or correction |
| Mapping | The profile version the run maps to; label mapping used |
| Provenance | Importer (authenticated user), imported-at time, source timestamps where supplied, source dataset and training-round provenance, file digest, row counts |
| State | Previewed, Validated, Accepting (n of N), Accepted, Rejected, Superseded |
| Validation results | Per-row outcome: matched, unmatched, unmapped label, duplicate row, out of scope, Study not current |
ExternalScreeningDecision. One source's decision on one Study for one profile in one run.
| Field group | Contents |
|---|---|
| Identity | Decision ID; (ProjectId, ProfileId, SourceKey, StudyId) plus a per-key version sequence |
| Content | Original label, mapped decision (Include, Exclude, Unsure), confidence or score where supplied, threshold applied where relevant |
| Study resolution | Source record identifier; how it was matched (SyRF Study ID, DOI, PMID, importer-confirmed title match); merge-lineage path when the identifier named a tombstoned original |
| Training context | onTrainingInput and onEvaluationInput flags against the configuration version's training-set record |
| Missing metadata | Explicit markers for anything the source did not supply |
- Versions and current pointer. A rerun or correction creates a new version for the same
(profile, source, Study)key; only the latest accepted version is current. Earlier versions stay in history. A source therefore holds at most one current decision per Study and profile. - Outcome composition.
ScreeningOutcomegainsmachineContribution∈ {None,Contributing,Sole} andexternalHumanContribution(boolean), derived at commit from the inputs the outcome rests on (PROPOSAL). - Outcome provenance, defined once (5 October harmonisation). A screening outcome's provenance
is kept separate from accepted-result authority (
AcceptedResultVersion.authority∈SingleAnnotator,HumanReconciled,Adjudicated,MergeResolved; RS §3.2). It has two parts: RS §3.8'sfinalSource(ProfileRule,Adjudicated,MergeResolved), which says how the outcome was resolved, and the composition fields above, which say which source classes it rests on. No extrafinalSourcevalue is needed for external or AI decisions: underContributingVoteorSoleScreenerthey resolve throughProfileRule, and a human resolution of a model Unsure throughAdjudicated. The earlier single list (CandidateAgreement,Reconciled,Adjudicated,Admin,Imported,LegacyUnknown) is retired for outcomes. It maps as follows:CandidateAgreement→ProfileRule;Reconciled(profile adjudication) →Adjudicated;Imported→ProfileRuleorAdjudicatedwith the composition fields set (andImportedstays a candidate provenance kind, §3.10);LegacyUnknown→ removed (Q-35; converted legacy outcomes are recomputed under the legacy-compatible profile, BC §3.4);Admin→ no value unless RS §12's Q-36 reading creates an outcome override. Legacy-gap states (BC §3.2) are a different thing and stay. - Proposed storage.
pmExternalScreeningRunandpmExternalScreeningDecision, keyed by project and run, with the current-pointer index on the decision key (PROPOSAL). - What they are not. Candidate sessions of a SyRF reviewer; votes from a person; accepted results.
3.14 Model training-set context¶
- What it is. Part of an
AIScreeningModelConfigurationversion: a description of the human screening used to train or evaluate the model, and, where known, a frozen list of the Studies and the exact human decision versions in the training and evaluation sets. - Rules. Outputs on training or evaluation inputs are flagged on each decision. They are not independent validation evidence of the model. They do not add the underlying human decisions a second time as new observations.
- Storage. Embedded in the configuration version, with the Study list as a referenced
attachment when large (
PROPOSAL). - What it is not. The S4 reviewer training step, its reference answers or its attempts (TI spec). It is not a model-training service.
3.15 Deduplication QC and parity evidence¶
- What it is. The P2 controls from amendment L that Q-37 approved: a QC sample of AutoConfirmed
groups shown for human confirmation; a reviewer action "Flag as possible duplicate of…" creating a
DuplicateReviewItem; the deduplication manifest (algorithm version, tier rules version, auto-confirmed versus reviewed share, reversals, QC sample result); the Publication privacy rule; and a pinned ASySD parity suite. - Changed by the owner session. A confirmed merge produces one consolidated current Study with reversible unmerge (DM spec), replacing the alias rule in amendment L rule 4.
- Thresholds. The QC share (5%, at least 20 groups), the parity pass conditions (identical AutoConfirmed groups, ProbableDuplicate pair-set F1 ≥ 0.99, sensitivity and specificity within 0.5 percentage points) and the performance target (80,000 citations in under an hour on Bramble) are proposed, not approved, and not achieved (D4-21).
- Storage.
pmDuplicateReviewItem,pmDedupAuditLedger(domain model); parity evidence as committed golden outputs in repository test data.
3.16 Agreement results store (AgreementResult)¶
- What it is. Reviewer-agreement figures per project, form or profile version, method version and study set, computed from canonical revisions by a bounded background job.
- Separation. Its own rebuildable store with a source watermark and a measured budget (D3-11). It is never a FEAT-024 family; FEAT-024 continues to exclude kappa.
- Source classes. Human independent (initial independent observations), human informed, external human, and machine source. Each class is shown separately; no figure mixes human and machine observations without an approved statistical definition.
- Storage.
pmAgreementResult, derived and rebuildable (domain model). - What it is not. A source of truth; an input to PRISMA; a progress statistic.
3.17 Statistics at the publication boundary¶
- What it is. The record of the statistics read a publication relied on: projection revision, source revision and digest of a FEAT-024 read that was Materialised-Fresh or pinned-Authoritative, or the authoritative count under Q-31(b), stored in the publication manifest (RD spec owns the publication command).
- Storage. A
pinnedStatisticsReadblock in the publication manifest (PROPOSAL). - What it is not. A new statistics family; a cached figure reused later.
4. What loads and writes when¶
Commands follow C18 (one Study-scoped transaction per Study, hybrid-logical-clock stamps, idempotency by command ID) and C19 (durable effects after commit).
W1. Priya freezes a PRISMA report.
| Aspect | Detail |
|---|---|
| Reads | All inputs listed in §3.2 at a watermark that satisfies the C11 as-of rule; the arithmetic identities I1 to I13; recorded mismatch explanations |
| Writes | One PrismaFlowSnapshot with its manifest and digests; a HistoryEvent "report frozen" |
| Transaction boundary | A single insert of the snapshot; the computation reads a causally closed cut and writes nothing else |
| Derived afterwards | Nothing changes in place. The snapshot list shows the new one as current; older snapshots show "superseded by snapshot n" in their header, computed on read |
W2. Priya enters or corrects an external count, including previous-review counts.
| Aspect | Detail |
|---|---|
| Reads | The search or project, existing ledger entries, SyRF decisions at the same phase (for the double-count refusal) |
| Writes | A new ExternalStepLedger entry; for a correction, the new entry names the one it supersedes |
| Transaction boundary | One ledger insert; refused before writing if the phase already has SyRF decisions for those records |
| Derived afterwards | The consistency check (identified at source minus removals equals imported) runs on read and shows a warning. Current report views recompute on read. Frozen snapshots are untouched |
W3. Priya withdraws search B, then reinstates it.
| Aspect | Detail |
|---|---|
| Reads | The search, its Citations, the Studies they identify, which of those Studies are identified by other current searches, active work on affected Studies (§7) |
| Writes | SearchWithdrawn on the search and its state; withdrawal of its ledger entries |
| Transaction boundary | The search document and its ledger entries in one transaction; per-Study pool re-evaluation follows as targeted processing (SP spec) |
| Derived afterwards | StagePoolDeparted events (reason "search withdrawn") for Studies no longer identified by any current search, written by ordered, idempotent processing. Current reports exclude the search and show the explanation line. Reinstating appends SearchReinstated and the same processing writes StagePoolEntered events where the filters match again |
W4. Priya corrects a search's documentation.
| Aspect | Detail |
|---|---|
| Reads | The current documentation version |
| Writes | A new documentation version with actor and time; the search's current pointer moves |
| Transaction boundary | The search document, with a base-version check |
| Derived afterwards | The methods summary of later snapshots uses the new version; frozen snapshots keep the version they pinned |
W5. Priya publishes a profile version that changes eligibility (reporting part only).
| Aspect | Detail |
|---|---|
| Reads | The draft and current profile versions; SyRF's "eligibility changed" suggestion; the protocol record |
| Writes | The ProtocolAmendment entry, inside the same publication operation that issues the profile version (RD and RS specs own the rest) |
| Transaction boundary | The amendment entry commits with the publication's final activation step; a publication without a required amendment is refused before any write |
| Derived afterwards | The methods summary of later snapshots lists the amendment with the profile version it belongs to |
W6. Tom attaches a PDF; Mei records Retrieved; later another Study is marked Not retrieved.
| Aspect | Detail |
|---|---|
| Reads | Study, current fullTextStatus, the caller's grant on a stage using the Study |
| Writes | Attaching writes only the PDF link and raises a suggestion. Mei's confirmation writes a Retrieved event; a Not retrieved action writes NotRetrieved with its reason |
| Transaction boundary | The Study and its ledger entry in one Study-scoped transaction |
| Derived afterwards | fullTextStatus projection; full-text step admission re-evaluated on read; PRISMA boxes 6, 7 and 8 computed in later snapshots |
W7. Ravi describes an AI screening model, then corrects its threshold note.
| Aspect | Detail |
|---|---|
| Reads | Existing configurations in the project |
| Writes | Version 1 of the configuration; later version 2 with the correction, actor and time |
| Transaction boundary | One version insert plus the head pointer move, with a base-version check |
| Derived afterwards | Nothing changes for existing profile versions, runs or decisions; they keep referencing version 1. The configuration page shows "version 2 is current; profile v3 still uses version 1" |
W8. Priya publishes a profile version with a source policy.
| Aspect | Detail |
|---|---|
| Reads | The draft profile version, the referenced model configuration version, existing decisions and outcomes on earlier profile versions (Q-26 impact), active work |
| Writes | The new ScreeningProfileVersion with its ScreeningSourcePolicy; the protocol amendment entry (§3.7); the publication record |
| Transaction boundary | The RD and RS publication protocol; the source policy is part of the immutable profile version |
| Derived afterwards | Outcome re-evaluation under the chosen Q-26 treatment; no external decision participates until a run is accepted against this version |
W9. Ravi uploads and validates a run.
| Aspect | Detail |
|---|---|
| Reads | The file; the profile version and its source policy; Study identities (SyRF ID, DOI, PMID, merge lineage); existing runs with the same source run ID or file digest |
| Writes | An ExternalScreeningRun in state Previewed, then Validated, with per-row validation results; no decisions become current |
| Transaction boundary | The run document only. Re-uploading the same file returns the existing run (idempotent by source, source run ID and file digest) |
| Derived afterwards | A preview listing matched rows, unmatched rows, unmapped labels, out-of-scope rows, Studies whose outcome would change and active-work impact (§7) |
W10. Priya accepts the run.
| Aspect | Detail |
|---|---|
| Reads | Per Study at commit: Study current state (re-resolved through merge lineage if needed), the profile's current version, the source's current decision on that Study, other decisions, the profile rules |
| Writes | Per Study: the new ExternalScreeningDecision version as current; the Study summary; a changed ScreeningOutcome with its composition; an AdjudicationTask when the rules route the result to adjudication; lifecycle transitions through StudyLifecyclePolicy; HistoryEvents for the outcome change. Run level: progress counter and final state |
| Transaction boundary | One Study-scoped transaction per Study, keyed by (runId, StudyId, decisionVersion) so a retry never duplicates; the run is an operation with resumable progress. No half-accepted Study can exist |
| Derived afterwards | Targeted pool re-evaluation of filters that depend on the profile, writing pool events with cause "accepted external screening decision" (ExternalScreeningAccepted, SP §3.12); notices to affected active reviewers (C19); agreement and statistics recompute under their own rules |
W11. Ravi reruns the model and imports the corrected output.
| Aspect | Detail |
|---|---|
| Reads | As W9 and W10, plus the superseded run |
| Writes | A new run with supersedesRunId; on acceptance, new decision versions; the old run becomes Superseded when all its rows are superseded |
| Transaction boundary | As W10 |
| Derived afterwards | Outcomes re-evaluated; the number of voters is unchanged because each source holds one current decision per Study |
W12. A model Unsure goes to adjudication.
| Aspect | Detail |
|---|---|
| Reads | The adjudication task, the exact version of the AI-model-generated screening decision (label, confidence, threshold, configuration version), any other candidate decision versions, the profile's blinding policy |
| Writes | An attributable Adjudicated outcome version listing its exact inputs; the adjudicator's identity; the rationale if the profile requires one |
| Transaction boundary | Study-scoped transaction (RS spec) |
| Derived afterwards | Pool re-evaluation; the AI-model-generated decision is unchanged; the outcome's machineContribution stays Sole or Contributing because the model output is one of its inputs |
W13. Priya generates an export.
| Aspect | Detail |
|---|---|
| Reads | A causally closed cut at the export watermark; definitions, versions and the export disclosure contract (C10) |
| Writes | The export job and its ExportManifest; files per the C11 ADR |
| Transaction boundary | Read-only on review data |
| Derived afterwards | Nothing; two exports at one watermark for the same requester authority are identical for versioned datasets |
W14. Dr Okafor opens the agreement view.
| Aspect | Detail |
|---|---|
| Reads | AgreementResult for the selected form or profile version and method version, with its watermark; the current source watermark |
| Writes | Nothing on read; a background job writes a new result when the source watermark has moved |
| Transaction boundary | The job writes one complete result per key; readers never see a mix of watermarks |
| Derived afterwards | The view labels freshness ("computed to 14:02; newer decisions are being included") |
W15. A form publication reads statistics at its boundary (D3-10).
| Aspect | Detail |
|---|---|
| Reads | Inside the publication fence, a FEAT-024 read that is Materialised-Fresh or pinned-Authoritative, or an authoritative count under Q-31(b) for named pilots or preview |
| Writes | The read's identity into the publication manifest |
| Transaction boundary | The publication's own protocol (RD spec) |
| Derived afterwards | Nothing; later statistics changes never alter the recorded read |
5. Rules¶
Reporting units and honesty
- RI-R01 Reports, exports and screens keep imported references, source documents and Studies as distinct units and label each count with its unit. Owner decision: Q-06b (E1), Q-23; consolidation §5.
- RI-R02 Report counts use established source-document identity. Where identity is partial or
missing, the figure carries its coverage and is labelled as records or as "one report per Study
assumed (identity not verified)". Owner decision Q-06b (amendment B); label text
PROPOSAL. - RI-R03 Study versions may hold several reference links (prepared multi-source links). No workflow groups distinct reports or investigations in this rollout, and no operation merges distinct reports silently. A conference abstract and its journal article stay separate Studies. Owner decision D4-08; consolidation §1, §2.
PRISMA priority and pool history
- RI-R04 Flow reporting prioritises Studies actually reviewed through a stage (M3) together with the eligibility justification recorded at review start. Ever-in-pool membership (M1) is kept as separate audit data and is never the reviewed count. Owner clarification, register "reporting priority is actual review through the pool"; consolidation §3.
- RI-R05 A filter departure is never reported as a screening exclusion; departures carry their reason category. Owner requirement, register "historical pool coverage"; consolidation §3.
- RI-R05a Exclusion reporting shows the count of distinct excluded Studies separately from the
per-reason counts. Per-reason counts may overlap, are labelled as overlapping when they do, and
are never summed into or presented as the excluded total. Where a primary reason is defined, each
excluded Study appears at most once in the primary-reason breakdown, and Studies without a
recorded primary reason are shown as such. Owner decision Q-22 (S3 clarification), OS-A17;
label text
PROPOSAL. - RI-R06 Evidence collected through another route that satisfies a stage is shown as satisfied elsewhere (M4) and never counted as reviewed in that stage; SyRF never creates a stage-specific review event to explain it. Owner clarification; stage filter model "Already sufficient evidence".
- RI-R07 Every measure counts distinct current Studies: re-entry never adds a Study, and tombstoned originals of a merge are never extra current Studies. Owner requirement (historical pool coverage); DM spec for lineage.
- RI-R08 Reports disclose the start of pool tracking and any baseline gap. Pre-tracking history is never fabricated. An eligible Study with no review shows "no review recorded" with no invented reason. Owner decision; consolidation §3.
- RI-R09 The mapping of these measures onto exact PRISMA boxes is specialist input (
T-SI-05) recorded before F6b. Until then, reports show the measures with their labels and the §3.3 reading is a proposal, not approved. Boundary from the consolidation §7 closing paragraph.
Frozen reports
- RI-R10 A frozen snapshot never changes; regenerating it gives identical numbers and digests. Corrections, amendments, withdrawals, merges and unmerges append and appear only in later snapshots. Owner decision Q-06b (amendment F), Q-33.
- RI-R11 Snapshots are computed only from authoritative records at a watermark; FEAT-024 rows
and FEAT-024 history are never inputs.
PROPOSAL(MS-11, existing package). - RI-R12 Each snapshot discloses missing information: reason coverage, retrieval not recorded, report identity coverage, pool-tracking coverage, reported external parts and machine-assisted share. Owner decision Q-06b; E3 AI expansion for the machine-assisted share.
Updated reviews and the external ledger
- RI-R13 Previous-review counts come only from explicitly supplied values in the ledger; the updated-review template switches on when such an entry exists; an unsupplied field shows "not supplied"; the full updated-review workflow stays deferred. Owner decision D4-11.
- RI-R14 Amendment K's ledger rules 1 to 9 apply: reported and computed parts kept apart in every manifest; derived fields #31 to #34 never reported; entry phase per search or import; corrections supersede. Owner decision Q-37.
- RI-R15 An external count is refused for records that already have SyRF decisions at that
phase, including accepted external and AI-model-generated decisions. Owner decision Q-37 (rule 6);
its extension to accepted external decisions is
PROPOSAL. - RI-R16 Entering or correcting external counts needs the PRISMA report capability; every change is audited; a consistency mismatch warns and never blocks entry, and blocks freezing until an administrator records an explanation. Owner decision Q-37; capability per the approved Q-03 matrix.
Search withdrawal
- RI-R17 Withdrawal appends an event, keeps Citations and evidence, and removes affected Studies from current pools through departure events with reason "search withdrawn". Reinstatement appends its own event. Owner decision Q-33; amendment J.
- RI-R18 Current reports exclude withdrawn-search records and say so ("Excluded from this
report: n records from withdrawn search B; m Studies are no longer identified by any current
search"). A Study also identified by a current search stays and is counted from that search's
records. Owner decision Q-33 (exclusion and explanation); the counting detail is
PROPOSAL.
Methods documentation
- RI-R19 Search documentation is versioned; each change creates a new version; snapshots and the methods summary pin the version they used. Owner decision D4-05.
- RI-R20 Each project has a protocol and registration record with an append-only amendment log. Owner decision D4-05.
- RI-R21 Publishing a screening-profile version that changes eligibility requires an amendment
entry. SyRF suggests whether eligibility changed; the publisher confirms or overrides with a
reason. Owner decision D4-05 (requirement); detection
PROPOSAL. - RI-R22 A change to a profile's source policy (adding or removing an external source, or
changing its role or scope) counts as an eligibility and selection-method change and requires an
amendment entry.
PROPOSAL.
Full-text retrieval
- RI-R23 Retrieval is recorded only by explicit Sought, Retrieved and Not retrieved actions with actor, time and reasons; retrieval status is separate from lifecycle. Owner decision D4-07.
- RI-R24 A PDF attachment raises a suggestion that a human confirms; it never records Retrieved by itself. Owner decision D4-07.
- RI-R25 Authorised administrators and reviewers with a grant on a stage using the Study can
record retrieval actions. Owner decision D4-07; capability name
PROPOSAL. - RI-R26 A collective title/abstract Include sets Sought through a system rule recorded with the
system actor.
PROPOSAL(amendment M default).
Exports
- RI-R27 Lane X1 delivers the comparison-level export, the codebook and RIS for a selected set; SyRF computes no effect sizes. Owner decision D4-09.
- RI-R28 Observations carry
extractionMethodwithgraph-estimatedas the default for a linked graph region in O1; graph digitisation is decided after O1 pilots. Owner decision D4-10. - RI-R29 Exports carry the exact definition, profile, model configuration and run versions used, plus the machine-source columns of §3.9; current settings are never substituted for the settings in force when a decision was imported. Owner decision, E3 AI-model metadata ownership.
Annotation-answer imports
- RI-R30 Annotation-answer imports keep provenance, earn target credit only when mapped to a SyRF reviewer, stay out of default independence statistics, never become gold automatically and pin the current question version. Owner decision D4-14 (E3).
- RI-R31 No importer is activated by this approval; each import lane needs its own brief, gate and flag. Owner decision; consolidation §5.
AI-model-generated screening decisions
- RI-R32 Documents and user-facing attribution say "AI-model-generated screening decision" and "AI screening model". Non-AI external sources keep their real source type. Owner decision, explicit AI-model terminology.
- RI-R33 The machine source has its own identity, separate from the importing user. It has no login, no membership and no permissions; attributing a decision to it grants nothing. Owner decision, E3 expansion approval and source identity.
- RI-R34 The project keeps versioned
AIScreeningModelConfigurationrecords holding the metadata in §3.11; anything not supplied is marked as such. Owner decision, E3 metadata ownership. - RI-R35 A profile version pins the exact configuration version and its source policy. Policy changes follow the profile publication and impact process and preserve earlier decisions and outcome history. Owner decision, E3 expansion; Q-26.
- RI-R36 A source counts only under its declared role,
ContributingVoteorSoleScreener, and only within its declared scope. Owner decision, E3 expansion. - RI-R37 Reruns and corrections create new versions of the same source's decision. Each source holds at most one current decision per Study and profile; repeated outputs never become extra independent screeners. Owner decision, E3 expansion.
- RI-R38 Label mapping to Include, Exclude and Unsure is explicit. A row whose label has no mapping is refused at validation. Thresholds are the values the project supplies; SyRF sets no default threshold. Owner decision (explicit mapping); thresholds per the consolidation boundary on unapproved numbers.
- RI-R39 Under a sole-screener policy a model Unsure is unresolved and enters the profile's human adjudication step (RS-R59; SP §3.4 step kinds). Not-excluded availability never makes it a collective Include, and pending adjudication is not an outcome. Owner decision, E3 expansion and S1 amendment.
- RI-R40 A human resolution creates a new attributable outcome whose inputs are the exact version of the AI-model-generated screening decision and any other candidate versions. The AI-model-generated decision is never changed. Owner decision, E3 expansion.
- RI-R41 As a contributing vote, an AI-model-generated screening decision follows the profile's versioned sufficiency, conflict and Unsure rules with their bounded escalation; there is no hard-coded extra vote. Owner decision, E3 expansion.
- RI-R42 Imports are previewed, validated and idempotent. Each source record resolves to a
current Study with recorded provenance; an identifier of a tombstoned original resolves through
merge lineage and records the path. Unmatched rows are reported and never create Studies. Owner
decision (preview, validation, idempotence, identity provenance); unmatched-row handling
PROPOSAL. - RI-R43 Imported decisions affect current outcomes and pool filters only after validated acceptance under the declared policy, and only for Studies and profile versions that are current under the ordinary eligibility, publication and history rules. Acceptance triggers the structured membership and impact history. Owner decision, E3 expansion.
- RI-R44 The configuration records training and evaluation membership with source versions.
Outputs on those inputs are flagged, are never independent validation and never add the
underlying human decisions again. By default they do not count toward sufficiency or agreement.
Owner decision (provenance and interpretation); the default exclusion is
PROPOSAL. - RI-R45 Reports, exports and the methods summary keep machine-assisted outcomes distinguishable from exclusively human outcomes, using the outcome composition fields. Owner decision, E3 expansion.
- RI-R46 Agreement statistics keep human independent, human informed, external human and machine
classes apart; no metric mixes them without an approved statistical definition. Owner decision,
E3 expansion; methods via
T-SI-02. - RI-R47 A permission-aware page shows model configurations, the profile versions and runs that use them, and each decision's provenance; exports and the methods summary expose the same method details. Owner decision, E3 metadata ownership.
Deduplication
- RI-R48 Deduplication preserves every import, keeps non-current records out of ordinary admission, applies the Publication privacy rule, offers a QC sample and a reviewer duplicate flag, records the manifest fields and validates parity; confirmed merges follow the consolidated Study model of the DM spec. Owner decision Q-37 (with D2-12 replacing the alias rule).
- RI-R49 Parity thresholds and the performance target are proposed, not approved; the release
decision needs the specialist parity method (
T-SI-04). Brief item D4-21.
Agreement and statistics
- RI-R50 Agreement starts with percent agreement and explicit denominators; multi-rater formulas ship only after statistical review; the default basis is initial independent observations; screening agreement is per profile with prevalence shown; an accepted or reconciled result is never an extra independent observation. Brief items Q-16 and D4-12 (recommendations carried as the treatment).
- RI-R51 Agreement figures live in a separate rebuildable store with a source watermark, a method version and a measured budget, outside progress statistics. Brief item D3-11.
- RI-R52 A publication boundary accepts a pinned authoritative statistics read as current evidence only when its source identity is recorded in the manifest. Brief item D3-10 (a).
- RI-R53 Project 0102, the existing statistics pilot, stays out of overlapping pilots until the canonical-commit-with-statistics check (C8-T07) passes. Brief item D3-10 (b).
- RI-R54 Multi-profile pilot statistics are served at profile granularity, live for R3b pilots until profile-grain families exist at F5. Brief item D3-10 ©.
- RI-R55 Preview pilots are exempt from the production materialisation dependency and use truthful authoritative counts. Brief item D3-10 (d).
- RI-R56 Production statistics readiness stays provisional: gate (b) failed on latency on 3 October 2026 (#3510), the X-STATS-b1 to b7 chain still applies, and the production target date after the 5 October staging activation is a target, not a commitment. Brief item D3-10; D1-03.
Specialist-dependent schemas and templates
- RI-R57 The event-count outcome schema is built only after a field-level scientific
specification is recorded (
T-SI-01); catalogue infrastructure and the legacy-compatible schema may come first. Brief item Q-17. - RI-R58 SYRCLE, CAMARADES and ARRIVE Essential 10 templates are published to the catalogue only
after methodologist verification of content and applicability (
T-SI-03). Brief item D4-06.
6. Authorisation, blinding and provenance¶
Capabilities. Each capability ships with the feature that uses it (Q-03 rule). Names are
placeholders (PROPOSAL).
| Action | Capability | Notes |
|---|---|---|
| Enter or correct external counts; freeze a PRISMA report | PRISMA report (approved Q-03 matrix) | Audited |
| Withdraw or reinstate a search | Manage searches | Shows active-work impact first (§7) |
| Edit search documentation; manage the protocol record | Manage searches; Manage protocol record | Versioned and audited |
| Record full-text retrieval | Record retrieval (administrators; reviewers with a stage grant) | D4-07 |
| Create or revise an AI screening model configuration | Design screening (profile design) | Never grants the model anything |
| Publish a profile version with a source policy | Publish (approved Q-03 subset) | Q-26 impact process |
| Upload and validate an external run | Import external screening decisions | Validation writes no current decisions |
| Accept a run | Accept external screening decisions | Separate from upload so a second person can accept (PROPOSAL) |
| View model metadata and decision provenance | View screening configuration | Read-only page |
| View agreement figures | Agreement capability (AG1) | Shows no candidate answers |
Authentication. Every command checks the caller at the server. An AI screening model never authenticates. An automated integration that delivers runs, if one is ever approved, uses an ordinary authorised service identity under the application authorisation contract; the model's attribution stays separate from that identity (E3 source identity).
Blinding.
- Human candidate identities follow the profile's reconciliation blinding: context-local labels, unpredictable order and no identity continuity across Studies (RS spec).
- Before an outcome exists, an external or AI decision is a candidate contribution: reviewers browsing their reviewer study pool never see it, exactly as they never see other reviewers' candidate decisions (O4 browsing clarification).
- Adjudicators see the exact AI-model-generated decision with its label, confidence and threshold. Whether the source type is shown under blinded adjudication is an open design point (§12, ambiguity A2).
- Agreement views show figures only; they never reveal candidate answers (AG1).
- Exports follow the disclosure contract (C10); model attribution is never an identity disclosure because the model is not a person.
Provenance. Every record carries real actor, on-behalf-of, command ID and stamp (C3). External decisions carry both the machine source (configuration version, run) and the importer and accepter as separate fields. Snapshots and exports pin every version they used. Structured history events follow C20.
7. Active-work impacts¶
| Action | Warned before commit | Rechecked at commit | Notified after | Apply anyway |
|---|---|---|---|---|
| Accept an external run | Studies whose outcome would change; pool entries and departures that follow; Studies with active reviewers or reconcilers (counts, names only with the Monitor capability, D3-20); dependent accepted results whose inputs change (Q-27) | Each Study's current state, profile version and source decision; rows whose inputs moved stop with a typed outcome and stay unaccepted | Affected active reviewers and authors of dependent work, with a concrete explanation suited to their access | Not offered; acceptance releases no reservations. Started work after a departure follows the continuation rules (SP spec) |
| Accept a rerun | As above, plus which current decisions the rerun replaces | As above | As above | Not offered |
| Withdraw a search | Studies that will leave current pools; active reviewers; drafts preserved; continuation rules | Which Studies are still identified by another current search | Affected active reviewers | Not offered |
| Publish a profile version that adds or changes a source policy | The Q-26 impact on existing decisions and outcomes; the amendment entry needed | The RD and RS publication recheck | Per the RD and RS specs | Per the ACD spec's general rule; never bypasses permissions or legal configuration |
| Record Not retrieved on a Study in a full-text step | Reviewers with started full-text work on that Study | Current retrieval status | Those reviewers | Not offered; drafts kept; continuation per SP spec |
| Revise a model configuration | Which profile versions use the old version (none change) | Base version | Nobody | Not applicable |
| Correct an external count | Frozen snapshots that used the superseded entry (they stay unchanged) | Base entry | Nobody | Not applicable |
The general preview pattern, wording and Apply anyway limits are specified in the AC and UX specs.
8. Failure and recovery¶
| Failure | What the user sees | Recovery |
|---|---|---|
| File cannot be parsed or is too large | "This file could not be read" with the line or size limit | Fix and re-upload; nothing written |
| Rows with unmapped labels, unknown Studies or out-of-scope Studies | Per-row results in the preview; counts per category | Fix the mapping in a new configuration version or the file; accept only valid rows (PROPOSAL) or reject the run |
| Profile republished between validation and acceptance | Rows stop with "profile version changed" | Re-validate against the current version; Q-26 rules decide how earlier mappings are treated |
| A Study merged or unmerged between validation and acceptance | Rows re-resolve through merge lineage; an unmerged Study stops with "needs re-resolution" | Importer confirms the new resolution; the path is recorded |
| Acceptance interrupted | Run shows "Accepting (n of N)" | Resume; per-Study idempotency keys prevent duplicates |
| Run attributed to the wrong configuration version | Detected after acceptance | Import a correction run with the right version; the wrong run is superseded and labelled; decisions are never edited |
| Snapshot fails an arithmetic identity | Freezing is blocked with the identity and its remainder | Record an explanation, or fix inputs and recompute |
| A regenerated frozen snapshot differs from its digest | An operator alert; the report shows "verification failed" | Investigate as a defect; never overwrite the snapshot |
| Agreement store lost or suspect | View shows "being rebuilt" | Rebuild from canonical revisions under a new watermark |
| Statistics stale at a publication boundary | The publication command takes a pinned authoritative read | Recorded in the manifest (RI-R52) |
| Restore after data loss (D2-13) | Manifests and reports record the history discontinuity | BC spec's recovery procedure |
| External count entered against SyRF-screened records | Refused with the reason | Enter counts only for records not screened in SyRF |
9. User flows and examples¶
The cast is fictional. Priya Shah administers the project "Neuroprotection in rodent stroke". Tom Reid and Mei Lin are reviewers. Ravi Patel runs the team's AI screening model outside SyRF. Dr Ada Okafor is a CAMARADES methodologist.
9.1 Reading a PRISMA report.
- Loading. Priya opens Data › PRISMA. A skeleton diagram shows while the current view is computed.
- Empty. In a new project with no searches the page says "No searches yet. Import a search or record identification counts to start the report."
- Pending. Mid-review, box "Records screened" shows 1,240 and a footnote "180 records released but not yet screened; 22 with unresolved outcomes (pending, conflict or awaiting adjudication)".
- Units. Hovering "Reports assessed" shows "Source documents. Report grouping across Studies was not performed; each Study is counted as one report."
- Freeze. Priya freezes the report for her protocol update. Identity I6 fails by 3; the dialog lists the three Studies (two awaiting adjudication, one with a conflict) and asks for an explanation. She records it and the snapshot freezes.
- Historical. A month later she opens the frozen snapshot. Its numbers are unchanged; the header says "Superseded by snapshot 4 (12 November)".
- Failure. If verification of a frozen snapshot ever fails, the header shows "Verification failed; contact support" and the figures are not presented as verified.
9.2 Updated review. The project updates a 2023 review. Priya adds a ledger entry "studies from a previous review version": previous studies 46; previous reports left blank. The diagram switches to the updated-review template. Box 16 shows "46 + 12 new = 58 studies"; the previous-reports cell shows "not supplied".
9.3 Withdrawing a search. Priya finds that search C ran with the wrong date limits. She withdraws it. The preview says 310 Studies came from search C; 260 are also identified by search A or B and stay; 50 will leave the current pools; two reviewers have started work on 3 of them and may finish under the continuation rule. After confirming, the current report shows "Excluded from this report: 498 records from withdrawn search C; 50 Studies are no longer identified by any current search". The frozen snapshot from last month still includes search C.
9.4 Explaining departures. In the full-text stage history Priya sees 14 Studies departed. The report breaks them down: 9 by a title/abstract Exclude outcome (after a corrected decision), 3 because a reconciled species answer no longer matched the filter clause "species = rat or mouse", and 2 because search C was withdrawn. Only the 9 appear as screening exclusions. Six Studies sat in the pool and were never reviewed because the stage was paused; they show "no review recorded".
9.5 Full-text retrieval. Tom attaches a PDF to S-204. A banner says "PDF attached. Mark full text as Retrieved?" Tom is busy and leaves it; the status stays Sought. Mei later confirms Retrieved. For S-219, Mei records Not retrieved with "author contacted, no response" and the date. In the next snapshot S-219 appears in boxes 6 and 7, not in box 8, and the export lists its reason.
9.6 Protocol amendment. Priya edits the title/abstract profile to exclude in vitro studies and publishes. SyRF marks the change "eligibility changed" because a derived-decision rule changed. Publishing asks for the amendment entry; she writes "Added in vitro exclusion after pilot screening" and links her registry update. The methods summary of the next snapshot lists the amendment with profile version 3.
9.7 AI model as a contributing vote.
- Training set. Tom and Mei screen 600 Studies in SyRF as usual. Ravi trains the team's classifier on those decisions outside SyRF.
- Configuration. Ravi creates "StrokeScreen classifier" version 1: provider "CAMARADES team",
version "2.1.0", intended use "title/abstract screening of rodent stroke studies", labels
include,exclude,uncertainmapped to Include, Exclude, Unsure, thresholds "score ≥ 0.80 include; ≤ 0.20 exclude; otherwise uncertain" (the project's values), and the training set as the 600 Studies with their decision versions. He marks "evaluation set" as not supplied. - Policy. Priya publishes profile version 4 with a source policy: StrokeScreen v1 as a
ContributingVotefor all remaining Studies. The profile rule is "two agreeing definite decisions". The amendment entry records the change of selection method. - Import. Ravi uploads the run for 4,000 Studies. The preview shows 3,990 matched, 6 unmatched (reported, not created), 4 already tombstoned originals resolved to their consolidated Studies, and 600 outputs flagged "on training input" that will not count toward sufficiency.
- Acceptance. Priya reviews the impact (no active reviewers affected) and accepts. Each Study now
holds one AI-model-generated screening decision. Tom then screens S-1301 and agrees with the
model's Include: two agreeing decisions, so the outcome is Included with
machineContribution = Contributing. - Conflict. On S-1302 Mei says Exclude and the model said Include. The profile routes conflicts to adjudication; Dr Okafor resolves it with the AI-model-generated decision and Mei's decision as inputs. The AI-model-generated decision itself is unchanged.
9.8 AI model as sole screener with Unsure. A second project uses StrokeScreen as
SoleScreener for a low-priority sub-question profile. Model Include and Exclude become outcomes
directly, labelled machine-only. Model "uncertain" on 140 Studies maps to Unsure; those Studies are
not excluded for availability, are not Included, and each gets an adjudication task for the
"Screening adjudicators" group. When a reviewer asks why a Study shows as pending, the
eligibility explanation answers "Awaiting adjudication of an AI-model-generated screening decision
(Unsure)".
9.9 Rerun. Ravi retrains the model (configuration version 2) and reruns it on the 140 Unsure Studies. The new run supersedes those rows. Studies whose adjudication is still pending get the new decision as current; the pending task shows "input changed: the AI screening model output was replaced", and the adjudicator sees both versions in history. Where adjudication had already resolved, the adjudicated outcome stays current, flagged "inputs changed", and the adjudicator is told; nothing changes until an adjudicator explicitly reconsiders it (RS-R58). The number of voters per Study stays one model plus the humans.
9.10 Exports. Priya exports screening decisions. Each row shows decisionSourceType
(Reviewer or AIScreeningModel), the configuration version, run ID, confidence and the threshold
applied; outcome rows show machineContribution. The codebook lists StrokeScreen versions 1 and 2
with their metadata as recorded at import, even though version 2 is current. The RIS export of
"excluded at title/abstract" contains the Citation raw fields of those Studies.
9.11 Agreement. Dr Okafor opens the agreement view. Human-independent percent agreement for the title/abstract profile is 91% (denominator 600 Studies with two initial independent decisions; Include prevalence 18%). Model-versus-human figures appear in a separate "machine source" panel marked "method pending statistical review" and exclude the 600 training-input Studies.
9.12 Statistics at publication. Priya publishes a form version on a staging pilot. The publication manifest records "statistics read: pinned authoritative, projection revision 812, source revision 4415, digest 9c1e…". On a preview environment the same publication records an authoritative count under Q-31(b).
10. Rollout and adoption¶
| Capability | Class | Release or lane | Flag decision | Dependencies |
|---|---|---|---|---|
| Reporting units, report identity coverage and labels | Baseline/MVP | P1 (Citations), P2 (Publication identity), R5a and R5b (labels) | Within each release's flag; reporting paths change observable output, so they are flagged | DM spec merge model; amendment N |
| Stage measures M1 to M5 and the PRISMA priority | Baseline/MVP | R5b reads SP's events, which start at R3a | R5b flag | C20; T-SI-05 recorded before F6b |
| Frozen snapshots and coverage disclosure | Baseline/MVP | R5b | R5b flag | F6b (C12, amendments B, E, F) |
| External ledger: identification and deduplication counts | Baseline/MVP | P1 | P1 flag | Amendment K |
| External ledger: other step types; previous-review counts | Baseline/MVP | R5b | R5b flag | D4-11 |
| Search withdrawal and reinstatement | Baseline/MVP | P1 | P1 flag | X-DEL join; SP departure events |
| Search documentation versions | Baseline/MVP | P1 | P1 flag | None |
| Protocol record and amendment log | Baseline/MVP | Before or with R3b, because R3b's profile publication must enforce the amendment rule (F5) | Small flagged slice | RD and RS publication |
| Full-text retrieval actions | Baseline/MVP | P1 (actions), R3a and R3b (admission), R5b (boxes) | P1 flag | PDF programmes for the suggestion hook |
| Dedup QC, reviewer flag, privacy, parity | Baseline/MVP | P2 | P2 flag | T-SI-04; DM spec |
| Agreement store (human classes) | Baseline/MVP | R5c | R5c flag | T-SI-02; D3-11 budget measured |
| Statistics boundary and pilot rules (D3-10) | Baseline/MVP | R2c (publication), R3b (profile grain) | FEAT-024 flags under their own approvals | X-STATS-a, X-STATS-b1 to b7; C8-T07 |
| Analysis-ready export, codebook, RIS | Baseline lane | X1 after O1, R4c and R5a | X1 flag | F6a |
| Estimated-from-graph provenance | Baseline/MVP | O1 | O1 flag | C14 |
| External and AI-model screening sources | Later opt-in lane after the first engine release; proposed lane ID XS1 (the rollout drafter names it) |
XS1, per-project opt-in | New flag, default off (PROPOSAL name externalScreeningSources): it changes authoritative outcomes, spans API and PM, and needs a kill switch and per-project enablement |
R3b (profile versions), R4p (adjudication), C20, C22, P1 identity matching, DM lineage |
ScreeningSourcePolicy slot in the profile-version shape |
Reserved shape only | F5 (with R3b) | No behaviour; refused if populated before XS1 | Avoids a later breaking change to profile versions (PROPOSAL) |
| Machine-source class in agreement, exports and methods summary | With XS1 | XS1 extends R5a, R5b and R5c | XS1 flag | Statistical definition for any mixed metric (T-SI-02) |
| Annotation-answer imports | Later lane after the first engine release; proposed lane ID XA1 |
XA1 | New flag, default off | R2a; D4-14 floors |
| Graph digitisation | Deferred | Decided after O1 pilots | None now | D4-10 |
| Investigation grouping of distinct reports | Deferred | None | None now | D4-08 |
| Event-count schema | Blocked on specialist input | O1 | O1 flag | T-SI-01 |
| RoB and reporting-quality templates | Blocked on specialist input | R1a catalogue | Catalogue publication rules | T-SI-03; ACD spec catalogue |
Pilots. Reporting features are piloted on the "PRISMA identification" and "Workflow routing" seed projects (acceptance criteria §6.3). XS1 adds a seed project with a synthetic model configuration and a synthetic run (no real model output, no real study content) under D3-14. No production pilot of any import starts without its own approval.
11. Acceptance evidence¶
Method codes follow acceptance criteria §1.4.
Every row is PROPOSAL until confirmed at its freeze gate, except where it restates an owner
decision as a testable assertion.
| ID | Evidence | Method | Amends |
|---|---|---|---|
| RI-AE01 | Fixture: two imports of one article produce 2 records, 1 duplicate, 1 report and 1 Study after merge; a conference abstract and its article produce 2 Studies and 2 reports with the "report grouping not performed" note | C | AC-R5b-10, AC-R5b-13 |
| RI-AE02 | A Study without a confirmed source-document key makes the snapshot show Partial report identity coverage with the unconfirmed count; no export labels the figure "verified reports" | I | AC-R5b-13 |
| RI-AE03 | No "link reports to one study" action exists on any route or endpoint; a StudyVersion with several reference links round-trips through export unchanged |
U, I | AC-R5b-15 (replaced) |
| RI-AE04 | Stage-measure fixture: a Study in the pool and never reviewed is in M1 and not M3; a Study satisfied elsewhere is in M4 and not M3; a Study that entered twice counts once in M1 | C | New (R5b) |
| RI-AE05 | A departure caused by a filter clause other than a screening outcome never appears in any exclusion count; each departure in a report has a reason category | C | New (R5b) |
| RI-AE06 | A project whose tracking began after review started shows the baseline date; no event earlier than the baseline is synthesised; an eligible Study with no review shows "no review recorded" | I | New (R5b) |
| RI-AE07 | After a merge, current counts include the consolidated Study once and no tombstoned original; after unmerge a new snapshot counts the restored Studies; the earlier frozen snapshot is unchanged | C | AC-P2-04 (wording), AC-R5b-02 |
| RI-AE08 | Regenerating a frozen snapshot from its manifest yields identical numbers and digests; a ledger correction made later appears only in the next snapshot | I | AC-R5b-02, AC-R5b-20 |
| RI-AE09 | An architecture test finds no FEAT-024 read in the snapshot computation path | U | AC-R5b-19 |
| RI-AE10 | Entering previous studies without previous reports switches the template, sets box 16 to new plus previous and shows "not supplied" for previous reports; no code path computes previous-review counts | I | AC-R5b-12 |
| RI-AE11 | Per-box combination fixtures pass; an external screening count for records with accepted external or AI decisions at that phase is refused | C | AC-R5b-03, AC-R5b-04, AC-R5b-16, AC-R5b-17 |
| RI-AE12 | Withdrawal fixture: a Study in withdrawn search C and current search A stays and is counted from A only; a Study only in C departs with reason "search withdrawn"; the explanation line appears; the frozen snapshot is unchanged; reinstatement re-enters matching Studies | I | AC-R5b-14, AC-P1-15, AC-P1-16 (withdrawal part) |
| RI-AE13 | Editing search documentation creates a new version; a snapshot frozen earlier still shows the old version in its methods summary | I | AC-P1-14 |
| RI-AE14 | The protocol record's amendment log is append-only: no endpoint edits or deletes an entry | I | AC-P1-14 |
| RI-AE15 | Publishing a profile version with a changed eligibility rule and no amendment entry is refused before any write; a source-policy change also requires one | I, E | New (R3b) |
| RI-AE16 | Retrieval: attaching a PDF leaves fullTextStatus unchanged and shows the suggestion; Sought, Retrieved and Not retrieved record actor, time and reason; the box 6, 7 and 8 fixture passes |
C, I, E | AC-P1-11, AC-R5b-18 (status to confirmed) |
| RI-AE17 | X1 fixtures pass for the comparison export, codebook and RIS round trip into EndNote and Zotero | C, E | AC-X1-01, AC-X1-02, AC-X1-03 (status to confirmed) |
| RI-AE18 | Linking a graph region defaults the observation's extractionMethod to graph-estimated, and exports carry it |
I | AC-O1 rows on provenance |
| RI-AE19 | Lane XA1 floor: an imported annotation answer not mapped to a SyRF reviewer earns no target credit, never appears in default independence figures and never becomes gold | I | §4.36 floors |
| RI-AE20 | A copy-deck guard spec rejects "model decision" in user-facing strings of the XS1 folders; attribution labels read "AI-model-generated screening decision" | U | New (XS1) |
| RI-AE21 | No account, membership or grant exists for any AI source; an attempt to authenticate as one is impossible by construction; importer and accepter are stored separately from the source | I | New (XS1) |
| RI-AE22 | After a configuration moves to version 2, decisions accepted under version 1 still reference version 1 in the UI, exports and codebook | I, C | New (XS1) |
| RI-AE23 | Changing a source policy creates a new profile version through the Q-26 impact flow; earlier outcomes keep their profile version | I, E | New (XS1) |
| RI-AE24 | Re-uploading the same file returns the existing run and writes nothing; unmatched rows are listed and no Study is created; tombstoned identifiers resolve through merge lineage with the path recorded | I | New (XS1) |
| RI-AE25 | A rerun replaces the source's current decision on each Study; the count of contributing sources per Study is unchanged | C | New (XS1) |
| RI-AE26 | Sole-screener fixture: Include and Exclude become machine-only outcomes; Unsure creates an adjudication task and the outcome stays pending; the adjudicated outcome lists the version of the AI-model-generated decision as input and that decision is unchanged | C, E | New (XS1) |
| RI-AE27 | Contributing-vote fixtures cover model plus one and two humans under the profile rule, including bounded escalation, with no extra vote added | C | New (XS1) |
| RI-AE28 | Outputs on training inputs are flagged and excluded from sufficiency and agreement in the default configuration | C | New (XS1) |
| RI-AE29 | PRISMA manifests, screening exports and the methods summary show machine contribution per outcome and the machine-assisted share per phase | I | AC-R5b-22 |
| RI-AE30 | Agreement output has separate human independent, informed, external human and machine classes; no figure combines classes | C | AC-R5c-06 |
| RI-AE31 | A validated but unaccepted run changes no outcome and no pool; acceptance writes pool events with cause "accepted external screening decision" | C, I | New (XS1) |
| RI-AE32 | Fault injection during acceptance followed by resume yields exactly one current decision per Study and no duplicate events | I | New (XS1) |
| RI-AE33 | QC sample, reviewer flag, manifest fields and Publication privacy pass; parity rows stay pending until T-SI-04 records the method and thresholds |
I, E, C | AC-P2-15, AC-P2-17, AC-P2-01r, AC-P2-13 |
| RI-AE34 | Rebuilding the agreement store gives identical figures for the same watermark and method version; full recompute and view load are measured against a budget agreed at the R5c freeze | I, B | AC-R5c-09 |
| RI-AE35 | D3-10: the publication manifest records the statistics read identity; project 0102 is outside overlapping pilots until C8-T07 passes; R3b pilots serve profile-grain screening statistics live; preview publications record authoritative counts | I, G | PI-R2a-05, PI-R2c-01, PI-R3b-01, AC-R2c-08 |
| RI-AE36 | The methods summary reports automation tools when external or AI sources contributed: model name and version, role, thresholds as supplied, training-set description and Unsure handling | I, V | AC-R5b-22 |
| RI-AE37 | The model metadata page refuses users without the view capability and shows configuration versions, using profile versions, runs and decision provenance to those with it | I, E | New (XS1) |
| RI-AE38 | O1's freeze record shows the T-SI-01 specification before the event-count schema is frozen |
G | AC-O1 schema rows |
| RI-AE39 | No SYRCLE, CAMARADES or ARRIVE template is published to the catalogue without the T-SI-03 sign-off record |
G | AC-R1a-09, AC-R1a-11 |
| RI-AE40 | Exclusion-count fixture: 10 excluded Studies, 5 with one reason, 4 with two and 1 with three; the report shows 10 distinct excluded Studies and per-reason counts summing to 16, labelled as overlapping; no view or export adds the per-reason counts into an excluded total; with a primary reason defined, each Study appears once in the primary-reason breakdown | C, I | AC-R5b-11 |
12. Brief items, specialist inputs and unapproved proposals¶
Entries this specification owns (treatment required).
| Entry | Required treatment | Tracker row |
|---|---|---|
| Q-23 | Apply the unit distinction of §3.1 and RI-R01 to RI-R03 in C12, R5a, R5b and X1; keep investigation grouping deferred | T-RI-00 |
| D4-08 | Specify the StudyVersion reference-link shape with sourceDocumentKey and basis; confirm Study-owned work; keep grouping deferred |
T-RI-00 (with T-DM-00) |
| Q-17 | Obtain the field-level event-count specification and the meaning of "variation" from Chris or CAMARADES methodologists | T-SI-01 |
| Q-16 and D4-12 | One methods specification: denominators, percent agreement, prevalence, initial independent observations, per-profile screening agreement, statistical review of multi-rater formulas, and how the machine-source class is reported | T-SI-02 |
| D4-06 | Methodologist verification of SYRCLE (with outcome-specific items), the CAMARADES checklist and ARRIVE Essential 10, including applicability, before catalogue publication | T-SI-03 |
| D4-21 | Specialist parity and benchmark method for the native ASySD port; thresholds below remain proposals | T-SI-04 |
| Pool history versus review through the stage | PRISMA box mapping for M1 to M5 and phase outcomes; also the box for AI sole-screener exclusions (ambiguity A1) | T-SI-05 |
| D3-10 | Prove source-pinned reads, profile granularity and protected pilots; keep production readiness provisional | T-RI-00 |
| D3-11 | Choose the agreement projection, its watermark and a measured budget, separate from progress statistics | T-RI-00 |
Proposed thresholds, not approved.
- ASySD parity: identical AutoConfirmed groups; ProbableDuplicate pair-set F1 ≥ 0.99; sensitivity and specificity within 0.5 percentage points of the pinned R package (D4-21).
- ASySD performance: 80,000 citations in under an hour on Bramble (D4-21).
- Deduplication QC sample: 5% of AutoConfirmed groups, at least 20 groups.
- Agreement store: full recompute for RV-DS-03 within 10 minutes; view load p95 within 2 seconds (AC-R5c-09).
- AI score thresholds: none. The project supplies its own values; SyRF proposes no default.
- The Not retrieved reason list of amendment M.
Proposals an owner may want to see. RI-R15 (external counts refused for records with accepted
external decisions); RI-R18 (counting a Study still identified by another search); RI-R21
detection of "eligibility changed"; RI-R22 (source-policy changes require an amendment); RI-R26
(automatic Sought); RI-R42 (unmatched rows never create Studies); RI-R44 (training-input outputs
excluded from sufficiency by default); the separate accept capability (§6); the reserved
source-policy slot at F5 (§10); the new names SearchDocumentationVersion,
ProtocolRegistration, ProtocolAmendment, SearchReinstated, reportIdentityCoverage,
machineContribution, externalHumanContribution and pinnedStatisticsRead; the lane IDs XS1
and XA1.
Ambiguities found (options and a recommendation; no owner answer is assumed).
- A1. Where AI sole-screener exclusions go in the PRISMA diagram. FEAT-011 field #8
excluded_automatic("records marked ineligible by automation tools", box 3) is defined from aRemovedByAutomationlifecycle status. A sole-screener AI Exclude is a profile outcome at the title/abstract phase. Options: (a) count it in box 3 as automation removal; (b) count it in box 5 as a screening exclusion with a machine-only breakdown; © let the specialist decide and store both computations in the manifest. Recommendation: ©, decided underT-SI-05before F6b, with the manifest always holding the machine-only breakdown so either mapping can be produced. - A2. Whether adjudicators see the source type under blinded adjudication. Options: (a) always show "AI screening model" as the source of a candidate decision; (b) hide the source type when the profile is blinded; © a profile setting. Recommendation: © with the default showing the source type, because the terminology decision requires explicit AI attribution and the adjudicator needs the confidence and threshold context; human identities stay blinded either way.
- A3. Whether outputs on training inputs may ever count. Options: (a) never count toward sufficiency or agreement; (b) count by default and flag; © count only when an administrator opts in per policy. Recommendation: (a) for the first lane (RI-R44), because counting them would repeat the human decisions the model learned from.
Harmonisation notes (5 October 2026).
- §3.13: the outcome
authoritylist (CandidateAgreement,Reconciled,Adjudicated,Admin,Imported,LegacyUnknown) is retired. Outcome provenance is now defined once as RS §3.8'sfinalSourceplus this page's composition fields, separate from the registry's accepted-result authority, with a mapping from the old values.LegacyUnknownis gone (Q-35 removed). - §13: the prisma-amendments §H, methodology §3.6, contracts C3 and C12 and domain-model bullets
follow the same split;
Importedsurvives only as a candidate provenance kind. - §3.3: departure reason categories now cite SP §3.5.3's codes; the unsupported "project-level removal" category is removed (project deletion writes no per-Study departure). M2 lists all of SP §3.7's release kinds. §3.5 and W10 cite SP's reason and cause codes.
- RI-R39 to RI-R41, W12, §6, §9.7 and RI-AE26: "model decision" replaced with "AI-model-generated (screening) decision" (superseded wording 14). RI-R39 cites the adjudication step in RS-R59 and SP §3.4.
- §9.9: added what happens when a rerun replaces an input after adjudication resolved (RS-R58).
- §3.1: the report identity key (
sourceDocumentKeyon reference links) is tied to RD §3.3's file links (sourceDocumentLinks[]), which may carry the same key. - §1: a model Unsure goes straight to adjudication only under a sole-screener policy; as a contributing vote it follows the profile's Unsure rules (RI-R39, RI-R41; RS §5.10).
13. Amendments to existing package documents¶
- prisma-amendments.md
- Summary table: B, E and F become "Approved (Q-06b, E1, 4 October)"; K becomes "Approved (Q-37)"; L becomes "Approved (Q-37) with rule 4 replaced by the consolidated Study model (D2-12)"; M becomes "Approved (D4-07)"; O becomes "Replaced by prepared multi-source links; grouping deferred (D4-08)".
- §A: rename
StudyEnteredPooltoWorkFirstReleasedand add that stage-pool membership events are separate (C20). - §B: add the report identity coverage values and the "one report per Study assumed" label.
- §K rule 6: extend the double-count refusal to accepted external and AI-model-generated
decisions (
PROPOSAL). - §K rule 7: state that previous-review counts are only ever supplied values (D4-11).
- §L rule 4: replace "merge is an alias" with a pointer to the DM spec and C21; keep rules 2, 8 and 9 with their thresholds marked "proposed, not approved".
- §O: mark as deferred and point to RI-R03.
- §H authority list: replace
CandidateAgreement,Reconciled,Admin,ImportedandLegacyUnknownwith RS §3.8'sfinalSourceand this specification's composition fields, using the mapping in §3.13. - New §P (proposed): external and AI-model screening sources in FEAT-011 terms (outcome
composition, the machine-assisted share, the box 3 versus box 5 question for
T-SI-05). - methodology-coverage.md
- §2 capability map, Screening row: move "Machine-learning prioritisation or automated exclusion" out of "Out of scope" and record E3's external and AI-model screening lane (prioritisation stays out of scope).
- §3.6: keep
Importedfor mapped human imports as a candidate provenance kind (with the independence declaration), not an outcome or accepted-result authority (§3.13); keep amendment K "included elsewhere"; add that configured external sources count byScreeningSourcePolicy(E3), superseding the reviewer-mapping requirement for them. - §4.1 to §4.3: add the machine-source class; state that formulas await
T-SI-02. - §7.2, §8.2, §11.4, §12.1: mark D4-05, D4-07, D4-11 and D4-09 decided; add versioned search documentation (RI-R19).
- §9.1: replace the alias text with a pointer to the DM spec; §9.3: label thresholds "proposed, not approved".
- §10.1: amendment O deferred (D4-08).
- §13.1: D4-14 decided-amended; annotation imports in lane XA1 after the first engine release.
- §14.1: the methods summary reports automation tools (RI-AE36).
- §16.1: update statuses from the §2 table of this specification.
- contracts.md
- C3 import provenance: add the external screening source fields and the machine source identity; rename "imported authority with independence declaration" to imported provenance with an independence declaration, restricted to mapped human imports.
- C11: replace "for a form bound to several stages, the most restrictive bound stage's BL1 applies (Q-28)" with "form-owned or profile-owned blinding" (superseded wording 4); add the screening export columns of §3.9 and the codebook's model configuration entries.
- C12: unit list gains report identity coverage; "entering screening" uses
WorkFirstReleased; add the stage measures and the specialist dependencyT-SI-05; amendments B, E and F decided; the deduplication bullet points to C21 instead of the alias; outcome composition fields; the authority values {CandidateAgreement, Reconciled, Admin, Imported, LegacyUnknown} becomefinalSourceplus the composition fields (§3.13); the rule list gains "no external decision changes a count before acceptance". - C14: keep estimated-from-graph as decided (D4-10); the Verified gold bullet is superseded (see the RS spec).
- New C22 (external and AI-model screening sources):
AIScreeningModelConfiguration,ScreeningSourcePolicy,ExternalScreeningRun,ExternalScreeningDecision, acceptance semantics, idempotency keys, identity resolution, outcome composition, agreement classes and the conformance tests behind RI-AE20 to RI-AE32. - acceptance-criteria.md
- AC-R5b-10: status from
assumption-A-11to confirmed under Q-06b, with RI-AE01 and RI-AE02 wording. - AC-R5b-12, AC-R5b-13, AC-R5b-14, AC-R5b-20: statuses to confirmed (D4-11, Q-23 alignment, Q-33, Q-06b).
- AC-R5b-15: retire and replace with RI-AE03 (prepared links, no grouping).
- AC-R5b-03, -04, -16, -17: status to confirmed (Q-37).
- AC-P1-11, AC-P1-14, AC-P1-15, AC-R5b-18: statuses to confirmed (D4-07, D4-05, Q-33).
- AC-P2-01r, AC-P2-13: stay pending, now on
T-SI-04with thresholds "proposed, not approved". - AC-R5c-06: stays pending on
T-SI-02; add the machine-source class (RI-AE30). - AC-R5c-09: status from pending-D3-11 to brief item with budget agreed at the R5c freeze.
- §4.36: X1 rows become confirmed (D4-09); add a lane section for XS1 with RI-AE20 to RI-AE32 and RI-AE37, and one for XA1 with RI-AE19.
- PI-R2a-05, PI-R2c-01, PI-R3b-01: status from pending-D3-10 to brief item with the treatments in RI-R53 to RI-R55.
- programme-integration.md §7: record
D3-10 (a) to (d) and D3-11 as brief items with the treatments above; restate that gate (b) failed
on latency on 3 October (#3510) and production readiness is provisional; add that machine-source
decisions are counted in progress statistics as their own source class, served live until a
FEAT-024 scope amendment covers them (
PROPOSAL). - domain-model.md: add
AIScreeningModelConfiguration,ExternalScreeningRunandExternalScreeningDecisionto the Screening Outcomes context; add the outcome composition fields; renameStudyEnteredPooltoWorkFirstReleased; replace the "Report-to-study link (amendment O)" row with the prepared reference links onStudyVersion; add the search documentation versions and the protocol record;AuthorityValueis split into accepted-result authority and outcome provenance as the RS spec's §13 says (verified (D4-03)andlegacyUnknownleave it). - integrated-plan.md: R5b, R5c, P1 and X1 rows take the statuses above; add lanes XS1 and XA1 with their dependencies (the rollout drafter fixes names and placement); R5b's MVP boundary adds the stage measures and the specialist dependency.
- open-questions-and-assumptions.md: Q-06b, Q-33, Q-37, D4-05, D4-07, D4-09, D4-10, D4-11 and D4-14 decided; Q-23 and D4-08 carry-forward alignment; Q-16, Q-17, D4-06, D4-12 and D4-21 specialist inputs; D3-10 and D3-11 brief items; A-11 retired by Q-06b.
- decision-register.md §2: add the rows in §14 of this specification.
- ux-strategy.md §3.1 and §7.8: PRISMA views show unit labels, coverage values and the machine-assisted share; add the model metadata page under Design (UX spec owns layout).
14. Superseded wording¶
| Old wording | New wording | Where the old wording appears today |
|---|---|---|
| "Model decision" | "AI-model-generated screening decision"; "AI screening model"; non-AI sources keep their real type | Earlier session drafts; any new text must avoid it (superseded item 14) |
| Every-ever-in-pool membership as the PRISMA reviewed count | Actual review through the stage is the reporting priority; pool history is separate audit data (RI-R04) | The register's "historical pool coverage" section before the owner's clarification (superseded item 16) |
| Every target-counted imported contribution must map to a SyRF reviewer (D4-14 original) | Configured external and AI screening sources count under the profile's ScreeningSourcePolicy; annotation imports keep the mapping rule |
methodology-coverage.md §13.1; acceptance-criteria.md §4.36 closing paragraph; C3 imported authority (superseded item 17) |
| Merge as an alias; reports and PRISMA "resolve aliases" | One consolidated current Study with reversible unmerge; reports count it once and never count tombstoned originals (DM spec) | prisma-amendments.md §L rule 4; C12 deduplication bullet; methodology §9.1 (superseded item 2) |
| Immediate full report or investigation linking (amendment O, D4-08 original) | Prepared multi-source links; grouping deferred (RI-R03) | prisma-amendments §O; methodology §10.1; AC-R5b-15 (superseded item 9) |
StudyEnteredPool |
WorkFirstReleased (renamed to avoid clashing with stage-pool entry) |
prisma-amendments §A; C12; domain model events |
| "Verified" gold authority and "single extraction, verified" export label (D4-03 original) | Target-one reconciliation with required human reconciliation; labels follow AcceptedResultVersion.authority (RS spec) |
C14 last bullet; methodology §4.4 and §5.8 (superseded item 11) |
| C11 "the most restrictive bound stage's BL1 applies (Q-28)" | Form-owned (annotation) or profile-owned (screening) reconciliation blinding | C11 blinding bullet (superseded item 4) |
| "Machine-learning prioritisation or automated exclusion … no lane" | External and AI-model-generated screening decisions are in scope in a later opt-in lane; prioritisation stays out of scope | methodology-coverage §2, Screening row |
15. Existing work reused¶
Three pieces of earlier work feed this specification. QM v2 PR-C (#2574) reserved export modes and built a schema sidecar for exports, and #2812 drafted a response-mode contract whose units and metadata types feed the outcome schemas. The harvest map is authoritative; nothing is ported or closed while the hold lasts.
| Entries | Verdict | Target section |
|---|---|---|
| H-API-11 | Adapt | §3.9 exports: the previous-versions mode with R2a (T-RD-02) and the as-of mode in R5a under the C11 as-of rule (T-RI-12) |
| H-API-12 | Adapt | §3.9 codebook and dataset labels (T-RI-08); C11 manifests, including R2a's previous-versions manifest (T-RD-02) |
| H-VAL-12 | Adapt | Units and metadata field types as input to the F-O C14 outcome-schema ADR (T-RI-11); its response modes go to C4 and E37 (T-RD-01) |
What this specification and C11 require that the earlier work lacks.
- Form-version selectors (
formId, sequence and session-version IDs), never stage scopes or a raw date; as-of uses a clock watermark under the C11 rule. - Codebooks keyed by question and form version, with compatibility class, option IDs, requiredness, the version each answer was given under, legacy-gap coverage labels from baseline conversion and dataset labels.
- Readers over canonical revisions, not over embedded legacy arrays.
- Server refusal of modes that are not yet enabled, behind their flags.