Reporting standards

The Lab can check a FlowScript document against 33 reporting checklists, from CONSORT and PRISMA to STROBE, TRIPOD, GRADE and AMSTAR 2. Each checklist runs as a set of executable checks over your typed model: it reconciles the numbers you declared (does the flow diagram add up, does the stated ICER follow from the costs and effects) and reports which fields the standard asks for that your document does not yet carry. Two places in the Lab use those checks. The Reporting standards pane scores the open document item by item, takes you to the line each item is about, and applies the repairs that ship with the checklist. The Standards bench turns a checklist into work: it starts you from a document already shaped for the standard, and proposes one thing to do about every item that is not yet reported, refusing out loud whenever the answer is a judgement or a fact only you have. This page documents both, the rules behind every verdict, and all 33 checklists with their versions, years, scope and starting documents.

At a glance

On Windows and Linux, read ⌘ as Ctrl.

How scoring works

What a checklist can and cannot judge

A checklist here is data plus checks over your compiled model. That makes two kinds of question decidable: arithmetic (do the counts in a flow diagram reconcile, does a stated ICER, interval or net benefit follow from the numbers beside it) and completeness (is the field the standard asks for present and filled in with something other than a placeholder). It does not grade writing, and it cannot tell whether a sentence in your manuscript says what an item wants. A high count of reported items is a statement about your figure model, not a guarantee about your paper.

The five verdicts

Unknown is a real answer and it counts as not met. On a figure model, most manuscript-shaped items land there. That is deliberate: a score should never read as "you may stop checking". If a check itself fails while running, the item is reported as Unknown and flagged, so a broken check lowers the score rather than disappearing from it.

No percentage

The pane shows met / applicable as two integers (for example 31 / 37), a bar with all four counted buckets side by side, and a sentence naming the unknown items as work you still have to confirm by hand. It never shows one confident percentage. If a report's own counts ever fail to add up to its denominator, the pane says so: "This report's counts do not add up to its own denominator — treat the numbers above as unreliable and read the items instead."

Repairs never invent data

Many items ship a fix: a source-to-source repair that writes an obvious placeholder into the right block (registry: "TODO — …", n: 0 // TODO, placeholder: true). Every check ignores a placeholder when deciding Reported. So a fix moves an item from Missing to Partial ("replace the placeholder") or, on checklists such as CHEERS and SQUIRE, to Unknown, but never to Reported. You still have to write the real value.

When scoring runs

Scoring runs only while the Reporting standards pane, the Advisor or the Standards bench is open, and it re-runs as you edit. Only the checklists that apply to your document are scored; the others are listed with a reason. Scoring needs a compiled model of your document. A document with ordinary validation errors still has one, built from the blocks that parsed, and is scored. When the compiler cannot read the document at all (an unterminated string while you are halfway through typing a quoted value is the usual cause), nothing is scored, and the Lab says so rather than reporting that no checklist applies.

Which checklists apply

Each checklist has a test for whether it governs your figure. The Lab orders the ones that apply by relevance, so the checklist you most likely came for is first, and keeps each extension next to its base (CONSORT's extensions follow CONSORT; PRISMA-S, PRISMA-ScR and the PRISMA flow arithmetic follow PRISMA).

"Names the tool" means a tool: attribute anywhere in the document with that value. "Mentions" means the word appears in a block kind, a block id (underscores and hyphens read as spaces) or any string value, so a block called cluster_a is enough to mention clusters. If nothing applies, the pane says: "No reporting standard claims this figure yet. The checklists are chosen from the document itself — declare a study design, arms, a cohort chain with exclusions, or an economic model, and the standards that govern it appear here."

The Reporting standards pane

Opening and closing it

• Open the document status popover (click the status glyph beside the document title, or press ⇧⌘M), and in its Reporting standards section click Score against CONSORT, PRISMA…. Clicking it again closes the pane.
• Press ⌘K and run Reporting standards… ("Score this figure against CONSORT, PRISMA and the rest — item by item, with the repairs that ship"), or choose Reporting standards under Go to.
• Close it with its × ("Close reporting standards") or Escape.

The pane opens in the column at the right of the canvas, which it shares with the Advisor, the methodologist, Comments, Revision history, Provenance and Publish. Only one of them shows at a time.

Header and the standards row

The header reads Reporting standards with a count of how many checklists apply ("N apply"), or an amber not scored chip when the compiler cannot read the document, and the document's title.

Below it, one chip per applicable checklist, most relevant first, each showing the checklist's name and its met / applicable fraction. Hover a chip for its version and scope. Click a chip to open that checklist.

The summary

For the open checklist:

When the compiler cannot read the document, the Lab clears the scores rather than keeping stale ones: the header shows not scored, the summary and tabs give way to "No checklist was run: the document does not compile." ("The engine scores a compiled model, and there is none — an unterminated string is the usual cause, and it happens every time a quoted value is half-typed. Nothing below is a verdict about your figure. Fix the source and the standards that govern it come straight back."), and every checklist is listed underneath as not run.

The three tabs

The To fix tab

Items are ordered so the quickest to resolve come first: items with a repair and a location, then items with a repair only, then items with a location only, then the rest. Within each group, Missing comes before Partial before Unknown, then checklist order. (This differs from a severity-only list on purpose: this list is for working through inside the editor.)

Each item shows:

• its verdict chip, its number as the standard prints it (such as 13a) and its title;
• line N with a crosshair when the verdict points at a place in your source. Click the item's header strip to put your cursor on that line in the Source shelf;
• the verdict's detail, in the checklist's own words, and any evidence it quotes from your document;
• "This item failed while running, so the verdict above is not a judgement — check it by hand." when the check itself failed;
• Show the fix when the checklist ships a repair for the item.

To apply a repair:

1. Click Show the fix (it becomes Hide the fix while open; one preview is open at a time). The pane previews the repair by running it on a copy of your document: a description, the lines it would add (prefixed +), any lines it would remove (prefixed -), and a note.
2. If the engine accepted the trial run, click Apply this fix. If it did not, the pane says "The engine recompiled this change and refused it, so it is not offered."
3. The repair is applied and a toast shows its note with Undo. ⌘Z also undoes it. If the repair is abandoned at this point, an error toast says so (for example "The fix was abandoned — your document is unchanged").

Every preview ends with: "A fix writes placeholders, never data. The item stays outstanding until you replace them." If the report is older than the checklist, the preview says: "This item's fix is no longer available — the report may be older than the checklist it came from."

When nothing is outstanding: "Nothing outstanding on this checklist: every applicable item is reported and none was left undetermined."

The Checklist tab

The checklist grouped into the standard's own sections (for example Title and abstract, Methods, Results), in first-appearance order, with items in checklist order. Each section header is collapsible and shows its own met / applicable fraction, an "N excluded" chip for not-applicable rows, or "none applies" when every row in the section is excluded. Each item shows its verdict, number, title, location, the requirement in plain words, and the verdict detail. An item with a repair that is not yet reported shows Fix available, which switches to To fix with that repair's preview open.

The Doesn't apply tab

Every checklist that was not offered, listed rather than hidden, each with its name and version, its item count, the reason ("Not offered for this figure: nothing in the document matched what this checklist looks for. Covers …"), and Start from this standard buttons for each starter that ships for it (hover a button for the starter's full title). The list opens with "N standards are not offered for this figure. They are listed with a reason rather than hidden — and any of them can be started from a compliant example." When the compiler cannot read the document, the list explains that none of the checklists was run, and each row says "Not run: the document does not compile, so this checklist's own test for whether it governs your figure was never called. It covers: …"

Starting from a compliant example

When the open checklist has a starter and you either have outstanding items or are not following that standard, the foot of the pane offers Start from a compliant example with one button per starter. A starter replaces your document's content. See Starting from a scaffold for what happens and how to undo it.

The Standards bench

Opening it

• Click Standards on the Lab's tool rail. Click it again to close the workbench sheet.
• Press ⌘K and choose Standards under Go to ("CONSORT, PRISMA and the rest as scaffolds and items you can insert").
• Click an Advisor proposal such as "Start from a CONSORT skeleton" or "Score this figure against STROBE". A proposal about one specific item opens the bench on that item, expanded and scrolled into view, with any filter cleared.

The bench has two shelves, switched at its top: Start from a standard and Improve this document. The strap line reads "compiled, never invented".

Standards bench: Start from a standard

This shelf lists all 33 checklists so you can start a document already shaped for one of them.

Search, filter and cards

If nothing matches: "No checklist matches those words in this field. Try one term at a time, or widen the filter to every field."

Each checklist is a card showing its name and version, a not a published standard badge for the two flowss syntheses, a scored badge when the open document has already been scored against it, and its scope. Expand a card to see:

• "N items on the checklist" and the fields it belongs to;
• the full citation, and a link to the standard's website where one exists;
• See what this document is missing, when the document has been scored against it, which switches to Improve this document on that checklist;
• the scaffold: its title, a badge naming where it came from, a scores no better than an empty document badge where that is true, a note saying exactly what the scaffold does and does not claim, a list of every line marked TODO ("N lines marked TODO — every one is yours to write") when it has placeholders, the full FlowScript, and Start a new document from this scaffold.

At the foot of the shelf, once you have opened a checklist the document has been scored against, a link reads "You have already scored this document against X — N outstanding" and takes you to Improve this document.

Where a scaffold comes from

The three generated structures:

AMSTAR 2's scaffold carries the badge scores no better than an empty document. Every AMSTAR 2 item is a judgement about a review you have read, so a document that answers none of them scores exactly what an empty one does. That is the honest result, not a defect; the scaffold still saves you writing out sixteen items. Add answer: yes, partial_yes, no or not_applicable to each item yourself.

Starting from a scaffold

A scaffold is a whole document (several blocks, the references between them and a view line), so it is never appended to your work. Appending it would rename its first block to avoid a clash and break every from: reference to it.

1. Expand the checklist's card and read the scaffold.
2. Click Start a new document from this scaffold (or Start from a compliant example / Start from this standard in the Reporting standards pane).
3. The scaffold replaces the content of the open document at once. A toast reads "Template loaded —" followed by the standard's id or the starter's title (for example "Template loaded — consort"), with Undo. There is no separate confirmation dialog (even though the small print under the button says the studio confirms first); Undo in the toast, or ⌘Z, brings your document back.
4. Data rows bound to the document from the Data bench are released, because they described the study that was there before. The Data bench says: "The rows bound to this document were released when the template replaced it — they described the study that was here before. Load the file again to switch the column checks back on."
5. If the open document already holds exactly that scaffold, nothing happens.

Tip: To keep your current document, create a new one first (⌘K New document), then start it from the scaffold.

Standards bench: Improve this document

This shelf works from a compliance report. For the checklist you choose, it lists every item that is not yet reported and proposes one thing to do about each: either a FlowScript snippet the bench has compiled and checked, or a stated refusal saying why the item is yours to write.

If nothing has been scored (no checklist applies, or the compiler cannot read the document), the shelf says "Nothing has been scored yet." and explains: "This shelf works from a compliance report: it reads the items a checklist did not find met and proposes one thing to do about each. Score the open document against a standard first, and the proposals appear here." A button offers Or start a document from a standard.

Choosing a checklist

Which checklist are you working through? lists every scored checklist with its outstanding count, such as "CONSORT — 6 outstanding", where outstanding counts Missing, Partial and Unknown items. Under it:

• a headline, such as "12 outstanding: the studio can start 5 of them, and 7 need a judgement or a fact only you have." (or "Nothing outstanding on this checklist.");
• for the two syntheses, an amber note that the checklist is not a published reporting standard but a synthesis of community checklists, declared as "synthesis 1".

Filters

How far a snippet has been checked

Every snippet is compiled before it is offered, and its holes are written as the checklist's own placeholders ("TODO — …", 0 // TODO, { not_yet_specified: 0 }), so nothing in it reads as an answer. The bench then scores it on its own with the real checks and withholds any snippet that would answer an item by itself. That is the check the Lab runs today, and the bench says so on screen:

Warning: Because snippets are not yet re-scored against your own document, re-open the item in the Reporting standards pane after you insert one. If it now reads Reported while the snippet still holds TODO markers, replace the markers before you rely on the score.

What an item shows

Collapsed: the verdict chip, the item number, its title, the badge above, and the checklist section. Expanded:

When you insert a snippet it is appended to the end of your document, any id that is already taken is renamed (nested ids included), the new block is selected, and a toast names the block kind and its id, such as "Inserted cohort 'approached'".

The reasons an attribute is left out of a snippet:

Refusals: "Yours to write"

A refused item has its own card headed Yours to write, the reason verbatim, and the line: "There is no button here on purpose, and no feature is missing. A bench that filled this in would be writing a method nobody used into your manuscript, and the checklist would tick it." The reasons are:

When a filter leaves nothing to show, the bench says which case you are in: "Nothing outstanding on this checklist. Every item it applies to is reported."; "No proposal on this checklist carries a snippet. Every outstanding item wants a judgement or a fact only you have — switch to “Yours to write” to read them."; or "The bench wrote a snippet for every outstanding item on this checklist. Nothing here is waiting on your judgement alone."

The 33 checklists

Item counts are the rows on each checklist, including any that may come back N/A. Version and year are as published (PRISMA 2020 was published in 2021). The last column is the scaffold title the Standards bench shows when you expand the checklist's card: a title ending "— starting from the … figure" reuses the base checklist's starter, and one ending "— appraisal structure" is a structure the bench generates.

Trials and protocols

Systematic reviews

A second PRISMA starter, PRISMA 2020 — flow diagram for an updated review, is offered in the Reporting standards pane for PRISMA and the flow arithmetic.

Observational, diagnostic and prediction studies

Specialised designs

Economic evaluation and quality improvement

Machine-learned prediction models

Critical appraisal and certainty

Checklists by field of work

CARE belongs to no field yet, so it appears only under Every field. Four fields of work in the Lab have no published reporting standard in this set (engineering reliability and risk, project delivery, business and finance, and the general figure); for them the standards views stay empty, which is a fact about the checklists, not a gap in your document.

Numbering that is flowss's own

Where a published checklist does not number its rows, or where a checklist is a flowss synthesis or consistency layer, the item numbers are flowss's and must not be quoted as published item numbers:

AMSTAR 2's 17 rows are its 16 published items plus one row for the overall confidence rating. RECORD's 13 rows use RECORD's own decimal numbering (1.1 to 22.1).

The example-data mark

Every starter is a complete study with realistic values, and every one is fictional. Each opens with a comment line such as "Every string below is EXAMPLE text from a fictional trial: replace it." While that line is in your source:

• the canvas shows Example data — not real. "This starter uses values from a fictional study — replace them with yours, then delete its "EXAMPLE text" notice to remove this mark from the figure and its exports." Click Got it to fold it to an Example data chip;
• every view carries a faint "EXAMPLE DATA" watermark and an amber Example data — not real label in its bottom-right corner, on screen and in every export;
• the submission bundle's preflight text opens with the same notice.

To remove the mark:

1. Replace every example value with your own.
2. Delete the comment line that carries the words "EXAMPLE text from a fictional …" (in most starters it begins "Every string below is EXAMPLE text from a fictional …" or "Every string is EXAMPLE text from …"; a few say "from fictional trials" or "from fictional studies"). The mark is triggered by those words anywhere in the source, in upper-case EXAMPLE, so check that no other line still contains them.
3. The mark disappears at the next redraw.

The three generated structures (ROBINS-I, ROBIS, AMSTAR 2) carry TODO markers instead of example values, so they are not stamped.

Tips

• Start from the scaffold of the standard you are reporting to, then replace every value. The scaffold is shaped so that each item has somewhere to go.
• Work down To fix in the Reporting standards pane first: its repairs know where in your document the text belongs. Then use Improve this document for the items no repair covers.
• Click an item's header to jump to the line it is about. A verdict with a location is one edit away.
• Treat Unknown as a to-do, not a pass. Its detail names the attribute that would make it decidable.
• Score an extension alongside its base, never instead of it.
• Replace every TODO a repair or snippet writes. Until you do, the item stays outstanding by design.
• Read the citation and follow the link on a checklist's card when you need the published wording; each item's requirement here is a paraphrase.

Limits and known constraints

• The checks judge your figure model, not your manuscript's prose. A sentence elsewhere in your paper can satisfy an item the Lab reports as Unknown.
• Only checklists whose own test claims your document are scored. There is no control to force-score a checklist that does not apply; its row in Doesn't apply explains why and offers its starter.
• PRISMA-NMA, PRISMA-IPD and the separate PRISMA 2020 for Abstracts checklist are not encoded.
• RoB 2's domain 2 is encoded for the effect of assignment only; the adherence variant gets Unknown for its algorithm. The RoB 2 variants for cluster-randomised and crossover trials, and ROBINS-I's 2024 successor, are not encoded.
• PRISMA's "included" box carries two counts (studies and reports), but a cohort has one n, so only the study count is checked.
• CLAIM's imaging-specific items are not part of the model-card synthesis.
• The AMSTAR 2 scaffold scores exactly what an empty document scores until you record your own answers. The ROBINS-I and ROBIS scaffolds can satisfy structural items (the declared domains, the figure, the caption) but leave every judgement to you.
• CHEERS item 16 is not applicable when there is no econ_model, and item 25 when engagement states there was no patient or stakeholder involvement.
• Starting from a scaffold or starter replaces the open document's content and releases any data rows bound to it.
• Snippets on Improve this document are checked on their own only, not re-scored against your document, so every one carries the snippet, unchecked badge; verify the item after inserting it.
• A snippet that belongs on a block you already have is appended as a second block; merge it into the existing block by hand.

Troubleshooting

The pane says "No checklist was run: the document does not compile." The engine scores a compiled model, and an unterminated string while you are typing a quoted value is the usual cause. Fix the source and the checklists come straight back.

My checklist is not in the standards row. Its test did not claim your document. Open Doesn't apply to see its reason, and add what the checklist looks for (see Which checklists apply), such as a second arm for CONSORT or a tool: robis for ROBIS.

I applied a fix and the item is still not reported. That is by design: a fix writes a placeholder. Replace the TODO with the real value.

Apply this fix is missing. The engine ran the repair on a copy of your document and refused it because it would have introduced an error. Make the edit by hand, following the item's detail.

A toast says "No automatic repair ships for … — this one is a manual edit". The item has no repair. Follow its detail in the pane.

A toast says "The document has to compile before a fix can be applied". Fix the errors shown in the document status popover's Validation section (⇧⌘M), then apply the fix.

Improve this document says "Nothing has been scored yet." Either no checklist applies to your document or the compiler cannot read it. Start from a standard, or add the blocks a checklist looks for, or fix the source.

I inserted a snippet and the item turned Reported although it is full of TODO markers. The bench checks snippets on their own, not inside your document, so in rare cases a marker-filled snippet supplies the one block an item was waiting for. Replace every TODO with the real value before you rely on the score.

The figure still says Example data — not real. Delete the starter's "EXAMPLE text from a fictional …" comment line. Replacing the numbers alone does not remove the mark.

I lost my work when I started from a scaffold. Click Undo in the "Template loaded" toast, or press ⌘Z.

Related pages

• Lab
• The FlowScript language
• Lab views and renderers
• The Data and Statistics benches
• Publishing, provenance and compliance in Lab
• Evidence
• Screening, extraction and synthesis
• Core concepts and glossary
• Copyright, licences and attributions