Skip to content
flowss
Start freeChecking your session
Decision matrix (weighted / Pugh) templates

Decision Matrix — Choosing an Evaluation Design

A research steering group weighs four designs for evaluating a falls-prevention programme on 12 wards — individually randomised, stepped-wedge, interrupted time series and matched cohort — with time to result and evaluation cost measured. The interrupted time series ranks first at 4.15 of 5, 0.48 ahead of the stepped-wedge trial, which overtakes only if control of confounding rises from 25% to 49.2% of the weight; the matched cohort is dominated. Illustrative values.

Decision matrix (weighted / Pugh)Research & AcademicDecision matrix (weighted / Pugh) syntax docs
Template previewDecision matrix (weighted / Pugh)
Falls-prevention programme — choosing the evaluation designFalls-prevention programme — choosing the evaluation designResearch steering group, 12 wards across two hospitals · 1 = weak, 5 = strongWeighted scoringScale 1–57 criteria4 optionsCRITERIONWEIGHTIndividuallyrandomised trialpatients randomisedwithin wardsStepped-wedgecluster trialwards cross over in4 stepsInterrupted timeseries36 months of routineincident dataRECOMMENDEDMatched cohortintervention vsmatched wardsControl of confounding25%5432Contamination between arms15%1543Feasibility on 12 wards15%2454Time to a resultmeasured, months · lower is better15%30 monthsscore 126 monthsscore 214 monthsscore 518 monthsscore 4Evaluation costmeasured, £k · lower is better10%£640kscore 1£480kscore 2.2£120kscore 5£210kscore 4.3Ethics & acceptability10%3454Generalisability10%3433Weighted scoreΣ weight × score, of 52.5551% of max3.6773.5% of max4.1583% of max3.2865.6% of maxRankhighest score first4th2nd1st3rdWeighted totalsEach bar is split into its criteria's weighted contributions (colours match the dots in the table); ranked best first.012345Interrupted time series4.1583%Stepped-wedge cluster trial3.6773.5%Matched cohort3.2865.6%Individually randomised trial2.5551%FINDINGS & SENSITIVITYROBUSTInterrupted time series ranks first at 4.15 of 5 (83%), 0.48 ahead of Stepped-wedge cluster trial.Sensitivity (robust): Stepped-wedge cluster trial draws level with Interrupted time series if the weight on “Control of confounding” risesfrom 25% to 49.2% (+24.2 points), the other weights rescaled in proportion — the smallest single-weight change that changes the winner.No single score change within the 1–5 scale lets another option draw level with Interrupted time series.Matched cohort is dominated by Interrupted time series: no better on any criterion and worse on 6, so it can be dropped without changing thedecision.Illustrative judgements for a fictional NHS-style programme; costs are evaluation costs only.

Make it your own.

title "Falls-prevention programme — choosing the evaluation design"
subtitle "Research steering group, 12 wards across two hospitals · 1 = weak, 5 = strong"
note "Illustrative judgements for a fictional NHS-style programme; costs are evaluation costs only."

option "Individually randomised trial" "patients randomised within wards"
option "Stepped-wedge cluster trial" "wards cross over in 4 steps"
option "Interrupted time series" "36 months of routine incident data"
option "Matched cohort" "intervention vs matched wards"

"Control of confounding"        weight 25 : 5 4 3 2
"Contamination between arms"    weight 15 : 1 5 4 3
"Feasibility on 12 wards"       weight 15 : 2 4 5 4
"Time to a result"   weight 15 unit "months" lower : 30 26 14 18
"Evaluation cost"    weight 10 unit "£k" lower     : 640 480 120 210
"Ethics & acceptability"        weight 10 : 3 4 5 4
"Generalisability"              weight 10 : 3 4 3 3