Designing adaptive or sequential experimental designs requires a meticulous and pre-specified approach to decision-making. The "decision rules" are the heart of these designs, dictating how the experiment changes or concludes based on accumulating data.

Here's a comprehensive guide to defining these decision rules, broken down by their common applications:

1. Defining Decision Rules for Adaptation

Decision rules are the pre-specified criteria that, when met, trigger a modification to the study design, an allocation change, or a stopping decision. They must be objective, based on accumulating data, and designed to maintain statistical validity and ethical conduct.

1.1. Core Principles for Defining All Decision Rules

Before diving into specific types, understand these overarching principles:

• Pre-specification: Every decision rule, the data it uses, the thresholds, and the resulting action MUST be fully detailed in the study protocol before the first subject is enrolled. Post-hoc changes can invalidate the study.


• Objectivity: Rules must be based on objective statistical criteria (e.g., p-values, confidence intervals, observed event rates, conditional power), not subjective judgment.


• Statistical Validity: The rules must preserve the desired Type I error rate (alpha) and Type II error rate (beta/power). This often requires specialized statistical methods (e.g., alpha spending functions, conditional power calculations, simulation).


• Feasibility: The data required for decision-making must be obtainable in a timely manner, and the resulting actions must be practically implementable.


• Ethics: Rules should prioritize patient safety, minimize patient exposure to ineffective or harmful treatments, and maintain clinical equipoise where appropriate.


• Transparency: The entire decision process, including the rules and the rationale, should be clearly documented and reported.


• Simulation: Rigorous simulation studies are essential to evaluate the operating characteristics (Type I error, power, average sample size, study duration) of any adaptive design with its defined decision rules.

1.2. Categories of Decision Rules

Decision rules typically fall into categories based on the type of adaptation they trigger:

A) Stopping Rules (for Efficacy, Futility, or Safety)

These rules dictate when to terminate the entire study early.

• Efficacy Stopping Rules:


• Purpose: To stop the study early if there's overwhelming evidence that an experimental treatment is superior (or non-inferior) to the control, making it unethical to continue enrolling subjects in the control arm or delaying access to the beneficial treatment.


• How to Define:


• Statistical Thresholds: Define a specific p-value threshold for the interim analysis that indicates efficacy. This threshold will be more stringent than the final analysis p-value (e.g., 0.001 instead of 0.05).


• Alpha Spending Functions: For multiple interim analyses, an alpha spending function (e.g., O'Brien-Fleming, Pocock, Wang-Tsiatis) allocates a portion of the total Type I error rate to each interim analysis. The decision rule for efficacy stopping will be based on the boundary defined by this function.


• Magnitude of Effect: Sometimes, an efficacy rule might also require the observed effect size (e.g., difference in means, hazard ratio) to be above a pre-defined clinically meaningful threshold, in addition to statistical significance.


• Timing: Specify at which interim analysis points (e.g., after 25%, 50%, 75% of planned enrollment) these rules will be applied.


• Example Rule: "If, at the 50% interim analysis, the observed p-value for the primary endpoint comparing Treatment A to Placebo is less than or equal to 0.0012 (O'Brien-Fleming boundary), the study will be stopped for efficacy, and Treatment A will be declared superior."


• Futility Stopping Rules:


• Purpose: To stop the study early if it's highly unlikely to achieve its primary objective even if continued to full enrollment, thereby saving resources and preventing patients from receiving an ineffective treatment.


• How to Define:


• Conditional Power: Calculate the probability of observing a statistically significant result at the end of the study, given the data observed so far. A common rule is to stop for futility if conditional power drops below a certain threshold (e.g., 20% or 30%).


• Predictive Power: Incorporates a prior distribution for the unknown effect size, providing a Bayesian perspective on the probability of success.


• Drift Parameter (for continuous outcomes): If the observed mean difference is too small at interim, it may be deemed futile.


• Observed Effect Size/Trend: If the observed effect size is in the "wrong" direction or substantially smaller than the minimum clinically meaningful difference, it might trigger futility.


• Timing: Specify when futility rules will be applied. Futility rules are often less stringent early in the study, becoming more stringent later.


• Example Rule: "If, at the 75% interim analysis, the conditional power to detect a significant difference at the final analysis (assuming the current trend continues) falls below 20%, the study will be stopped for futility."


• Safety Stopping Rules:


• Purpose: To stop the study early if an experimental treatment is causing unacceptable harm or an adverse event rate that significantly exceeds expectations or the control group.


• How to Define:


• Thresholds for Adverse Events (AEs): Define specific rates or counts for severe AEs (SAEs), dose-limiting toxicities (DLTs), or deaths that would trigger a stop. These can be absolute numbers or rates compared to the control arm or historical data.


• Pre-specified Safety Metrics: e.g., "If the incidence of Grade 3 or higher liver toxicity in Treatment B arm exceeds 15% AND is statistically significantly higher (p < 0.01) than the control arm at any interim analysis, the study will be stopped."


• Data Monitoring Committee (DMC) Authority: While rules guide, DMCs often have overriding authority for safety. The rule specifies the trigger for DMC review.


• Example Rule: "The study will be stopped for safety if, at any interim analysis, the observed rate of Grade 4 cardiovascular adverse events in any active treatment arm exceeds 5% and is statistically significantly higher (Fisher's exact test p < 0.01) than the placebo arm."

B) Treatment Allocation Rules (Response-Adaptive Randomization)

These rules change the probability of assigning a subject to a particular treatment arm based on accumulating efficacy or safety data.

• Purpose: To increase the number of patients allocated to better-performing treatments, or decrease allocation to worse-performing/toxic treatments, often improving ethics and efficiency.


• How to Define:


• Target Allocation Ratio: Define the desired asymptotic allocation ratio (e.g., eventually 80% to the best arm).


• Statistical Model: Use a statistical model (e.g., Bayesian Dirichlet-multinomial model for binary outcomes, multi-arm bandit algorithms) that updates the probability of success for each arm after each new patient's outcome.


• Adaptive Randomization Algorithm: Specify the algorithm (e.g., 'play-the-winner,' 'bandit-based,' 'outcome-based sequential randomization') that translates the updated probabilities into new randomization weights.


• Randomization Procedure: Detail how the updated weights are used for the next block of randomization or for individual subject assignment.


• Initial Period: Often, a fixed randomization (e.g., 1:1:1) is used for an initial period to gather initial data before adaptation begins.


• Example Rule: "After the initial 100 subjects are randomized 1:1:1, for every subsequent block of 10 subjects, the randomization ratio will be updated using a Bayesian Dirichlet-multinomial model. The probability of assigning a subject to Treatment X will be proportional to [posterior probability of success for Treatment X]^eta, where eta = 2, using success/failure data accumulated for all prior subjects. This ensures more subjects are allocated to arms with higher observed success rates."

C) Sample Size Re-estimation (SSR) Rules

These rules allow adjusting the final sample size during the study.

• Purpose: To ensure adequate statistical power given true effect sizes or nuisance parameters (e.g., variance) that differ from initial assumptions, without over-enrolling subjects if initial assumptions were overly conservative.


• How to Define:


• Type of Re-estimation:


• Blinded SSR: Based on pooled data (not revealing treatment group differences). Used to re-estimate nuisance parameters like variance (for continuous outcomes) or event rates (for binary outcomes). This is generally safer for Type I error control.


• Unblinded SSR: Based on observed treatment effect differences. This allows for more precise power adjustment but requires careful pre-specification (e.g., using conditional power) to avoid Type I error inflation.


• Trigger for Re-estimation: Specify at which interim analysis points SSR will occur (e.g., after 50% of planned events).


• Decision Criteria:


• For Blinded SSR: "If the estimated pooled variance at the 50% interim analysis differs by more than 20% from the initial assumption, recalculate the required sample size using the new variance estimate. If the new sample size is larger, enroll up to a maximum of [X] additional subjects. If smaller, proceed with the original sample size unless a futility rule is met."


• For Unblinded SSR: "At the 70% interim analysis, if the observed conditional power to detect the original hypothesized effect size (with 80% power) is between 50% and 90%, the sample size will be increased to achieve 80% conditional power. If conditional power is below 50%, consider stopping for futility (separate rule). If above 90%, proceed with the original sample size. An alpha spending function will be used to adjust p-values for the final analysis."


• Maximum Sample Size: Always specify a maximum possible sample size to control costs and duration.


• Example Rule (Blinded): "At the 50% interim analysis, the independent DMC will unblind the variance of the primary endpoint from the combined treatment groups. If the re-estimated variance is different from the protocol-specified variance by more than 15%, the final sample size will be re-calculated to maintain 90% power, provided the new total sample size does not exceed 150% of the original plan."

D) Dose Adaptation Rules (Especially in Phase I/II)

These rules adjust the dose of an experimental treatment based on observed toxicity and/or efficacy.

• Purpose: To find the maximum tolerated dose (MTD) or the optimal biological dose (OBD) efficiently and safely.


• How to Define:


• Dose Escalation/De-escalation Algorithm:


• Rule-Based (e.g., 3+3 design): "Start with 3 patients at Dose Level 1. If 0/3 DLTs, escalate to next dose. If 1/3 DLTs, enroll 3 more at current dose. If 0/3 DLTs (0/6 total), escalate. If 1/6 DLTs (total 1/6), escalate. If 2/3 DLTs (total 2/3) or 2/6 DLTs (total 2/6), de-escalate or declare MTD below current dose."


• Model-Based (e.g., CRM, BOIN): Specify the statistical model (e.g., Bayesian logistic regression for CRM) that estimates the dose-toxicity curve and recommends the next dose. Specify the target toxicity rate (e.g., 25%).


• Stopping Rules: Define criteria for declaring the MTD, or stopping due to excessive toxicity or insufficient efficacy.


• Cohort Size: How many patients are evaluated at each dose level before a decision is made.


• Over-Dose Protection: Rules to prevent excessive toxicity (e.g., "If >2 patients at current dose experience DLT, no further escalation and consider de-escalation").


• Example Rule (BOIN): "The BOIN design will be used with a target DLT rate of 25%. Cohorts of 3 patients will be enrolled sequentially. If the observed DLT rate in a cohort is below the 'under-dosing' boundary, escalate to the next dose level. If it's above the 'over-dosing' boundary, de-escalate. Otherwise, remain at the current dose. The study will stop when the required sample size for the MTD estimation (e.g., 30 patients) is reached, or when the maximum number of DLTs at the lowest dose level is reached (e.g., 3 DLTs in 6 patients at Dose Level 1)."

E) Arm Selection/Dropping Rules (Multi-Arm Multi-Stage Designs)

These rules allow dropping inferior treatment arms or selecting superior ones at interim stages.

• Purpose: To efficiently screen multiple experimental treatments against a control, discarding unpromising arms early to focus resources on the most promising ones.


• How to Define:


• Comparison Method: Specify how arms are compared (e.g., against the common control, or against the "best" remaining experimental arm).


• Elimination Thresholds: Define statistical criteria for dropping an arm. This could be a p-value threshold, a lower bound of a confidence interval (e.g., if the lower bound of the 95% CI for the difference vs. control is below a futility margin), or a conditional power threshold.


• Number of Stages: Specify how many interim analyses (stages) for arm selection will occur.


• Alpha Control: MAMS designs require specific adjustments (e.g., using Dunnett's test variants or alpha spending functions across arms and stages) to control the family-wise error rate.


• Example Rule: "At the first interim analysis (Stage 1), after 100 patients per arm, any experimental arm for which the observed efficacy (e.g., response rate) is not statistically significantly better than the control (one-sided p > 0.15) or whose lower bound of the 95% confidence interval for the difference vs. control is below a pre-specified futility margin of -5% will be dropped from the study. All remaining arms will proceed to Stage 2 with additional enrollment."

1.3. Practical Steps for Defining Decision Rules

• Clearly Define Objectives: What are you trying to optimize (e.g., reduce sample size, increase power, find MTD, select best arm)?


• Select Adaptive Design Type: Choose the appropriate adaptive design framework (e.g., group sequential, multi-arm multi-stage, response-adaptive randomization, Bayesian adaptive).


• Identify Interim Analysis Points: Determine when and how many times data will be reviewed for potential adaptations (e.g., after every 'x' subjects, at 'y%' of total information).


• Specify Data to be Used: Which endpoints (primary, secondary, safety) will inform which decision rules?


• Determine Statistical Methods for Each Rule:


• For stopping: Alpha spending functions, conditional power, specific safety thresholds.


• For allocation: Bayesian models, bandit algorithms.


• For sample size: Re-estimation formulae for variance/effect size.


• For dose: CRM, BOIN, 3+3.


• For arm dropping: Multiple comparison procedures, futility margins.


• Quantify Thresholds: Set the numerical values for p-values, CI bounds, conditional power, effect sizes, event rates. These are critical and drive the design's operating characteristics.


• Detail Actions: Explicitly state what happens when a rule is met (e.g., "stop study," "change randomization ratio to X:Y:Z," "enroll 'N' more subjects," "drop Arm C").


• Conduct Extensive Simulations: This is non-negotiable.


• Simulate the design under various scenarios (true effect sizes, true variances, different DLT rates).


• Evaluate Type I error rate, power, average sample size, study duration, number of patients on suboptimal treatments.


• Refine rules and thresholds based on simulation results to achieve desired operating characteristics.


• Write the Statistical Analysis Plan (SAP): Document all rules, their statistical justification, and the procedures for implementing them in detail.

By systematically defining these decision rules and rigorously testing them through simulation, you can build robust and efficient adaptive experimental designs that maintain statistical integrity while being ethically responsive to accumulating data. This process often requires the expertise of an experienced biostatistician.