This is an excellent request! Mixed methods are powerful in implementation science because they allow you to understand what happened (quantitative outcomes) and why it happened (qualitative insights).

Let's break this down into the two main parts.

Part 1: Measuring Implementation Outcomes Quantitatively

The Proctor et al. (2011) taxonomy of implementation outcomes is the gold standard here. For each outcome you listed, I'll provide a definition, common quantitative metrics, data sources, and key considerations.

General Considerations for Quantitative Measurement of Implementation Outcomes:

• Standardized Measures: Where possible, use or adapt existing, validated scales (e.g., for acceptability, appropriateness, feasibility). This enhances comparability across studies.


• Data Sources: Be strategic about where you'll get the data (e.g., electronic health records, program logs, surveys, direct observation).


• Baselines: Whenever possible, collect baseline data to compare changes over time.


• Reliability & Validity: Ensure your chosen measures are reliable (consistent) and valid (measure what they're supposed to measure).


• Context: Tailor your metrics to the specific intervention, setting, and target population.


• Denominator: Clearly define the denominator for all proportions and rates (e.g., "X% of eligible providers," "Y% of patients who received the intervention").

Implementation Outcomes & Quantitative Metrics:

1. Adoption

• Definition: The proportion of eligible settings, providers, or clients who intend to initiate or actually initiate use of an intervention.


• Quantitative Metrics:


• Proportion of Organizations/Sites: (Number of organizations adopting / Total number of eligible organizations) x 100%.


• Proportion of Providers/Staff: (Number of eligible providers adopting / Total number of eligible providers) x 100%.


• Absolute Number of Adopters: Count of new users (sites, providers, patients) over a specified period.


• Rate of Adoption: (Number of new adopters / Time period) per eligible unit.


• Enrollment Rates: For patient-level interventions, the percentage of eligible patients who enroll.


• Data Sources:


• Program enrollment logs or registries.


• Organizational administrative data.


• Electronic Health Records (EHRs) – e.g., number of providers who placed a specific order, or number of patients receiving a new service.


• Surveys of organizations/providers (asking about their intent to adopt or actual adoption).


• Considerations: Clearly define "eligible" and "adopting." Is "adoption" defined by attending training, using it once, or consistently using it?

2. Fidelity

• Definition: The degree to which an intervention was implemented as intended by the developers.


• Quantitative Metrics:


• Adherence to Components:


• (Number of intervention components delivered / Total number of components designed) x 100%.


• Score on a fidelity checklist (e.g., sum of points for delivered components).


• Dose Delivered:


• Number of sessions completed, duration of sessions, frequency of contact.


• Percentage of prescribed dose achieved (e.g., [Actual dose delivered / Intended dose] x 100%).


• Competence: Ratings of provider skill or quality of delivery using standardized observational scales.


• Adaptations: Number or type of adaptations made (can be quantified by coding qualitative descriptions).


• Data Sources:


• Direct observation of intervention delivery (e.g., live or recorded sessions).


• Self-report checklists completed by implementers.


• Client/patient report of services received.


• Program logs, administrative data (e.g., number of times a specific protocol was followed in an EHR).


• Intervention manuals/protocols for comparison.


• Considerations: Requires a clear, manualized description of the intervention. Distinguish between adherence, dose, and competence. Decide how to score adaptations (e.g., necessary vs. unnecessary, critical vs. minor).

3. Sustainment

• Definition: The extent to which an intervention is maintained or institutionalized in a setting after external supports (e.g., funding from a research grant) are removed.


• Quantitative Metrics:


• Continued Use: (Number of sites/providers still using the intervention at X months/years post-implementation / Number of sites/providers who adopted) x 100%.


• Duration of Use: Average number of months/years an intervention remains active.


• Budget Allocation: Percentage of organizational budget allocated to the intervention over time.


• Staffing Levels: Number of staff dedicated to the intervention post-funding.


• Integration into Routine Practice: Scores on surveys assessing the degree of institutionalization (e.g., "organizational routinization scale").


• Data Sources:


• Longitudinal follow-up surveys with sites/providers.


• Organizational administrative data (budget reports, staffing records).


• EHR data (continued use of specific tools or services).


• Program logs.


• Considerations: Defining the "post-implementation" phase and the specific time points for measurement is crucial. What constitutes "sustained"?

4. Penetration

• Definition: The integration of an intervention within a service setting and its subsystems. Often refers to the reach within an adopting organization. (Distinct from adoption, which is about whether an organization takes it up at all).


• Quantitative Metrics:


• Reach within Adopting Units: (Number of eligible individuals within adopting sites who received the intervention / Total number of eligible individuals within adopting sites) x 100%.


• Coverage: Percentage of the target population within an adopting system who are exposed to the intervention.


• Usage Rates: Frequency or intensity of intervention use by patients/clients in an adopting setting.


• Spread: Number of departments or teams within an organization that have integrated the intervention.


• Data Sources:


• EHRs (e.g., number of patients seen by a specific service, or number of patients who received a specific screening).


• Patient tracking systems.


• Program logs, administrative data from adopting sites.


• Surveys of staff or patients within adopting organizations.


• Considerations: Requires clear definition of the target population within the adopting unit/system. How deep does the intervention need to be integrated to count?

5. Acceptability

• Definition: The perception among implementation stakeholders (e.g., intervention recipients, implementers) that a new treatment, service, practice, or innovation is agreeable, palatable, or satisfactory.


• Quantitative Metrics:


• Scores on Standardized Scales:


• Acceptability of Intervention Measure (AIM): A 4-item scale (Likert-type) assessing how acceptable an intervention is perceived to be.


• Treatment Acceptability/Adherence Scale (TAAS): Used in some contexts.


• Satisfaction Scores: Mean scores on Likert-type scales (e.g., "On a scale of 1-5, how satisfied are you with this intervention?").


• Willingness to Recommend: Percentage of stakeholders who would recommend the intervention to others.


• Dropout/Refusal Rates: (Number of individuals refusing/dropping out / Total number offered) x 100% (lower rates suggest higher acceptability).


• Data Sources:


• Surveys administered to implementers, recipients (patients), and administrators.


• Exit interviews (can be quantified by coding responses or using scales).


• Considerations: Who is assessing acceptability (patients, providers, administrators)? What aspects of the intervention are they assessing? Ensure anonymity for honest responses.

6. Appropriateness

• Definition: The perceived fit, relevance, or compatibility of the intervention or innovation with a given practice setting, an innovation's host, or an implementation issue.


• Quantitative Metrics:


• Scores on Standardized Scales:


• Intervention Appropriateness Measure (IAM): A 4-item scale (Likert-type) assessing the perceived fit and relevance.


• Relevance Scores: Mean scores on Likert-type scales (e.g., "To what extent do you feel this intervention is relevant to your practice?").


• Compatibility Scores: Mean scores on Likert-type scales (e.g., "This intervention is compatible with our existing workflows.").


• Data Sources:


• Surveys administered to implementers, recipients, and administrators.


• Structured interviews with stakeholders.


• Considerations: Similar to acceptability, consider who is assessing and what aspects of appropriateness are being measured (e.g., fit with organizational culture, patient needs, workflow).

7. Feasibility

• Definition: The extent to which an intervention can be successfully carried out within a given setting or system.


• Quantitative Metrics:


• Scores on Standardized Scales:


• Feasibility of Intervention Measure (FIM): A 4-item scale (Likert-type) assessing the practicality and ease of implementation.


• Resource Utilization:


• Time spent on implementation (e.g., staff hours, training hours).


• Cost of implementation (e.g., staff time, materials, technology).


• Burden Scores: Mean scores on Likert-type scales assessing perceived workload or difficulty.


• Rate of Completion of Tasks: Percentage of preparatory or implementation tasks completed.


• Number of Barriers Encountered: Can be quantified by coding reported barriers and counting frequencies.


• Data Sources:


• Surveys administered to implementers and administrators.


• Time logs, cost accounting data.


• Project management records.


• Qualitative data (e.g., interviews on barriers) coded quantitatively.


• Considerations: Feasibility can relate to financial, human, or logistical resources. It's often assessed prospectively before full-scale implementation or concurrently during piloting.

Part 2: Designing Mixed Methods Approaches

Mixed methods designs for implementation science typically follow Creswell's typologies. The key is how the quantitative (Quan) and qualitative (Qual) data are integrated to answer your research questions.

General Principles for Mixed Methods Design:

• Rationale: Clearly state why you're mixing methods (e.g., triangulation, explanation, exploration, development).


• Timing: Are the methods conducted sequentially (one after the other) or concurrently (at the same time)?


• Weighting: Is one method given more prominence than the other (e.g., Quan > Qual, Qual > Quan, or equal)?


• Integration: At what stage are the data merged or connected? (e.g., during data collection, analysis, interpretation).


• Research Questions: Frame your questions so that they explicitly invite both quantitative and qualitative data.

Mixed Methods Designs & Application to Implementation Outcomes:

1. Sequential Explanatory Design (Quan -> Qual)

• Purpose: To use qualitative data to explain, elaborate on, or explore unexpected or noteworthy findings from the initial quantitative phase.


• Design: Collect and analyze quantitative data first. Then, based on those results, design and collect qualitative data to gain deeper understanding.


• Integration: Connect the qualitative data to specific quantitative findings during interpretation.


• When to Use:


• When you have clear quantitative hypotheses about implementation outcomes.


• When you want to understand why certain outcomes were high or low, or how they came to be.


• To explain variations in implementation outcomes across sites or providers.


• Example Research Questions & Application:


• Overall RQ: How and why did the implementation of the new patient safety checklist vary across hospital units?


• Quan RQ (Phase 1): What are the adoption rates, fidelity scores, and perceived appropriateness of the new patient safety checklist across N hospital units?


• Measures:


• Adoption: % of eligible units using the checklist.


• Fidelity: Score on a 10-item checklist completion during direct observation or chart review.


• Appropriateness: Mean IAM scores from unit staff.


• Analysis: Calculate descriptive statistics, compare means/proportions across units.


• Qual RQ (Phase 2, based on Quan findings): What organizational, provider, or intervention factors explain the observed variations in checklist adoption, fidelity, and appropriateness, especially in units with notably high or low outcomes?


• Integration:


• If Quan shows low adoption in Unit A but high in Unit B: Conduct in-depth interviews with staff and managers in Unit A and Unit B to explore perceived barriers (e.g., lack of time, poor fit with workflow, perceived lack of leadership support) and facilitators (e.g., champions, clear communication) respectively.


• If Quan shows high appropriateness scores but low fidelity: Use focus groups to understand if staff believe it's a good idea, but face practical challenges in doing it correctly (e.g., training gaps, resource constraints).


• Analysis: Thematic analysis of qualitative data, looking for themes that directly explain the quantitative results.

2. Convergent Parallel Design (Quan + Qual, simultaneous)

• Purpose: To simultaneously collect and analyze both quantitative and qualitative data to provide a comprehensive understanding of a phenomenon by converging or triangulating findings.


• Design: Collect and analyze quantitative and qualitative data independently during the same time frame. Then, merge the results during the interpretation phase.


• Integration: Compare and contrast the findings from both datasets to see if they corroborate, diverge, or complement each other.


• When to Use:


• When you want to corroborate quantitative findings with qualitative depth.


• To gain different perspectives on the same implementation outcomes.


• To develop a richer, more nuanced understanding than either method could provide alone.


• Example Research Questions & Application:


• Overall RQ: What is the acceptability and feasibility of a new telehealth intervention for chronic disease management, and how do providers and patients perceive these aspects?


• Quan RQ (Simultaneous): What are the mean acceptability and feasibility scores among N providers and M patients using the new telehealth intervention?


• Measures:


• Acceptability: AIM scores for providers and patients.


• Feasibility: FIM scores for providers (e.g., ease of use, time burden).


• Analysis: Calculate descriptive statistics, conduct t-tests/ANOVAs.


• Qual RQ (Simultaneous): What are providers' and patients' experiences and perceptions regarding the usefulness, ease of use, and integration challenges of the new telehealth intervention?


• Measures: Semi-structured interviews or focus groups with a subset of providers and patients, exploring their experiences with the intervention, perceived benefits, challenges, and general satisfaction.


• Analysis: Thematic analysis to identify common themes.


• Integration:


• Compare Quan scores with Qual themes: If quantitative AIM scores are high, do the qualitative interviews reveal specific aspects that are highly valued (e.g., convenience, effectiveness)?


• If Quan FIM scores are moderate: Do qualitative interviews reveal specific barriers (e.g., technical glitches, lack of training, integration with EHR) that contribute to the moderate scores?


• Look for divergence: If patients report high satisfaction (Quan) but providers report high burden (Qual themes), this highlights important tensions to address. The data sets provide different, but related, insights into the same concepts.

3. Sequential Exploratory Design (Qual -> Quan)

• Purpose: To use qualitative data to explore a phenomenon first, and then use the findings to develop and test quantitative measures or hypotheses in a subsequent quantitative phase.


• Design: Collect and analyze qualitative data first. Use these findings to inform the development of a quantitative instrument (e.g., survey, checklist) or to identify key variables for a subsequent quantitative study.


• Integration: The qualitative findings directly shape the design and content of the quantitative phase.


• When to Use:


• When little is known about a particular implementation challenge or outcome.


• To develop new, context-specific measures for implementation outcomes.


• To identify key constructs or themes from the ground up before measuring them broadly.


• Example Research Questions & Application:


• Overall RQ: What are the key factors influencing the sustainment of a peer support program for new mothers in rural clinics, and how prevalent are these factors across multiple clinics?


• Qual RQ (Phase 1): What factors do clinic staff and program participants identify as critical for the sustainment of the peer support program?


• Measures: In-depth interviews and focus groups with staff (nurses, administrators) and participants from a few clinics where the program has been in place for some time. Explore challenges, successes, perceived value, resource allocation, and program adaptations.


• Analysis: Thematic analysis to identify emergent themes related to sustainment (e.g., perceived value, staff turnover, funding, leadership support, community engagement).


• Quan RQ (Phase 2, based on Qual findings): To what extent are the key sustainment factors identified in the qualitative phase present or perceived across a larger sample of rural clinics implementing the peer support program?


• Integration: Based on the qualitative themes (e.g., "leadership support," "adequate funding," "staff champions," "integration into workflow"), develop a survey instrument with Likert-type scales or yes/no questions to quantitatively measure the prevalence and perceived strength of these factors related to program sustainment.


• Measures: Administer the newly developed survey to a larger sample of rural clinics. Also, collect quantitative sustainment data (e.g., continued program operation, budget allocation, staffing levels) for each clinic.


• Analysis: Descriptive statistics for the survey items, correlation/regression analysis to see which factors are significantly associated with higher rates of program sustainment.

Key Considerations for All Mixed Methods Designs in Implementation Science:

• Theoretical Framework: Always ground your study in an implementation science framework (e.g., CFIR, RE-AIM, PRISM, EPIS). This will help you select relevant implementation outcomes, guide your research questions, and interpret your findings within a broader theoretical context.


• Sampling: Be thoughtful about how your quantitative and qualitative samples relate. Are they the same people? Different but linked?


• Data Analysis: Plan your analysis concurrently for both data types, considering how you will manage and analyze mixed data effectively.


• Integration Points: Clearly articulate where and how you will integrate your data in each phase of the study. This is the "mixed" part!


• Reporting: Be transparent about your methods and how the different datasets contributed to your overall understanding.