Using Cohort Performance Data to Drive Curriculum Redesign

Calibrate performance reports reveal strengths and gaps. Learn how to turn these insights into actionable curriculum improvements.

In today’s rapidly evolving healthcare landscape, academic programs must continually adapt to ensure that graduates are prepared for the realities of clinical practice. Whether training physicians, PAs, or nurse practitioners, the challenge remains the same: how do we identify where learners excel and struggle, and then translate those insights into meaningful curriculum changes?

The Calibrate Curricular Performance Summary Report and Curricular Performance Report provide exactly that opportunity. When used together, these tools move beyond anecdotal impressions, providing educators with concrete evidence to guide curriculum design and renewal.

What Calibrate Reports Show

Curricular Performance Summary Report: This report aggregates the same data across disciplines, body systems, and clinical focus areas. By summarizing patterns across multiple items, it highlights broader trends in performance that might otherwise be obscured by item-level variation.

Curricular Performance Report: This detailed report drills down to individual items, learning objectives, and teaching points. For each cohort of students, faculty can view completion rates, accuracy, and the distribution of learners across performance categories (Excellent, Competent, Unsure, Novice, and Warning). In essence, it shines a light on exactly how well students performed on each item. Given that items are linked to individual Aquifer learning objectives and teaching points within clinical cases, this report highlights the strengths and weaknesses of this cohort.  

Applying Data to Curriculum Design

1. Spotting Areas of Strength

When performance is consistently strong—for example, if students excel in cardiovascular diagnosis across multiple items—it suggests that current teaching strategies, sequencing, or clinical exposures in that domain are effective. Recognizing and reinforcing what’s working is just as valuable as identifying gaps. As an early adopter of Calibrate, Anne Walsh, Clinical Associate Professor at the Chapman University Physician Assistant Program, integrated Calibrate into an elective rotation and used its cohort and student reports to refine implementation and focus remediation efforts. She discovered that over 60% of students improved from early to late assessment—evidence the reporting loop directly shaped course delivery.

2. Identifying Gaps in Competency

Low accuracy percentages or a high proportion of “novice” responses clustered around specific body systems or skills should prompt a closer look. If students repeatedly underperform on gastrointestinal management questions, for instance, this may signal:

• Insufficient case exposure
• Lack of alignment between preclinical and clinical teaching
• A need for supplementary resources or simulation experiences

By triangulating these findings with faculty observations and student feedback, programs can prioritize where to invest limited curricular time and energy.

3. Aligning With Accreditation Standards

Accreditation bodies emphasize the importance of competency-based education and the use of outcomes data to demonstrate effectiveness. The cohort reports provide ready-made evidence of student performance that can be mapped to competencies such as patient care, medical knowledge, and clinical reasoning. Integrating these data into curriculum committee discussions can strengthen both the design process and accreditation documentation.

4. Supporting Interprofessional Learning

Because the reports span multiple clinical disciplines and body systems, they can inform conversations across programs. For example, suppose both NP and PA cohorts within the same program show similar weaknesses in musculoskeletal diagnosis. In that case, it may be an opportunity to develop a shared interprofessional learning activity.

5. Monitoring Change Over Time

Most importantly, the reports can serve as benchmarks. After making curricular adjustments—adding a workshop, restructuring a course sequence, or embedding more deliberate practice opportunities—subsequent cohorts’ data can reveal whether those changes had the desired effect.

Making It Practical

• Bring the data to curriculum meetings.
  Begin by reviewing the summary report to identify high-level trends, and then drill down into the detailed report for context and potential root causes.
• Engage faculty champions.
  Share targeted excerpts with course directors or clerkship leaders whose content aligns with the identified gaps.
• Use the data for faculty development.
  If performance issues cluster around teaching points that require nuanced clinical reasoning, targeted faculty workshops on teaching those skills may be warranted.

Encouraging a Culture of Continuous Improvement

Curriculum design and redesign are not one-time events but an ongoing cycle of assessment, reflection, and adjustment. These Calibrate reports give educators actionable, evidence-based insights to fuel that cycle.

For MD, DO, PA, and NP programs alike, these tools transform performance data into a roadmap for building stronger curricula—ensuring that the next generation of healthcare professionals is prepared to meet the challenges of modern practice.

Ready to implement these insights? Schedule a curricular consult with our team to explore how performance data can directly inform your program’s next steps.