Calibrate assessments are designed to leverage the concepts of test-enhanced learning and spaced repetition to drive self-directed learning in clinical students. Granular reports provide visibility for faculty into student knowledge gaps and certainty misalignment, enabling meaningful mid-clerkship feedback, informing coaching, and providing structured remediation.

Calibrate-question-certainty-rating

Calibrate assessments are:

  • Shorter, more frequent assessments that can be given twice during a clerkship or course to leverage spaced learning 
  • Single answer multiple-choice exams with a certainty rating for each question
  • 25-45 questions long and take approximately 1 hour
  • Comprehensive to all cases in each Aquifer course
  • Mapped to learning objectives and teaching points in Aquifer cases, which cover the national curriculum for STFM, AUR, COMSEP, and CDIM.
  • Developed, written, and peer-reviewed by teams of trained medical educators.
  • Easy to administer (no proctoring required)

Calibrate is Assessment for Learning

Formative
Informs student learning
Summative
Provides a score

HOW CALIBRATE HELPS

Calibrates Knowledge & Certainty

Identifies True Knowledge Gaps
  • Detailed reports go beyond right or wrong answers
  • Identifies areas of strength to expand upon during clinical experiences
  • Provides new dimensions for diagnosing and addressing problem areas
    • Under confidence
    • Over confidence
    • Knowledge gaps
Identify-knowledge-gaps

Drives Efficient Self-Directed Learning

  • Informs efficient study plans
  • Links to relevant, trusted content to address gaps & find answers
  • Supports accreditation standards on self-directed learning
    • LCME 6.3
    • COCA 6.7
Drives self-directed learning

Informs Meaningful Feedback and Coaching Conversations

  • Aligns students & faculty around a common tool
  • Provides objective data to drive conversations
  • Supports accreditation standards for formative assessment and feedback
    • LCME 9.7
    • COCA 9.5
Informs feedback and coaching

Aligns to National Clerkship Curriculum

  • Trust that your students are learning your clerkship’s core curricular objectives
  • Highlights areas of curricular strength and areas for improvements
  • Supports accreditation standards for curricular content and management
    • LCME 7.0 & 8.0
    • COCA 6.11
aligns to national curriculum

Easy to Adopt

Trusted Tools, Quality Content
  • Imbedded within the overall Aquifer learning system
  • Easy to administer
  • No proctoring required
  • Delivers highest quality assessments developed by your trusted peers and expert educators
Easy-to-adopt

Ways to Use Calibrate

  • Efficient study planning

  • Inform coaching

  • Mid-clerkship feedback

  • Detect curricular gaps

  • Structured remediation

  • Exam preparation

EVIDENCE-BASED FORMATIVE ASSESSMENTS

Calibrate is assessment for learning. Moving beyond the traditional summative assessment of learning, test-enhanced learning is a pedagogical approach in which students are continuously assessed and re-assessed as a formative process.

Research has shown that the “retrieval practice” of taking multiple formative assessments over time enhances students’ ability to recall, retain, and apply information over traditional study methodology. The studied effects are consistent for all health professions, learner levels, formats and learning outcomes (Green et al 2018). Test-enhanced learning is useful for helping students identify relatively weaker areas in need of further study, as well as areas of relative strength on which to build further, allowing for more focused and efficient study methods.

We know that medicine is more than just knowing a collection of facts. Clinicians in training must be able to integrate the information that they’ve acquired and apply that knowledge to a unique patient and to develop a diagnostic and therapeutic plan. To accomplish this, students must develop the skills of a master adaptive learner–a clinician who continuously utilizes the metacognitive approach to self-regulated learning that ultimately leads to the development of adaptive clinical expertise (Cutrer et al. 2017, Academic Medicine; Cutrer et al. 2018, Medical Teacher).

Aquifer calibrate confidence gapUNCOVERING OVERCONFIDENCE

With a certainty rating for each question, Calibrate aligns knowledge and certainty, and provides visibility through detailed reports. Arming students with this added level of information delivers a more accurate picture of their level of understanding, and ability to address true gaps.

learning-process-adaptive-expertise

PREVIEW: THE POWER OF CALIBRATE REPORTS

Calibrate reports deliver instant, actionable results to students, and provide a unique look at cohort and student summary performance for faculty.

Key Element: Clinical Learning Calibration

A calculation that considers accuracy, certainty, and efficiency to summarize student performance.

Clinical Learning Calibration
Student Performance Reports

Student reports provide detailed information to illuminate knowledge gaps and drive efficient study planning. Reports include:

  • Coaching suggestions & next steps
  • Summary performance & comparison between early & late assessments
  • Option to share reports directly with faculty coaches
Calibrate-student-report-summary
Detailed Performance & Links to Learning Resources

For each teaching point and learning objective, students can see their accuracy and certainty, along with the relevant system, whether they have already completed the related Aquifer case, and link directly to the teaching point or full Aquifer case. Reports include:

  • Detailed performance by learning objective/teaching point, including accuracy and certainty ratings
  • Direct links to the relevant teaching point & Aquifer case for self-directed learning
  • Sort the report to easily view areas that were not accurate or lacked certainty

Note: Faculty do not have access to detailed student performance to preserve a safe space for learners.

Aquifer Calibrate student performance details
For Students & Coaches: Debrief & Learning Plan Worksheet

PREPARATION & REFLECTION TO DRIVE MEANINGFUL CONVERSATIONS

A critical component of Calibrate is the student debrief worksheet, which walks students through the process of analyzing and reflecting on their assessment results, noting important patterns where they exist, and creating an actionable learning plan to both leverage strengths and address gaps and areas of miscalibration.

We recommend that students share the worksheet with coaches—along with the Coach's Report—to help both participants prepare for a productive discussion.

Calibrate-Debrief-Learning-Plan
For Coaches: Student Performance Summary

OBJECTIVE DATA TO FUEL COACHING CONVERSATIONS

Students have the option to share a summary of their performance directly with a preceptor, faculty member, resident, tutor, or peer. Along with the summary data about the student's assessment, coaches also receive coaching tips and access to quick, efficient tools to help them support the student in a meaningful way.

Aquifer Calibrate Coaching Report
For Faculty: Cohort Performance Reports

HIGHLIGHTING CURRICULAR GAPS

Compare cohort performance across early and late administrations of the assessment, and find curricular gaps easily by seeing cohort performance at a granular level, associated with each teaching point and learning objective covered.

Aquifer-calibrate-reports
Aquifer Calibrate faculty reports cohort detail performance
For Faculty: Student Performance Summary Reports

IDENTIFY AT-RISK STUDENTS

High-level student performance summaries support early identification of students in need of coaching or remediation.

The Learning Progress flag is an overall summary of student performance.

Aquifer Calibrate faculty report student performance summary

Access Calibrate

“Giving our Curricular Partners early access to our new tools and pilot program is like nothing we’ve ever done before. We’re really inviting these programs into our innovation space and counting on them to provide feedback to help us optimize these new learning tools before we extend availability across medical and health professions education.

We’re so excited to bring them into the fold and truly partner with them, working together to build what’s coming next.”

Leslie Fall, MD, Aquifer Chief Executive Officer

APPLY NOW: PILOT PROGRAM

The Aquifer Calibrate Pilot Program is now accepting applications from MD and DO confirmed Aquifer Curricular Partner Programs. This phase of the pilot will begin on October 1, 2022. 

Participating in the Aquifer Calibrate pilot will provide the unique opportunity to become a true partner in Aquifer’s innovation process. As we prepare to debut this unique, transformative new assessment system, we will be looking for faculty and administrator partners to conduct the assessments with their students and provide early feedback to inform optimizations. Pilot schools will also have access to faculty development sessions on coaching as part of this program.

Pilot phases:

– MD & DO Student Beta Test (Summer 2022)

– MD & DO Curricular Partner Program Pilot (Applications now open, pilot access begins October 1)

– PA & NP Curricular Partner Programs Pilot (late 2022/early 2023)

LEARN MORE

WEBINAR - Aquifer Calibrate: Formative Assessment for Efficient & Effective Clinical Learning

Find out how Aquifer Calibrate supports students and faculty with evidence-based approaches to build more efficient, effective learners.

Podcast: Coaching Conversations – Tips & Tools to Empower Learners

Dr. Sherilyn Smith provides tips on how to prepare for and conduct coaching conversations that develop students into lifelong learners who are able to create plans, take time to reflect, identify personal learning gaps, and are motivated to fill those gaps. Dr. Smith also discusses how Aquifer’s new Calibrate formative assessment system benefits faculty and students by providing information that can inform and enhance their coaching conversations. Full show notes can be found here, or explore more episodes from the Aquifer Educator Connection Podcast.