Calibrate assessments are designed to leverage the concepts of test-enhanced learning and spaced repetition to drive self-directed learning in clinical students. Granular reports provide visibility for faculty into student knowledge gaps and certainty misalignment, enabling meaningful mid-clerkship feedback, informing coaching, and providing structured remediation.
Calibrate assessments are:
- Shorter, more frequent assessments that can be given twice during a clerkship or course to leverage spaced learning
- Single answer multiple-choice exams with a certainty rating for each question
- 25-45 questions long and take approximately 1 hour
- Comprehensive to all cases in each Aquifer course
- Mapped to learning objectives and teaching points in Aquifer cases, which cover the national curriculum for STFM, AUR, COMSEP, and CDIM.
- Developed, written, and peer-reviewed by teams of trained medical educators.
- Easy to administer (no proctoring required)
Calibrate is Assessment for Learning
Formative
Informs student learning
Summative
Provides a score
HOW CALIBRATE HELPS
Calibrates Knowledge & Certainty
Identifies True Knowledge Gaps
- Detailed reports go beyond right or wrong answers
- Identifies areas of strength to expand upon during clinical experiences
- Provides new dimensions for diagnosing and addressing problem areas
- Under confidence
- Over confidence
- Knowledge gaps
Drives Efficient Self-Directed Learning
- Informs efficient study plans
- Links to relevant, trusted content to address gaps & find answers
- Supports accreditation standards on self-directed learning
- LCME 6.3
- COCA 6.7
Informs Meaningful Feedback and Coaching Conversations
- Aligns students & faculty around a common tool
- Provides objective data to drive conversations
- Supports accreditation standards for formative assessment and feedback
- LCME 9.7
- COCA 9.5
Aligns to National Clerkship Curriculum
- Trust that your students are learning your clerkship’s core curricular objectives
- Highlights areas of curricular strength and areas for improvements
- Supports accreditation standards for curricular content and management
- LCME 7.0 & 8.0
- COCA 6.11
Easy to Adopt
Trusted Tools, Quality Content
- Imbedded within the overall Aquifer learning system
- Easy to administer
- No proctoring required
- Delivers highest quality assessments developed by your trusted peers and expert educators
Ways to Use Calibrate
Efficient study planning
Inform coaching
Mid-clerkship feedback
Detect curricular gaps
Structured remediation
Exam preparation
Evidence-based Formative Assessments
Calibrate is assessment for learning. Moving beyond the traditional summative assessment of learning, test-enhanced learning is a pedagogical approach in which students are continuously assessed and re-assessed as a formative process.
Research has shown that the “retrieval practice” of taking multiple formative assessments over time enhances students’ ability to recall, retain, and apply information over traditional study methodology. The studied effects are consistent for all health professions, learner levels, formats and learning outcomes (Green et al 2018). Test-enhanced learning is useful for helping students identify relatively weaker areas in need of further study, as well as areas of relative strength on which to build further, allowing for more focused and efficient study methods.
We know that medicine is more than just knowing a collection of facts. Clinicians in training must be able to integrate the information that they’ve acquired and apply that knowledge to a unique patient and to develop a diagnostic and therapeutic plan. To accomplish this, students must develop the skills of a master adaptive learner–a clinician who continuously utilizes the metacognitive approach to self-regulated learning that ultimately leads to the development of adaptive clinical expertise (Cutrer et al. 2017, Academic Medicine; Cutrer et al. 2018, Medical Teacher).
UNCOVERING OVERCONFIDENCE
With a certainty rating for each question, Calibrate aligns knowledge and certainty, and provides visibility through detailed reports. Arming students with this added level of information delivers a more accurate picture of their level of understanding, and ability to address true gaps.
PREVIEW: THE POWER OF CALIBRATE REPORTS
Calibrate reports deliver instant, actionable results to students, and provide a unique look at cohort and student summary performance for faculty.
Key Element: Clinical Learning Calibration
A calculation that considers accuracy, certainty, and efficiency to summarize student performance.
Student Performance Reports
Student reports provide detailed information to illuminate knowledge gaps and drive efficient study planning. Reports include:
- Coaching suggestions & next steps
- Summary performance & comparison between early & late assessments
- Option to share reports directly with faculty coaches
Detailed Performance & Links to Learning Resources
For each teaching point and learning objective, students can see their accuracy and certainty, along with the relevant system, whether they have already completed the related Aquifer case, and link directly to the teaching point or full Aquifer case. Reports include:
- Detailed performance by learning objective/teaching point, including accuracy and certainty ratings
- Direct links to the relevant teaching point & Aquifer case for self-directed learning
- Sort the report to easily view areas that were not accurate or lacked certainty
Note: Faculty do not have access to detailed student performance to preserve a safe space for learners.
For Students & Coaches: Debrief & Learning Plan Worksheet
PREPARATION & REFLECTION TO DRIVE MEANINGFUL CONVERSATIONS
A critical component of Calibrate is the student debrief worksheet, which walks students through the process of analyzing and reflecting on their assessment results, noting important patterns where they exist, and creating an actionable learning plan to both leverage strengths and address gaps and areas of miscalibration.
We recommend that students share the worksheet with coaches—along with the Coach's Report—to help both participants prepare for a productive discussion.
For Coaches: Student Performance Summary
OBJECTIVE DATA TO FUEL COACHING CONVERSATIONS
Students have the option to share a summary of their performance directly with a preceptor, faculty member, resident, tutor, or peer. Along with the summary data about the student's assessment, coaches also receive coaching tips and access to quick, efficient tools to help them support the student in a meaningful way.
For Faculty: Cohort Performance Reports
HIGHLIGHTING CURRICULAR GAPS
Compare cohort performance across early and late administrations of the assessment, and find curricular gaps easily by seeing cohort performance at a granular level, associated with each teaching point and learning objective covered.
For Faculty: Student Performance Summary Reports
IDENTIFY AT-RISK STUDENTS
High-level student performance summaries support early identification of students in need of coaching or remediation.
The Learning Progress flag is an overall summary of student performance.
Access Calibrate
“Giving our Curricular Partners early access to our new tools and pilot program is like nothing we’ve ever done before. We’re really inviting these programs into our innovation space and counting on them to provide feedback to help us optimize these new learning tools before we extend availability across medical and health professions education.
We’re so excited to bring them into the fold and truly partner with them, working together to build what’s coming next.”
Leslie Fall, MD, Aquifer Chief Executive Officer
PILOT PROGRAM
Aquifer Calibrate is currently undergoing pilot testing. Calibrate will be available with your 2023-24 subscription to Family Medicine, Internal Medicine, Pediatrics, and Radiology starting July 1, 2023.
LEARN MORE
Podcast: Coaching Conversations – Tips & Tools to Empower Learners
Dr. Sherilyn Smith provides tips on how to prepare for and conduct coaching conversations that develop students into lifelong learners who are able to create plans, take time to reflect, identify personal learning gaps, and are motivated to fill those gaps. Dr. Smith also discusses how Aquifer’s new Calibrate formative assessment system benefits faculty and students by providing information that can inform and enhance their coaching conversations. Full show notes can be found here, or explore more episodes from the Aquifer Educator Connection Podcast.
