Formative Assessments In Development

Aquifer is developing new formative assessments for test-enhanced learning. As the non-profit leader in clinical learning, Aquifer is pleased to create high-quality assessments for learning to accompany clinical rotations for Family Medicine, Internal Medicine, Pediatrics, and Radiology. 

Aquifer’s new assessments combine the concepts of test-enhanced learning and distributed practice to help build more efficient, effective learners. The new assessments are:

  • Single-best answer multiple-choice exams with a certainty rating for each question.
  • Shorter, more frequent assessments that can be given twice during a clerkship or course.
  • 25-45 questions long and take approximately 1 hour to complete.
  • Easy to administer (no proctoring required). Assigned students complete the exam on their own, or as directed by their program.
  • Comprehensive and mapped to learning objectives and teaching points in Aquifer cases, which cover the national curriculum for STFM, AUR, COMSEP, and CDIM.
  • Developed, written, and peer-reviewed by teams of trained medical educators.
  • Included with your 2021-22 course subscription.

For Students

Calibration & Self-Directed Learning

Performance reports will allow students to:

  • Calibrate knowledge & confidence by providing performance details on accuracy and certainty linked to learning objectives and teaching points.
  • Inform efficient study planning by delivering granular feedback early and throughout a clerkship or course.
  • Provide self-directed learning opportunities with links directly to Aquifer cases that cover the content linked to each question.

For Faculty & Administrators

Identify At-Risk Students & Curricular Gaps

Performance reports will:

  • Bring to light gaps in curriculum by showing student performance on questions linked to learning objectives and teaching points.
  • Summarize the overall performance of individual students to identify at-risk students.
  • Support structured remediation and self-directed learning by providing direct links to Aquifer cases and suggestions for supporting student learning.

Learn More

We’re excited to share a detailed update and ask for your feedback as we hone the development of these new assessment tools.

Aquifer Consortium Assessment Leads for each course, led by Valerie J. Lang, MD, MHPE, Aquifer Academic Director for Assessment, worked to recruit and train medical educators from across the country to develop single best-answer multiple-choice questions for our new assessments. All new questions are mapped to key learning objectives and teaching points in Aquifer’s cases for each course. Because Aquifer’s courses cover the national curriculum (STFM, AUR, CDIM, and COMSEP), this provides a strong correlation between the new formative assessments and national curriculum standards.

Question authors were trained by assessment experts, and content was extensively peer-reviewed and edited during regular calls and day-long seminars at national meetings. These new questions will be added to Aquifer’s existing multiple-choice exams to provide psychometric validation prior to piloting formative assessments. Valid questions from our existing exams will also be used in the new formative assessments.

Technology development is currently underway. Aquifer plans to enter a robust pilot testing phase for Family Medicine, Pediatrics, and Radiology in 2021. Internal Medicine will be pilot tested after the completion of the question validation process.

If you are interested in learning more about pilot testing these innovative new exams, please complete our survey.


Existing Exam Update: Available Through July 2022

Our summative 100-question multiple-choice exams for Aquifer Family Medicine, Pediatrics, and Radiology will continue to be available in their current format until June 2022. Aquifer will no longer offer a summative exam option beyond June 2022.

The Aquifer Internal Medicine Clinical Decision-Making Exam, formerly the Key Features Exam, will not be impacted and will continue to be available at no cost for subscribers of Aquifer Internal Medicine. In addition, formative assessment tools to accompany our Internal Medicine course are in development.


Frequently Asked Questions

In August 2018, we surveyed both current Aquifer exam users and subscribers who don’t currently use our exams. We learned that:

  • Subscribers use Aquifer exams in a variety of ways. Some programs use the exams as a summative assessment, some as national exam practice, and some for formative assessment or pre-clerkship feedback.
  • Our subscribers would like more high-quality formative assessment tools that can be used as assessments for learning, providing immediate feedback to students to help them focus their learning.  About one-half of the respondents to our surveys told us that they would require their students to take formative assessment if a high-quality option was available.
  • About 80% of respondents agreed that offering formative assessment with granular feedback would promote students’ learning regarding patient care as well as improve student outcomes on the Shelf and National Board Exams
  • Immediate feedback, comprehensive curriculum coverage, and customization were highly valued as the most important aspects of formative assessment.

We believe the transition to formative-based assessment, directly aligned to the teaching content in our cases, will allow us to employ our assessment questions in a way that uniquely and proactively impacts student clinical learning.  Learn more about test-enhanced learning and why Aquifer chose to develop evidence-based formative exams.

As a non-profit organization, Aquifer’s mission is to enable learning that matters by building students’ essential knowledge, clinical reasoning and decision making skills. Our Aquifer Family Medicine, Pediatrics, and Radiology exams were originally developed to provide alternatives to NBME Shelf and AMSER exams that more closely aligned with each discipline’s curricula. As the quality of NBME Shelf exam coverage of national curricula has improved over the years, the value Aquifer contributes to the community through our own summative exams has diminished. In addition, clerkship directors told us that they want high quality, rigorous formative assessment for their students, but there are no resources that are directly linked to the national curriculum.

Aquifer courses are developed in collaboration with our national partners (STFM for Family Medicine, COMSEP for Pediatrics, AUR/AMSER for Radiology, AAIM/CDIM for Internal Medicine) to provide comprehensive coverage of their national curriculum. NBME Subject exams, like Aquifer courses, are designed to test a broad range of knowledge within the subject. Also like Aquifer courses, the Subject exams concentrate heavily on application and integration of knowledge rather than on recall of isolated facts. Students using Aquifer courses gain significant experience in applying knowledge in a virtual clinical context. Additionally, each Aquifer case is followed by a set of self-assessment questions to provide students with additional practice answering multiple-choice questions and to reinforce key concepts from the case.

Be sure to view our latest update and share your feedback on assessment format, administration, and reporting—and read about Aquifer’s evidence-based approach to building formative assessments for test-enhanced learning.

Feel free to reach out to Dr. Bridie Napier, Aquifer’s Director of Curricular Operations, at [email protected], with questions about this project. Please let us know if you would like to be included in helping us develop and pilot test these new tools.