New frontier for evaluating dental students

Think communication, trust and giving oral health care advice

A patient sits in a dental chair anxiously awaiting a bone graft procedure. He wonders if the procedure will hurt and how he will feel once it’s over.

Amanda Heydari, an OHSU School of Dentistry periodontic resident, walks into the exam area and sits down next to the patient. She confidently explains to the patient how he’ll receive nitrous oxide to sedate him during the procedure and the other comfort steps. The patient visibly relaxes.

Meanwhile, a periodontology professor is nearby, listening to the exchange to ensure the resident is covering all the necessary patient care steps. Later, the professor will record the resident’s progress using a new type of assessment tool.

“I’m graduating in a month, and I want to be able to do these procedures myself,” said Heydari, a member of the Class of 2025. She appreciates the new tool that is evaluating her in the moment while she cares for patients.

Flying a plane

The OHSU School of Dentistry evaluates its students and residents through a mixture of daily clinical assessments and a review of presentations.

But that is changing.

Faculty at the School of Dentistry and School of Medicine are using a new assessment technique for students that focuses on people skills instead of stand-alone technical skills.

Think communication, trust and giving health care advice.

The process is like learning to fly a plane.

“You fly a certain number of hours with your flight instructor. You’re sitting in the plane whole time, but every 10 seconds the flight instructor is correcting something that you’re doing wrong,” said Tracy Bumsted, M.D., M.P.H., professor of pediatrics and associate dean of undergraduate medical education, OHSU School of Medicine.

The student is getting their required hours, but they may not be demonstrating the level of proficiency and trust necessary to perform the task without supervision.

Building blocks

Dentistry, like other health care programs, has long used competencies to evaluate a student’s learning proficiency. Since the 1970s, higher education institutions in the United States have used skills and abilities learned in the classroom as a yardstick for being ready to care for patients.

But now there’s another solution on the horizon.

A small group of dental professionals from across the U.S. are spearheading an evolving concept that evaluates overall emotional intelligence—an approach that is redefining student readiness for practicing clinical dentistry.

“This will change graduation assessments,” said Phillip Marucha, D.M.D, Ph.D., professor of periodontology, OHSU School of Dentistry. He’s part of a national workgroup formed four years ago by the American Dental Education Association, or ADEA.

The concept is called Entrustable Professional Activities and was coined by Theodorus (Olle) ten Cate, Ph.D., emeritus professor of medical education at Utrecht University in the Netherlands and visiting chair of surgery at the University of California, San Francisco, School of Medicine.

Entrustable Professional Activities, or EPAs, are building blocks that are completed over the span of a student’s training. They are learned activities, responsibilities and people skills that students will eventually carry out on their own.

Examples include diagnosing a patient, taking a patient history, communicating with peers or advising a patient.

Dental feedback

The School of Dentistry is exploring aspects of EPAs to use in its doctor of dental medicine, or DMD, curriculum in preparation for folding EPAs into its assessment process. The school is already using the technique in its advanced dental education program for periodontics.

The areas being explored to add to the DMD program are:

• clinical situations where students get immediate faculty input
• types of data to collect
• a data platform shared with higher education dental institutions across the U.S.

“The oral health work group is focusing on filling in the knowledge, skills and attitudes for dentistry to build the EPAs the same way that the Association of American Medical Colleges has done. We are also working with [Olle] ten Cate to build an oral health-oriented course,” said Marucha, referring to the architect of EPAs.

The group began penciling out EPAs for oral health care about two years. If adopted, they would apply to dental education across the nation, Marucha added.

Trust is paramount

“EPAs use the currency of trust. Trust is a fickle thing, and it’s very hard to quantify, but it really is when you get down to it, trust between the medical practitioner and the patient is what fuels most everything in medicine,” said Logan Jones, M.D., assistant professor of medicine, OHSU School of Medicine.

“(Olle) ten Cate outlines trust as knowing yourself and your skills and being humble enough to refer things to others when they are beyond your skill set. This is an important element of whether you can trust somebody,” said Marucha.

Trust, as defined by ten Cate, refers to one’s competence and abilities; integrity and honesty; reliability, as it relates demonstrating consistent behavior; and humility, as in knowing one’s limitations.

“This serves both education and patient care,” said ten Cate. “In this respect, trust is essential. Every day, supervisors consider whether to delegate professional activities to trainees. They must trust them to perform these with reasonable chances of success,” he states in a 2005 paper, “Entrustability of Professional Activities and Competency-based Training” from The Association for the Study of Medical Education.

A new way

The old way of evaluating students by observing them doing isolated specific tasks no longer guarantees that they are ready the day they graduate to be oral health care providers or medical residents, experts say.

Currently, dental and medical schools are modeled on the amount of time the average student needs to train. Some people reach that competency early in their learning, while others take longer.

The advanced dental education program in periodontics started piloting an EPA model in 2022. It became an official assessment tool during the 2023-2024 academic year. There are 12 Entrustable Professional Activities that evaluate independence, quality and complexity. The EPAs are based on competency standards set by the Commission on Dental Accreditation.

The purpose is to evaluate the students’ ability to practice safely and independently while demonstrating ongoing competence.

The 12 EPAs include:

• communication and collaboration with a health care team
• planning and performing a dentoalveolar surgery and development of an implant site
• diagnosis, examination, treatment planning for complex periodontal and implant patients
• tooth extraction and socket management

Learners are reviewed on:

• independence
• quality
• procedural complexity
• infection control
• professionalism and ethics
• communication and compassion
• legal and regulatory knowledge

Faculty use their mobile devices to tap into a Smartsheet application to access the rating system. There are detailed instructions to aid in scoring the complexity level. A dashboard generates high-level scores and charts to observe results, trends and even red flags.

“This process is optimal for formative and summative holistic assessment,” said Yota Stathopoulou, D.D.S., D.M.D., Ph.D., director of the periodontics graduate program. “It helps us assess the technical and surgical skills of our residents, their level of independence and overall readiness to practice.”

The benefits:

• residents get immediate feedback through a dashboard
• learner progress is tracked in each EPA
• assessments are easy to understand
• relevant evaluation of skills to perform as a periodontist
• stress reduced by eliminating single high-stake clinical competency
• timely feedback on level of independence and quality of performance
• program director can track performance of all residents in each EPA
• progress can be shared with faculty

With more than 1,000 data points in the software for the advanced education program in periodontics so far, the outcomes compare with other traditional evaluation methods, according to Stathopoulou.

OHSU School of Medicine

The School of Medicine joined a 10-school pilot initiated by the Association of American Medical Colleges in 2014, using Core Entrustable Professional Activities (EPAs) for Entering Residency.  The group explored what the process would look like to use EPAs within a medical curriculum. By 2017, the school developed a custom data collection platform and launched a digital feedback tool to capture in-the-moment, low-stakes assessments.

The Workplace-based Assessments, or WBAs, that were created are now the primary method for gathering formative feedback about a student’s clinical performance:

• WBAs are the primary method for gathering formative feedback about a student’s clinical performance.

• WBAs involve brief, real-time surveys where assessors report the level of supervision provided to the learner.
• Scale ranges
  • Level one: the supervisor fully handles the activity
  • Level four: the student independently performs the task with their supervisor available just in case

“The benefit for students is that collecting WBAs provides a structured framework for receiving real-time feedback from their supervisors, which helps them improve their skills,” said Jones.

The school also established an entrustment group, made up of faculty and educational staff, to oversee the ongoing refinement of EPAs and award badges to students who demonstrate consistent trust.

“My vision is that proving entrustment in EPAs in some form will be part of the graduation requirements for the School of Medicine. However, because we’re still pursuing validation of this process, we’re simply requiring students to engage in collecting WBA data to allow us to further study this,” said Jones.

Questions remain

Although the AAMC pilot project has since wound down, the School of Medicine continues to explore EPAs and is engaged with medical schools nationwide to study how EPAs can be reliably used and potentially supplant the time-based benchmarks for educational promotion in the medical field.

“Even though the medical education community in the U.S. has been doing this [EPA work] for almost two decades, it’s still a very green field,” said Jones.

There are questions yet to be answered:

• How is useful data generated?
• How should data be interpreted?
• Should students be released from courses early if they successfully accumulate all the expected assessments?

“How do we get meaningful data and feedback, not just for the student to learn, but how do we capture it for our processes?” said Jones.

Redefining training

OHSU is at the forefront of leveraging its educational strengths to explore how the EPA framework can redefine health profession training. The School of Medicine and the School of Dentistry are working together to usher in a new approach that aligns training with real-world readiness.

“It’s a way that we can prove to our populace that our professions are training people and releasing them to care,” said Jones.

---

Story by Rhonda Morin, APR

Photos by Patrick Lee