Society of Cardiovascular Anesthesiologists

Exam Standards for Transesophageal Echocardiography

This year, like others, the Perioperative TEE Certification Exam Committee of the NBE received feedback from people who took the TEE exam. Unlike other years, some of the feedback for this year's exam was negative, focusing on the perceived degree of difficulty as well as the conditions of the exam setting itself. The committee has always regarded the perioperative TEE exam process to be an extremely important one for perioperative physicians choosing to demonstrate and measure their echocardiographic skills. The exam committee has diligently worked at maintaining a high level of credibility and fairness to the examination process and has worked with the National Board of Medical Examiners from the beginning to assure this goal. A few points about this process are worthy of further explanation and comments.

Standard Setting

The pass/fail decisions on the NBE-PTE examination are based on a criterion-related standard. This means that a certain level of proficiency is required of each candidate to pass the examination. This level depends only on the number of questions answered correctly and does not depend on what proportion of the candidates pass. Thus, it is possible for all candidates taking an examination to pass if they all had the required skills, while it is also possible for no candidates to pass if none had the necessary proficiency level. How was this minimum required proficiency level set?

In May 1998, the task force for certification in perioperative TEE (then under the auspices of SCA) undertook a standard setting study to determine the passing standard on the NBE-PTE examinations. Eight members of this task force served as panelists. First, the borderline candidate (who would know just enough to pass) was discussed at length and defined. Then, independently, they went through a complete examination (200 questions) question by question and gave estimates of each question's difficulty as it appeared to the borderline candidate. The results of the study were summarized and discussed at length by the committee members. Finally, a decision was made on the standard. The decision was implemented and fixed on the reference score scale that is used to report scores to the candidates.

Each year, the candidates' scores are equated and placed on the reference score scale. Equating is the process by which the effects of examination difficulty are removed from the scores. Thus, candidates who took a more difficult form of the examination will not be penalized and those who took an easier form will not be given unfair advantage. If a candidate's score is equal or greater than the minimum passing score required (currently 444), he/she passes. If the score is less than the minimum passing score required, he/she fails. There is no set quota for the number of candidates passing at any time. The total passing rates were 76% in 1998, 72% in 1999, and 79% in 2000.

Exam Conditions

The Perioperative Question Writing Committee of the NBE (originally the SCA Task Force for Certification in Perioperative TEE) has always tried to honor the wishes of the prospective examinees (principally members of the SCA) by offering the Perioperative TEE Certification Exam in the city and at the time of the SCA Annual Meeting.

The trade off for doing this has been to accept the need to adjust to the hotel or convention center facility constraints. This has unfortunately always added an unknown variable to the examination. These adventures have included birds flying around in the rafters of the auditorium in Seattle, a water pipe bursting at the Sheraton Hotel Convention Center in Chicago and, last but I am sure not least, the fire alarm and video monitor blackout in the Orlando Dolphin Hotel.

The unpredictability of these events is unfortunately the only thing that is predictable. I fear this variable of unpredictable conditions will continue to occur if we continue to have future exams in hotels and convention centers. The alternative would be to have the exam in a test center with controlled conditions. There is such a site in Philadelphia at the NBME headquarters and would be available if desired. The trade off would be convenience for SCA members who wish to go to the annual meeting and take the Echo Exam during the same time. The decision to plan the location and setting of future exams will be strongly influenced by your feedback and advances in technology as computerized exams may be offered in the near future.

Gladly the performance of this year's exam by the examinees does not appear to have been influenced by the conditions. We apologize for any inconvenience the delay for starting the exam in Orlando may have caused.

Finally, I would like to close by saying that it has truly been an honor and privilege to have been a part of the perioperative TEE certification exam process. Since the beginning while under the auspices of the SCA Task Force on Perioperative TEE Certification through the negotiations with the ASE Exam, Inc. and now as a part of the National Board of Echocardiography, I have been lucky to have worked with gifted and passionate professionals. Because of their dedication, the process can move onward. I am confident this process will continue to improve, as it will now be under the leadership of Dr. Jonathan Mark, who will take over as Chairman of the Perioperative TEE Certification Exam Question Writing Committee.

Thank you. It has been an honor.

Respectfully,
Solomon Aronson, MD, FACC, FCCP

Newsletter