Background

Healthcare education has historically operated under the adage of, “See one, Do one, Teach one,” a phrase coined after Halsted’s depiction of early surgical residency training.¹ In reality, Duvivier et al. found that medical students require repetitive practice to reach competency.² For optometry students, competent operation of the binocular indirect ophthalmoscope is a skill that requires many hours of practice. Binocular indirect ophthalmoscopy (BIO) is a difficult skill because it requires knowledge of anatomy, proper alignment of the optics and fine motor dexterity, all while navigating an inverted retinal image. Traditionally, students reach competency through classroom instruction, hands-on laboratory sessions with faculty and many hours of practice with peers.

Ericsson et al. hypothesized the path to mastery was not solely based on the amount of time spent practicing, but the amount of practice time devoted to targeting specific aspects of performance, which he termed deliberate practice.³ Deliberate practice requires students to:

1. break the task down to the individual skills required to become competent
2. engage in tasks that provide immediate feedback on performance
3. persevere with increasingly more challenging tasks until the intended cognitive or psychomotor skills are achieved

Figure 1. Depiction of what the user sees in the Eyesi Indirect Ophthalmoscope headset.7 Click to enlarge

Figure 2. Students using the Eyesi Indirect Ophthalmoscope. Click to enlarge

The use of simulations in healthcare education has increased due to concerns for patient safety, necessity of exposing students to rare clinical scenarios and a need for repeated practice of clinical skills within a standardized experience. Issenberg et al. defined simulation-based medical education (SBME) as an individualized learning opportunity for students to acquire and practice clinical skills in an environment that imitates real patient encounters, anatomic regions or representative clinical tasks.⁴ High-fidelity medical simulations facilitate learning through timely feedback and engagement in activities that reinforce and challenge the student, contain well-defined learning objectives and are representative of clinical practice. McGaghie et al. performed a meta-analysis and concluded that SBME with deliberate practice was superior to traditional clinical medical education in achieving skill acquisition.⁵ Hayden et al. found that incorporating high-quality simulation experiences could replace up to half of traditional clinical education opportunities with no deleterious effect on outcomes.⁶ Advances in virtual reality have provided more realistic and immersive environments for SBME. The Eyesi Indirect Ophthalmoscope is a high-fidelity virtual reality simulator for training of the BIO skill. When wearing the Eyesi headset, the student is able to view a binocular rendering of the anatomical structures of the retina (Figures 1 and 2). The Eyesi curriculum contains four modules designed for students to independently learn basic BIO navigation and documentation and visualize common pathologies such as age-related macular degeneration and diabetic retinopathy.⁷

During the spring quarter of the 2015-2016 academic year, first-year students were tasked to complete Tiers A and B within the Eyesi Indirect Ophthalmoscope curriculum. The purpose of this study was to evaluate whether these students were able to perform BIO to a competent level following 10 weeks of independent study with the Eyesi Indirect Ophthalmoscope.

 Methods

Previous class-year instruction

Figure 3. Scoring rubric for the Posterior Pole Mini Proficiency (PPMP) and the Pre-Lab Mini Proficiency (PLMP). Click to enlarge

For the three previous academic years (2013-14, 2014-15, and 2015-16), the BIO skill was first introduced to students in the Ocular Health Procedures I course in the fall quarter of their second year. In this course, students received a total of four hours of lecture (two hours of lecture per week) along with six hours of laboratory instruction (three hours per week) on the BIO skill. In addition, students were provided with after-hours practice sessions staffed by third-year optometry teaching assistants seven days a week for three hours each session. During the third week, students were administered the Posterior Pole Mini Proficiency (PPMP), which was used to evaluate their competency in obtaining full views of the posterior pole of the retina. Figure 3 shows the rubric in which students were evaluated with a total of 10 points possible and requiring a minimum score of 7.5 points to have demonstrated competency.

Introduction of the Eyesi BIO simulators

During the spring quarter of the 2015-2016 academic year, first-year optometry students were tasked with completing Tiers A and B within the Eyesi curriculum in the Optometric Clinical Services I course. The intent of placing the simulators in the first-year curriculum, prior to traditional instruction, was to reduce the need for faculty instruction on the BIO skill.

Students completed an online learning module and a hands-on demonstration designed to familiarize them with basic handling of the simulators. In addition, students reviewed an online video tutorial on how the BIO skill is performed on a live patient and were allocated three hours of practice time every other week for a 10-week period to complete the tiers of the Eyesi Indirect Ophthalmoscope curriculum.

In Tier A (Figure 4), students were taught the basic handling of the indirect ophthalmoscope as well as how to maneuver to obtain the desired views. They were tasked to obtain a view of the retina (starting with posterior pole) and to move the viewfinder crosshairs over geometric shapes within the retinal image (Figure 5). In Retina Screening, students learned the location of anatomical structures within the retina by searching for and placing the viewfinder crosshairs over specific anatomical structures. In Retina Documentation, students were tasked to find and document the location, orientation and size of abstract objects in order to understand the true location of the inverted images (Figure 5). At the end of each training case, students were provided a report card on how well they completed the task (Figure 6).

In Tier B (Figure 7), students were provided with images of normal retinas from various cases. They worked through the teaching and exam modes, which tasked them to identify anatomical features and classify characteristics of healthy retinas. Students who needed additional time to complete the tiers were provided access to the simulators during non-scheduled hours. The instructor of record of the Optometric Clinical Services I course evaluated the students’ progress via the online portal to ensure all students completed the assigned tiers by the end of the spring quarter.

Figure 4. Description of Tier A within the Eyesi Indirect Ophthalmoscope simulator curriculum.7 Click to enlarge

Figure 5. Example of abstract objects in Tier A.7 Click to enlarge

Figure 6. Example of end-of-case feedback report to the student. Click to enlarge

Figure 7. Description of Tier B within the Eyesi Indirect Ophthalmoscope simulator curriculum.7 Click to enlarge

After the summer break, the students (now second-year) were enrolled in the Ocular Health Procedures I course during the fall quarter of the 2016-2017 academic year. At the start of the fall quarter, the students were administered a Pre-Lab Mini Proficiency (PLMP) prior to receiving any laboratory instruction on the BIO skill. The PLMP was identical to the PPMP, and the same rubric was used for grading (Figure 3). Students then received the same traditional BIO instruction and practice sessions as described above for previous classes, including taking the PPMP during the third week.

Data collection and analysis    

The mean scores on the PPMP for the three previous class years were compared to the mean score of the 2016-2017 class using ANOVA. Also, the mean score on the PLMP exam for the 2016-2017 class was compared to its mean score on the PPMP using a paired t-test. The percentage of students who demonstrated competency (at least 7.5 of 10 points) on the PPMP (and PLMP for the 2016-2017 class) was recorded for the four classes.

Results

During the spring quarter of the first-year, 99 students completed Tiers A and B within the Eyesi curriculum. During the subsequent fall quarter, the students were enrolled in the Ocular Health Procedures 1 course where 5% of the class reached competency (defined as a minimum score of 7.5 out of 10 points on the PLMP). The mean score for the class was 2.46±2.08 on the PLMP (Table 1).

Table 1. Click to enlarge

After instruction and practice, students took the PPMP where the 2016-2017 class achieved a mean score of 8.55±1.94 (paired t-test p<0.001) and 82.82% of the class reached competency. Based on ANOVA, with statistical significance set at a p-value less than 0.05, the PPMP score was not statistically different (p=0.283) from the scores of the previous classes.

Discussion

Based on the comparison of the PLMP score to the PPMP score, as a standalone instruction method for teaching the BIO skill, the EyeSi curriculum was not able to teach students to obtain a full, in-focus and stable view of the posterior pole to a competent level. These results differ from previous reports in which a positive learning outcome was associated with the incorporation of the EyeSi Indirect Ophthalmoscope.^8-13 Rai et al. found that novice ophthalmology residents who received instruction through the EyeSi Indirect Ophthalmoscope outperformed their colleagues who had received traditional BIO training with didactic lecture and practicing BIO under supervision.¹² This conclusion was based on the residents’ performance score derived from the EyeSi Indirect Ophthalmoscope, which factored in accurate documentation, amount of retina visualized, elapsed time and the ability of the residents to identify and maintain views of posterior segment structures. However, it is likely that the familiarization of training on the simulators had an effect on performance scores because the conventional training group also had improvement in performance following an opportunity to receive training on the simulators. Anderson et al. reported a greater number of students achieving a 100% score in visualizing the mid-periphery and posterior pole with their BIO assessment when they were assigned to complete Tier A of the EyeSi curriculum as a supplement to traditional faculty-led instruction.¹¹ In their subsequent course administration, a portion of Tier A was assigned to the students prior to the course, similar to our course design. Anderson et al. observed that student learning had been accelerated. As a result, to better challenge and further assess students’ BIO ability, they altered their assessment to include full views in nine peripheral locations.¹¹ In our study, the timing of the PPMP was the same as in previous course administrations; therefore, we were unable to assess whether the students’ competency in BIO had been accelerated.

Issenberg et al. identified feedback as the single most important feature of simulation-based medical education.⁴ After completing Tiers A and B of the EyeSi curriculum (Figure 4 and 7), students learned to obtain views of the retina with the indirect ophthalmoscope. However, the limitation of the EyeSi curriculum was its inability to provide feedback on the other dimensions of the PLMP rubric (Figure 3), such as hand-washing, ability to hold a full view for at least three seconds, manipulation of a patient’s lids and lashes and adjustment variables such as chair height and patient fixation angles.

Limitations

A limitation of our study was the timing of the PLMP. Students’ skills were subject to degradation over the summer break. It is possible that a larger proportion of students would have passed the PLMP if it had been administered at the conclusion of the spring quarter of their first-year. Although the instruction of the BIO skill was similar across each course administration, the cohorts of students were different, which could have affected the outcome of our study.

Conclusion

The EyeSi Indirect Ophthalmoscope is a high-fidelity simulator that provides a realistic student training experience, opportunity for continuous practice, tracking of skill acquisition and introduction to a wide array of common and rare vitreoretinal pathology. The simulator adopts the principles of deliberate practice, which may accelerate competency attainment and assist in reducing the achievement gap between students’ performance. Cham and Cochrane hypothesized that introduction of the EyeSi Indirect Ophthalmoscope would help students reach technical competency sooner, which would reduce teaching workloads by minimizing the need for intensive faculty-led instruction on the BIO procedure.¹³ Based on our work and the work of Anderson et al. and Rai et al., the EyeSi Indirect Ophthalmoscope is best utilized as an adjunct to faculty-led instruction, not as a replacement for faculty instruction.^11,12 Additional applications for the EyeSi Indirect Ophthalmoscope simulator may be as a tool to reduce erosion of skill over time, to simulate clinical decision-making (Eyesi curriculum Tiers C and D) and to reduce the rigors and consequences associated with repeated dilations on one another.

Acknowledgments

We thank Stanley Woo, OD, MS, MBA (former Dean of the Southern California College of Optometry at Marshall B. Ketchum University) and the estate of the late Joseph F. Taylor, OD (1949 graduate of the Los Angeles College of Optometry) for expanding optometric education by providing the opportunity and resources to acquire the Eyesi simulation technology.

References

1. Cameron JL. William Stewart Halsted. Our surgical heritage. Ann Surg. 1997;225(5):445-458.
2. Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VR, van der Vleuten CP, Scherpbier AJ. The role of deliberate practice in the acquisition of clinical skills. BMC Medical Education. 2011;11(1):101. doi:10.1186/1472-6920-11-101
3. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008;15(11):988-994. doi:10.1111/j.1553-2712.2008.00227.x
4. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10-28. doi:10.1080/01421590500046924
5. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence: Acad Med. 2011;86(6):706-711. doi:10.1097/ACM.0b013e318217e119
6. Hayden JK, Smiley RA, Alexander M, Kardong-Edgren S, Jeffries PR. The NCSBN National Simulation Study: a longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. Journal of Nursing Regulation. 2014;5(2):S3-S40. doi:10.1016/S2155-8256(15)30062-4
7. VRmagic Holding AG. Eyesi indirect ophthalmoscope. Eyesi Indirect Ophthalmoscope – VRmagic [accessed 2017 Dec 18]. Available from: https://www.vrmagic.com/fileadmin/downloads/simulator_brochures/Eyesi_Indirect_Brosch%C3%BCre_170926_EN_WEB.pdf
8. Leitritz MA, Ziemssen F, Suesskind D, et al. Critical evaluation of the usability of augmented reality ophthalmoscopy for the training of inexperienced examiners. Retina (Philadelphia, Pa). 2014;34(4):785-791. doi:10.1097/IAE.0b013e3182a2e75d
9. Singh P, Deuchler S, Schaefer H, Fassbender S, Kohnen T, Koch FH. Indirect ophthalmoscopy: training with conventional hardware versus the Eyesi indirect ophthalmoscopy simulator. Invest Ophthalmol Vis Sci. 2014;55(13):279.
10. Chou J, Kosowsky T, Payal AR, Gonzalez Gonzalez LA, Daly MK. Construct and face validity of the EyeSi indirect ophthalmoscope simulator. Retina. 2017;37(10):1967-1976.
11. Anderson HA, Giannoni AG, Berntsen DA. Student performance and perceptions following incorporation of Eyesi indirect simulators into the optometric curriculum. Optometric Education. 2017 Winter-Spring;42(2).
12. Rai AS, Rai AS, Mavrikakis E, Lam WC. Teaching binocular indirect ophthalmoscopy to novice residents using an augmented reality simulator. Can J Ophthalmol. 2017;52(5):430-434. doi:10.1016/j.jcjo.2017.02.015
13. Cham KM, Cochrane, A. Educator’s Podium. Using technology to enhance student learning and clinical teaching outcomes. Optometric Education. 2017 Winter-Spring;42(2).

 Save article as PDF