After-Hours Practice Time and Optometric Theory and Methods Laboratory Success in Remote Online Learning

Introduction

At the onset of the COVID-19 pandemic in March 2020, the majority of education institutions suspended face-to-face in-person teaching, closed their campuses and moved their course curricula to remote online learning.^1,2,3,4 This transition posed a problem for clinical laboratories in optometric curricula. Clinical laboratories involve hands-on learning of procedural skills with in-person preceptor instruction. This hands-on nature proved challenging for faculty transitioning to virtual platforms and for student learning.²

Online learning has been around for some time; however, its use in practice-based learning has been limited until the recent pandemic. While online learning and blended teaching has been effective in many courses at different institutions,^5,6 practice-based learning, or hands-on learning, in clinical laboratories has been the traditional method of delivering education in various fields of medicine. As with any learning strategy, some students excel while others struggle, and the transition to online learning during the COVID-19 pandemic was no different. Several studies have looked at how blended learning, simulated learning environments, clinical simulators and other strategies influence student education.^1,5,6

Many institutions have implemented forms of virtual learning within lab-related courses. A study published in 2016 assessed the effect of short-term exposure to a simulated learning environment on student clinical subjective refraction performance. Fourth- and fifth-year optometry students enrolled in the 5-year dual Bachelor of Vision Science and Master of Optometry degree program at the University of Aukland were assessed. Results demonstrated a positive influence for the less-experienced students.⁷ In another study, the University of Auckland conducted a survey to evaluate whether their students, using a blended teaching approach of either a static or an interactive website for Ocular Anatomy and Physiology, obtained better examination scores than students in the previous year who only had access to a static website. They also assessed the students’ perceptions and impressions of the interactive vs. static website. They concluded that although access to the interactive website did not significantly improve the students’ grades, their responses to the interactive website were positive. However, there were critical comments regarding the absence of educator assistance when using the online tools. This led the researchers to further conclude that for their Ocular Anatomy and Physiology program, a blended style that includes the educator as the main administrator of student learning is necessary.⁸

Pacific University College of Optometry also assessed blended learning, specifically in the optometric procedural courses in the second year of the program.⁵ The study compared blended strategies with strictly in-class instruction from previous years and found no significant difference in exam scores between in-class and online teaching.⁵ A study at New England College of Optometry involving third-year students compared the effectiveness of an interactive multimedia learning module vs. traditional learning in preparation for in-person B-scan procedural training. Use of the interactive multimedia module resulted in a 20% improvement in test scores and significantly positive feedback on surveys.⁶

Transitioning to a purely online learning environment poses many challenges to both students and faculty. Students are unable to complete practical assessments and receive appropriate grades for courses requiring hands-on procedural skills.¹ Remote online learning also involves a steep learning curve as students and faculty must quickly familiarize themselves with the technology and delivery methods of the curriculum,^1,2 which can include video conferencing platforms, collaboration and communication apps, remote proctoring solutions and others.^3,4 An added limitation to these strategies is the inability to practice the skills on a variety of eyes. This reduces students’ competency in the subject matter as they are exposed to less variable scenarios.

Due to the nature of the optometric skills taught in laboratory settings, in-person guidance and instruction has always been the primary method of training across all optometric education institutions. There had not been cause or reason to analyze the teaching and learning of these skills remotely. The pandemic provided a unique opportunity to assess whether online instruction of optometric techniques could be a viable alternative to in-person delivery with equivalent levels of success. The aim of this study was to demonstrate how the number of in-person practice hours correlated with pass rates in different optometry skills after a semester of virtually learning each of these techniques.

Methods

The study was approved by the Nova Southeastern University Institutional Review Board. Each participant provided signed informed consent after discussion about the nature and purpose of the study and guaranteed anonymity of responses. First-year optometry students at Nova Southeastern University completed their first semester of optometry school from home in fall 2020 due to protocols set because of the COVID-19 pandemic. They were taught the Optometric Theory and Methods (OTM) lecture course and laboratory virtually via Zoom with instructional lab videos and live demonstrations. Weekly 2-hour labs were held via Zoom for each lab section with topics taught concurrently with the lecture component.

Virtual lectures were delivered over the period of 1 hour for color vision, 1-2 hours for stereoacuity and 2 hours for extraocular muscle (EOM) testing and saccades. Students were provided handheld laboratory equipment for performing the skills at home on roommates or family members. In the virtual lab sessions, 1 hour was dedicated to color vision, 1 hour to stereoacuity and 2 hours to EOM testing. Lab manuals with videos were provided to the students for each skill. Three preceptors were assigned to Zoom breakout groups to address any questions and to see if students had a grasp of the theory. Students paired up and went through the motions of administering the skills, and they were required to complete a worksheet of the activity. Students completed assignments where they recorded themselves performing preliminary skills (color vision, stereoacuity and EOM movements) at home and then critiqued one another’s performances along with instructor feedback.

Keratometry instruction was delivered via Zoom for approximately 3 hours and further discussed in a virtual lab session for 2 hours in the small breakout groups. Diagrams of the instrument labeling all knobs and dials were provided. Worksheet activities and video demonstrations were provided to guide the students, although they could not practice this skill hands-on because they did not have access to this instrument at home.

Retinoscopy content was taught over the course of 3 hours with 6 hours of virtual lab time. Lab manuals and video demonstrations were provided. Students were guided through retinoscopy using schematic eyes, taught how to simulate various spherical and spherocylindrical refractive errors and educated on how to check their work in neutralizing the reflex. They also had to complete a worksheet that guided them through the skill. Preceptors were available virtually to direct them through the activity.

Subjective refraction instruction was provided for 3 hours via Zoom prior to 4 hours of lab sessions. The lecture component included demonstrations of the technique and, again, worksheets were provided during the lab sessions for students to complete in order to understand the steps of refraction. Instructors assisted in breakout rooms to walk through these techniques and provide feedback.

Figure 1. Survey provided to first-year optometric students at Nova Southeastern University inquiring whether they passed each skill on the first attempt and the number of hours they spent practicing throughout the semester and during the week prior to the exam. Click to enlarge

The students returned to campus in January 2021 and were able to perform these techniques in the laboratory with guidance from three lab instructors. The skills included keratometry, color vision, stereoacuity, EOM movements and saccades, retinoscopy and subjective refraction. Due to strict COVID-19 protocols during this time, students were split into eight lab sections. Each section was assigned weekly evening practice times during which students could sign up for optional 1.5-hour practice sessions. One instructor was present during these after-hours practice sessions to oversee lab activities and ensure safety and social distancing. Due to the inability to properly assess their competence in these skills virtually, testing the students via practical examination format was pushed into the spring semester of 2021. Students were given 1 month of in-person practice time before they were examined on the preliminary and keratometry techniques. Retinoscopy and subjective refraction were tested later in the semester after approximately 3.5 months to allow students more time to practice and utilize actual lab equipment during the semester.

After the spring semester of 2021, students were given a 5-minute survey (Figure 1) inquiring how many hours on average they signed up to practice during the semester and how many hours they practiced the week of their practical exams. They were also asked to voluntarily report which skills they passed or failed on the first attempt.

Data analysis

The pass rate in each OTM skill was calculated and the differences in pass rates were assessed with Chi-square test. Linear regression was applied to explore the association between pass rate and the number of hours spent on each OTM skill. A p-value less than 0.05 indicates statistical significance.

Results

The overall pass rate in keratometry (0.51) was significantly lower than the pass rate in other OTM skills (0.77 in color vision, p < 0.01; 0.81 in stereopsis, p < 0.01; 0.79 in EOM, p < 0.01; 0.76 in retinoscopy, p < 0.01; and 0.77 in subjective refraction, p < 0.01; Chi-square test). Of the 104 students testing, 53.7% failed keratometry in part due to incorrect documentation, whereas 46.3% failed solely due to inaccurate technique or running out of time.

The average number of after-hours practice throughout the semester was analyzed for preliminary examination skills (Figure 2). Linear regression was applied to reveal the relationship between pass rate and the number of hours spent. Sample sizes less than 9 were not included in the analysis. There was no significant relationship between hours spent and pass rate for color vision (Figure 2B), stereopsis (Figure 2C) or EOM (Figure 2D). However, there was a significant linear relationship between the pass rate and hours spent on practice for retinoscopy (Figure 2E) and subjective refraction (Figure F). Pass rate increased as more hours were spent on practice.

We also analyzed the relationship between pass rate and hours of practice during the week before the test (Figure 3). It should be noted that the pass rate for subjects who spent only 1 hour was low in all OTM skills. There was a dramatic elevation in pass rate for subjects who spent 2 or more hours (Figures 3A to 3F). Due to the small sample sizes of subjects who only spent 1 hour, the dramatic increment did not reach significance. Linear regression was applied to analyze the relationship between pass rate and the number of hours spent. Sample sizes less than 9 were not included in the analysis. There was no significant linear relationship between the pass rate and hours spent on practice for color vision (Figure 3B), retinoscopy (Figure 3E) or subjective refraction (Figure 3F). However, there was a significant relationship between hours spent and pass rate for stereopsis (Figure 3C) and EOM (Figure 3D). Pass rate decreased as more hours were spent on practice the week before the test. In addition, a significant correlation (r = 0.7338, p < 0.001) was found between after-hours spent during the semester compared to the week of the checkout.

Figure 2. Average number of hours spent practicing in the Optometric Theory and Methods lab during the semester. A solid line indicates a significant regression trend (p < 0.05). Symbol size is proportional to the sample size. Samples with a size smaller than 9 were not included in the analysis. Click to enlarge

Figure 3. Hours spent practicing in the Optometric Theory and Methods lab the week before the test. A solid line indicates a significant regression trend (p < 0.05). Symbol size is proportional to the sample size. Samples with a size smaller than 9 were not included in the analysis. Click to enlarge

Discussion

Based on the COVID-19 safety protocols during this academic year, students were limited in the number of hours they were able to practice in the OTM lab during spring 2021. Students were separated into eight lab sections to reduce the number of individuals in the lab and ensure appropriate social distancing. Prior to COVID-19, the lab was open during evenings and weekends for students to practice at any time with any other student in their class. With slightly more than 100 individuals per class year, students had a large variety of eyes to practice on throughout the semester. The COVID-19 precautions limited students to one or two 1.5-hour practice sessions per week, and they were only allowed to practice with students in their own lab group, which consisted of 14-16 students each. For the entire semester, students were permitted to practice only on a maximum of 16 pairs of eyes, which greatly reduced their exposure to different ocular health conditions and refractive errors. However, students were supervised by an optometry faculty member or resident during each of these practice sessions to offer more support and ensure compliance with safety guidelines.

To our knowledge, no previous studies have evaluated the relationship between student practice time and pass rates. Prior to the pandemic, monitoring which students took advantage of after-hours practice, how long they spent practicing, and who they practiced with was not possible. Pandemic guidelines mandated the monitoring of all these parameters for contact-tracing purposes. During after-hours practice time, students were provided instructor supervision while in the lab, which was not the case prior to COVID-19. This allowed students an additional resource for structured feedback while performing optometric skills so that questions could be answered and errors identified and corrected instantaneously. Before the pandemic, students were practicing on their own after normally scheduled lab time for as long as they desired without observation or suggestions for improvement from a licensed practitioner.

These optometric skills can be separated into three main categories with preliminary tests (color vision, stereoacuity and extraocular muscle movements) as one category, keratometry as another category, and retinoscopy and subjective refraction as the third category. There are differences in the results across each of these categories. Results revealed high pass rates for preliminary exam skills across the board, which could be due to a lack of need for this skill to be performed in the lab setting with faculty supervision. Equipment required for this testing is mainly portable, and the techniques can be performed at home. Partners can easily modify their responses in order to provide more practice opportunities and troubleshoot how to assess abnormal results. However, keratometry, retinoscopy and subjective refraction all require instrumentation, including a keratometer, phoropter and acuity chart, as well as human eyes, that are available only on campus.

Despite the increased hours of practicing, keratometry still had a higher fail rate compared with the other skills, which could be due to multiple factors. Keratometry was one of the first skills tested after only 1 month of practicing on campus along with the preliminary skills. Despite the high pass rates for the preliminary techniques, keratometry posed a different challenge for students. Because this skill cannot be performed from home and requires use of a keratometer in the lab, students had a limited 1-month timeframe to practice this skill prior to being evaluated. In addition, failure was based on both accuracy of technique as well as proper documentation. The percentage of students failing due to documentation errors was higher compared to the rate of students failing due to inaccuracy of the skill or not completing the skill within the allotted timeframe. This could be due to students using after-hours practice sessions to focus mainly on performance and efficiency and less on documentation because the latter can be reviewed and learned from home. Instructor guidance during these sessions emphasized precision in measurement rather than in recording findings correctly. Because student practice time was restricted, the best use of this lab time was believed to be in performance rather than documentation.

During the week of both practical exams, results demonstrated that 1 hour spent practicing was not sufficient for passing the practical examination. Utilizing the lab for 2 hours during the week of the practicals had the highest pass rates. However, practicing more than 2 hours the week of the practicals did not appear to improve pass rates. These trends were consistent among all skills tested. This indicates that there is an ideal amount of time for students to practice during this stressful week. Spending 3 or more hours practicing during the week of the practical exam may result in burnout for students where they are not improving in technique performance. This may also be due to students fatigue from longer days, less sleep and more stress resulting in lower pass rates.

Performance in optometric skills requires strict dedication to practicing in the lab diligently and consistently. Based on survey responses, students who practiced frequently throughout the semester also likely practiced the most during the week of their practical examination. This study demonstrated that certain skills require more practice and instructor guidance than others. The results demonstrated that preliminary skills can be conducted at home with minimal supervision and equipment. Keratometry, retinoscopy and subjective refraction require more hands-on learning with the use of a laboratory setting. Students benefit from supervision, guidance and instant feedback on skill performance from a licensed practitioner during this time. This study also revealed that more emphasis should be placed on documentation and ensuring that students also practice reading measurements appropriately and recording accurately.

One of the limitations of this study was that the survey simply inquired about the number of hours spent practicing but did not assess the breakdown of time spent on each of the skills. A few students in the class have parents or relatives in the optometric field and may have had the advantage of access to office equipment during the remote sessions in fall 2020. However, based on informal student feedback, the majority of students moved to residences near campus to attend the virtual semester in order to better focus on coursework away from home. General trends from instructor feedback indicated that students tended to practice the skill of the week during their after-hours session early in the semester. They then practiced all skills together closer to their practical exam weeks as they became more proficient. However, there was no formal assessment of the exact number of hours spent on each technique after-hours. A future study may be conducted to specify how many hours were spent practicing each of the skills individually.

An additional limitation to the study was the lack of comparing this data to a class year prior to the pandemic when students had full access to the lab and all hands-on learning. Although previous class years had unlimited access to the lab and could practice on any classmate, they lacked the added advantage of having an instructor present during all after-hours sessions to provide guidance and immediate feedback. A future study may collect this data for comparison purposes.

Another limitation of this study was that students were asked to self-report the information. Data can be skewed if students did not report truthfully or accurately. Inquiring about how many hours they practiced in the lab is considered an estimation with room for error. A future study may review the sign-in sheets and accurately calculate each student’s exact number of hours in the lab throughout the semester. Students also self-reported whether they passed or failed each skill, which may lead to participants falsifying their responses to avoid shame or disappointment. However, when the number of students who failed due to documentation errors on keratometry was reviewed, it correctly matched the number of students reporting passing or failing of this skill. Student pass or fail responses were compared via grading reports for each skill and confirmed consistency between the self-reported grades and actuality. This provides investigators with greater confidence in the accuracy of self-reporting.

Conclusion

This study indicated that students can achieve high success rates in performing preliminary optometric skills despite the skills being taught virtually and with minimal instructor feedback. However, keratometry, retinoscopy and subjective refraction are skills that required lab instrumentation and consistent after-hours practicing in order for students to successfully pass.

References

1. Jonuscheit S, Lam AKC, Schmid KL, Flanagan J, Martin R, Troilo D. COVID-19: ensuring safe clinical teaching at university optometry schools. Ophthalmic Physiol Opt. 2021 Jan;41(1):144-156.
2. Schmid KL, Backhouse S, Cochrane AL, et al. A snapshot of optometry teaching in Australia and New Zealand in response to COVID-19. Clin Exp Optom. 2021 Aug;104(6):723-727.
3. Twa MD. COVID-19 Pandemic-driven innovations in optometric education. Optom Vis Sci. 2020 Oct;97(10):831-832.
4. Ramani KK, Hussaindeen JR. Optometric education in the post-COVID-19 era: a time of forced change! Indian J Ophthalmol. 2021 Mar;69(3):746-750.
5. Goodwin D, Hua LV, Hayes JR. Blended learning in optometric clinical procedures instruction. Optometric Education. 2014 Winter-Spring;39(2):58-64.
6. Biffi EZ, Woodbury M. Interactive multimedia learning vs. traditional learning in optometry: a randomized trial, B-scan example. Optometric Education. 2019 Summer;44(3).
7. Woodman-Pieterse EC, De Souza NJ, Vincent SJ. The influence of a novel simulated learning environment upon student clinical subjective refraction performance: a pilot study. Clin Exp Optom. 2016 Jul;99(4):342-9.
8. Acosta ML, Sisley A, Ross J, et al. Student acceptance of e-learning methods in the laboratory class in Optometry. PLoS One. 2018 Dec 13;13(12):e0209004.