An Evaluation of Student Use of the Southern College of Optometry Clinical Grading Program as Means of Reflection and Learning

Introduction

One aspect of teaching in the clinical programs of a professional program, such as optometry, is the need to inform students about their performance. Grading rubrics can include Likert-scale items and open-text boxes to provide the students’ feedback. It is hoped that students will then absorb that feedback, reflect on it and take some action that helps them grow as clinicians and critical thinkers. The question of learning and the actions taken based on feedback during clinical grading has, to the best of our knowledge, not been explored in the optometric literature. Faculty often spend hours per week, sometimes with dedicated scheduled time, completing grades, but the question of student impact remains unexplored. This study explored students’ use of the clinical grading system at Southern College of Optometry (SCO), their attitudes toward the feedback provided, the impact on reflection and learning and the actions they took in response to that feedback. A better understanding of these aspects of clinical grading will provide SCO and other optometric institutions with information to analyze and enhance their clinical grading systems, thereby strengthening their role as a teaching tool.

Experiential Learning

History/Key Characteristics

The experiential learning theory originated in the work of educator John Dewey. Dewey¹ espoused the idea that learning occurs through experience. Dewey’s work became the foundation for what is now known as progressive education. According to Dewey,¹ not all experiences produce learning; some may miseducate, leading to experiences that are less than meaningful, in which learning or growth does not occur. For learning to occur, the learner must connect with the experience and the experience must be meaningful.²

Figure 1. Kolb’s Experiential Learning Theory5.Click to enlarge

Kolb³ defined experiential learning as the process whereby knowledge is created through the transformation of experience. The Kolb learning model (Figure 1) consists of four stages through which learners progress during the learning process: concrete experience, reflective observation, abstract conceptualization and active experimentation.⁴

Learning is effective when the learner progresses through these four stages. The model is more complicated than a circular cycle, however. It comprises two modes of grasping experiences (concrete experience and abstract conceptualization) that are oppositional and two modes of transforming experiences (reflective observation and active experimentation) that are also oppositional. The four learning modes are defined as follows:

• Concrete experience: The ability to be open and willing to involve oneself in new experiences
• Abstract conceptualization: The ability to create concepts that integrate their observations with previous knowledge, experiences and logically sound theories
• Reflective observation: The ability to reflect on what was observed during the new experience.
• Active experimentation: The ability to try out or use the theories they learned to make decisions and solve problems.

The learning cycle begins with the individual having concrete experience, which is the basis for observation and reflection. The reflections are integrated into abstract concepts from which implications can be drawn and actively tested and experimented on by future experience.

Relevance of Experiential Learning to Clinical Grading

The four Kolb^6,7 experiential learning cycle stages concerning the clinical optometric learning experience highlight the following connections (Figure 2). Students are assigned a patient in the clinic and perform an examination, including a history. The students present their data and work with faculty members to develop a differential diagnosis and a plan. Treatment is initiated, be it glasses, medication, therapy, a minor surgical procedure, or referral. This is the concrete experience aspect of Kolb’s learning cycle, which students encounter through patient care.

Figure 2. Kolb’s Experiential Learning Theory with Connections to Clinical Experiences/Grading.Click to enlarge

Both during and after the examination, the students reflect on the encounter. This reflection begins with rethinking the case as they complete the examination record. By considering the choices made in test selection, how they approached and handled the encounter and the treatment provided, the student begins self-reflection on the concrete experience they just completed.

Verbal discussion between the faculty and the student, and clinical grading, facilitate the self-reflection process. During and after the clinical experience, there is often a verbal discussion between the faculty member and the student. This is an opportunity for case discussion and a time for the faculty member to guide the student and critique their performance. Clinical grading, which includes written and numerical feedback, should be considered part of the reflective component. Each clinical encounter is graded, offering faculty a structured opportunity to guide the student in the learning process for that patient and provide feedback and suggestions for future learning. The faculty member, of course, uses it in this manner, offering productive feedback to enhance reflection, and the student must be open to hearing the feedback and using it as part of their reflection.

The student builds on the reflection to improve in some way. This abstract conceptualization can occur by building knowledge by researching a topic, speaking with a friend or faculty member, or practicing/enhancing a skill. The student self-directs their learning, chooses to practice a given technique, or learns more about a given topic or condition regarding the previous patient encounter.

The faculty member’s feedback and the self-directed learning it generates hopefully help the student integrate and actively experiment with their new knowledge or approach and test it in future patient experiences. This experience will then, in turn, lead to the process of starting anew.

In discussing the concept of assessment in experiential learning theory, Kolb³ stated, “It views much assessment in education as problematic because it is overly focused on declarative content knowledge rather than learning skills: and it is not holistic, emphasizes cognitive knowledge at the expense of interpersonal and applied competencies.” Despite this seemingly negative view of assessment related to experiential learning, it is necessary to document learning outcomes and fuel the reflective fire. As Coyle, et al.⁸ noted, “Students will learn more if instruction and assessment are integrally related. Providing students with information about particular qualities of their work and what they can do to improve it is crucial for maximizing learning.” Feedback, when coupled with practical application, helps students not only to learn content but also to enhance critical thinking.⁹

This study explored the student use of clinical grading system at SCO, their attitudes toward the feedback provided, the impact on reflection and learning and the actions they took based on that feedback.

Methods

A 15-question survey was created by the author based on experiences in clinical grading (Appendix). This survey has not been validated or used in previous studies. The survey was sent electronically via Microsoft Forms to third-year (135) and fourth-year (45) students at SCO. This represents the current third-year class and the fourth-year students who are completing their internal rotation. Both groups are currently using the described clinical grading system. The survey comprised multiple-choice and Likert-scale-type questions. The Likert scales had either four or five choices, depending on the question. Of the Likert-scale questions, nine of 12 used a four-point scale, while the remaining three used a five-point scale. The specific questions and scales used are available in the Appendix. The student responses were anonymous. No incentive was provided for the completion of this survey. The survey was open for 10 days and collected responses in MS Forms. Descriptive statistical analysis was performed using MS Forms. The study was approved by the Institutional Review Board of SCO.

Results

The response rate was 75% (135/180), with fourth-years showing a higher rate (97.8) than third-years (67.4%). The satisfaction with the SCO clinical grading process was as follows: “very satisfied” (5.2%), “satisfied” (67.9%), “neither satisfied or dissatisfied” (24.6%), and “dissatisfied” (2.2%).

Fifty percent of students reported checking their grades at least daily (17% more than once a day; 33% once per day). An additional 36% checked several times a week, while 14% did so only once a week. Regarding the time spent reviewing grades, 71% reported spending less than 3 minutes per session (17% spent less than 1 minute, 54% spent 1-2 minutes), while 27% spent 3-5 minutes.

In comparing the students’ examination of the various aspects of the grade (categories and comment box), there are different distributions, but overall, the patterns are similar. (Figure 3) Students indicated that they viewed the comment box “always” or “often” 98.5% of the time, compared to 76.3% for the categories. Of note, a tiny percentage did not view the categories (2.2%) or the comment box (0.7%).

Figure 3. How often students use certain aspects of the grading process.Click to enlarge

The students’ comment box showed a different pattern of use. Only 20.9% selected the “always” or “often” option, and 50.7% selected “sometimes.” While this shows that close to three quarters of the students took the opportunity to use their comment box, the remainder (28.4%) did not.

Only 7.4% of students selected “disagree” or “strongly disagree” when asked if the clinical grading process could be used as a means of learning. Close to 83% selected “agree” or “strongly agree.”

Taking action based on the grade was overall positive. “Always” or “often was selected 77.6% of the time, and “sometimes” was 21.6%. Only one respondent indicated that they “never” took action. The actions taken showed a wide distribution (Figure 4). The most common answers selected were “Google a topic”, “look up your class notes”, “ask a friend”, and “ask a faculty member.” A total of 467 actions were taken, averaging 2.59 per respondent. The number of actions ranged from zero to eight.

Figure 4. Different actions taken by the students in response to faculty feedback.Click to enlarge

Several factors were assessed for their perceived impact on learning (Figure 5). Faculty comments were reported to positively impact learning “always” or “often” by 79.7% of students compared to a negative impact of 10.5%. Responses indicating “never” further illustrated this contrast between positive vs. negative impact (0.8% vs. 38.1%). Close to 90% of students indicated that the timeliness of the grading impacted their learning. This underscores the perceived importance of prompt feedback in clinical education, as it adds interpretation and significance. In comparison, 98.5% felt that the quality of the faculty comments affected their learning through the clinical grading process.

Figure 5. The impact of different aspects of feedback on learning via the grading process.Click to enlarge

Discussion

This study examined the clinical grading process at SCO and how students used various aspects, including faculty feedback and the grading system, as a means of learning. In early 2023, the grading system was updated to make it more efficient and provide greater transparency for students and faculty. The high student approval indicates that those changes effectively create a satisfactory experience.

One factor that affects students’ ability to reflect on their grades is the timeliness of faculty in completing those grades.^10-12 While the burden to complete the grade falls on the faculty, the burden to view the completed grade, including the feedback provided, falls on the student. How often and for how long students are entering the grading platform to view their grades, and the timeliness of faculty completing the grade, must be considered in this equation. For example, a faculty member might efficiently complete their grades, but the student might not view them promptly, reducing the opportunity for reflection. Conversely, the student may often check their grades daily, but the faculty may be lax in their duties and complete the grade a week later.

On the student side of the equation, 50% reported checking their grades at least once a day. Fourth-year students, who have more clinical experience in patient care and are more knowledgeable, might check their grades less, but that would be a false assumption. Among students who checked their grades weekly, eight of 19 were fourth-year students and 11 were third-year students. Conversely, of the 23 students who viewed their grades more than once daily, eight were fourth-years. When examining the time spent reviewing grades, 38 of 44 fourth-year students spent 1-2 minutes or less per session, compared to 58 of 91 third-year students. Third-year students spend more time reviewing their grades. While the frequency of checking grades is student-specific, a pattern emerges: third-years spend more time viewing their grades when they access them.

Timely feedback is identified by Romani and Krackov¹³ as one of their 12 imperatives for feedback. The current study supports this fact. The poor timeliness of the feedback can disrupt the experiential learning cycle,¹⁴ thereby hindering reflection and, in turn, impacting the learning opportunity. Timely feedback is especially crucial for students requiring remediation, those with limited clinical efficiency, or those with identified academic deficiencies.¹²

Although the survey results do not identify which specific aspects of the grade most influence students’ reflective processes, it is likely that the faculty comment boxes play a greater role, as they provide narrative feedback on what students did well and what they can improve. Faculty often provide links to additional resources as well. Faculty often include links to additional resources in their narrative feedback. In contrast, Likert-scale category ratings (e.g., documentation, technical skills and data interpretation) reflect performance along a continuum from ‘below expected’ to ‘above expected.’ While the numbers are still relatively high for both aspects of the grade, more students looked at the faculty comment boxes regularly than the categories. The impact on learning from the quality and nature of feedback provided in the faculty comment box will be discussed below.

Reflection is a critical aspect of clinical teaching and has been shown to help develop self-regulated learning skills that can enhance competence, humanism and professionalism.¹⁵ According to Winkel et al., there is support for reflection as a means to “advance knowledge, guide future learning, deepen understanding of complex concepts and explore emotionally challenging situations.¹⁵ According to Ledvinka,¹⁴ “we are helping them [students] to develop tools for lifelong learning,” through reflective practice. Looking more closely at the process’s reflection component raises the question of whether clinical grading sufficiently supports reflective learning. A lack of reflection may indicate surface learning, whereas critical reflection demonstrates deeper engagement.

This question was evaluated in two ways in the survey. In 2023, one of the new aspects of the grading system was the creation of a student-only comment box developed in response to student feedback. In this box, which students complete before sending the grade request to the faculty member who worked with them, they can ask questions, request resources, or inform the faculty member who worked with them about something that occurred during the examination. In examining the definition of reflection put forth by Branch and Paranjape,¹⁶ which defines reflection in medicine as “to include consideration of the larger context, the meaning and implications of an experience or action,” the student comment box could be considered such an opportunity. It is positive to see that almost ¾ of the students use the comment box in some fashion. In this aspect of the grading system, a proportion of students need to see the comment as a learning opportunity and take advantage of it often enough.

The opportunity for reflective practice concerns students’ actions after receiving feedback through their grades. The most selected actions included Googling a topic, looking at class notes and asking a friend or a faculty member. Practicing a skill was selected less often but was chosen by slightly over 40% of students. Interestingly, 11 of the students who indicated that they “never” used the student comment box but instead selected the option to “ask a faculty member.” It can be inferred that they are asking the faculty member in person, outside the grading process. Despite the actions taken, it is unclear whether they were prompted by faculty feedback or initiated independently. Given that reflection can be facilitated rather than purely self-directed, this distinction is important to explore in the future.

Another major issue with reflection in the clinical grading process is the quality of feedback. Comments may be superficial, focusing on technique or diagnosis, but they do not provide the student with enough direction to reflect and grow. All but two students (98.5%) indicated that the quality of the faculty comments impacted their learning. This can be interpreted as having either a positive or a negative influence. While the lack of feedback or low-quality, overly general feedback can have a negative impact, specific, actionable feedback can have a positive impact.¹⁷ If one of the goals of the clinical grading system is to provide feedback that students can use to build confidence and engage in meaningful reflection on what they did well or poorly, providing high-level, specific feedback with actionable points is crucial.

Several studies have examined the characteristics of the feedback given in medical clinical grading. Canavan et al.¹⁸ found that “Most feedback comments were positive, self-oriented and lacked actionable information that would make them useful to learners.” Pulito et al.¹⁹ reviewed 331 evaluation forms from medical students’ clerkship rotations. Not one comment was made regarding history-taking and physical examination skills, ordering or performing lab tests, or surgical technical skills; these are actionable growth areas where specific feedback would be most beneficial to student growth. Comments related to these three topics are likely to have the greatest impact and offer the greatest opportunity to enhance reflection. Lye et al.²⁰ conducted a study of pediatric medicine clerkships and collected 261 evaluations from 157 students. Only 311 (31%) comments were related to clinical performance, and only 134 of those offered specific details.

In the current study, students reported that faculty comments can negatively impact learning. Learners vary in their responses to criticism, which may hinder their ability to process feedback and reflect on their strengths and weaknesses.¹⁷ “Students with inaccurate self-assessment may struggle to interpret and apply feedback effectively.²¹ Self-esteem also impacts how students receive feedback, as students with high self-esteem are more likely to engage in positive reflection. Those with poor self-esteem interpret the criticism as negative and personal in nature.²²

Clynes,²³ however, also reported that negative feedback did not harm the mentor-student relationship. As fewer than 10% of students selected “often” or “always” for the impact of negative feedback on learning, one can extrapolate that, on the whole, students appear to internalize negative feedback constructively and that it is not negatively impacting their learning. Interestingly, a relatively equal percentage of third- and fourth-year students chose both options, indicating that receiving feedback for a longer period did not alter their reactions to the negative feedback.

Clinical grading can give students confidence and boost their emotional well-being.²⁴ In the current study, nearly 100% of the students indicated that the faculty comments had a positive learning impact. Skinner et al.²⁵ found that students’ confidence and interpersonal skills improved following experiential learning opportunities. Giving constructive criticism is intended to boost self-esteem and motivation. Veloski et al.,²⁶ in a systematic review of assessment, feedback and clinical performance of physicians, identified 220 potential studies. Of 41 studies that met the inclusion criteria, 29 (70%) reported a positive effect of feedback on clinician performance.

While overall, the students surveyed felt that the type of feedback, negative or positive, did not impact their ability to learn via the SCO clinical grading process, specific changes in performance before and after the feedback were beyond the scope of this survey. Zahid et al.²⁷ found a significant difference in performance pre- and post-surgical procedures in a training lab following negative and positive feedback. The positive feedback group improved by 39 seconds, compared to only 4 seconds in the negative feedback group. When asked about their performance, those in the positive feedback group rated their performance as average to above average. In contrast, those in the negative feedback group said they were average to below average. When questioned about future performance, those in the positive feedback group expected to do better, while those in the negative feedback group ranged from expecting to perform extremely well to expecting no change. This study highlights how feedback type may influence both immediate performance and students’ expectations of future performance.

Limitations

This study is limited to a single grading system at a single college of optometry. As such, the findings may not be generalizable to other programs or grading structures. It is possible that students at another college of optometry with a different clinical grading system and a different frequency of feedback may not respond similarly. The qualities and optimal timing for facilitating reflection and student action were not studied but understanding them would be beneficial.

Conclusion

The clinical grading program at SCO is part of the student’s educational paradigm. They use and view the system as a means of learning and reflection and take actionable steps based on the feedback they receive. The quality and timeliness of feedback impact learning. Faculty feedback has a greater positive impact than a negative one. Further study will focus on the nature and quality of faculty feedback, with the goal of providing insight into how student reflection can be enhanced.

Appendix

Survey Questions