OptomCAS | OPTOMETRY ADMISSION TEST     

Optometric Education

The Journal of the Association of Schools and Colleges of Optometry

Optometric Education: Volume 44 Number 2 (Winter-Spring 2019)

PEER REVIEWED

The Student Learning Objectives Initiative
in a Doctor of Optometry Degree Program:
a Report of Student and Faculty Perceptions

Ida Chung, OD, MSHE, and Stephanie Amonoo-Monney, MPA, AALHE

Abstract

This study reports on the perceptions and opinions of Doctor of Optometry students regarding use of student learning objectives (SLOs), prediction of exam questions based on SLOs, use of SLOs by teachers, helpfulness of SLOs, and alignment of exams with SLOs. From the teachers’ perspective, this study reports on the perceived difficulties of constructing SLOs, their actual use, delivery of information about them, and instructions to students about using SLOs. A majority of students utilize lecture-based SLOs for exam preparation and report SLOs are most valuable when they are specific and linked to exam questions. Most faculty find writing SLOs easy and the majority linked their exam questions to SLOs.

Key Words: learning objectives, course objectives, Doctor of Optometry students, student perceptions, optometry curriculum

Background

The concept of student learning outcomes, also known as student learning objectives (SLOs), behavioral objectives, learning goals, goal focusing, or relevance instruction, has been discussed in higher education for many decades.1-7 SLOs are short statements directing students’ attention to the salient points that the teacher wants them to master from lectures, reading assignments or other learning materials and activities. Setting student learning outcomes is the first step in a five-step process of making changes to a course to improve student achievement.8 This is followed by the teacher referring to the outcomes during each class and writing assessments that directly link back to the learning outcomes.9 Benefits are gained when students test themselves utilizing the lecture-specific learning objectives as they prepare for examinations.10,11

In Academic Year 2015-2016, Western University of Health Sciences College of Optometry launched the student learning objectives (SLOs) initiative. The SLOs initiative required faculty to provide to students learning objectives for each and every lecture. In contrast to course learning outcomes, which define on a macro level what students should be able to do at the end of a course, lecture-specific SLOs define what students should be able to do at the end of a lecture (topic), the instructional intent being at a more micro level.12 A number of observational and experimental studies have demonstrated the usefulness of objectives to enhance the depth of learning,13 or the efficiency of discovering what should be learned.14 If the instructor can articulate what he or she wants students to gain from various learning opportunities, students can either acquire those facts, skills or attitudes at a higher level or acquire them more efficiently than students whose learning is not influenced by SLOs. Often, teachers inform students of expectations in a course and the means by which students can achieve the learning outcomes by including learning outcomes in their syllabi.4 When SLOs are incorporated into a syllabus, they can be taken directly from the syllabus as a learning resource for each class meeting.15

As part of ongoing faculty development, faculty members were provided with specific guidelines, rubrics and advice to facilitate the construction of high-quality SLOs (Appendix A). The SLOs were based on a pyramid of learning outcomes. The foundation of the pyramid is the eight Western University of Health Sciences Institutional Learning Outcomes (ILO). These are aligned with 27 Program Learning Objectives (PLO), which are based on the desired Attributes of Students Graduating from Schools and Colleges of Optometry promulgated by the Association of Schools and Colleges of Optometry in 2011.16 Each of the 27 PLOs serves as the foundation for Course Learning Outcomes (2 to 5 per course credit hour, a required part of each course syllabus), which form the basis for the requirement of 3 to 6 SLOs provided by each instructor for each hour of lecture.17,18 Faculty were advised to include the lecture-specific learning objectives in the first slide of each lecture. If faculty incorporated SLOs into the syllabus, for each class meeting the SLOs could be taken directly from the syllabus as a student learning resource. Finally, in the ideal case, the instructor aligns every examination item with one of the lecture-specific SLOs. The ultimate objective of this initiative is to enhance students’ knowledge, skills and attitudes, as articulated in the PLOs and ILOs.

A year after the initiation of the student learning objectives initiative, the effort is highly successful. Virtually all lectures include learning objectives. We now seek to better understand how students and faculty are using the SLOs. The specific aims of the project are: (1) increase understanding of how graduate health professional students utilize lecture-based SLOs in their examination preparation, (2) increase understanding of the process and obstacles in having faculty include SLOs for every lecture, and (3) measure how well the course assessments reflect the SLOs in terms of percentage of test questions.

Methods

All didactic courses administered in the fall of 2015 for the first, second and third years of the Doctor of Optometry curriculum were evaluated for inclusion of student learning objectives for every lecture. Courses with student learning objectives for every lecture were included in the study. Student learning objectives were identified from review of lecture materials posted on the college’s course management system. Sixteen of 18 courses met the criteria.

One study investigator performed an independent evaluation of all exam questions and SLOs for each of the 16 courses to identify the following parameters: total number of SLOs, number of SLOs tested, number of test questions not linked to a SLO, and number of questions for each SLO.

Courses covered various disciplines, including basic biomedical science, vision science and clinical science. We utilized a mixed method research method. Two surveys were developed, a student survey and a faculty survey consisting of qualitative and quantitative questions. The student survey asked if students used the learning objectives, liked using learning objectives and found them beneficial for test preparation. The faculty survey asked how they utilized lecture SLOs in their courses, and what if any obstacles they encountered in writing SLOs. Questions asked in the survey were specific only to courses students had already completed that had listed student learning objectives.

The study received approval from Western University of Health Sciences’ Institutional Review Board. The participants were provided a description of the study and informed consent was assumed if the participants completed the survey. Survey participants were informed their participation was voluntary and they could withdraw at any time without retribution. All participants were assured their responses would be kept confidential and data would be reported in aggregate form in future presentations and publications. The surveys were administered via e-mail to students and faculty from March 1 through March 30 to accommodate absences from campus during spring break. All courses included in the survey were completed during the prior fall semester. The questionnaires included both closed and open-ended questions to provide an understanding of perspectives as well as to corroborate the findings of the survey results. The student survey was administered online via Qualtrics to all first-, second-, and third-year students (n=244) enrolled in the courses included in the study. The faculty survey was administered online via Qualtrics to faculty (n=13) who teach the courses included in the study. Faculty members were asked in their survey to select ordered responses on a five-point Likert scale in response to how difficult or easy it was to write student learning objectives for their respective courses. The survey also prompted faculty to provide comments to the questions, but comments were not mandatory.

Students were asked to select ordered responses on a three- and five-point Likert scale for five questions that included whether they used SLOs in exam preparation, if SLOs were included in their courses, and whether they were able to predict exam questions from the SLOs provided. The student survey also asked for comments about their learning experiences in the courses that included student learning objectives for each lecture. A simple descriptive analysis was used to report the results of the quantitative portion of the survey. A qualitative method of content analysis was used, classifying texts referencing distinct ideas with code names and grouping similar codes to define themes. The clustered frequencies were grouped as themes describing the range and relative weighting of students’ comments. The aim was to illustrate and complement the results of the quantitative analyses using corroboration to confirm the consistency of students’ perception.

Results

The response rate for the student survey was 82.8% (N=202) and the response rate for the faculty survey was 100% (N=13), resulting in a representative sample of respondents.

Student survey

Students were asked to respond to survey questions about SLOs provided in their courses on the ordered Likert scale. Figures 1 and 2 show the results. Figure 1 shows the responses to the first three questions. Figure 2 shows the responses to the last two questions condensed into three categories from five.

Student survey comments

A total of 96 comments were received. Three themes revealed included the usefulness/non-usefulness of SLOs, the non-use of SLOs and non-alignment of test questions to SLOs. The following are highlights from the survey relative to these themes. Thirty-nine students stated that when the SLOs were provided they were helpful, albeit in some courses more than others. Their comments indicated that the learning objectives provided were a useful guide for studying and exam preparation. Nine students stated the SLOs were not useful or helpful. Their comments indicated that the SLOs provided were too general and/or too broad. Ten students indicated they did not use the learning objectives provided. There were 19 comments related to test materials not being aligned with learning objectives provided. A sample of students’ verbatim comments captured through written responses and relative to these themes is provided in Appendix B.

Faculty survey

Faculty responses to questions relating to writing SLOs were captured. Forty-six percent reported that it was easy or very easy; 23% reported it was neither difficult nor easy; and 31% reported that it was difficult or very difficult to write SLOs. Faculty responses to four yes-or-no questions are shown in Table 1.

Faculty survey comments

The second part of the survey prompted faculty to provide comments related to obstacles in writing SLOs, which they identified as determining the key content vs. the outline, the level of detail to include in the objectives, and selecting the best level of Bloom’s Taxonomy to use. Seven faculty members also shared comments about writing SLOs and using them in exam construction. They shared their perceptions about SLOs, how they wrote SLOs, and they provided some reflections about writing SLOs. Verbatim comments are provided in Appendix B.

Review of Learning Objectives and Exam Questions

The results from a review of all test questions and how they matched with SLOs for all 16 courses are summarized in Table 2. Courses varied from 2 to 4 credit hours. The total number of SLOs for each course ranged from 8 to 183, with an average of 87.8. The percentage of SLOs tested (i.e., there was at least one exam question matched to each SLO) ranged from 26% to 100%, with an average of 63.5%. The total number of test questions per course ranged from 87 to 159, with an average of 101 questions. Of these test questions, the percentage of test questions linked to at least one SLO ranged from 91.7% to 100% with an average of 97.9%. The average number of questions per SLO tested was 2.7, with a range of 1.2 to 12.8.

Discussion

First study aim

The first aim of this study was to further our understanding of how graduate health professional students utilize lecture-based student learning objectives in their examination preparation.

This is the first study to document the use of lecture-specific student learning objectives in a Doctor of Optometry program, and only one of three studies to examine use of SLOs in an active classroom setting. Our study found 87.6% of optometry students utilized lecture-specific SLOs to prepare for examinations either always or sometimes, and 12.4% did not. A study involving medical students that included four years of preclinical and clinical courses included only course learning outcomes, without any lecture-specific SLOs.19 In that study, 53% of students agreed course objectives helped them to pass tests. A study involving optometry students found 19 of 22 students (86%) who were surveyed utilized the objectives in learning, including for self-testing; however, the results were based on use of objectives in a course manual for one course only, and response rate was low.14

In our study, 49% of the students who reported utilizing SLOs for exam preparation found SLOs useful, while 15.4% did not. A similar percentage (50.5%) of students reported the exams were aligned with the SLOs. Students found most helpful those objectives that were not too general or too vague, consistent with other studies.19 Additionally, 90% of students predicted possible questions on the exam based on the SLOs provided in each lecture. In other words, objectives that are specific and exam questions that are directly linked to those objectives are the ones students find most helpful in guiding learning. By ensuring SLOs and exam questions are aligned, faculty can drive what students study. This finding is consistent with previous findings that students’ attention to objectives and their actual utilization of provided objectives varies, with a direct positive correlation to increased attention to objectives that are tested 9,20-23 This has implications for teaching in that the instructors’ ability to communicate to students the relevance of the lecture material (i.e., material will be tested) increases student attention and engagement which can result in increased retention of material.23

The 12.4% of students who did not use SLOs for exam preparation and the 15.4% who utilized but did not find SLOs helpful in exam preparation reported several reasons: the test not reflecting the objectives, the objectives not being specific enough, and not knowing how to utilize the SLOs. The latter may indicate that students do not always choose learning strategies that benefit them, an observation others have made.2

Furthermore, our study found that although 100% of faculty provided learning objectives for each lecture, only 34% of students reported professors sometimes (and 1% reported never) provided SLOs for each lecture, and 77% of students reported faculty directly highlighted to students how exam questions align with the lecture. These results point to the need for faculty to increase student awareness of the existence of learning objectives for each lecture, a point confirmed by Stark14 in stating student attention to objectives is necessary as a first step to utilization. Another factor to consider is whether students with certain characteristics are more likely to utilize and benefit from learning objectives for learning. Holt has reported that men (more than women) and medical students with sensing-type Myers-Briggs preferences benefitted most from using learning objectives, as measured by test performance.3 We did not collect this information for this study.

Second study aim

The second aim of this study was to increase understanding of the process and obstacles involved in having faculty include student learning objectives for every lecture.

More faculty found writing SLOs easy (46%) than difficult (31%). Although some faculty members reported that writing SLOs was difficult, a majority of them (85%) reported no obstacles. For the 15% of faculty who reported obstacles, one was determining the appropriate level of detail for guiding student learning while not inadvertently confining the content faculty wished to teach. Faculty have previously reported similar concerns, such as feeling their autonomy is threatened or they are rushed to cover all topics to meet the objectives.4 Faculty comments in our study also revealed selecting the appropriate cognitive level (Bloom’s Taxonomy) for each lecture to be an obstacle. The reported obstacles support the need for more faculty training, and the assistance required by each faculty may vary. Our study is in line with others that have shown higher education faculty are not necessarily trained as teachers and may require extensive training to implement practices that enhance student learning.24

Faculty may use lecture-specific learning objectives as a guide for their examination preparation, which doubles as a blueprint to assist students in examination preparation. The majority (77%) of faculty reported they implicitly informed students they will utilize SLOs for exam construction and instructed students to use the SLOs to guide their exam preparation. This was confirmed by a higher percentage (87.6%) of students who reported using SLOs for examination preparation. This is an important point because even if objectives are present, students may not implicitly know how to use them in learning, and thus faculty’s role in pointing this out to students is important.25 Interestingly, 15% of faculty members reported not using their SLOs to write exam questions, which matches the percentage who reported obstacles to writing SLOs. They may or may not be the same faculty members. SLOs should be broad enough only to accommodate changes in course content over time. Providing SLOs that are too general would be hard to measure and not useful in guiding students to obtain the relevant content in the course. Determining outcomes for each lecture can prove challenging and may require several attempts to properly accomplish.

Third study aim

The third aim of this study was to measure how well the course assessment test questions reflected the student learning objectives.

The number of SLOs for a single course ranged from a low of 8 to a high of 183. For a 2-credit-hour course, 90 SLOs are expected (based on a guideline of 3 learning objectives per lecture hour). For a 4-credit-hour course, 180 SLOs are expected. That a course has only 8 SLOs likely means the objectives are too broad, and therefore less likely for students to find them useful for exam preparation.

If student course performance is measured by test performance, it is important for the test items to accurately reflect the student learning objectives.5 Our study found 100% of test questions were tied to at least one SLO for 7 out of 16 courses. Ninety-eight percent of the test questions were linked to a SLO for 15 of the 16 courses. The high percentage of linkage of exam questions to SLOs provides confidence to the instructors that the material tested reflects the material covered in lectures. Although only a negligible number of test questions (2%) were not linked to a SLO, on average only 63.5% of the SLOs were tested, with a high of 100% to a low of 26% for a single course. A slightly lower percentage (50.5%) of students reported the exams were aligned with the SLOs provided by instructors in each lecture. Additionally, 85% of faculty reported using their SLOs as a guide to write exam questions. Because constructive alignment of learning objectives and test items is important to ensure what is being taught is being measured,26 faculty whose test items are not 100% aligned with learning objectives may benefit from feedback and support.

This study demonstrates the feasibility of initiating a uniform student learning objectives initiative in an established optometry curriculum. Faculty compliance with this initiative during its first year was 89%. This contributes to the feasibility of analyzing and generalizing the study data. The high compliance rate suggests the faculty development program implemented prior to the SLOs initiative was successful in providing the necessary training in instructional approach, including how best to facilitate the incorporation of lecture-specific learning objectives for exam preparation. On the other hand, the finding that faculty assessed 63.5% of the SLOs indicates the need to provide faculty with feedback for continuous improvement of curricular content.

Strengths and limitations

This study is the first to document use of lecture-specific SLOs in a variety of basic, vision and clinical science courses in a healthcare curriculum across 3 years. Well-written lecture-level student learning objectives have important student learning benefits: they enhance student learning by directing students to the essentials, they help students to focus their study on the material teachers most want students to gain from their courses, and they assist teachers in determining appropriate content for exam questions.

One limitation of the study is that it did not evaluate how students utilized the learning objectives or whether they learned more through this process, only whether they used the objectives or not. This study also does not provide information on whether students with certain personality types are more likely to benefit from using SLOs for exam preparation. Without information on actual test performance or specific students, this study cannot directly confirm or reject these hypotheses.

Future implications

Well-written lecture-specific student learning objectives can be utilized to monitor student performance. This is accomplished by linking specific test items to stated learning outcomes and tracking student achievement. Through this process, strong SLOs can be helpful in identifying redundancies and omissions in the curriculum and for monitoring student learning.

Conclusion

Student learning objectives provided for each lecture assisted approximately 88% of doctorate-level health professions students in exam preparation. Students reported that SLOs were useful as long as they are not too general or broad. Objectives that are specific and linked to exam questions are most valuable in advancing student learning. Almost half of the faculty reported that writing learning objectives was easy, but nearly a third found it difficult, indicating a need for continuous feedback for faculty development. Our study found the majority of faculty utilize specific learning objectives for test construction, and most learning objectives are tested on exams. The incorporation of lecture-level SLOs can be impactful for student learning and is recommended as part of the curricular pedagogy for health professions curriculum. Whether SLOs actually change student learning outcomes needs further investigation.

Acknowledgements

The authors wish to acknowledge Daniel Kurtz, OD, PhD, FAAO, for his intellectual contribution and support in the development of this manuscript.

References

  1. Deterline WA. The secrets we keep from students. In: Kapfer MB, editor. Behavioral objectives in curriculum development: Selected readings and bibliography. Englewood Cliffs: Educational Technology Publications; 1971. pp. 3-8.
  2. Abate MA, Stamatakis MK, Haggett RR. Excellence in curriculum development and assessment. Am J Pharm Educ. 2003;67(3):1-22.
  3. Holt JT, Ghormoz J, Sung YJ, White MW, Szarek JL. Medical student benefit from learning objectives correlates to specific Myers-Briggs types. Med Sci Educ. 2015;25:249-254.
  4. Bahous R, Nabhani M. Faculty views on developing and assessing learning outcomes at the tertiary level. J General Education. 2015;64(4):294-309.
  5. Anderson HM, Moore DL, Anaya G, Bird E. (2005). Student learning outcomes assessment: a component of program assessment. Am J Pharm Edu. 2005;69(2):256-268.
  6. Prideaux D. The emperor’s new clothes: from objectives to outcomes. Med Educ. 2003;34:168-9.
  7. Webb EM, Naeger DM, Fulton TB, Straus CM. Learning objectives in radiology education: why you need them and how to write them. Acad Radiol. 2013;20:358-63.
  8. Walvoord BE (2010). Assessment clear and simple. A practical guide for institutions, departments and general education. SE Jossey-Bass.
  9. Raghubir KP. The effects of prior knowledge of learning outcomes on student achievement and retention in science instruction. J Res Sci Teach. 1979;16(4):301-4.
  10. Rushin JW, Baller W. The effect of general objectives defined by behavioral objectives on achievement in a college zoology course. Coll Stud J. 1981;15(2):156-61.
  11. Mast TA, Silber DL, Williams RG, Evans GP. Medical student use of objectives in basic science and clinical instruction. J Med Educ. 1980;55(9):765-72.
  12. Harden RM. Learning outcomes and instructional objectives: is there a difference? Medical Teacher. 2002;24(2):151-5.
  13. McCrudden MT, Schraw G. Relevance and goal-focusing in text processing. Educational Psychology Review. 2007;19(2):113-139.
  14. Stark LR. Communicating educational objectives in an optometry course. Optometric Education. 2017;42(3):35-42.
  15. Crossman JM. Using your syllabus as a learning resource [Internet]. Faculty Focus; 2014 June 9 [cited 2018 November 4]. Available from: https://www.facultyfocus.com/articles/teaching-and-learning/using-syllabus-learning-resource/.
  16. Smythe JL, Daum KM. Attributes of students graduating from schools and colleges of optometry: a 2011 report from the Association of Schools and Colleges of Optometry. Rockville (MD): Association of Schools and Colleges of Optometry; 2011 October 11.
  17. Maki PL. Assessing for learning: building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing, LLC; 2004.
  18. Nichols JO, Nichols KW. The departmental guide and record book for student outcomes assessment and institutional effectiveness. 3rd ed. New York: Agathon Press; 2000.
  19. Mast TA, Silber DL, Williams RG, Evans GP. Medical student use of objectives in basic science and clinical instruction. J Med Educ. 1980;55(9):765-72.
  20. Duchastel P. Learning objectives and the organization of prose. J Educ Psychol. 1979;71(1):100-6.
  21. Tobias S, Duchastel PC. Behavioral objectives, sequence, and anxiety in CAI. Instr Sci. 1974;3(3):231-42.
  22. Jiang L, Elen J. Why do learning goals (not) work: a reexamination of the hypothesized effectiveness of learning goals based on students’ behaviour and cognitive processes. Educ Technol Res Dev. 2011;59(4):553-73.
  23. Dew R, Goscinski A, Coldwell-Neilson J. Towards a framework for aligning learning outcomes, academic literacies and assessment criteria. Educ Inf Technol. 2016;21:401-423.
  24. Banta TW. Can assessment for accountability complement assessment for improvement? [Internet]. Washington (DC): Association of American Colleges & Universities; 2007 Spring [cited 2018 November 4]. Available from: https://www.aacu.org/publications-research/periodicals/can-assessment-accountability-complement-assessment-improvement.
  25. Bain K. What the best college teachers do. Cambridge, MA: Harvard University Press; 2004.
  26. Biggs J, Tang C. Teaching for quality learning at university. 3rd ed. New York: Open University Press; 2007.

    Appendix B. Click to enlarge

 Save article as PDF

Dr. Chung [ichung@westernu.edu] is the Associate Dean of Academic Affairs at Western University of Health Sciences College of Optometry. She received her Doctor of Optometry degree from the State University of New York College of Optometry and her Master of Science degree in Higher Education from Drexel University. She completed her residency training in the area of Pediatrics and Binocular Vision at the Eye Institute of the Pennsylvania College of Optometry at Salus University.

Stephanie Amonoo-Monney is the Director of Program Assessment and an Adjunct Faculty member at Western University of Health Sciences College of Optometry. She received her Master of Public Administration degree from Villanova University. Her research interests include the practice of assessment and evaluation in higher education.