Getting to Know ChatGPT – Notes from Our Initial Conversations

Generative artificial intelligence (AI) is the term used for algorithms that can generate new content including text, audio, images and video.¹ As you likely know, a recent breakthrough in generative AI is the chatbot ChatGPT (Chat Generative Pre-trained Transformer). ChatGPT (https://chat.openai.com) is able to answer a wide range of complex questions and perform many advanced tasks. It is based on an enhanced version of GPT-3, a speech-processing AI model developed by OpenAI. GPT-3 is equipped with a staggering 175 billion parameters, making it the largest and most complex language model ever.²

It has been widely reported that within 2 months of becoming available to the public, ChatGPT attracted more than 100 million users. Curious as to whether this powerful new tool could, or should, be used by optometry educators and students, and inspired by scholarly contributions from health care,^3,4 we used multiple prompts to test the bot’s capabilities in February 2023.

Opportunities and Caveats

Table 1. Click to enlarge

Table 1 lists inputs/prompts we used to explore ChatGPT’s capabilities along with our general assessments of the resulting outputs/responses. We found that the technology can serve as a useful virtual assistant to optometry educators for certain routine day-to-day tasks, potentially freeing up time for higher-level tasks. In addition, because ChatGPT converses naturally on knowledge and comprehension, it may enable educators to focus more on facilitating student learning, for example, by empowering students to tackle certain basic learning activities on their own. Similarly, using ChatGPT as a self-directed learning tool is a potential way students can take ownership of their learning, which is key to their success in optometry school and their lifelong success as optometrists. Prompted with custom queries, ChatGPT can provide personalized interactive support, real-time feedback, and assistance with self-assessment and revision.⁵

However, if educators and students are to contemplate uses such as learning facilitation and self-directed learning, they must be aware of ChatGPT’s current shortcomings. For example, while the bot is able to predict the next possible word in a sentence, it does not seem to understand whether the information is true. Some of its output may be false, vague, biased and devoid of depth.⁶ Also, the output is not always the same even if the same prompt is repeated, and responses seem to be more accurate in the recall of facts and concepts (i.e., lower-order Bloom’s taxonomy) than for higher-level, more cognitive questions.⁴

Given the current shortcomings of ChatGPT, some education institutions have partially or completely banned students from using it on school servers.⁷ Institutions and educators who are encouraging use of the tool to improve learning⁸ should actively guide students with regard to its potential benefits⁸ and limitations. For example, much like case discussions and team-based projects, ChatGPT could be used to promote critical-thinking skills, but only with educator coaching on the need for critical analysis of the authenticity of the information it provides. Additionally, because ChatGPT has been shown to write essays⁹ and answer questions well enough to pass medical exams, now is a good time for institutions to reinforce the dire consequences associated with plagiarism and cheating.

Conclusion

The functionality and authenticity of ChatGPT will undoubtedly change rapidly. As time goes on, it will likely impact the field in many ways. Already, it has put educators on alert for ChatGPT responses in students’ written work. Furthermore, assessments of student competencies that require them to reproduce knowledge without interpretation or application are likely to have a limited role in the future. In the meantime, based on our initial exploration of its utility as summarized here, this AI tool can be cautiously embraced in optometry education.

Acknowledgements

We used optometry-related prompts to test ChatGPT (Table 1); however, this article was not written by ChatGPT. The authors have no financial or proprietary interest in any materials mentioned herein.

References

1. Gozalo-Brizuela R, Garrido-Merchan EC. ChatGPT is not all you need. A state of the art review of large generative AI models. arXiv:2301.04655v1 [cs.LG]. 2023;1-22. doi: https://doi.org/10.48550/arXiv.2301.04655.
2. Brown TB, Mann B, Ryder N, et al. Language models are few-shot learners. arXiv:2005.14165. 2020. doi: https://doi.org/10.48550/arXiv.2005.14165.
3. Gordijn B, Have HT. ChatGPT: evolution or revolution? Med Health Care Philos. 2023 Mar;26(1):1-2. doi: 10.1007/s11019-023-10136-0.
4. Antaki F, Touma S, Milad D, El-Khoury J, Duval R. Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings. Ophthalmol Sci. 2023 May 5;3(4):100324. doi: 10.1016/j.xops.2023.
5. Firat M. How ChatGPT can transform autodidactic experiences and open education? [Internet]. OSF Preprints; 2023. Available from: osf.io/9ge8m.
6. Cano YMY, Venuti F, Martinez RH. ChatGPT and AI text generators: should academia adapt or resist? [Internet]. Boston, MA: Harvard Business Publishing – Education; c2023 [cited 2023 Feb 1]. Available from: https://hbsp.harvard.edu/inspiring-minds/chatgpt-and-ai-text-generators-should-academia-adapt-or-resist.
7. Schwartz EH. ChatGPT is banned by these colleges and universities [Internet]. Washington, DC: Voicebot.ai; c2023 [cited 2023 Feb 10]. Available from: https://voicebot.ai/2023/02/09/chatgpt-is-banned-by-these-colleges-and-universities/.
8. Villasenor J. How ChatGPT can improve education, not threaten it [Internet]. New York, NY: Scientific American; c2023. Available from: https://www.scientificamerican.com/article/how-chatgpt-can-improve-education-not-threaten-it/.
9. Stokel-Walker C. AI bot ChatGPT writes smart essays – should professors worry? Nature. 2022 Dec 9. doi: 10.1038/d41586-022-04397-7. Epub ahead of print.