Introduction / Context
Recent developments in artificial intelligence (AI) tools, such as ChatGPT, have triggered reactions in higher education, ranging from fear about impacts on academic integrity, to concerns about the tool’s effectiveness, to opportunities to innovate. Navigating these developments as part of teaching and learning is critical (Eaton, 2022) and some considerations could support new ways to ethically use these tools.
There are many relevant AI tools that affect course design and assessment; this resource focuses on ChatGPT as it has sparked the current conversations. Further, this resource focuses on assessment, particularly how to engage with your teaching team and students to understand what ChatGPT is, how it may be used in student work and how it may be used as part of assessment.
Where to start
Addressing ChatGPT in assessment requires building trust, communicating, collaborating, and connecting with colleagues, students, and your teaching team.
Ask ChatGPT to help convert assessment points into grades or transform short notes and comments about the students’ assignments into coherent and cohesive messages that could be reviewed before final submission.
Castelvecchi. D. (2022, December 8). Are ChatGPT and AlphaCode going to replace programmers? Nature (London). https://www-nature-com.ezproxy.lib.ucalgary.ca/articles/d41586-022-04383-z
CESE NSW What Works Best in Practice. (n.d.) A Teacher’s Prompt Guide to ChatGPT. https://usergeneratededucation.files.wordpress.com/2023/01/a-teachers-prompt-guide-to-chatgpt-aligned-with-what-works-best.pdf
Eaton, S. (2022). Sarah’s thoughts: Artificial intelligence and academic integrity. Learning, Teaching and Leadership: A blog for educators, researchers and other thinkers by Sarah Elaine Eaton, PhD. https://drsaraheaton.wordpress.com/2022/12/09/sarahs-thoughts-artificial-intelligence-and-academic-integrity/
Eaton, S., & Anselmo, L. (2023, January 12). Teaching and Learning with Artificial Intelligence Apps. Taylor Institute for Teaching and Learning, University of Calgary. https://taylorinstitute.ucalgary.ca/teaching-with-AI-apps
Gordijn, B. & Have, H. T. (2023). ChatGPT: evolution or revolution? Medicine, Health Care, and Philosophy. 1-2. https://doi.org/10.1007/s11019-023-10136-0
Hemsley, B., Power, E., & Given, G. (2023, January 18). Will AI tech like ChatGPT improve inclusion for people with communication disability? The Conversation. https://theconversation.com/will-ai-tech-like-chatgpt-improve-inclusion-for-people-with-communication-disability-196481
Kumar, R., Mindzak, M., Eaton, S. E., & Morrison, R. (May 17, 2022). AI & AI: Exploring the contemporary intersections of artificial intelligence and academic integrity [Conference Session]. Canadian Society for the Study of Higher Education Annual Conference [Online]. http://hdl.handle.net/1880/114647
Mills, A. (2023). What to do about artificial intelligence text generators. [Webinar]. College of Marin. https://docs.google.com/presentation/d/1P5nSOm1g3CvsoPEga4fSnzjMiLtGg7Bf/mobilepresent?slide=id.p2
Mollick, Ethan R. and Mollick, Lilach, New Modes of Learning Enabled by AI Chatbots: Three Methods and Assignments (December 13, 2022). Available at SSRN: https://ssrn.com/abstract=4300783 or http://dx.doi.org/10.2139/ssrn.4300783
Taylor, J. (2023, February 1) ChatGPT maker OpenAI releases ‘not fully reliable’ tool to detect AI generated content The Guardian. https://www.theguardian.com/technology/2023/feb/01/chatgpt-maker-openai-releases-ai-generated-content-detection-tool
Develop explicit guidelines for grading in your courses’ assignments and assessments for your teaching team and provide a rationale for these choices.
How are you going to address ChatGPT in your grading and feedback?
Plan how your team is going to discuss ChatGPT with your students and how/if it will be used in their assessment.
Can students use ChatGPT in their work and in what ways?
How will students be asked to disclose if they have used ChatGPT?
Experiment with feedback prompts created by ChatGPT.
Can they be modified and used to assist with improving qualitative comments?
Co-create criteria in your rubric that addresses the use of ChatGPT, or search for potential answers ChatGPT could provide to your assignment so the teaching team is aware of this tool’s outcomes.
Experiment using ChatGPT to improve your comments and feedback.
Discuss what you and your students know about ChatGPT.
What uses would be regarded as academic misconduct? What uses could improve teaching or learning?
Co-create guidelines regarding how and when to use and document ChatGPT (Eaton & Anselmo, 2023).
How can ChatGPT be used to reflect ethical learning?
Review examples of appropriate and inappropriate use of ChatGPT.
Agree upon a style to document the use of ChatGPT in your assignments (citation, highlighting, discussion in text, etc.) and indicate this in your rubrics.
Help students recognize ChatGPT’s limitations (e.g., questions about grammar accuracy, citations, text cohesion, authorship, depth, reasoning, and the quality of text) (Kumar et al., 2022; Mills, 2023).
Experiment with AI-generated content with your students by annotating and critiquing an assignment, or by experimenting with the quality of various inputs and prompts.
Identify ways to use ChatGPT to interact with students over their assessments.
How can you identify and articulate to your students which stages of the assignment or assessment your students could use ChatGPT and how those sections will be graded?
Create assignment instructions with the help of ChatGPT in various ways of representation to address students’ diverse needs such as formatting and text length.
The capabilities of AI tools and the reaction in higher education are changing rapidly. Take advantage of the flexibility to help inform your future approaches.
- Work with your teaching team to reflect on how ChatGPT has been a factor in assessing your students' work.
- Gather evidence for how ChatGPT was used in your teaching, how students experienced ChatGPT, how they used it, and how it impacted assessment.
- Use this reflection to inform your assessment in this course and consider how you might adjust your assessments in future course offerings.