A First Response to Assessment and ChatGPT in your Courses
This summary was prepared by Lorelei Anselmo, Tyson Kendon, and Beatriz Moya at the Taylor Institute for Teaching and Learning at the University of Calgary.
Introduction / Context
Recent developments in artificial intelligence (AI) tools, such as ChatGPT, have triggered reactions in higher education, ranging from fear about impacts on academic integrity, to concerns about the tool’s effectiveness, to opportunities to innovate. Navigating these developments as part of teaching and learning is critical (Eaton, 2022) and some considerations could support new ways to ethically use these tools.
There are many relevant AI tools that affect course design and assessment; this resource focuses on ChatGPT as it has sparked the current conversations. Further, this resource focuses on assessment, particularly how to engage with your teaching team and students to understand what ChatGPT is, how it may be used in student work and how it may be used as part of assessment.
Where to start
Addressing ChatGPT in assessment requires building trust, communicating, collaborating, and connecting with colleagues, students, and your teaching team.
Working with your teaching team
How could you use ChatGPT to support your assessment?
ChatGPT is not a replacement for student work or for authentic assessment. When used intentionally, it may be used to assist your teaching team with improving communication and managing your assessment time. For example, ChatGPT could help provide clear feedback as it can perform some tasks, such as expanding short sentences or comments, modelling more professional communication, and correcting errors (Hemsley et al., 2023).
What are your expectations?
As a teaching team, identify what your approach to assessment will be, and your thoughts on artificial intelligence.
Discuss what your team expects from your students and how they might use ChatGPT:
- What uses would be regarded as academic misconduct? What uses would not be considered academic misconduct? Consider that not all artificial intelligence uses are academic misconduct (Eaton, 2022).
- What red flags could potentially signal the presence of ChatGPT text (e.g., absence of subjective opinions or insights, generic language, and repetitive phrases)? It is not possible to definitively determine whether or not a text was generated by ChatGPT, even when using other AI tools. (Taylor, 2023)
- Refer to the academic misconduct policy and plan for the next steps considering the student academic misconduct guidelines.
Discuss ways your team might use ChatGPT to assist with grading:
- What procedures could be delegated to ChatGPT to expedite assessing and giving feedback while protecting students?
- How can the teaching team address potential issues of bias and grade responsibly? (UNESCO, 2022).
Example
Ask ChatGPT to help convert assessment points into grades or transform short notes and comments about the students’ assignments into coherent and cohesive messages that could be reviewed before final submission.
Related content
Articles and resources for ChatGPT
Additional readings
Castelvecchi. D. (2022, December 8). Are ChatGPT and AlphaCode going to replace programmers? Nature (London). https://www-nature-com.ezproxy.lib.ucalgary.ca/articles/d41586-022-04383-z
CESE NSW What Works Best in Practice. (n.d.) A Teacher’s Prompt Guide to ChatGPT. https://usergeneratededucation.files.wordpress.com/2023/01/a-teachers-prompt-guide-to-chatgpt-aligned-with-what-works-best.pdf
Eaton, S. (2022). Sarah’s thoughts: Artificial intelligence and academic integrity. Learning, Teaching and Leadership: A blog for educators, researchers and other thinkers by Sarah Elaine Eaton, PhD. https://drsaraheaton.wordpress.com/2022/12/09/sarahs-thoughts-artificial-intelligence-and-academic-integrity/
Eaton, S., & Anselmo, L. (2023, January 12). Teaching and Learning with Artificial Intelligence Apps. Taylor Institute for Teaching and Learning, University of Calgary. https://taylorinstitute.ucalgary.ca/teaching-with-AI-apps
Gordijn, B. & Have, H. T. (2023). ChatGPT: evolution or revolution? Medicine, Health Care, and Philosophy. 1-2. https://doi.org/10.1007/s11019-023-10136-0
Hemsley, B., Power, E., & Given, G. (2023, January 18). Will AI tech like ChatGPT improve inclusion for people with communication disability? The Conversation. https://theconversation.com/will-ai-tech-like-chatgpt-improve-inclusion-for-people-with-communication-disability-196481
Kumar, R., Mindzak, M., Eaton, S. E., & Morrison, R. (May 17, 2022). AI & AI: Exploring the contemporary intersections of artificial intelligence and academic integrity [Conference Session]. Canadian Society for the Study of Higher Education Annual Conference [Online]. http://hdl.handle.net/1880/114647
Mills, A. (2023). What to do about artificial intelligence text generators. [Webinar]. College of Marin. https://docs.google.com/presentation/d/1P5nSOm1g3CvsoPEga4fSnzjMiLtGg7Bf/mobilepresent?slide=id.p2
Mollick, Ethan R. and Mollick, Lilach, New Modes of Learning Enabled by AI Chatbots: Three Methods and Assignments (December 13, 2022). Available at SSRN: https://ssrn.com/abstract=4300783 or http://dx.doi.org/10.2139/ssrn.4300783
Taylor, J. (2023, February 1) ChatGPT maker OpenAI releases ‘not fully reliable’ tool to detect AI generated content The Guardian. https://www.theguardian.com/technology/2023/feb/01/chatgpt-maker-openai-releases-ai-generated-content-detection-tool
Which strategies could you consider for your courses?
As a teaching team, collaborate to find ways that ChatGPT can be used ethically in assessment.
Develop explicit guidelines for grading in your courses’ assignments and assessments for your teaching team and provide a rationale for these choices.
How are you going to address ChatGPT in your grading and feedback?
Plan how your team is going to discuss ChatGPT with your students and how/if it will be used in their assessment.
Can students use ChatGPT in their work and in what ways?
How will students be asked to disclose if they have used ChatGPT?
Experiment with feedback prompts created by ChatGPT.
Can they be modified and used to assist with improving qualitative comments?
Example
Co-create criteria in your rubric that addresses the use of ChatGPT, or search for potential answers ChatGPT could provide to your assignment so the teaching team is aware of this tool’s outcomes.
Example
Experiment using ChatGPT to improve your comments and feedback.
Working with your students
How can ChatGPT impact your students' learning and your teaching?
Develop high-quality student-teacher relationships by creating opportunities to discuss the ethical use of artificial intelligence tools. The existence of ChatGPT may require more discussion on the purpose of assessments and the connection to learning outcomes.
What are your expectations?
Discuss ChatGPT with your students, include how it will affect their work and how it will affect your assessment of their work.
Discuss what you and your students know about ChatGPT.
What uses would be regarded as academic misconduct? What uses could improve teaching or learning?
Co-create guidelines regarding how and when to use and document ChatGPT (Eaton & Anselmo, 2023).
How can ChatGPT be used to reflect ethical learning?
Example
Review examples of appropriate and inappropriate use of ChatGPT.
Example
Agree upon a style to document the use of ChatGPT in your assignments (citation, highlighting, discussion in text, etc.) and indicate this in your rubrics.
Which strategies could you consider for your courses?
Discuss potential uses of ChatGPT in assignments and assessments and the ways in which you might use ChatGPT to support your work.
Help students recognize ChatGPT’s limitations (e.g., questions about grammar accuracy, citations, text cohesion, authorship, depth, reasoning, and the quality of text) (Kumar et al., 2022; Mills, 2023).
Experiment with AI-generated content with your students by annotating and critiquing an assignment, or by experimenting with the quality of various inputs and prompts.
Identify ways to use ChatGPT to interact with students over their assessments.
Example
How can you identify and articulate to your students which stages of the assignment or assessment your students could use ChatGPT and how those sections will be graded?
Example
Create assignment instructions with the help of ChatGPT in various ways of representation to address students’ diverse needs such as formatting and text length.
Reflect
The capabilities of AI tools and the reaction in higher education are changing rapidly. Take advantage of the flexibility to help inform your future approaches.
- Work with your teaching team to reflect on how ChatGPT has been a factor in assessing your students' work.
- Gather evidence for how ChatGPT was used in your teaching, how students experienced ChatGPT, how they used it, and how it impacted assessment.
- Use this reflection to inform your assessment in this course and consider how you might adjust your assessments in future course offerings.