Exploring Artificial Intelligence and Assessments

A guide for instructors considering artificial intelligence in their assessments.

Authors: Lorelei Anselmo, MEd & Dr. Tyson Kendon, PhD

Last modified: October 10, 2023

The way digital technologies are designed, used, and presented in the public discourse have implications for teacher assessment practices, as they potentially transform how the teachers grade students' work and consider what assessment entails in the new technological landscape.

Farazouli et al., 2023, p.4

This short guide provides some context on Generative Artificial Intelligence for instructors, as well as some suggestions for ethically addressing artificial intelligence in teaching and learning. There are also some risks and limitations you may need to consider as well as some guiding questions and approaches which may help mitigate those risks and limitations.

A First Response to Assessment and ChatGPT in your Courses

Teaching and Learning with Artificial Intelligence Apps

The good, the bad and the artificial: Bringing artificial intelligence into the classroom

Artificial Intelligence

A catch-all term for techniques to help computers solve problems the way people would or to model computational problem solving on biological brains.

Generative Artificial Intelligence

Techniques to allow AI systems to produce works based on the data they have been trained on. These can be text, images, translations, sound, video or more. Other types of AI systems might do things like classify inputs.

Large Language Model

A type of AI system that generates text based on the probability of words in its training data and the text that it has already generated.

Informed Use: Prior to using the tool, research who or what company developed the tool, how it was developed, how it works, what functions it can perform, and what limitations and/or risks it presents.

Transparent Use: In your work, indicate in detail and expressly which tool you used and how you used it.

Ethical Use: The manuscripts must distinguish what was written or produced directly by you and what was generated by an AI tool.

Responsible Use: Limit the use of AI tools to early stages of research, to inspire or suggest directions, not to produce content that will later be included in your deliverables. In any use of GAI, it is your responsibility to verify all content.

Gutiérrez, J. (2023) Guidelines for the Use of Artificial Intelligence in University Courses. Version 4.3, Universidad del Rosario.

AI systems may not fit into your teaching and may be problematic. The following are some factors to consider when deciding whether to incorporate AI in your assessments:

Lack of understanding context and nuance

Generative AI might not be trained on the specific context related to your course. Generative AI might provide incorrect or oversimplified answers that could mislead students.

Inaccuracies and bias in information

Al models can inherit biases present in their training data which could perpetuate or amplify existing biases, leading to discriminatory output.

Privacy and data security concerns

Use of Generative AI might involve sharing proprietary information and student data with third parties. This raises concerns with data security and privacy breaches.

Lack of feedback and guidance

Generative AI might provide feedback that is not correct, context-specific or helpful for students to advance and succeed in their learning.

Limited interactivity and discussion

Generative AI might appear to be performing as a conversational tool: however it is not able to provide meaningful and reliable information on which to base critical thoughts or deep analysis on course-related topics or problems.

Overreliance on technology

An overreliance of Generative AI might reduce a learners’ ability to develop transferable, creative, and critical thinking skills

Use clear guidelines and expectations

Provide students with clear guidelines about the role of Generative AI in assessments. Make it clear what Generative AI can and cannot do and specify how it could be used as a resource, emphasizing that Generative AI is a tool to aid understanding not a replacement for critical thinking.

Incorporate human grading and review

While Generative AI can assist in certain aspects of student work, human graders should play a significant role in evaluating student work, providing context-specific feedback.

Provide feedback on AI-generated responses

When Generative AI responses are used in assessments, offer feedback on the quality and accuracy of those assessments. This helps students understand the flaws in AI-generated responses and encourages them to critically evaluate the information provided.

Discuss how AI-assisted assignments will be assessed

Demonstrate to students how AI-generated responses will be evaluated including providing an explanation of the extent to which AI was used in the assignment and indicate how students cross-referenced information with reliable sources.

Address bias and fairness

Include discussion on the ethical use of AI and encourage students to critically assess the information provided by AI to consider its accuracy, bias, and implications.

Mix AI-assisted and non-AI assisted questions

Design assessments that incorporate a mix of questions suited for AI-assistance and questions that require deep analytical skills. This allows learners to practice with GAI while still allowing for the development of critical thinking and creativity skills.

Provide balanced resources

Ensure that students have access to a variety of resources, including AI tool, textbooks, and other materials so students learn to make informed decisions about when to use Generative AI and when to rely on other sources.

Cornell University Center for Teaching and Innovation (2023). Generative Artificial Intelligence for Education and Pedagogy. https://teaching.cornell.edu/generative-artificial-intelligence/cu-committee-report-generative-artificial-intelligence-education?mibextid=hubsqH

Communicate and set expectations with your students.

Consider: Have a conversation with your students about what AI is and where, or if, it fits within your course. Include how you intend to use generative AI and how your students can use it and what conditions must be met if they do.

Provide opportunities to model and practice ethical use of AI in learning.

Consider: Introduce an assignment by demonstrating how Generative AI can be used as a brainstorming tool. Include examples of how to cite Generative AI use correctly.

Demonstrate a proactive approach to assessments.

Consider: Include both formative and summative assessments that align with course learning outcomes and focus on process rather than product.

Focus on creativity and critical-thinking skills.

Consider: Design assessments that require specific skills or topics directly related to the course and can be demonstrated in multiple formats.

Use technology judiciously.

Consider: Review when and if it is appropriate to use Generative AI in your courses. Ensure students are aware of the risks and limitations of using Generative AI.

Incorporating GAI into assessments can offer benefits and challenges for both instructors and students. Here are some guiding questions for instructors to think about when considering the use of GAI in their assessments:

What are the specific learning outcomes or goals that I want students to achieve with this assessment? How can Generative AI help me better align the assessment with these objectives?
What type of assessment am I planning to use AI for? Which aspect of the assessment will Generative AI enhance or automate?
How will I ensure fairness and avoid bias in Generative AI assessments? Have I considered potential biases in the training data or AI algorithms? What steps can I take to mitigate bias and ensure fairness in the assessment process?
How will students be informed about the use of Generative AI in assessments? Are there any ethical or privacy concerns that students should be aware of?
How will I ensure the ethical and responsible use of Generative AI in assessments? Have I considered the ethical implications if using Generative AI in grading or assessments?
How can GAI enhance the feedback process for students? Can Generative AI help in making feedback timelier and more accurate?
How will I assess the effectiveness of Generative AI in improving assessments? Will I conduct surveys or gather feedback from students?

Artificial Intelligence Libguide

101 Creative Ideas to Use AI in Education

Farazouli, A., Cerratto-Pargman, T., Bolander Laksov, K., & McGrath, C. (2023). Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices. Assessment and Evaluation in Higher Education. https://doi.org/10.1080/02602938.2023.2241676

Gutiérrez, J. (2023) Guidelines for the Use of Artificial Intelligence in University Courses. Version 4.3, Universidad del Rosario. https://forogpp.files.wordpress.com/2023/02/guidelines-for-the-use-of-artificial-intelligence-in-university-courses-v4.3.pdf

Mishra, P., Warr, M. & Islam, R. (2023). TPACK in the age of ChatGPT and Generative AI, Journal of Digital Learning in Teacher Education, 39(4), 235-251. https://doi.org/10.1080/21532974.2023.2247480

National Academic Integrity Network (2023). Generative artificial intelligence guidelines for educators. https://www.qqi.ie/what-we-do/engagement-insights-and-knowledge-sharing/national-academic-integrity-network

Exploring Artificial Intelligence and Assessments

Related content

Artificial Intelligence Terminology