Don't redesign, just reflect

A student critical reflection learning tool for GenAI use in assessments

Illustration of a head with computer graphics around.

Authors: Brenda McDermott, PhD, and Lorelei Anselmo, MEd

Last modified: May 21, 2024

Overview

Goal

This resource provides a student self-assessment tool that supports integrating emerging technologies, such as generative AI (GenAI), into their assignments. Course instructors can use this tool to adapt existing assignments and enhance teaching practices. 

When looking to integrating emerging technology into assignments, this student self-assessment tool helps course instructors support students’ learning with the technology. This student self-assessment tool encourages critical reflection on the use of emerging technology within a framework of learning. As technology continues to advance, course instructors and students benefit from exploring how GenAI influences learning.

Audience

This resource is for academic staff, graduate teaching assistants, and staff to support their conversations with students regarding students’ use of GenAI applications in their assignments.   

Related content

Maximizing learning with effective GenAI prompt writing: An instructor’s resource for students

STRIVE: Emerging Considerations When Designing Assessment for AI Use

A First Response to Assessment and ChatGPT in your Courses

Exploring Artificial Intelligence and Assessments

SAMR Model

The SAMR model helps consider the integration of emerging technology. The SAMR model considers how technology impacts teaching and learning through each of the following four levels: substitution, augmentation, modification, and redefinition (Puentedura, 2013). The following is an overview of how SAMR intersects with students’ learning through technology. 

The first two levels of the SAMR model focus on enhancing learning, while the last two support the redefinition of the task provided. The SAMR model can be used to help students reflect on their technology use and its role in their learning. Rather than focusing on the idea of technology replacing students learning, this model situates the ideas of how the technology supports and expands their learning. 

S illustration of a keyboard letter.

Substitution

The first level of the framework involves using the new technology in the same way as another tool. In this way, the only difference is switching one tool for another one. Student use of the tool has not substantively changed their learning output.

For example, students may choose to use a screen-reader to support their comprehension of a written text. The use of technology hasn’t changed the text.

 

The letter "A" illustration of a keyboard letter.

Augmentation

The second level of the framework notes the technology's use, but it allows the learner to do something better. In other words, the use of the tool adds to the expected learning outcome.

For example, a student may use a tool for reading content that allows for built-in definitions. As a result of this modification, the student can add to their comprehension by selecting the definition of unfamiliar words.

M illustration of a keyboard letter.

Modifications

The third level of the framework describes how the use of the technology allows for changes to increased efficiency and effectiveness. As a result, the orientation task can be redesigned in the new context.

For example, a student may use a GenAI tool to add headings and subsections to existing publications or documents to allow the reader a non-linear way of interacting with the text.

R illustration of a keyboard letter.

Redefinition

The last level of the framework articulates when technology allows for new learnings not conceivable without technology use.

For example, the student may use a Generative AI tool to summarize and present the content of the reading as an infographic or pictorial image.

Summary

Substitution

Definition: Exchanging one GenAI tool for another tool.

Example: Using GenAI to verify sentence structure.

Augmentation

Definition: Using GenAI to enhance parts of a learning task.

Example: Using GenAI to build an outline for an essay.

Modification

Definition: Using GenAI to develop parts of a learning task. 

Example: Using GenAI to create case studies to analyze.

Redefinition

Definition: Using GenAI to co-create parts of an assignment.

Example: Using GenAI to co-design an image to include in an assignment.

Considerations

Once a course instructor has determined whether GenAI is acceptable to use in their course assessments, the next conversation with students may focus on the ways in which GenAI use is permitted. 

The following four considerations may help course instructors in these conversations that impact a student’s decisions on what ways GenAI tools may be used in an assessment:

Intention matters

Technology is used by students to support their learning rather than replacing their action. Therefore, students should be using the technology to allow for the modification or redefinition of the assessment task.

Active engagement

Students should remain in control of learning, supported not supplemented by the technology. In this way, the student is not relying on technology as a substitution for their effort. Rather the student is considering how they, through interaction with the technology, are moving towards the intended course learning outcomes.

Responsibility for accuracy

Students are responsible for ensuring the accuracy of the information and approaches. In this way, the technology may augment the output, but the students remain responsible for the curation on the confirmation of outputs.

Evaluation for learning

Students reflect on the technology to support their metacognition. In this interaction, the student considers how technology allows them to achieve new learning outcomes. In particular, it prompts them to consider how learning technology allowed them to redefine their learning.

These guiding questions may be provided to students to assist with their decision-making process when using GenAI in their assessment submissions. 

Intention matters

How is using GenAI in my assessments supporting my learning?

Active engagement

In what ways is using GenAI assisting me with achieving the intended course learning outcomes and assignment objectives?

Responsibility for accuracy

How have I ensured the accuracy of the GenAI output? 

Evaluation for learning

In what ways has using GenAI elevated my learning in the course?

Student critical reflection learning tool and SAMR

Using the SAMR model as a critical reflection learning tool for students to self-assess their GenAI-assisted assessments may result in the following benefits:

  • ensuring that technology-supported assessments are used with intention and align with course learning outcomes;
  • providing students with opportunities to develop competencies in GenAI tools; and,
  • enabling students to develop critical thinking, problem-solving, and self-reflection skills that may benefit students in their future careers. 

Student critical reflection learning tool for GenAI use in assessments

This student critical reflection learning tool is adapted from Cubero (2024) and considers GenAI to include any AI tools that generate new content (text, images, audio, video, code, etc.). 

Rather than re-designing the assessment, course instructors could use this tool for bonus points when students are using GenAI and for providing justification for how a student moves up the SAMR levels.

AI may be used by students as a direct substitute without significant changes, offering minimal enhancements to the assignment, often replacing sections of AI-generated content.

  1. Acceptable GenAI use by student

    GenAI tools can be used by students to make improvements to original student work in support the final output. 

    GenAI replaces other tools/resources that the student might have in this context (such as spellcheck).

  2. Example student explanation

    For this assignment, I drafted the full document without any assistance from AI. I had 10 sentences that I thought were too wordy in the document. I asked [tool] to review these sentences and let me know how I could make them shorter.  For example, [before/after]. 

    I learnt that I could remove most of the adverbs or -ly words to make them easier to follow. 

GenAI can be used by students to enhance various aspects of the assignment, slightly improving its quality by adding GenAI generated insights or functionalities without fundamentally changing the assignment’s nature.

  1. Acceptable GenAI use by student

    GenAI tools can be used for brainstorming, creating structures and generating ideas for improving work.

  2. Example student explanation

    I had never written a reflective assignment before, so I used [Tool] to explain the assignment [see screenshot].  I also read through some of the suggested sources. I drafted some key points and asked the tool to draft an outline. 

GenAI can be used in specific parts of the assessment and is used effectively to significantly improve the assignment’s quality, depth, or structure be blending GenAI content seamlessly and substantially enhancing the overall assignment.

  1. Acceptable GenAI use by student

    GenAI is used to evaluate the quality of the student work and to provide the student with formative assessment before submission. 

    GenAI is used to encourage student reflection and further revision of the preliminary drafts. 

  2. Example student explanation

    I was unsure of the quality of my reflexive assignment. I uploaded my paragraph on media consumption and asked [tool] for the response to the following: Identify key learning from the experience. Evaluate how well it was supported. Provide options for more details or clarification. 

    Based on that feedback, I entered a prompt to redraft my first and last sentence of the paragraph.

GenAI is used in any aspect of the assessment to redefine the assignment fundamentally setting new standards or approaches by using AI to co-create entirely new approaches to the assignment.

  1. Acceptable GenAI use by student

    GenAI tools are used to co-create a student’s work to enhance student creativity. The student is responsible for the final output including all AI-generated content.

  2. Example student explanation

    I was unsure about how to structure my reflection statement, so I used [tool] to provide an outline based on the assignment prompt. I also follow up on the suggested links the tool provided.

    I did a rough outline of the assignment and then I prompted [tool] to evaluate the outline based on the key areas outlined in the rubric.

    Using that feedback, I draft the various sections of the reflection. I again used [tool] to evaluate the quality of the written expression. 

    Lastly, I return to the [tool] and prompt it to grade the assignment based on the rubric, making final adjustments in specific areas. 

    The links to AI chat(s) may be required as part of the final submission as well as a reflection on statement on how GenAI supported their assessment. 

Download template

A SAMR-GenAI critical reflection tool can be used to supplement a course instructor’s current assessments; thereby, not requiring a re-design of existing course assessments. If a course instructor has determined that GenAI is permissible in their course assessments, they can adapt the student critical reflection learning tool to their teaching context. 

Final thoughts

This resource may help course instructors in supplementing their current assessments by allowing them to incorporate GenAI student use without having to redesign each assessment.  In addition, this resource may serve as tool to assist both course instructors and students as they navigate emerging technologies in an ever-changing educational environment.

This adaptable critical reflection tool serves to support academic integrity conversations and GenAI assisted assessments by highlighting student agency and skill development that results from the intentional use of GenAI tools (Perkins et al., 2023).  We recommend that students are asked to cite the GenAI tools used in their reflections. Screenshots of prompts and outputs could also be used as appendices to their reflection statements. 

Supplementary resources

King’s College London (2023). King’s guidance on generative ALI for teaching, assessment and feedback. https://www.kcl.ac.uk/about/strategy/learning-and-teaching/ai-guidance

Moorhouse, B.L., Yeo, M.A., & Wan, Y. (2023). Generative AI tools and assessment: Guidelines of the world’s top-ranking universities. Computers and Education Open, 5, 100151-. https://doi.org/10.1016/j.caeo.2023.100151

References

Cubero, V. (2024). Can I use AI on this assignment? Generative AI acceptable use scale. https://www.canva.com/design/DAF3bSmWIBI/7e7FK5jaH2ripTBSaHIdpA/view

Kumar, V. S., & Boulanger, D. (2021). Automated Essay Scoring and the Deep Learning Black Box: How Are Rubric Scores Determined? International Journal of Artificial Intelligence in Education, 31(3), 538–584. https://doi.org/10.1007/s40593-020-00211-5

Monash University. Learning and teaching: Teach HQ. Generative AI and assessment. https://www.monash.edu/learning-teaching/teachhq/Teaching-practices/artificial-intelligence/generative-ai-and-assessment

Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2023). Navigating the generative AI era: Introducing the AI assessment scale for ethical GenAI assessment. Cornell University. https://doi.org/10.48550/arxiv.2312.07086

Puentedura, R.R. (2013). SAMR model substitution, augmentation, modification, redefinition.  https://d1pf6s1cgoc6y0.cloudfront.net/5fdcf2f73b804107b4fa3f2b6177affa.pdf