Authors
Resource type
Use type
Tools
Tags
Project name
Lesson Overview
In this lesson, students interact directly with several LLMs to evaluate model performance and think critically about implications of model output for their work, their peers, and for society.
NOTICES:
- Students <13 years old should not engage this lesson due age restrictions.
- The previous lesson in this sequence, Lesson 4.3, introduces skills that benefit from the critical thinking activities introduced in this lesson.
Total Lesson Time: 60 minutes
Learning Objectives:
- Making counterfactual explanations can help probe an LLM to identify (a) parts of a prompt that have the greatest impact on the output and (b) parts of a prompt that lead to biased output.
- LLMs can cultivate confirmation bias, thus (a) they are not great tools for learning or research, (b) they can lead to over-reliance and even dependency.
Vocabulary Introduced: confirmation bias, dependency
Pacing:
- Opening (5 min)
- Mini-lesson: LLM evaluation (10 min)
- LLM output evaluation activity (10 min)
- Mid-workshop interruption: Sharing noticings (10 min)
- LLM output evaluation activity continued (10 min)
- Reflection & Discussion (10 min)
- Closing (5 min)
Planning Guide
Preparation Needed: 15-20 minutes
Prep Needed for Teaching In-Person:
- Communicate with school admin and families, well in advance of implementing this lesson, to let them know that you will be using LLMs in class, specifically models such as chatGPT and Gemini.
- Give students access to chatGPT, Gemini, and any other LLMs you would like them to explore.
- Give students access to the LLM evaluation activity guide either as a link to the digital version or as a printed worksheet.
Log in or register to view attachments and related links, and/or join the discussion. If you are already logged in, scroll to the bottom of this page for the links.
Activity Usage
Copyright held by MIT STEP Lab
License: CC-BY-NC under Creative Commons
These materials are licensed as CC-BY-NC 4.0 International under creative commons. (For more information visit https://creativecommons.org/licenses/by-nc/4.0/). This license allows you to remix, tweak, and build upon these materials non-commercially as long as you include acknowledgement to the creators. Derivative works should include acknowledgement but do not have to be licensed as CC-BY-NC. People interested in using this work for for-profit commercial purposes should reach out to Irene Lee at [email protected] for information as to how to proceed. Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)
Attribution:
This unit was created by Katherine (Kate) Moore of MIT for the Everyday AI PD project, which created the Developing AI Literacy (DAILy) 2.0 curriculum.