Lesson 5. High-risk and unacceptable AI uses in education
Reflection task: Identifying high-risk and unacceptable AI uses
Read the scenario below.
The “Heroes” adult education centre is considering three AI tools:
- 1. Tool A creates general lesson examples and activity ideas for educators.
- 2. Tool B analyses learner attendance, online activity and assignment scores to label learners as “low risk,” “medium risk,” or “high risk” for dropping out.
- 3. Tool C uses learners’ webcam images during online assessments to estimate whether they are stressed, distracted or confident.
Step 1: Classify the risk
For each tool, decide whether it seems to be:
- – lower risk;
- – needs caution;
- – high-risk;
- – unacceptable or prohibited.
Step 2: Explain your reasoning
Answer the following questions:
- Which tool has the least direct impact on learners?
- Which tool could influence how learners are supported or treated?
- Which tool may affect assessment conditions or learner dignity?
- Which tool raises concerns under the EU AI Act?
- Which tool should not be used in normal educational practice?
Step 3: Decide what should happen next
For each tool, decide what the centre should do:
- – Can it be used with basic checking?
- – Does it need organisational approval and safeguards?
- – Does it need a legal/data protection review?
- – Should it be rejected?
Suggested answer direction
- – Tool A is likely lower risk if it uses no personal learner data and staff review the content.
- – Tool B needs strong caution and may become high-risk if the labels affect learner support, placement, progression, or access to opportunities.
- – Tool C is likely unacceptable in normal educational practice because it attempts to infer learners’ emotional states in an educational setting.