An AI feature can appear overnight in a service a school already uses. The interface may look familiar, but the information flow may have changed. Student prompts, uploaded work, behavioural data and generated responses can create new privacy and security questions even when the underlying product has already passed procurement.
The Australian Framework for Generative AI in Schools places privacy, security and safety among its core principles. The practical implication is that approval should depend on what data is necessary, where it travels, how long it remains and who can use it.
1. What information is genuinely required?
Begin with purpose. If a tool generates practice questions, does it need a student’s full name, date of birth or detailed profile? Data minimisation means collecting only what the educational task requires. Optional fields and convenient integrations can expand exposure without improving learning.
2. Are prompts and uploads used to improve the model?
Schools should distinguish between processing information to deliver the service and using it to train or improve a broader system. Vendor language may describe this as service improvement. Ask for a direct answer covering student prompts, files, feedback and metadata, including whether settings can disable secondary use.
3. Where is information stored and who can access it?
Location, subcontractors and support access may affect legal and policy obligations. A diagram of the data flow is often more useful than a general assurance. It should show the school, the primary provider, model providers and other processors.
4. How is information deleted?
Retention should have a clear duration and purpose. Ask what happens when a student leaves, a teacher deletes a conversation or the school ends the contract. Backup copies and derived data should be addressed, not only the visible account.
5. Can the school explain the tool to families and students?
Transparency is a practical test of understanding. If staff cannot explain what the tool does with information in plain language, the assessment is incomplete. Students should also know what not to enter and how to report an unexpected response.
Privacy is only one part of responsible use
A narrow review can confirm where data is stored and still miss how an AI system affects students. A 2024 systematic mixed-studies review of 40 empirical studies on responsible AI in education identified privacy and security alongside fairness, transparency, agency, autonomy and the obligation to avoid harm. These concerns are connected. A system may protect a student’s identity while producing an explanation that is difficult to challenge, or use limited data while still encouraging teachers to defer too much judgement to an automated recommendation.
Schools can make the five questions more useful by pairing each data answer with an educational consequence. If a provider retains prompts, what classroom practice will prevent identifiable information being entered? If a model can change without notice, who will review its output after an update? If staff cannot see why a recommendation was made, will the tool be excluded from consequential decisions?
This is also why approval should be matched to a defined use, not a product name. A service approved for brainstorming lesson ideas has not automatically been approved to analyse student work, create learning plans or make wellbeing recommendations. The data, risk and required oversight can change with the task even when the interface remains the same.
Research: systematic review of responsible human-centred AI practices in education.
Approval is not permanent
AI services change quickly. A scheduled review, change-notification clause and named owner help ensure that approval follows the product rather than its old description. High-risk uses need closer scrutiny and relevant privacy, legal, security and educational expertise.
Begin tomorrow
Choose one AI-enabled service already in use and answer the five questions on a single page. Mark every answer as confirmed, assumed or unknown. The unknowns become the next vendor questions; assumptions become items to verify. Do not mistake a privacy policy link for completed due diligence.
The goal is not to eliminate all data use. It is to ensure that each use is necessary, understood and proportionate to the learning benefit.
Reference: Australian Framework for Generative AI in Schools.
Latest posts by John Bigelow (see all)
- Using the 2026 NAP ICT Literacy Option as a School Improvement Tool - October 9, 2026
- Will a New Digital Tool Reduce Teacher Workload—or Move It? - October 8, 2026
- Before Approving an AI Tool: Five Student-Data Questions for Schools - October 7, 2026
You must be logged in to post a comment.
There are no comments
Add yours