Schools’ legal duties stem from in loco parentis — the obligation to supervise students and protect them from foreseeable harm. Schools comply by setting policy, selecting curriculum and overseeing instruction. This liability framework relies on traceability to evaluate fault. Generative artificial intelligence disrupts this structure by removing control, including traceability and predictability over output, creating significant litigation implications for schools and insurers.
Not all AI systems carry the same risk. Two types of AI predominate in schools: rule-based systems, which may be insurable, and genAI systems, which may not be. Although both are subsets of narrow AI, trained to perform specific tasks, genAI presents materially different risk and oversight challenges.
The key difference is the degree to which schools can exercise control and operational oversight, including vetting, monitoring, supervising, and auditing the technology to ensure safety and compliance with laws and policies. Certain technical properties — explainability, predictability, traceability and repeatability — determine how readily an AI system integrates into a school’s governance framework and aligns with educational objectives and legal obligations.
Rule-based AI systems operate within defined parameters: for example, a math program that guides students through problem sets with predictable outputs. These systems exhibit high explainability, predictability, traceability, and repeatability. As a result, their outputs can be constrained by school controls, vetted before deployment, monitored during use and audited afterward. This ensures that student-facing content will predictably comply with district curriculum and policy. Rule-based outputs can be traced to identifiable individuals and institutions, enabling accountability through traditional legal analysis and liability apportionment among potentially liable parties. Thus, these systems fit into existing frameworks governing schools’ liability.
GenAI poses fundamentally different challenges because it lacks the explainability, predictability and traceability of rule-based systems. Systems like ChatGPT and Gemini generate novel content in real time across an unbounded output space, and their black-box and non-deterministic nature prevents schools from exercising meaningful institutional authority over outputs during the content-formation phase at school scale. Neither AI use policies nor vendor contracts can remedy that. These frameworks presuppose adult users who manage their own engagement, evaluating outputs against an existing knowledge base and retaining meaningful ability to disengage within a structure that provides recourse, conditions that do not apply to minors in institutionally authorized access environments.
Some genAI vendors provide channels for schools to report harmful content. Whether those channels trigger vendor obligations to investigate, remediate or prevent recurrence depends on contractual terms, and many agreements impose no such obligations. After-the-fact reporting alone is unlikely to discharge a school’s duty of care when harmful exposure has already occurred and reporting does not remedy or prevent it from happening again.
When schools provide students with access to genAI, the system explains concepts, guides reasoning, provides feedback, shapes learning, and, over time, shapes students’ foundational cognitive capabilities, including critical thinking and understanding. These are instructional functions, meaning genAI is an instructional actor in practice. Traditional instructional actors, that is, teachers, are licensed, supervised, evaluated and subject to established legal duties and professional accountability standards. GenAI systems are not subject to any of those requirements yet perform the same function. Labeling genAI systems a “tool” in a school policy does not change its functional role or the applicable legal framework. Calling it by a different name will not mitigate legal risks applicable to schools for this functional instructional actor’s output.
The above factors raise significant liability and coverage concerns for schools that offer students access to genAI. Some insurers have concluded that genAI systems fail to meet insurability criteria because schools provide access to systems whose outputs lack predictability and traceability, including an inability to prevent exposure to harmful content. Schools nevertheless continue to grant students access to the technology on school-issued devices.
GenAI outputs that are difficult to predict, trace and constrain increase the risk of students encountering inappropriate content, including pornographic or discriminatory material. Parents reasonably expect schools to prevent such exposure, yet existing technical safeguards in these systems do not eliminate this risk.
Lawsuits allege genAI content has caused psychological harm and, in serious cases, encouraged self-harm or violence. Harmful output typically reaches students before adults can intervene. This is problematic for schools granting students genAI access.
Who is accountable when system output causes harm? The student is not culpable when exposure does not arise from misuse — that is, the student acted within the district’s acceptable use policy, which governs students’ conduct, not system output. Vendor agreements typically include liability limitations and indemnification provisions intended to shift risk to school districts. Those agreements may not bar design defect claims by students against vendors, but that avenue of recovery remains untested and does not limit a school’s potential liability. This paradigm may leave schools, including administrators, teachers and board members, bearing full liability under their inherent duties to supervise students and keep them safe from foreseeable harm.
Plaintiffs will argue that the school’s duty of care extends to content generated by the systems it authorized. Since these systems lack traceability and predictability, defenses cannot rely on arguments that proper policy was followed: district policy cannot constrain AI output, and schools know this. Therefore, liability may hinge on whether granting genAI access constituted a breach of duty, and current climate favors such a finding, whether decided by a judge or jury.
Legal claims of student harm from the proper use of school-authorized genAI can be anticipated. Schools and insurers should evaluate now whether granting genAI access, given its lack of predictability and traceability, can withstand breach-of-duty challenges and coverage disputes. Schools should consider whether policy can adequately address these systems’ unknowns, whether they can leverage vendors to retain liability contractually and whether they can confirm with insurers which, if any, aspects of genAI are covered.
Pending clear risk analysis, including of state laws, district policies, vendor contracts, and clear guidance on insurability, schools may consider pausing student access to genAI.
Emily Fernandez is a New York-based partner and trial attorney at Wilson Elser. She is also a member of the Distraction-Free Schools Policy Project. She can be reached at [email protected]. Jill Coleman is the founder of the Digital Childhood Council and a child safety governance specialist focused on institutional risk, accountability, and structural gaps in digital environments. She can be reached at [email protected]. Jana Slavina Farmer is a White Plains, New York-based partner on Wilson Elser’s IP & technology team and one of the chairs of the firm’s data use and AI governance practice. She can be reached at [email protected].
Business Insurance is a singular, authoritative news and information source for executives focused upon risk management, risk transfer and risk financing.
Never miss important news: Become a Business Insurance Online subscriber today
Copyright 2026. BUSINESS INSURANCE HOLDINGS
Leave a Reply