Will AI significantly improve education or is it exaggerated?21 Sourced Claims, Contested Facts & Evidence

Topic summary

The strongest case that Will AI significantly improve education or is it exaggerated? will transform society.

Key research questions

• Investigate "Will AI significantly improve education or is it exaggerated?" from this distinct point of divergence: Why it will transform society.

Important claims and supporting citations

• A World Bank RCT in Nigeria found that students using an AI tutor (Microsoft Copilot/GPT-4) outperformed a control group by 0.31 standard deviations on a combined assessment.

  • https://the-decoder.com/ai-tutoring-helps-nigerian-students-gain-two-years-of-learning-in-six-weeks/
  • https://pershmail.substack.com/p/ai-is-maybe-sometimes-better-than
  • https://impactaieducation.substack.com/p/a-harvard-randomized-controlled-trial

• Khan Academy's AI tutoring system required a dedicated calculator tool for math and pre-processing pipelines for visual content interpretation to function reliably.

  • https://arxiv.org/pdf/2506.18710v1
  • https://arxiv.org/pdf/2506.18710
  • https://arxiv.org/html/2506.18710v2

• In the Harvard physics RCT, students using the AI tutor achieved a higher median post-score (4.5) compared to students in the active learning classroom (3.5).

  • https://the-decoder.com/ai-tutoring-helps-nigerian-students-gain-two-years-of-learning-in-six-weeks/

• On the Cross-Domain Pedagogical Knowledge benchmark, model performance at $0.10 per million tokens increased from 50% in April 2024 to 82% in June 2025.

  • https://arxiv.org/pdf/2506.18710v1
  • https://arxiv.org/pdf/2506.18710
  • https://arxiv.org/html/2506.18710v2

• A full year of Khanmigo costs $44 and allows for up to ten children per subscription, making it significantly cheaper than one hour of human tutoring.

  • https://www.gettingsmart.com/2025/11/25/can-tutoring-and-technology-finally-solve-blooms-two-sigma-problem/
  • https://pershmail.substack.com/p/ai-is-maybe-sometimes-better-than
  • https://impactaieducation.substack.com/p/a-harvard-randomized-controlled-trial

• In a partnership between Google DeepMind and Eedi, over 75% of AI-generated messages for tutoring sessions did not require major revisions by human tutors.

  • https://arxiv.org/pdf/2506.18710v1
  • https://arxiv.org/pdf/2506.18710
  • https://arxiv.org/html/2506.18710v2

• A 2025 survey of 2,232 US public school teachers found that 60% had used AI tools for work, with weekly users estimating they save 5.9 hours per week.

  • https://news.gallup.com/poll/691967/three-teachers-weekly-saving-six-weeks-year.aspx

• Testing of 97 LLMs on The Pedagogy Benchmark shows accuracy ranges from 28% (Llama-3.2 1B) to 89% (Gemini 2.5 Pro).

  • https://arxiv.org/pdf/2506.18710v1
  • https://arxiv.org/pdf/2506.18710
  • https://arxiv.org/html/2506.18710v2

Competing explanations

• Recent studies on AI-powered tutoring show mixed results, with some researchers noting difficulties in isolating the specific impact of AI chatbots from broader program effects.