The rapid proliferation and superb capabilities of widely available LLMs has ignited intense debate within the educational sector. On one side they offer students a 24/7 tutor who is always available to help; but then of course students can use LLMs to cheat! I’ve seen both sides of the coin with my students; yes, even the bad side and even at the university level. While the potential benefits and problems of LLMs in education are widely discussed, a critical need existed for robust, empirical evidence to guide the integration of these technologies in the classroom, curricula, and studies in general. Moving beyond anecdotal accounts and rather limited studies, a recent work titled “The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis” offers one of the most comprehensive quantitative assessments to date. The article, by Jin Wang and Wenxiang Fan from the Chinese Education Modernization Research Institute of Hangzhou Normal University, was published this month in the journal Humanities and Social Sciences Communications from the Nature Publishing group. It is as complex as detailed, so here I will delve into the findings reported in it, touching also on the methodology and delving into the implications for those developing and deploying AI in educational contexts. Into it: Quantifying ChatGPT’s Impact on Student Learning The study by Wang and Fan is a meta-analysis that synthesizes data from 51 research papers published between November 2022 and February 2025, examining the impact of ChatGPT on three crucial student outcomes: learning performance, learning perception, and higher-order thinking. For AI practitioners and data scientists, this meta-analysis provides a valuable, evidence-based lens through which to evaluate current LLM capabilities and inform the future development of Education technologies. The primary research question sought to determine the overall effectiveness of ChatGPT across the three key educational outcomes. The meta-analysis yielded statistically significant and noteworthy results: Regarding learning performance, data from 44 studies indicated a large positive impact attributable to ChatGPT usage. In fact it turned out that, on average, students integrating ChatGPT into their learning processes demonstrated significantly improved academic outcomes compared to control groups. For learning perception, encompassing students’ attitudes, motivation, and engagement, analysis of 19 studies revealed a moderately but significant positive impact. This implies that ChatGPT can contribute to a more favorable learning experience from the student’s perspective, despite the a priori limitations and problems associated to a tool that students can use to cheat. Similarly, the impact on higher-order thinking skills—such as critical analysis, problem-solving, and creativity—was also found to be moderately positive, based on 9 studies. It is good news then that ChatGPT can support the development of these crucial cognitive abilities, although its influence is clearly not as pronounced as on direct learning performance. How Different Factors Affect Learning With ChatGPT Beyond overall efficacy, Wang and Fan investigated how various study characteristics affected ChatGPT’s impact on learning. Let me summarize for you the core results. First, there was a strong effect of the type of course. The largest effect was observed in courses that involved the development of skills and competencies, followed closely by STEM (science/Technology) and related subjects, and then by language learning/academic writing. The course’s learning model also played a critical role in modulating how much ChatGPT assisted students. Problem-based learning saw a particularly strong potentiation by ChatGPT, yielding a very large effect size. Personalized learning contexts also showed a large effect, while project-based learning demonstrated a smaller, though still positive, effect. The duration of ChatGPT use was also an important modulator of ChatGPT’s effect on learning performance. Short durations in the order of a single week produced small effects, while extended use over 4–8 weeks had the strongest impact, which did not grow much more if the usage was extended even further. This suggests that sustained interaction and familiarity may be crucial for cultivating positive affective responses to LLM-assisted learning. Interestingly, the students’ grade levels, the specific role played by ChatGPT in the activity, and the area of application did not affect learning performance significantly, in any of the analyzed studies. Other factors, including grade level, type of course, learning model, the specific role adopted by ChatGPT, and the area of application, did not significantly moderate the impact on learning perception. The study further showed that when ChatGPT functioned as an intelligent tutor, providing personalized guidance and feedback, its impact on fostering higher-order thinking was most pronounced. Implications for the Development of AI-Based Educational Technologies The findings from Wang & Fan’s meta-analysis carry substantial implications for the design, development, and strategic deployment of AI in educational settings: First of all, regarding the strategic scaffolding for deeper cognition. The impact on the development of thinking skills was somewhat lower than on performance, which means that LLMs are not inherently cultivators of deep critical thought, even if they do have a positive global effect on learning. Therefore, AI-based educational tools should integrate explicit scaffolding mechanisms that foster the development of thinking processes, to guide students from knowledge acquisition towards higher-level analysis, synthesis, and evaluation in parallel to the AI system’s direct help. Thus, the implementation of AI tools in education must be framed properly, and as we saw above this framing will depend on the exact type and content of the course, the learning model one wishes to apply, and the available time. One particularly interesting setup would be that where the AI tool supports inquiry, hypothesis testing, and collaborative problem-solving. Note though that the findings on optimal duration imply the need for onboarding strategies and adaptive engagement techniques to maximize impact and mitigate potential over-reliance. The superior impact documented when ChatGPT functions as an intelligent tutor highlights a key direction for AI in education. Developing LLM-based systems that can provide adaptive feedback, pose diagnostic and reflective questions, and guide learners through complex cognitive tasks is paramount. This requires moving beyond simple Q&A capabilities towards more sophisticated conversational AI and pedagogical reasoning. On top, there are a few non-minor issues to work on. While LLMs excel at information delivery and task assistance (leading to high performance gains), enhancing their impact on affective domains (perception) and advanced cognitive skills requires better interaction designs. Incorporating elements that foster student agency, provide meaningful feedback, and manage cognitive load effectively are crucial considerations. Limitations and Where Future Research Should Go The authors of the study prudently acknowledge some limitations, which also illuminate avenues for future research. Although the total sample size was the largest ever, it is still small, and very small for some specific questions. More research needs to be done, and a new meta-analysis will probably be required when more data becomes available. A difficult point, and this is my personal addition, is that as the technology progresses so fast, results might become obsolete very rapidly, unfortunately. Another limitation in the studies analyzed in this paper is that they are largely biased toward college-level students, with very limited data on primary education. Wang and Fan also discuss what AI, data science, and pedagogues should consider in future research. First, they should try to disaggregate effects based on specific LLM versions, a point that is critical because they evolve so fast. Second, they should study how students and teachers typically “prompt” the LLMs, and then investigate the impact of differential prompting on the final learning outcomes. Then, somehow they need to develop and evaluate adaptive scaffolding mechanisms embedded within LLM-based educational tools. Finally, and over a long term, we need to explore the effects of LLM integration on knowledge retention and the development of self-regulated learning skills. Personally, I add at this point, I am of the opinion that studies need to dig more into how students use LLMs to cheat, not necessarily willingly but possibly also by seeking for shortcuts that lead them wrong or allow them to get out of the way but without really learning anything. And in this context, I think AI scientists are falling short in developing camouflaged systems for the detection of AI-generated texts, that they can use to rapidly and confidently tell if, for example, a homework was done with an LLM. Yes, there are some watermarking and similar systems out there (which I will cover some day!) but I haven’t seem them deployed at large in ways that educators can easily utilize. Conclusion: Towards an Evidence-Informed Integration of AI in Education The meta-analysis I’ve covered here for you provides a critical, data-driven contribution to the discourse on AI in education. It confirms the substantial potential of LLMs, particularly ChatGPT in these studies, to enhance student learning performance and positively influence learning perception and higher-order thinking. However, the study also powerfully illustrates that the effectiveness of these tools is not uniform but is significantly moderated by contextual factors and the nature of their integration into the learning process. For the AI and data science community, these findings serve as both an affirmation and a challenge. The affirmation lies in the demonstrated efficacy of LLM technology. The challenge resides in harnessing this potential through thoughtful, evidence-informed design that moves beyond generic applications towards sophisticated, adaptive, and pedagogically sound educational tools. The path forward requires a continued commitment to rigorous research and a nuanced understanding of the complex interplay between AI, pedagogy, and human learning. References Here is the paper by Wang and Fan: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis. Jin Wang & Wenxiang Fan Humanities and Social Sciences Communications volume 12, 621 (2025) If you liked this, check out my TDS profile. The post What the Most Detailed Peer-Reviewed Study on AI in the Classroom Taught Us appeared first on Towards Data Science.

During an excavation, amidst the Patagonian winds and hard rock, a fossil began to turn green. It was an unexpected reaction: the adhesive applied to protect the bones, fragile after millions of years beneath the ice, had interacted with plant matter trapped in the rock’s cracks. This greenish hue earned the fossil the nickname Fiona, like the ogre from Shrek.But Fionais much more than a ogre-themed name. It is the first complete ichthyosaur ever excavated in Chile and, even more remarkably, the only known pregnant female from the Hauterivian — a stage of the Early Cretaceous dating back 131 million years. Her skeleton, discovered at the edge of the Tyndall Glacier in Torres del Paine National Park — an area increasingly exposed by glacial retreat — belongs to the species Myobradypterygius hauthali, originally described in Argentina from fragmentary remains.The discovery, led by Judith Pardo-Pérez, a researcher at the University of Magallanes and the Cabo de Hornos International Center (CHIC), and published in the Journal of Vertebrate Paleontology, offers an unprecedented glimpse into ancient marine life — from how these majestic reptiles reproduced to how they adapted to oceans vastly different from those of today.An Ichthyosaur Maternity Ward in Patagonia(Image Courtesy of Irene Viscor)So far, 88 ichthyosaurs have been found on the Tyndall Glacier. Most of them are adults and newborns. Two key facts stand out: food was abundant, and no other predators were competing with them.Fiona, who measures nearly 13 feet long, is still encased in five blocks of rock. Despite the challenge, she was transported to a local clinic, where CT scans allowed researchers to study her skull and body. Her species was identified thanks to one of her fins. “There’s no other like it in the world,” says Pardo-Pérez. The limbs were remarkably elongated, suggesting this animal was built for long-distance swimming.Inside her, there were more surprises. One of them was her stomach contents, which revealed what may have been her last meal: tiny fish vertebrae. But the most striking find was a fetus, about 20 inches long, already in a position to be born.“We believe these animals came to Magallanes — the southern tip of Chilean Patagonia — from time to time to give birth, because it was a safe refuge,” Pardo-Pérez says. “We don't know how long they stayed, but we do know that mortality was high during the first few days of life.”One of the big unanswered questions is where they went next, as there are no records of Myobradypterygius hauthali, apart from a piece of fin found in Argentina. The most abundant remains come from southern Germany, but those date back to the Jurassic period, meaning they’re older.Palaeontologist Erin Maxwell suggests, “In many modern ecosystems, species migrate to higher latitudes during the summer to take advantage of seasonally abundant resources and then move to lower latitudes in winter to avoid harsh conditions,” she explains. “We believe Mesozoic marine reptiles may have followed similar seasonal patterns.”Sea Dragon GraveyardThe environment where Fiona was discovered — dubbed the "sea dragon graveyard" — also has much to reveal.According to geologist Matthew Malkowski of the University of Texas at Austin, the Hauterivian age is particularly intriguing because it coincided with major planetary changes: the breakup of continents, intense volcanic episodes, and phenomena known as "oceanic anoxic events," during which vast areas of the ocean were depleted of dissolved oxygen for hundreds of thousands of years.One such poorly understood event, the Pharaonic Anoxic Event, occurred around 131 million years ago, near the end of the Hauterivian, and still raises questions about its true impact on marine life. “We don't have a firm grasp of how significant these events were for marine vertebrates, and geological records like that of the Tyndall Glacier allow us to explore the relationship between life, the environment, and Earth’s past conditions,” Malkowski notes.Evolution of IchthyosaursReconstruction of Fiona. (Image Courtesy of Mauricio Álvarez)Don't be misled by their body shape. “Ichthyosaurs are not related to dolphins,” clarifies Pardo-Pérez. Although their hydrodynamic silhouettes may look nearly identical, the former were marine reptiles, while the latter are mammals. This resemblance results from a phenomenon known as convergent evolution: when species from different lineages develop similar anatomical features to adapt to the same environment.Ichthyosaurs evolved from terrestrial reptiles that, in response to ecological and climatic changes, began spending more time in the water until they fully adapted to a marine lifestyle. However, they retained traces of their land-dwelling ancestry, such as a pair of hind flippers — absent in dolphins — passed down from their walking forebears. They lived and thrived in prehistoric oceans for about 180 million years, giving them ample time to refine a highly specialized body: their forelimbs and hindlimbs transformed into flippers; they developed a crescent-shaped tail for propulsion, a dorsal fin for stability, and a streamlined body to reduce drag in the water. Remarkably, like whales and dolphins, “ichthyosaurs had a thick layer of blubber as insulation to maintain a higher body temperature than the surrounding seawater and gave birth to live young, which meant they didn’t need to leave the water to reproduce,” explains Maxwell.Whales and dolphins also descend from land-dwelling ancestors, but their transition happened over a comparatively short evolutionary timespan, especially when measured against the long reign of the ichthyosaurs. “Their evolution hasn't had as much time as that of ichthyosaurs,” notes Pardo-Pérez. “And yet, they look so similar. That’s the wonderful thing about evolution.”Read More: Did a Swimming Reptile Predate the Dinosaurs?Fossils on the Verge of DisappearanceOne of the key factors behind the remarkable preservation of the fossils found in the Tyndall Glacier is the way they were buried. According to Malkowski, Fiona and her contemporaries were either trapped or swiftly covered by underwater landslides and turbidity currents — geological processes that led to their sudden entombment.But the good fortune that protected them for millions of years may now be running out. As the glacier retreats, exposing fossils that were once unreachable, those same remains are now vulnerable to wind, rain, and freeze-thaw cycles, which crack the surrounding rock. As vegetation takes hold, roots accelerate erosion and eventually conceal the fossils once again.“While climate change has allowed these fossils to be studied, continued warming will also eventually lead to their loss,” Maxwell warns. In Fiona’s story, scientists find not only a record of ancient life, but also a warning etched in stone and bone: what time reveals, climate can reclaim.Article SourcesOur writers at Discovermagazine.com use peer-reviewed studies and high-quality sources for our articles, and our editors review for scientific accuracy and editorial standards. Review the sources used below for this article:María de los Ángeles Orfila is a science journalist based in Montevideo, Uruguay, focusing on long-form storytelling. Her work has appeared in Discover Magazine, Science, National Geographic, among other outlets, and in leading Uruguayan publications such as El País and El Observador. She was a fellow in the 2023 Sharon Dunwoody Mentoring Program by The Open Notebook and often explores the intersections of science, culture, and Latin American identity.