Identifying the truth about whether generative AI and LLMs are suffering from cognitive decline.gettyIn todays column, I examine the recent spate of claims that generative AI and large language models (LLMs) supposedly suffer from cognitive decline. This certainly sounds juicy. One related disconcerting factor is that comparing AI to the human condition associated with cognitive decline is an inapt analog and sadly another sign of excessive anthropomorphization of AI. In any case, the crux of the contention is that AI is presumably in some kind of state of decline regarding mustering computational and mathematical forms of exhibited intelligence.Lets talk about it.This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).Human Cognitive DeclineBefore we unpack the AI side of this matter, we ought to first establish the human facets of cognitive decline since thats the barometer being used in these emerging claims.According to the American Psychological Association (APA), the definition of cognitive decline consists of this aspect (excerpted from APA online dictionary):Reduction in one or more cognitive abilities, such as memory, awareness, judgment, and mental acuity, across the adult lifespan.The presence and degree of decline varies with the cognitive ability being measured.Cognitive decline is a part of normal healthy aging, but a severe decline is not normative and could be symptomatic of disease.It is the primary symptom of disease-induced dementias, such as Alzheimers disease.Im sure that weve all encountered someone who has been undertaken by dementia. They typically will be slow to process their thoughts and might at times wander from a given conversation. Forgetfulness is relatively common. The person gropes for words and has a tough time with a persistent train of thought.As noted above, cognitive decline can range from minor to wholly significant. The aging process has a natural impact on cognition and leads to a semblance of cognitive decline. If you meet someone at the age of 40 and carry out various intelligence tests and revisit them when they are 80 years of age, the chances are that they will perform less capably on the intelligence test.It would seem somewhat intuitive to suggest that people as they age are gradually going to reach a point in their later years that entails cognitive decline. Not everyone does so to the same degree. Not everyone does so at the same age. Nonetheless, it isnt surprising that for example, a 40-year-old when they are 80 years old is likely to show signs of cognitive decline from that early base test to their latest intelligence test.Generative AI And LLMsShift gears and focus on modern-day generative AI and LLMs. I will provide some quick background about the general nature of AI so that we are all on the same table as we get further into the unpacking of the premise of AI and cognitive decline.First, the usual way of developing generative AI consists of doing a massive data training effort that involves scanning a wide swath of content on the Internet. The approach entails doing mathematical and computational pattern-matching of how humans write and make use of words. After performing this large-scale pattern matching, the AI system is tuned and then released for public usage. The AI appears to be fluent in natural language and can engage in human-like dialogue.For more in-depth details on the building of AI, see my discussion at the link here.Second, the AI makers are rapidly progressing their generative AI models. New techniques and ways to optimize the underlying pattern-matching are a fast-moving realm of research and practice, see for example breakthroughs that Ive depicted at the link here and the link here. The aim is to make the AI bigger, faster, and more responsive.Though an AI maker might decide to take an existing version and extend it, the likelier approach consists of starting over with the newest capabilities and performing the data training all over again. This makes abundant sense. Rather than being trapped by the prior less capable restrictions that were used to do the data training on a prior model, you are bound to have better luck by doing the data training anew.Thus, if I share with you version 1.0 of an LLM that I made, and then I tell you to consider switching to my version 2.0, the odds are that I started the 2.0 from scratch and did not simply add onto version 1.0. I might have carefully studied 1.0 to see where it has issues and lacks capabilities, but I am probably not going to use that version to strive forward with my next version.In that sense, version 1.0 and version 2.0 are not the same. They are built on different premises, though certainly have similarities since they have much of the same internal infrastructure.Please keep this in mind since it is going to be quite a sticking point in a moment.Claims About AI Cognitive DeclineBanner headlines have been touting that AI is suffering from cognitive decline. This is eye-catching. Wow, AI is just like humans and has cognitive decline, thats the overt message or at times implied assertion of these pieces.Here are some recent headlines along those lines:Shocking Flaw Found In AI: Like Humans, It Shows Signs Of Dementia With Age (StudyFinds, December 19, 2024).Digital Dementia? AI Shows Surprising Signs of Cognitive Decline (SciTechDaily, December 18, 2024).Older AI Models Show Signs Of Cognitive Decline, Study Shows (LifeScience, February 16, 2025).A close look at such postings indicates that they are primarily based on a research study that sought to examine various generative AI and LLMs to see whether there was cognitive decline involved.The study is entitled Age against the machinesusceptibility of large language models to cognitive impairment: cross sectional analysis by Roy Dayan, Benjamin Uliel, and Gal Koplewitz, BMJ, December 19, 2024, and made these salient points (excerpts):In this study, we evaluated the cognitive abilities of the leading, publicly available large language models and used the Montreal Cognitive Assessment to identify signs of cognitive impairment. Older large language model versions scored lower than their younger versions, as is often the case with human participants, showing cognitive decline seemingly comparable to neurodegenerative processes in the human brain (we take older in this context to mean a version released further in the past).Analysis Of The ClaimTime to do some unpacking.Lets think about this broadly. If I asked you to assess my version 1.0 LLM and compare its performance on some form of intelligence test to my version 2.0, what would you likely expect to see as the results?The odds are that version 2.0 would do better than version 1.0.Why?Because I have hopefully done something valuable in the added techniques and optimization that would make the new one better than the old one. Thats common sense. People expect that a newer model is going to be better than the prior one, else they might as well not switch to the newer one.Okay, under that reasonable assumption, suppose I gave an intelligence test to my version 1.0, and gave it again later on to my version 2.0. What should happen? Version 1.0 ought to score lower than the newer 2.0 because I sought to make 2.0 better than 1.0. and was likely successful in doing so.Since my 1.0 has scored lower than my 2.0, would you ergo conclude that my LLM has suffered a cognitive decline?Nope.That wouldnt make logical sense. The 2.0 is better than the 1.0. Thats good. The 1.0 is less capable on the intelligence test because it was my earlier version and not as advanced as my 2.0. Period, end of story.An assumption that a prior model, because it is said to be older, can be used to argue that a decline or degradation has occurred, in comparison to a newer model, defies common sense when you think of the progression involved in advancing AI from model to model over time. The mental trap comes into the picture when you start trying to do a one-to-one analogy to how humans operate, which Ill mention more shortly.Other ConsiderationsThere are more facets to unpack.When I give an intelligence test to a human at the age of 40, it is a one-time snapshot. They are never that precise age again since time moves on. I give that same person the same intelligence test around forty years later, at the age of 80, and see how they do. The chances are they wont do as well (all else being equal). They have suffered some amount of cognitive decline, possibly minimal or possibly a lot.It is the same person, tested over time.Now lets ponder the AI side.I give an intelligence test to my 1.0 version that I released in say September 2024 and do the test at that time. I give that same test to the same 1.0 version about six months later, around March 2025. The odds are that the test results will be approximately the same. I suggest this to be the case since though the AI maker is probably making some minor improvements to the 1.0, it roughly remains about the same. If anything, it will probably score slightly higher on the test in March 2025.I release my version 2.0 in December 2024 and give it an intelligence test. Later on, in March 2025, I give it the same intelligence test. The odds are that the score will be roughly the same, though it is possible that I made minor improvements during those months of usage, and it could score slightly higher on the subsequent test.We have tested two different AI versions. Trying to compare the test results of the 1.0 to the 2.0 version is potentially problematic. They are not necessarily the same versions. This is unlike the idea of one person that has one mind, and we test them over time.Even in the case of the same version, such as focusing solely on 1.0, the chances are that the AI has been improved over time and will perform better on the test. That doesnt indicate a decline, it instead indicates improvement.I trust that you get my gist.Revisiting The Decline ConsiderationNow that weve gotten that matter addressed, Id like to explore this topic with a fresh eye.At the outset, lets put aside the word cognitive since that is usually reserved to refer to humans. The same goes for the word think which is popularly used when referring to AI but again is overloaded with meaning since we use that word when describing human thought and cognition.I will simply revert to indicating that we can be interested in whether AI can be devised to improve over time or whether it might be devised such that it declines in capability over time. Notice that I didnt have to use the word cognitive or think in expressing that consideration.Is it possible that AI such as generative AI or LLMs might worsen or decline over time in their capabilities at responding to user prompts?Yes, this is indeed possible.Heres how.Suppose that we take an LLM and decide to give it some augmented data training. Our goal is that we hope the additional data will boost the AI. You can undertake such an effort readily using retrieval-augmented generation (RAG), as I describe at the link here.But imagine that inadvertently the data used for this augmented effort was lousy data. It is replete with falsehoods and mistakes. The proper way to do things would be to have examined and screened the data beforehand. Assume for the sake of discussion that the data was just used as is. No sensibility checks and balances were performed.What might happen to the LLM that is augmented in this manner?Well, if we administer tests and those tests perchance tap into the pattern-match realm that involves that lousy data, the odds are that the AI will do less well on the test. The AI will be answering based on flawed data.Voila, the AI has declined in performance.Fine Tuning And Forgetting You might be tempted to question whether having merely added lousy data to AI was the only way that the AI might decline in performance over time.There are plenty of other avenues.For example, suppose an AI maker opts to tweak their LLM.One aspect that has come up frequently is that people want to have the so-called right-to-forget when it comes to generative AI, see my coverage at the link here. This entails having the AI maker delete out facets that are in the AI, that someone believes ought to be forgotten about them. There is a great deal of controversy over this matter and not everyone agrees that having AI be adjusted this way is the proper ethical or legal way to go (others insist that it is).Anyway, envision that an AI maker is adjusting and refining their generative AI over time. They ought to do so with a careful hand. I mention this because they could lop out useful aspects that are crucial to their generative AI. If they arent mindful of the changes they are making, there is a solid chance that the AI will perform worse than it did before.That would be a considered example of generative AI or LLM that has declined or performed worse over time.Final Thoughts To Mull OverA twist to that aspect is that the changes to the AI dont necessarily have to be undertaken by the human hand. Some of the latest AI has been devised to be so-called self-healing, self-repairing, self-improvement, self-reflection, etc., see my coverage at the link here. Its a dual-edged sword. The AI might improve itself. Yay. But theres no guarantee that the upside is the only result. The AI might undercut itself. Sad face.Again, you would contend that the generative AI has declined or worsened over time.The most popular twist on this consists of using synthetic data for data training of generative AI and LLMs. It goes like this. Right now, the data on the Internet that is used for data training is substantially human-crafted writing. Generative AI meanwhile is being used to pump out responses, i.e. data, of which some or a lot is being posted to the Internet. This is referred to as synthetic data.Some research studies have suggested that if you data train generative AI on principally synthetic data, this can lead to what is known as catastrophic model collapse. The idea is that you are using cloned data and the AI gets weaker and less viable accordingly. Ive examined the truth of this claim, see the link here.So, yet another pathway to having AI decline over time.A final comment for now.You might be familiar with Mark Twain and his famous satirical remarks about his supposed death in 1897 (he actually passed away in 1910), namely that such reports were greatly exaggerated. Its a clever remark by the vaunted writer and humorist. Theres a lesson to be learned from that insight. Be cautious when you hear or read about AI and various eye-catching claims.Dig in and find out the real story.