www.forbes.com
Prompt engineering gains a new handy technique known as atom-of-thoughts (AoT), here's the ... [+] nitty-gritty.gettyIn todays column, I showcase a vital new prompting technique known as atom-of-thoughts (AoT) that adds to the ongoing and ever-expanding list of prompt engineering best practices.Readers might recall that I previously posted an in-depth depiction of over fifty prompt engineering techniques and methods (see the link here). Top-notch prompt engineers realize that learning a wide array of researched and proven prompting techniques is the smartest way to get the most out of generative AI and likely garner you some good bucks for your skilled deeds.The newly devised atom-of-thoughts is reminiscent of the famed chain-of-thought (CoT) method, especially regarding CoT decomposition variations (see my discussion at the link here).AoT is brand new and provides an intriguing and useful twist that is worthwhile leveraging.Lets talk about it.This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).Chain-of-Thought Is FamousThe place to begin is to quickly bring up what chain-of-thought is all about. Doing so will handily set the stage for atoms-of-thought.Here we go.Avid users who are especially experienced in using generative AI and large language models (LLMs) know that one of the handiest prompting techniques consists of asking the AI to do a chain-of-thought (CoT) processing approach (I cover the basics at the link here).All you need to do is tell the AI to proceed on a stepwise basis, and the AI will then showcase various logical steps that it performed to reach an answer. In any given prompt, you can say something such as that the AI should proceed by using a step-by-step approach or just about any similar suggestion. The AI will get your drift and shift into CoT mode.Studies show that the use of CoT tends to stir generative AI toward better answers. This is partially because of the AI slowing down to carefully specify each step of a solving process. Most of the AI makers have tilted their AI toward being fast, more so than being necessarily accurate or correct. By giving a prompt that explicitly tells the AI to do CoT, you are giving the AI permission to methodically attempt to answer your query or cautiously solve the problem you have presented.Another advantage of using CoT is that you can inspect the reasoning steps that the AI displays to you.This is useful for two primary reasons. First, the steps shown are essentially an explanation of how the AI arrived at an answer. Sometimes, having the AI explain the basis for arriving at a generated response can be hugely useful. Second, upon personal inspection of the steps, you might discern that something went amiss in one or more steps. This gives you a helpful dose of skepticism about whether the answer the AI generated is right.The main downside of frequently leaning into CoT is that it takes a bit longer for the AI to process your queries and then return a result. This is referred to as latency. CoT usually extends or lengthens the latency. A somewhat related downside is that the processing time is likely to cost more because you are using up more processing cycles to arrive at a response.Generally, if the cost you are paying is low and you are willing to wait a few extra seconds, chain-of-thought is a bucko-prompting technique. The AI makers eventually realized that utilizing CoT is such a good idea that they have opted to make it an always-on feature. The latest generative AI reasoning models enforce the use of CoT.As a precaution, it is best to not ask for CoT in a prompt if the generative AI that you are using is already set up to automatically invoke CoT; troubles can arise (see my explanation at the link here).Atom-Of-Thoughts ArisesThere are many offshoot prompting techniques that make use of CoT as a launching pad. For example, there is logic-of-thoughts (see LoT described at the link here), tree-of-thoughts (see ToT at the link here), skeleton-of-thoughts (see SoT at the link here), and others.Atom-of-thoughts is also an offshoot of chain-of-thought.The big difference is that atom-of-thoughts instructs the AI to do a divide-and-conquer when working on the steps associated with generating an answer. The aim is to divide the steps into a semblance of atoms, namely individual steps that dont rely on any other steps. As much as possible, the most elemental steps are to be surfaced, and then they can each be separately resolved. After doing so, they are brought back together to reach a final answer.Why do this?One crucial aspect is that the conventional chain-of-thought tends to be a bloated affair. Heres what I mean. The AI usually keeps all the steps in a large baggage area and consumes internal memory somewhat needlessly. There is a default assumption that all the steps need to be collectively in a semi-active state. But that might not be truly needed.If the steps can be placed into individual parcels, then the history associated with all the steps isnt particularly actively required. The AI can just tackle each step on its own merits. All the other floating around stuff is not pertinent to that specific step. Keep away all the non-pertinent junk and focus on a single step.AoT Considerations And CaveatsIm sure you are already saying to yourself that this approach is nice, but if the steps intricately interrelate with each other and are wholly dependent on each other, the AI cant sensibly parcel them into individualized, concentrated steps.You nailed it.A noteworthy facet is that there are some classes of problems that the AoT will be useful on and others where it wont. AoT is not a silver bullet. It is a new tool in the toolbox. You dont use a hammer when a screwdriver is warranted. AoT has its place and time. The mainstay is when the underlying steps can be worked on rather independently.You might not have any ready means of knowing whether a problem that you give to the AI is suitable for AoT. Generally, if the problem is seemingly large and complex, it is a reasonable bet that an AoT might be beneficial. If the problem is tiny and requires few steps, Im not sure that using AoT is going to get much bang for the buck.Of course, if you are aware at the get-go that the steps are likely to be separable, boom, go ahead and invoke AoT. Even if you arent sure, thats somewhat okay since the AoT will be willing to essentially perform a customary CoT; its just that you havent gained anything via the AoT invocation.Categories that are very likely for AoT would be when using generative AI for deriving mathematical proofs, producing programming code, and for highly structured reasoning tasks. Less likely categories would be creative writing tasks, overall conversational engagement, etc.Those are general rules of thumb.Depending on how AoT is implemented in a particular generative AI and LLM, those rules might vary, so make sure to look at the documentation for whichever AI you are using to decide whats best when it comes to AoT usage.One other handy benefit is that the AI can potentially do the processing in parallel and radically speed up the time needed to get you an answer. There is some irony there. On the one hand, invoking any CoT usually takes more processing cycles and costs more. But if the AI is established to simultaneously use multiple processors at once, the individualized steps can be done extremely quickly since they are executed at the same moment in time. Parallelism is a lot harder when steps are interrelated, such that the AI must execute a step, wait for a result, and then do the next step. AoT plays into the possibility of exploiting parallelism in the hardware.Invoking Atom-Of-ThoughtsSome generative AI apps have a built-in feature to perform AoT, while others allow an add-on that does AoT. Many generative AIs dont natively yet have a direct means of performing AoT, though you can use a prompt that might get you close to it.You are welcome to use this prompt that Ive devised:Atom-of-thoughts fundamental prompt: Break down the problems that I enter into their most atomic, self-contained steps feasible. Each step should be as independent as possible, requiring minimal reliance on previous steps. If any of the steps can be executed in parallel, structure them accordingly. Perform each step separately, then reintegrate all results to produce a coherent final response.A few points about this templated sample are worth pointing out.First, you dont have to use the representative prompt word-for-word as shown. There isnt any magical wording going on. You can vary the wording as befits your style and preference. Mainly, keep things short and direct. Too long of an instruction can sometimes befuddle the generative AI.Second, the odds are that the AI is going to say yes, it will proceed as youve asked. Thats good news and possibly bad news.Unfortunately, you dont know for sure what it will actually do. If the AI isnt especially capable of AoT, it will mimic the AoT, and youll possibly get a kind of AoT or maybe a half-baked version. It wont be wrong per se, it will just be a fakery or simulated attempt at AoT. Sometimes, something is better than nothing. You be the judge.Some Example Problems Tackled By AoTSuppose that I used the above AoT establishing prompt and then gave the AI this problem to be solved.My entered prompt: Using the Atom-of-Thoughts (AoT) approach that I described to you, go ahead and solve the quadratic equation 3x - 5x + 2 = 0.Generative AI response: I will go ahead and break the solution process into the most atomic, self-contained steps. Each step will be performed independently if feasible. I will then reintegrate the steps to produce a final answer for you.You might be rusty about solving quadratic equations. No worries. There are about a half dozen steps to solve this easy equation. Of those steps, a few can be performed independently of the others. I knew that was the case and thus gave the AI an easy AoT candidate to work with.The two independently computed roots are x = 1 and x = 2/3. Easy-peasy.Lets try another example that might be a more transparent candidate for AoT.My entered prompt: Using again the Atom-of-Thoughts (AoT) approach that I described to you, determine whether the statement 'All cats are mammals, and some mammals are not cats; therefore, some cats are not mammals is logically valid. Show me the steps you end up using to solve this problem.Generative AI response: Step 1. Independently interpret the first premise that all cats are mammals. Step 2. Independently interpret the second premise that some mammals are not cats. Step 3. Independently interpret the claimed conclusion that some cats are not mammals. Step 4. Bring together the prior step interpretations. The statement "Therefore, some cats are not mammals" is a claim that contradicts Step 2. If all cats are mammals, then it is logically impossible for some cats to be not mammals. Step 5. The structure of the conclusion attempts to reverse the logical relationship established in Step 2. This is a logical fallacyspecifically, a misinterpretation of categorical logic. It is therefore not logically valid.The AI parsed the statement that I gave and opted to independently assess each of the separable premises. Once this was completed, the AI brought together those assessments to try and figure out whether the statement was logically valid. Those first steps could be executed in parallel. The final step was dependent upon the results of those prior steps.Admittedly, those were quite simple queries. Using AoT probably didnt add much value. If the word problem was a lot longer and more convoluted, AoT would likely be a plus. The same would have happened if I had given the AI a complex mathematical equation. I was just aiming to showcase some illustrative examples.Additional Benefits Of AoTI think it is useful to emphasize some bonuses of using AoT.I already noted that you can potentially have generative AI exploit parallel execution. Multi-threaded AI systems and distributed processing architectures for AI are pretty much where we are headed. Ergo, this works out well (see my coverage at the link here).Moving on, you might be familiar with the gloomy and doomy nature of AI hallucinations, consisting of circumstances where the AI generates falsehoods that arent factually grounded. AI hallucinations are a big-time problem for generative AI and LLMs. Using AI on serious and life-critical aspects is undermined by the chance that at any moment an AI hallucination might be encountered, and the user wont readily know that it has occurred. For more about the many efforts to try and beat back AI hallucinations, see my discussion at the link here.Some speculate that the chances of AI hallucinations arising during CoT and AoT are generally lessened. We dont know for sure that this is the case. It might be wishful thinking. There is some logic as to why the stepwise method might help reduce the odds of AI hallucinations.The logic is this. The AI will hopefully perform each step well and independently verify the step when it does the execution. Sometimes, conventional CoT is so jumbled that if a step goes awry, the result cascades into the rest of the problem-solving process. One bad apple spoils the barrel. The thinking is that in AoT, there is a heightened chance that by closely performing each step independently, the AI is reducing those cascading odds.Sounds upbeat. Well need to see how empirical research studies bear out on this. Im keeping my fingers crossed.Research On AoTSpeaking of research, a handy new paper that introduces AoT was recently posted online and is entitled Atom of Thoughts for Markov LLM Test-Time Scaling by Fengwei Teng, Zhaoyang Yu, Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo, arXiv, February 17, 2025, making these key points (excerpts):Large Language Models (LLMs) achieve superior performance through training-time scaling, and test-time scaling further enhances their capabilities by conducting effective reasoning during inference. However, as the scale of reasoning increases, existing test-time scaling methods suffer from accumulated historical information, which not only wastes computational resources but also interferes with effective reasoning.To address this issue, we observe that complex reasoning progress is often achieved by solving a sequence of independent sub-questions, each being self-contained and verifiable.These sub-questions are essentially atomic questions, relying primarily on their current state rather than accumulated history, similar to the memoryless transitions in a Markov process.Based on this observation, we propose Atom of Thoughts (AOT), where each state transition in the reasoning process consists of decomposing the current question into a dependency-based directed acyclic graph and contracting its sub-questions, forming a new atomic question state.This iterative decomposition-contraction process continues until reaching directly solvable atomic questions, naturally realizing Markov transitions between question states.Notice that they leverage Markov modeling with the AoT. If you are interested in how generative AI and LLMs can advantageously lean into Markov chains, see my explanation at the link here.Use AoT Gracefully And With AplombI urge you to consider adding AoT to your prompt engineering toolkit.That being said, your best bet would be to do what it takes to get to Carnegie Hall -- practice, practice, practice. You should set aside some focused time to try out the AoT prompt. Use a variety of problems. Examine what works and what doesnt. Decide how and when youll use the new prompting technique.Please dont mindlessly try to use a screwdriver if a hammer is the better choice.The final comment for now will go to Albert Einstein: Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning." Make sure to question when to use AoT, along with questioning yourself when you dont use AoT and should have done so.Yes, indeed, the important thing is not to stop questioning.
