
Prompt Engineering For Advanced Multi-Agent AI Prompting
www.forbes.com
Best practices in prompt engineering for the latest advances in generative AI and agentic AIs.gettyIn todays column, I identify and showcase a new prompting approach that serves to best make use of multi-agentic AI.The deal is this. We are increasingly going to witness the advent of agentic AI, consisting of generative AI and large language models (LLMs) that perform a series of indicated tasks. Turns out that there are going to be quite a number of these agentic AIs. The question then becomes how you can suitably compose prompts that will invoke the right set of agentic AIs to get whatever work you want done.There could very well be dozens, hundreds, or thousands of agent AIs to call upon. Your prompting must make sure to hit the nail on the head, otherwise, youll potentially invoke agentic AIs that you didnt need to engage, plus you might miss the boat and fail to invoke agentic AIs that you should have involved.Lets talk about it.This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).Prompt Engineering Meets Agentic AIReaders might recall that I previously posted an in-depth depiction of over fifty prompt engineering techniques and methods, see the link here. Top-notch prompt engineers know that seriously learning a wide array of researched and proven prompting techniques is the best way to get the most out of generative AI and likely garner you some good bucks for your skilled deeds.A new twist to the prompt engineering field is the emergence of agentic AI.Ill get started by discussing what agentic AI consists of.Imagine that you are using generative AI to plan a vacation trip. You would customarily log into your generative AI account such as making use of ChatGPT, GPT-4o, o1, o3, Claude, Gemini, Llama, etc. The planning of your trip would be easy-peasy due to the natural language fluency of generative AI. All you need to do is describe where you want to go, and then seamlessly engage in a focused dialogue about the pluses and minuses of places to stay and the transportation options available.When it comes to booking your trip, the odds are you would have to exit generative AI and start accessing the websites of the hotels, amusement parks, airlines, and other locales to buy your tickets. Few of the major generative AI available today will take that next step on your behalf. It is up to you to perform those nitty-gritty tasks.This is where agents and agentic AI come into play.In earlier days, you would undoubtedly phone a travel agent to make your bookings. Though there are still human travel agents, another avenue would be to use an AI-based travel agent that is based on generative AI.The agentic AI has the interactivity that you expect with generative AI. It also has been preloaded with a series of routines or sets of tasks that perform the efforts of a travel agent. Using everyday natural language, you interact with the agentic AI which works with you on your planning and can proceed to deal with the booking of your travel plans.Voila, this agentic AI proceeds to undertake a series of efforts to fulfill your travel booking request.Multi-Agentic AI Is Even BetterIf one agentic AI is a good thing, we might as well up the ante and aim to leverage a multitude of agentic AIs. The world is headed toward having a zillion agentic AI agents that do this or that. I assure you; this is going to make everyones head spin. There will be more AI agents than you can poke a stick at.The good news is that there will be plenty of agentic AIs to pick from. Riches aplenty. The bad news, as it were, will be that trying to determine which ones to invoke is going to be a bear.Imagine that you enter a prompt that invokes a slew of agentic AI agents. If youve inadvertently invoked some that dont need to be used, thats bad for you. The odds are that youll need to pay for their usage, regardless of whether it was of use to you. There is also a solid chance that an agent AI that wasnt relevant could end up messing with whatever you were trying to undertake.The other side of that coin is when your prompt doesnt invoke AI agents that you should have engaged. Oopsie, now the problem will be that your solving efforts might falter. A crucial AI agent that should have gotten underway was not activated. Like a weak link in a long chain, the lack of invoking even one of the needed AI agents might produce either no answer or a faulty answer.For my latest coverage on the emerging role of said-to-be orchestration for multi-agentic AI, see the link here.Making Sure To Sit In The Right SeatDuring my classes on prompt engineering, I emphasize that when it comes to composing prompts that are associated with multi-agent AI, you can think of the situation as consisting of two major approaches.The two approaches are:(1) Drivers seat. It is on your shoulders to directly identify in your prompt which AI agents you want to invoke. Make sure to name them properly. Make sure to specify the needed sequence, etc. If you are ambiguous, the chances are that the agentic AIs wont get suitably engaged.(2) Passengers seat. You tell generative AI in your prompt the overall nature of the task or tasks that you want to have undertaken. There is no need to specify which AI agents are to be used. You leave that up to the generative AI to decide.There are tradeoffs between those two approaches.The drivers seat is handy because you are explicitly stipulating the AI agents and how they are to be made use of. In a sense, this lessens the chances of some messiness if you were to allow generative AI to take the reins and ascertain which agents ought to be used. You are driving the car. Period, end of story.Being a driver of a car can be burdensome.Instead, sometimes its great to be a passenger. You simply say where you want to go and let the generative AI figure out the rest for you. This entails broadly stating what task or tasks are to be performed. The generative AI will hopefully do a bang-up job of selecting suitable agents and getting them going correctly.Sometimes you should use the drivers seat style, other times you should use the passengers seat style. It all depends on the circumstance at hand.Rules Of The Two ApproachesIn a moment, I will walk you through some illustrative examples so that you can plainly see what the prompting looks like.Before we get there, I would like to add some rules of thumb that I explain in my courses. There are five rules that are associated with each of the two approaches. Please know that many more rules are possible. Ive condensed this to the key five rules per each approach.My Drivers seat rules are as follows.Category 1: User is in the drivers seat (manual selection of AI agents)(1) Make sure to explicitly indicate the name of the AI agent and what you want it to do for you.(2) Think carefully beforehand about the sequence of AI agents that youll want to invoke.(3) Find out what inputs and outputs they each have and whether they data-connect with fellow agents.(4) Dig into the possible actions each agent performs and ascertain what overlap there might be.(5) Practice using the AI agents so that youll be ready when you truly need to use them.My Passengers seat rules are as follows.Category 2: User is in the passenger seat (supervising AI makes agent selections)(1) Specify the task you want to have undertaken and dont worry about which AI agents need to be used (let the supervising AI handle that).(2) Tell the supervising AI about any priorities or issues that ought to be on the table.(3) If you think it is valuable to know, ask which agents the supervising AI has decided to make use of.(4) The agents might provide explanations but usually only if requested, so give the supervising AI a heads-up before you let it proceed on the task at hand.(5) Expect to iterate with the supervising AI to get things aligned and properly settled.You will see these come into play in the examples I am about to showcase.Multi-Agent ScenarioLets lay out a multi-agent AI scenario that is relatively straightforward. This will allow me to then indicate various prompts associated with the situation at hand.The domain Ive chosen has to do with using generative AI and agentic AI to help with doing coding, programming, or developing software. As you look at the examples, generalize since the same precepts apply to using agentic AI in other domains. I just picked coding because it is fresh on my mind and something that I use AI with all the time.Envision that there are these five AI agents that you can invoke:(1) CodeFixer: An AI agent that debugs and optimizes programming code.(2) CodeReviewer: An AI agent that reviews code for best practices and maintainability.(3) BugHunter: An AI agent that Identifies security vulnerabilities and logic errors in code.(4) PerfAnalyzer: An AI agent that evaluates code performance and recommends optimizations.(5) DocWriter: An AI agent that generates documentation for codebases.It is important for you to know something about the nature of the AI agents and any conditions associated with them. Lets consider those various aspects.You can use any of those AI agents. There isnt a requirement that you use them, instead, they are merely available in case you would like to use them. Each one is a distinct or separate agent. The good news is that they readily share data with each other. They will automatically do so, and you dont have to give any specific instructions to them in that regard.None of the AI agents will automatically invoke another one. It is up to the mainstay invoker to specify a sequence of agents to be invoked when needed, and the mainstay invoker stipulates what the desired sequence is.There is some overlap among the AI agents. For example, CodeReviewer reviews code and could potentially identify security vulnerabilities during that review process, while BugHunter explicitly seeks to find security vulnerabilities and will doggedly try to discover them. They both touch upon security. Another example is that CodeFixer optimizes code, and so does PerfAnalyzer. Keep in mind those overlaps and decide which of the AI agents you think will best befit your needs.Drivers Seat ExampleThe easiest showcase consists of my taking the drivers seat. In my prompt, I will tell generative AI which of the agent AIs are to be invoked. In a sense, generative AI is merely a conduit and will proceed to execute the agentic AIs as I have stipulated in my prompt.Here we go.My entered prompt: I need some help with my Python script. It runs, but I think its a bit slow and might have some bugs. Please run agent CodeFixer. After doing so, run agent BugHunter. Next, run PerfAnalyzer since I want to then see if the code runs faster and seems to be hopefully bug-free. Thats it, let me know once the sequence has been completed.Generative AI response: I will invoke in this order CodeFixer, BugHunter, and then PerfAnalyzer. Once they have completed, I will show you the result. Did you want to also run DocWriter so that youll get a full set of documentation?My entered prompt: Youve got the correct sequence, thanks. No need to additionally run DocWriter at this time. I might use it after Ive seen the results of the other three AI agents.Generative AI response: Got it, proceeding.Observe that generative AI sought to clarify what I wanted to have done and gave me a helpful recap. This is useful since I might have blundered or forgotten to invoke an agentic AI that would be important to use. Likewise, generative AI might have misinterpreted my prompt and improperly laid out what needed to be done.Generative AI tried to double-check my request and suggested that an additional agentic AI be run. I told the generative AI that running the additional AI agent wasnt necessary in this instance.Nice to see that AI had my back.Happy face.Passengers Seat ExampleIn this next example, I want to do the same thing I did above, but this time I am going to let generative AI determine which agentic AIs to invoke. I am in the proverbial passengers seat. Leave the driving to AI. Thats sometimes a huge convenience.Here we go.My entered prompt: I need some help with my Python script. It runs, but I think its a bit slow and might have some bugs. Could you invoke AI agents that will help me out?Generative AI response: Got it. I interpret your prompt to mean that I am to run these AI agents in this sequence: CodeFixer, BugHunter, PerfAnalyzer, and then DocWriter. I believe that will sufficiently find bugs, figure out where the code can be sped up, and do a performance analysis after making coding changes, plus youll get a full set of documentation. Shall I proceed?My entered prompt: Thanks for telling me what you are planning to do. I dont need the full set of documentation currently so nix that part of the effort.Generative AI response: Okay, I wont run DocWriter which would have produced the documentation. The others I will go ahead and get going. Once they complete, Ill let you know.You can see that the generative AI told me which of the various available AI agents it was anticipating running. Whether you care to know or not, thats up to you. You can explicitly tell the generative AI to simply get going and not explain what it is going to do.Generally, my preference is to have generative AI tell me what it is planning to do. In this instance, the generative AI was going to run the DocWriter agent. I dont need that right now. Running it would have needlessly increased my billing charges -- I only run DocWriter once Im sure my code is done and ready to be shipped.Avoid Passengers Seat Of Inadequate DirectionIf you are opting to use the passengers seat approach, make sure to be overt about wanting to use agentic AI. I mention this because your prompt might otherwise be overly vague, and the generative AI wont catch your drift.Look at this example.My entered prompt: I need some help with my Python script.Generative AI response: I am happy to help you with your Python script. There are several AI agents that I have available to undertake debugging and optimizing code, reviewing code, finding security issues, doing performance analyses, and producing documentation. Ill need to know more about what kind of help you are interested in. What do you have in mind?My entered prompt: Im not exactly sure what help I need. All I know is that the code runs slowly and seems to have some bugs in it.Generative AI response: Got it, I know what to do. I will make use of a series of agents and get back to you with their results. Once you see the results, we can discuss whether there are other agents that I might further employ to help you with your Python code. How does that seem?My entered prompt: Thats a great plan. Please get underway.In this case, the generative AI luckily opted to ask me for some well-needed clarification.The problem with my vague help me with my Python script is that the generative AI could have gone in a wide array of directions. Maybe the generative AI would try to solve the issue directly and not invoke any AI agents. Perhaps the generative AI would simply sympathize with my plight and say something trite like dont get down about needing help writing your code.Remember, using generative AI is like a box of chocolates you never know for sure what you might get out of a prompt. Try to always be as specific as you can.Latest Research On Multi-Agent AIThe AI research community is pushing mightily ahead on the intricacies of multi-agent AI. You can expect new insights to arise nearly daily. I strive to bring the especially notable ones to your attention.Speaking of which, a recent research paper entitled AgentRec: Agent Recommendation Using Sentence Embeddings Aligned to Human Feedback by Joshua Park and Yongfeng Zhang, arXiv, January 23, 2025, made these salient points (excerpts):Multi-agent systems must decide which agent is the most appropriate for a given task.However, most research that has been done on this subject often executes these agents in a rigid order and can only allow for a certain class of questions as there is no robust way of flexibly determining which agents out of a large selection of unrelated agents are most suited to perform the given task.To solve this issue, we contribute AgentReca scalable, fast, and efficient method of selecting an agent to perform a task given a natural language prompt.The following agents are: (1) tech support agent, (2) cooking agent, (3) math agent, (4) gaming agent, (5) therapy agent, (6) reading agent, (7) health agent, and (8) fitness agent. Some agents were selected to be as unrelated as possible from other agentssuch as the cooking agentwhereas some agents were selected to have a soft overlap of their expertise domain such as the health and fitness agents.The dataset represented 8 agents of varying general topics with 1,250 prompts each, making the total number of prompts 10,000.By using sentence embeddings, we are able to extract the semantic content of a natural language prompt relevant to selecting an agent and compare it to a labeled corpus of synthetically generated prompts that specific agents are expected to be able to answer in under 300 milliseconds for 8 agents with a top-1 test accuracy of 92.2%.I thought you might find this particular study of keen interest due to the use of a specialized method for selecting AI agents.Another factor I liked was that they did an interesting experiment consisting of eight AI agents and invoking them via thousands of prompts. Doing this in the large helps to identify what works and doesnt work in the long run.Generative AI Becoming VersedAn imperative aspect of the above research study is that there is a solid chance of purposely getting generative AI to be especially good at selecting which AI agents ought to be run for a given request by a user. We can lean into the pattern-matching facility of generative AI to do so.It goes this way.Feed lots of examples of prompts and subsequent agentic AI choices into generative AI for a data training exercise (essentially, pairs that align). Generative AI will hone in on which prompts are best fitting for which agentic AI selections. After enough data training, the idea is that the prompt you enter as a passengers seat style approach will be suitably matched to which agentic AIs should be invoked. Thus, the generative AI will do better than mere chance since it has been specially data-trained accordingly.I expect well see a lot more of this as a means of inching generative AI toward the best or some optimal selection of AI agents.Now that you know about invoking multi-agent AIs and the kinds of prompts to be used, I would urge you to consider practicing doing so. Practice makes perfect. Practice, practice, practice.Admittedly, practicing is a bit challenging right now because few of the major generative AI apps are directly allowing users to invoke AI agents. Some of the generative AI apps will invoke them on your behalf, but not even necessarily tell you when this is being done, nor which ones are being invoked. It is all a mixed bag. Rapid changes are happening.Best of luck and you might keep the famous words of Abraham Lincoln in mind: The best thing about the future is that it comes only one day at a time.That equally applies to the advent of multi-agentic AI.
0 Commenti
·0 condivisioni
·74 Views