How to Build a Self-Improving AI with Agentic RAG and Flowise

9.51k views3491 WordsCopy TextShare

Leon van Zyl

Agentic RAG is a powerful approach for building AI solutions that are able to self-improve their res...

Video Transcript:

Are you tired of your chatbot giving you wrong answers? Today, we'll be building a self-improving AI application using a genetic rag and Flowwise. That's right, no more frustrating AI hallucinations or irrelevant responses.

In order to build this application, we will use sequential agents within Flowwise. With sequential flows, we can add multiple agents to a single AI application. And since we want our application to correct itself, we will be using different agents for different functions.

But first, let's start with a simple demo of this application. As per usual, we will be using my Oak & Barrel restaurant as an example. Oak & Barrel is a fictitious restaurant that sells steaks and sushi.

So if a user asks the question like, "What are your current specials? " Our assistant will be able to answer that question. And if we have a look at the conversation history, the retrieval tool reached out to our knowledge base to fetch information about the current specials.

We then have this conditional agent that checked if the retrieved documents is indeed related to the user's question. And if so, it instructs the process to generate a response. And then this final generate agent provides this response to the user.

Now let's ask something that's completely unrelated to the restaurant. Like, "Do you sell crypto? " Let's send this.

And this time, if we have a look at the conversation history, the retrieval agent reached out to the tool, which did return some information from the knowledge base. And looking at this, there is no mention of crypto anywhere. Now this could lead to inaccurate answers or hallucinations.

So when we go down with the process, we can now see that the conditional agent determined that the documents fetched from the vector store are not relevant to the user's question, and is therefore calling the rewrite agent to try and rephrase the question. The rewrite agent then rephrases the question in an attempt to get a different output. Then finally, we do get a response saying that "Akenbarrel does not sell cryptocurrency.

" This might be an extreme example. Realistically, what would happen is a user would ask a question that is indeed related to the restaurant, but because of the way it's phrased, the documents returned from the vector store is not really related, in which case the rewrite agent will take the user's question, rephrase it differently, and hopefully a more suitable set of documents will be returned and injected into the context. Now let's go ahead and build this a gigantic RAG app together.

In the Flow-wise dashboard, go to Agent Flows, and if you don't see Agent Flows in the menu, then please ensure to update your Flow-wise instance to the latest version. And by the way, if you don't want to deal with installing or updating Flow-wise yourself, then check out their fully managed cloud service instead. The link is in the description of this video, and by using my link you will be supporting my channel.

Let's create our Agent Flow by clicking on Add New, Save this project, let's give it a name like "Self-improving RAG", and let's save this. Let's start by adding a new node to the canvas, and under Sequential Agents, let's add the start node. If you're new to sequential agents, then I highly recommend checking out my fundamentals video over here.

Let's also add a chat model, so let's go to Chat Models, and I'm going to use the Chat OpenAI node, but of course the choice of model is up to you. Just make sure the model is powerful enough to run agents and have tool use capabilities. Under Model Name, let's select GPT40, I'll also select my credentials, and I'll set the temperature to a low value like 0.

2. Now let's think about what this application needs to do. First, we want to call an agent that is able to retrieve information about a restaurant from a knowledge base.

After that, we will have agents that will score the response from the knowledge base, and then determine if the query should be rewritten, or if an answer can be generated. But let's first focus on the agent and our knowledge base. So let's start by adding a new node to the canvas, and let's add an agent node.

Let's call this agent something like Customer Support, then for the system prompt, let's enter something like, you are a customer support agent for a restaurant called Oak and Barrel, that always answers questions with the most relevant information using the tools at your disposal. And then we'll just specify that it's got this restaurant info tool available. Let's save this prompt, let's also connect our start node with this agent.

Let's also add this restaurant info tool, so under Add Nodes, let's go to Tools, and within Tools, let's add the Retriever tool. We can use the Retriever tool to fetch information from a vector store. So let's attach this Retriever tool to our agent, and for the tool description, let's enter Search and Return Documents Related to the Restaurant.

Great! Now our agent has a tool that I can call to fetch information about the restaurant. Let's continue working on this tool.

First, we need to attach a vector store to this tool, and within vector stores, let's add an in-memory vector store, and let's attach it to our tool. Now the vector store requires two inputs as well. Let's start with the Embeddings input.

So under Add Nodes, let's go to Embeddings, and because I'm using OpenAI, I'll simply add the OpenAI Embeddings node, and let's attach it to the vector store. And I'm also going to select my credentials, and that should be everything we need for the Embedding node. Now we can attach our knowledge base.

There are many different integrations that you can choose, but what I like to do lately is, if you go back to the dashboard, you can go to Document Stores, and within Document Stores, you can add a new document store, give it a name, and then click Add. And when you open the document store, you are now able to add multiple document loaders to the store. So as an example, I'll select a .

docx file, I'll then upload the OakenBattle knowledge base, like so, then under Text Splitter, I'll simply select the Recursive Character Text Splitter to chunk this document, and I'll set a chunk size of 250 characters, and an overlap of 20. These sizes are completely up to you. So let's go ahead and preview the chunks, and we can see a list of all the chunks and the content that they contain.

Let's click on Process, and that should be it. If we wanted to, we could add more document loaders, as many as we want, and if we ever wanted to remove information from the knowledge base, we can simply delete the entry over here. Let's go back to our Agent Flow, and let's add the document store.

Within the document store, we can also select the document store that we just created, and let's attach the document store to our in-memory vector store, like so. Let's test this flow out by adding an end node to the canvas. Let's go to Add Nodes, under Sequential Agents, let's grab the end node, and let's attach our agent to this end node, like so.

So now we have a very simple Rack chatbot that we can test out. Let's save this, let's click on Chat, and let's ask a question related to the restaurant, like, "Do you sell steaks? " And we can see that the customer support agent deadcalled the restaurant info tool and retrieved this context from the vector store, and we are getting the correct answer back.

So now that this is working, we now need to add a second agent that will be responsible for determining if the answer from the knowledge base is related to the user's question, and then conditionally call the rewrite agent or the generate agent. Now in order to conditionally call a process, we can go to Add Nodes, and under Sequential Agents, we have the choice between a condition node or a condition agent node. With a condition node, we can take a simple value, like a yes or no, and then branch the user off into different parts, or we could use the intelligence of an LLM to determine the path for us.

Let's select the condition agent node and add it to the canvas. I'm also going to break the connection between the end node and our first agent, and then let's pass the output from our first agent to this condition agent, like so. Let's give our conditioned agent a name, like, "Check if docs are relevant," and let's simply attach the end output to our end node.

We will have a look at this next output in a minute. Now let's have a look at this condition agent. Let's go to Additional Parameters, and here we can see a default system prompt and a default human prompt.

None of these are related to our application, so I'm actually going to go ahead and clear these fields. For the system prompt, let's enter, "You are a grader assessing relevance of a retrieved document to a user question. " Here is the retrieved context.

This value within curly braces is actually a placeholder for a dynamic value, and we will inject the result from the knowledge piece into this placeholder in a minute. Here is the user's question, and again we have a variable for question, which we will link to the user's question. If the document contains keywords or semantic meaning related to the user's question, grade it as relevant.

Give a binary score of "yes" or "no" to indicate whether the document is relevant to the question. Remember, always use the extract tool to output only "yes" or "no. " Your prompt doesn't have to be exactly like this, but just ensure that you inject the context and the user question into this prompt.

Let's save this, and just to reinforce these rules, I'm also going to add a human prompt, which is very similar to the system prompt. Let's save this, and now let's hook up those variables. Let's click on "format prompt values," and now we can see our placeholders for context and question.

For the context, we want to grab the output of the previous agent node, which is this sequential agent zero. If you were wondering how I knew what the agent's technical name was, I'll quickly show you. Let's close these pop-ups, and when I hover over this agent node, I can see this info button over here, and this will give me the unique name for this agent, which was sequential agent zero.

Great, let's go back to the condition node, let's go to prompt values, and now let's also assign the question. Let's click on "edit," and let's select the user's question. Let's close this pop-up, and then finally, let's set one value in this JSON structured output.

Let's add an item, let's call it "score," let's make this an enum, and the possible values are either "yes" or "no," and the description is "graining score. " Just to explain what's happening here, in the system prompt, we're telling the agent that it needs to check if the context is relevant to the user's question and then output a value of "yes" or "no. " What we have to do is store this "yes" or "no" value somewhere so that we can conditionally call different paths.

Otherwise, this is just simply a string output from the agent, which isn't really helpful in this case. So by defining this JSON structured output, the agent now has a variable that it can store the "yes" or "no" answer in. If we close this pop-up, we can now click on "condition," and here we can set those different paths.

So I'll actually add two items, and I'm actually going to do this a little bit different, and these two paths represent the generate path as well as the rewrite path. For the generate path, we want to check if some variable is equal to the value of "yes. " The variable that we want to check is actually the variable that we created within this structured output, which we called "score.

" Remember, the score value will contain the result from this agent execution. So within the condition on the variable, let's select flow. output, and let's replace this with the name of that variable, which we called score.

Then if score is equal to the value of "yes," then we'll follow this generate path. Or if the flow. output score is the value "no," then we will follow the rewrite path.

Let's save this, and you will now notice that our condition node gives us a generate path as well as a rewrite path. So let's add the logic for the generate and rewrite nodes. Let's go to add nodes with the "insequential agents.

" We can either use an agent node or an LLM node. I'm actually going to use the LLM node, as we don't need any tool calling for these specific agents. Let's attach the generate output to this LLM node, and let's call this guy generate.

I'm going to copy this node, and let's attach the rewrite output to the LLM node, and let's call this node rewrite. Excellent. Let's also attach end nodes to these two agents.

So I'll just copy this end node, let's attach it, and let's add an end node to this LLM node as well. Great. For the generate node, let's click on additional parameters, and let's enter something like "respond with the word generate.

" Let's also go to the rewrite node, and let's say "respond with the word rewrite. " We will work on these two nodes in a minute, but let's first test this process out to ensure that our conditional agent is working. Let's go to chat, and let's ask something that's relevant to the restaurant, like "What are your specials?

" Right, so our customer support agent was able to fetch information about the specials from the knowledge base, and we can see that the condition agent correctly determined that the generate path should be followed, and therefore the generate agent, this guy over here, output the word generate. Let's ask a question that's not related to the restaurant, like "Do you sell crypto? " But now we run into a very strange issue.

But what you'll notice is that our conditional agent decided to call the generate path as well. Although we know, the question and the context from the knowledge base is not relevant. Now, why is that?

I intentionally made this mistake to demonstrate the difference between using the agent node combined with an LLM node using a tool node. At the moment, we are grabbing the response from the agent node, so this response over here, and passing that to the condition agent node as context. And if you think about it, this text here is actually related to the user's question.

The user asked if the restaurant sells crypto, and this response is saying no. We do not sell crypto, so the two are definitely related. However, the response from the tool, in other words, the document from the vector store, is not related to the question at all.

So what we should be doing is comparing the response from the vector store to the user's question, and not the response from the agent, so this text over here. Now, how do we do that? First, let's delete the agent node.

Let's add two new nodes to the canvas. First, let's add an LLM node, and let's also add the tool node. Let's add the start node to our LLM node, and I'm also going to move all of these nodes up just to create a bit of space, like so.

Then let's put this tool node next to the LLM node, and there we go. Let's attach the LLM node to the tool node, and let's attach the tool node to the condition node, like so. So for the LLM node, let's call it customer support again, just like we did with the agent node.

In fact, it's going to be very similar to the agent node as well. When we click on additional parameters, we can go ahead and enter the exact same system prompt that we passed to the agent node earlier. And for the human prompt, let's simply add a placeholder for the user's question.

Let's click on format prompt values, and let's select question, like so. All right, let's move on to the tool node. Let's call the tool node retrieve, and then all we have to do is take our retriever tool and attach it to the tool node.

And I'm actually going to move these guys up, like so. Great. So now the tool node will reach out to the vector store and return the documents, and we can then take the documents and pass them into this condition node as context.

So within the condition node, let's go to additional parameters, let's go to format prompt values, and let's change the context from sequential agent to the sequential tool nodes output instead. So now instead of receiving some text from an agent, we will now receive the documents themselves from the vector store, and then compare that to the user's question. Let's test this out.

So in the chat, let's try what are the current specials. We can see the retriever tool fetch this information from the vector store, and the conditional agent decided to generate the response. Let's ask an invalid question.

Do you sell crypto? And of course, the retriever tool fetched information from the vector store. Those documents were passed to the conditional agent, which determined the document is not related to the user's question, and is now therefore triggering the rewrite path.

Great. That is exactly what we wanted. So hopefully, you now have a better understanding of when to use tool nodes, instead of agent nodes with tool calling.

Now let's implement our generate node. This one's quite simple. Within additional parameters, change the system prompt to "You are an assistant for question answering tasks.

Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. " And then we have placeholders for the user's question and the context.

Then it's also enter a human prompt, given the user question and context, answer the user's query. Let's click on format prompt values for the question. Let's select the question from the chat window.

And for the context, let's use the sequential tool nodes output. Great. Let's close this, and that's everything we need to do for the generate node.

Let's go to our rewrite agent. Let's click on additional parameters. Let's change the system prompt to something like, "You are a helpful assistant that can transform the query to produce a better question.

" Let's save this. Then in the human prompt, let's enter, "Look at the input and try to reason about the underlying semantic intent or meaning. Here is the initial question.

Formulate an improved question. " Let's save this. Let's go to prompt values.

Let's change the question to question. And that's all we have to do for this node as well. So what will happen with the rewrite node is that this lnM over here will try to rewrite the user's question in a different way, hoping we'll get a different answer.

Now, obviously, we don't want to stop the process at this point. So I'm going to delete this end node. So after this lnM has rewritten the question, we want to loop back to the customer support agent over here.

So let's simply copy the name of this node and paste it into the loop node, like so. And that's actually everything we need to do to build this application. Then in the chat, let's ask, "Do you sell crypto?

" So the retriever reached out to the vector store and retrieved this document over here, which is not related to the user's question. Therefore, the rewrite path was called, and the rewrite agent has rewritten the question to something like this, which produces a final output of "No, Oaken Barrel does not sell cryptocurrency. " So this is extremely helpful if the user's question is indeed related to the restaurant, but is perhaps not phrased in the correct way.

If you liked this video, then please hit the like button and subscribe to my channel for more Flow-wise content. I'll see you in the next one. Bye bye.