Forget OpenAI! Flowise Tool Agents Work with ANYONE (ft. Claude 3)

10.32k views2446 WordsCopy TextShare

Leon van Zyl

Flowise recently added a Tool Agent to the platform. The Flowise Tool Agent allows you to attach any...

Video Transcript:

I get these types of comments all the time. Why are you only using OpenAI in your tutorials? The reason is actually quite simple.

For a very long time, OpenAI was pretty much the only provider that supported tool use or function calling. So if you wanted to create an agent that was able to call custom tools and functions, you were very much limited using the GPT-4 models or the OpenAI Assistance API. This is therefore why I tailored my tutorials around OpenAI.

Things have changed recently and there are more and more models supporting tool use. So this is why Flow-wise decided to deprecate the current tool use and function agents. And instead, they've decided to implement a single tool use agent, which will work with any model that supports function calling.

As per their post on X, this includes Mistral, Gemini, Claude, Grok, and obviously OpenAI. So in this video, I will show you how you can create a tool agent using the Claude 3 models, but feel free to swap this out for any other model that supports function calling, like the ones mentioned earlier. This agent will be able to perform retrieval from a knowledge base, like a Word document or a PDF doc, and it's also able to call tools.

So what you'll need to follow along is an anthropic API key. To get your API key, go to anthropic. com, click on API, click on Get Started Now, sign into your account.

And the nice thing about anthropic is that they do give you some free credits. And this is unfortunately something that OpenAI stopped doing. And that also seems to be why you guys want to move away from OpenAI as part of your learning process.

So after signing in, you should receive your free credits. To generate the API key, go to Get API Keys, then click on Create Key, give your key a name, like YouTube Tutorial, and create your key. Copy this key, then back in Flow-wise, go to Credentials, click on Add Credential, click on Anthropic API, give your credential a name, like Tool Agent Key or whatever you want, and paste in your API key.

Please ensure to use your own as I will delete my key after this video. Click on Add. Now let's create this flow.

Go to Chat Flows, then click on Add New. Let's save this. Let's give it a name.

I'll call mine Claude Tool Agent, and let's click on Save. As with any flow, we need either a chain or an agent. And in this example, we will add a Tool Agent.

So let's click on Add Notes, then under Agents, scroll all the way down to Tool Agent. Let's add this to the canvas. Great.

Now this note takes in a list of tools as well as a memory node. Memory is simply for keeping track of the conversation history. And we also have to supply a chat model that supports function calling or tool use.

Let's start by adding the chat model. Let's click on Add Notes, then we can close Agents. Let's go to Chat Models.

And because we want to add a Claude 3 model, we need to add this Chat Anthropic node to the canvas. Let's attach our chat model to the agent, like so, and under Credentials, let's select the Tool Agent Credential that we just created. And under Model Name, we have a list of a few models, Iku, Opus, and Sonnet.

You can see the differences between these models in the Anthropic documentation. Iku is a really cheap model that is fast and surprisingly accurate. And on the right-hand side, we have the most powerful model called Opus.

And Sonnet sits in the middle and is a good middle ground between performance, cost, and reasoning abilities. And if you look at how these models compared to the GPT and Gemini models, they pretty much outperform these other models in every other metric. So enough of that, let's continue with our flow.

Under Model Name, let's select Opus, which is the most powerful model. Let's change the temperature to something like 0. 2.

You will also notice this Allow Images Uploads toggle, and this is available because these Claude models are also multimodal and they have vision capabilities. This means that if you enable this toggle and save the chat flow, you will be able to upload images in the chat and ask questions about that image. But we will not use this in this tutorial, but I just thought it's a very cool feature to share with you.

Let's also go ahead and add the memory node. So let's go to Add Nodes, let's go to Memory, and let's simply add the buffer memory node. And let's connect this memory node to our agent.

Great, now for the tools. In this demo, I'm going to add three tools to this agent, but feel free to add as many tools as you want. In fact, if you go to Add Nodes and Tools, you can see a whole list of available tools, and you also have this custom tool that will allow you to write custom code for all you developers out there.

And with this custom tool, you can get the agent to perform pretty much anything you want. If you would like to learn more about things like the custom tool or the chain tool, then check out these other videos on my channel. Let's keep things simple for this demo.

Let's add a calculator to the canvas. I'll simply add some of these tools to the right-hand side of this agent, and we can attach a tool to the agent by clicking on the output from this tool and adding it to the tool input on the agent. Our tools work is when we ask the agent a question, the agent will see if there are any tools available that it can use to potentially answer our question.

So if we ask it a math question, it will see that we've supplied a calculator tool, and it will attempt to use that tool to answer the question. We can test that out now by saving the flow, and in the chat, let's first check if this is actually working. Let's simply write, hello, and we do get a response back from the Anthropic API, which means this is working.

Now let's ask a math question, like what is 500 times 10,000, just as a stupid little example. We get this response back, but what you will also notice is this little button up here showing us that the calculator tool was used, and if we click on this, this will show the inputs into the tool as well as the tool output. This can be very useful.

Let's also add the ability for the agent to search for information online, effectively perform a sort of a Google search. Let's go to add nodes, then within tools, let's add the SERP API node, like so, and then let's attach this node to the tools as well. For this node to work, we need to provide credentials.

So let's click on create new, let's give it a name like Claude tool agent, or whatever you want, and for the SERP API key, we can get the key from SERPAPI. com. Simply register or sign into your account, and don't worry though, there are no costs involved as we will be using the free tier.

From your dashboard, simply copy this API key and then paste it into Flow-wise, like so, and let's click on add. Let's save this chat flow, and our agent should now be able to search the internet. So let's ask a question like, what is Langsmouth?

This took a few seconds to execute, but we did get our response back and we can see that the search tool was indeed used with the input of Langsmouth, and we can also see the tool output over here. Now let's also add the ability for the agent to answer questions from a knowledge base. To demonstrate this, I've created this fictitious Q&A document for a restaurant called the Oak and Barrel, and this is a very simple document that contains some general information about this business.

So let's see how we can attach this knowledge base to our agent. Now I've already covered RAG chatbots in other videos, so please be sure to check out those videos as well if you get lost during this process. But what I want to do instead is focus on a very specific tool that will allow us to attach our vector store to the agent, and that is called the Retriever tool.

So let's add this to the canvas, and let's attach our tool to the agent, like so. We will set up all the RAG nodes in a second, but what this tool needs is a name and a description. For the name, I'm going to call it Restaurant Information, and the description is very important as this will be used by the agent to determine when this tool needs to be called.

In theory, you could have multiple retrievers from different data sources, and the description will tell the agent when these tools need to be called. Let's enter something like, use this tool when asked questions about the restaurant, food items, specials, and contact information. We can also enable return source documents to see where the model retrieved this information from.

Now we need to attach a Retriever, which is effectively our vector store. It's going to add nodes. Let's scroll down all the way to the bottom to vector stores, and let's add the in-memory vector store.

Feel free to use Pinecone or any other vector store that you want. It's attached this to the Retriever tool, and now the vector store requires an embeddings function as well as a document uploader. Let's start with the document.

Let's go to add nodes. Let's go to document loaders, and I'm going to use this docx file uploader because my knowledge base is a Word document. Let's attach this.

Let's select that file, and let's also add a text splitter to chunk the data. So let's go to text splitters. Let's add the recursive character text splitter, and let's attach our text splitter to the document uploader.

Let's change the chunk size to 200 with an overlap of 20. And lastly, let's add the embeddings function. This is something that a lot of you have been asking about as well, is which embeddings function to use.

You can use pretty much any embeddings function or model that you want. Just be sure to use the correct dimensions. And in fact, you can even use the open AI model if you want, but that is not what people do in this video.

Instead, if we have a look at the anthropic documentation, and if we scroll down, we can see this section that says that anthropic does not offer its own embedding model, but they do recommend that we use voyage AI for the embeddings function. And that is what we'll use in this video. So in our flow, let's go to add nodes, then let's go to embeddings.

Within embeddings, let's add the voyage AI embeddings node. And let's attach it to the vector store. Now, I know what some of you might be thinking.

You might be wondering what the dimensions would be if you had to use a persistent vector store like Pinecone. And I will show you that in a minute, but let's first get this simple example working. The voyage AI embeddings node requires credentials.

So let's click on this dropdown. Let's click on create new. Let's give our credential a name like Claude tool agent.

And now we have to populate this voyage API key. We can get the key by going to voyageai. com, then sign in, then from this dashboard, click on create new API key.

I will call mine Claude tool agent and click on create API key. Go ahead and copy this key, then paste it into flow wise and click on add. And then lastly, let's change the model name from voyage two to a voyage large two.

Let's save this chat flow. And that is actually all we have to do to chat with documents using our agent. You can simply swap out the document loader for any other file type and then simply absurd the data into your vector store.

In fact, let's absurd this data now by clicking on absurd. Let's click on absurd. So this is now chunked the data and added 29 records to the vector store.

Let's go ahead and test this out in the chat by asking what are the current specials? And looking at our FAQ document, we would expect the answer to be 50% off all stakes and happy hour between four and 6 p. m.

And looking at our chat, that is exactly what we received. We can also see that the restaurant information tool was called as well as the input and the output from our retriever. Now let's say you used a different vector store like pine cone.

Typically the embedding model will determine the dimensions that you have to enter when creating a vector store. As an example, if I log into pine cone and create a new index, you typically have to enter the dimensions for this index. And with open AI, it was something like 1536.

Now, because we're using the voyage AI embeddings function, what would the dimensions be based on the model that we selected here? Well, the answer is I simply went back to the anthropic documentation and I'm referring to the documentation that suggested that we use voyage in the first place. And in this documentation, if you scroll down, there is the section called available voyage models.

Here we can see the voyage large two model, which we used in this demo and this embedding dimension should be 1536. Or if you used any of the smaller models, you can simply use 1024 as the dimensions. So we can simply copy this value and then use that to create our vector index.

I hope you enjoyed this video and I would like to know what other models you would like me to cover in future videos. Please hit the like button, subscribe to my channel and also have a look at this other video where we create a rag chatbot using open AI.