Add Flowise to ANYTHING! Flowise API Crash Course

19.92k views4691 WordsCopy TextShare

Leon van Zyl

#flowiseai #flowise In this video we will look at how to use the Flowise API endpoints to connect ...

Video Transcript:

Flowise offers several API endpoints that we can use to integrate our chat flows into custom applications or third party platforms. These APIs allow us to access and use our Flowwise chat flows from outside of Flowwise. But you might be wondering how this is different to the embed API.

We can use the embed API to add these little chat bubbles and chat windows to websites. Whereas the API endpoints allow us to access the chat flows from everywhere else. For example, we could call these APIs from custom code, which will allow us to add things like user authentication and session management.

Or we could also integrate these into workflow automation tools like Zapier and Make. Another popular use case is to call these APIs from AI chat flow platforms like BotPress and VoiceFlow. This video is slightly more technical than the previous videos in the series.

We won't write any code in this video and we won't be integrating this into any specific platform as there are way too many different options to consider. So instead, we will purely focus on how we can use these API endpoints. In order to test these APIs, I will be using a tool called Postman.

So if you would like to follow along and download and install Postman for your operating system. Let's also have a look at what we'll be doing in this video. We will start off by calling this chat flow by using the API endpoint.

We will also have a look at using that endpoint to override some of the values in this chat flow. And that will allow us to dynamically make changes to any chat flow using the API endpoint. Secondly, we will also have a look at calling this rag chat flow that will allow us to chat with custom data.

And we will also have a look at using the API to trigger the observing of this data as well as passing in files via the API endpoint. So let's start with a simple LLM chain. All this chain does is generate a company name based on a product description.

And this variable is called product. And if we look at the prompt values, I'm simply mapping the input from the user to product. And if we call this in the chat, and let's pass in something like balloons, we do get a response back.

Now, let's say we wanted to call this chat flow from outside of Flow-wise. What we could do is click on API endpoint, and here we can see a few options for Python, JavaScript, and Curl. Let's say you were writing a Python application and you would like to call your Flow-wise chat flow from Python.

You could simply copy this code. And the same thing goes for JavaScript. But let's say you are working on some Zapier or Voiceflow application.

You could simply go to the scroll tab to see the API endpoint that you need to call as well as the header information, as well as the fields that you need for the body. So what we could do is copy this URL, and then in Postman, click on Add, then paste in that URL, and change this from get to post. And we can see the type of request over here, and this is a post request.

Then we also have to set the header content type to application JSON. We can do that by going to Headers. Then we can create a new key and call it content type with a value of application JSON, and in the body, we can simply change this to raw, and we can then copy this value in the body and then paste it into this field.

So let's test this by clicking on Send, and we can see these values coming back, and this text field includes the response from our chat flow, which is trying to generate a business name. So let's change this to something like Cakes. Let's send this, and we are getting our response back.

So let's go back to Flow-wise for a second, and let's close this popup. When we go to Settings and View Messages, we can see that each time we call the API, a new conversation is being generated. Once we have a look at conversation chains, I will show you how you can reuse the same session ID to simply append messages to the same conversation, and that will allow us to simply continue with an existing conversation instead of generating new conversations with each API call.

So let's have a look at one of the most important properties on this API, and that is the override config property. Let's go back to this API popup, and let's check this Show Input Config checkbox, and here we can see a lot of additional information, which might seem intimidating, but this is actually really easy to use. First, we can see all the nodes on our canvas, and at the moment, we only have three, which is the chat model, the Allyl-in chain, as well as the prompt template.

We can use the override config property to override any of the properties for any of these nodes. So let's say as a silly example that we wanted to change the model for this chatOpenAI node dynamically, and at the moment, this model is set to GPT 3. 5 Turbo, but let's say that using the API, we want to override this value to GPT 4.

Now, let's go back to this popup, and let's see how we can override these values. First, we can add another property to this payload called override config. We can then add a colon, and then curly braces to specify further properties.

Now, let's find the property name for the model name. Let's expand this chatOpenAI node again, and we can see the name of the specific properties in this name column. Let's copy this model name property.

Let's add it to our payload. We can then specify a value with colon, and in quotes, let's pass in a value of GPT 4. Let's go ahead and run this.

We received our response, but how do we know that this was actually using GPT 4 and not GPT 3. 5 Turbo? This is where an analytics tool like Langsmouth comes into play.

If you're unfamiliar with Langsmouth, I've already created a video on how to use it, and it is free to use. We can enable Langsmouth by going to configuration, then analyze chat flow, then under Langsmouth, I'm just going to select my Langsmouth credentials. Let's give it a project name like API demo.

Let's enable this. Let's click on save. Let's save the chat flow.

So now that we have Langsmouth enabled, let's run this query again. In Langsmouth, I can see this API demo project, and if I click on LLM chain, we can see the model that was used by clicking on this metadata tab, and in here, we can see that the model was indeed GPT 4. We can override some other properties as well.

For example, we can override this prompt template completely. So instead of generating a product name, let's say we want to generate a joke instead. So in order to override this prompt template value, we can get the property name by going to this popup, show input config, then on the prompt template itself, we can see two properties, the template as well as the template values.

To override the template, let's copy the name of this property, then in postman, let's pass in a template of tell me a joke about product, and product is the variable that we are mapping from the user input. Let's change the user input from cakes to dogs, and let's run this. And we can see in the response that we are now getting a joke instead of a business name.

We can also pass in a specific value for a variable by copying this prompt values property. Let's also add that to the override config, and let's change this one product to subject, and prompt values is actually an object as well. And in this object, we have to specify the name of the variable that we are trying to override, and in this example, this is subject, which has a value of, let's say cats.

And now when we run this, we are now receiving a joke about cats. So now we have completely overwritten both the template as well as the variable that we are using. Of course, you can use both variables by simply using both of these variables in this template string.

And we can also see that behavior in Langsmouth. And if we look at the prompt that we used, we can see that we've actually received two variables for product and subject. And the prompt template was indeed changed to tell me a joke about cats.

So you might also be wondering that at the moment, we are simply passing in a value to override a template and prompt values. But in the example where we have multiple prompt templates on the same flow, how does Flowwise know which template to override for which node? As an example, you might recall that in the series, we had a look at something called prompt chaining where we had multiple LLM chains in the same flow.

To change the value for a specific node is actually very easy. So let's assume we actually had multiple prompt templates on this canvas. So let's simply grab another prompt template and just place it randomly on the canvas.

Now, let's say we wanted to change the prompt template for this node only and not this one. We can do this very easily by getting the node ID by hovering over any node and then clicking on this info button. And this value in this yellow background is actually the node ID.

We can simply copy this node ID. So let's say we wanted to change the template for the specific node only. What we can do is remove this string value and replace it with an object or curly braces.

And now in quotes, we can specify the specific node that we'd like to change and then we can specify the new value. Like tell me a joke about subject. And if we wanted to, we could add additional nodes to that very same object.

So let's get the node ID for this node. Let's copy it. Let's add a comma.

It's passing the value of the node name. And now we could specify its prompt template and we could follow this exact same pattern for any node and for any property. Now let's move on to conversation chains.

So here I have a very simple conversation chain using the chat open AI model with a buffer memory node. So the thing about conversation chains is when we send messages, those messages will start a new conversation. And ideally within a conversation, we want to achieve two things.

We want the ability to continue with an existing conversation by appending messages to that conversation. And secondly, we want to be able to recall information from the chat history. So if I tell the model what my name is and continue the conversation days later, I want the model to be able to recall that information.

And at the moment, we are simply creating a new conversation for each and every API call, which is not ideal. There is something very unique that we can do with conversation chains using the API endpoint. Let's have a look at this.

Let's start by sending a message like hello. And now you will notice that with conversation chains, we actually get this session ID property back and we can use this session ID property to continue with an existing conversation. For example, I can now copy the session ID property along with its value and we can add this to the override config object.

Let's now change the message to something like how are you? And let's send this. We get our response back and you will also notice that the session ID that is being returned is the same as a session ID that we passed in over a year.

Now, if we go back to our review messages, we can see that conversation over a year with that session ID and looking at the messages, we can now see that these messages are indeed being appended to this list. So let's also send something like, my name is Leon, let's send this. Let's refresh this view messages page.

And indeed we can see those messages being appended. Next, let me show you a limitation with this conversation and I'll show you two possible solutions. Although I've told this model that my name is Leon, if I asked it what is my name using the same session ID, the model will actually tell us that it does not know our name.

And if I refresh this again, we can actually see this information is still in the chat. According to the developers from Flow-wise, this is currently a feature that is being worked on. And by the time you watch this video, this issue might be resolved natively within Flow-wise.

But thankfully there are two very easy ways to work around this limitation. The first option is to simply add a new property to your payload called history. And history is an array of values and each value will either represent a response from the AI or a user message.

So it's basically mimicking this list of messages over here. So this means we could add an object and these objects have two properties, the message, which is my name is Leon. And secondly is the type of message.

And this will be the user message. For the AI messages, we could simply add another object. We have to specify the message property for this with a message like, "Hi Leon, nice to meet you.

" And this type is the API message, which was the response from the model. And one very important thing to note is that the messages contained in this history property will not be appended to the final conversation. This is simply used as context for the AI model.

So if I now send this, the model is able to recall our name. But don't worry though, I am going to show you a super simple way of dealing with the chat history in a minute. So you might be wondering how on earth you will be able to fetch this chat history from your own applications in order to build up this history array.

Thankfully, that is really simple to do. Let's create a new session in Postman and then paste in the following URL. So first it needs to start with your server.

And by the way, I am running a local instance of Flow-wise for this demo, but in a production application where you need to access Flow-wise from something like Voice Flow, you need to use an instance of Flow-wise that is available in the cloud. So simply paste in your server details, then slash API V1 chat message slash, and then your workflow ID is something that you can copy from this prediction URL like so, and let's paste it in here. Now in order to fetch the messages for a specific session ID, we need to specify a key called session ID with a value.

And for the value we need to pass in the session ID. I'll simply copy it from this previous payload and let's paste it in here without this quote. Now when I run this, we can see in the response all of the messages related to that session ID.

And this contains everything we need to build up that array, like the role as well as this content property. But I'm actually going to show you the easiest way to deal with chat history. So I'm going to remove this history array completely and back in Flow-wise, let's actually remove this buffer memory node and let's go down to memory and let's grab this up stash Redis backed chat memory node and let's connect this to our chain and let's select our credentials.

If you're new to this, I've already created a video on using this up stash Redis backed chat memory node and this is free to use. And I highly recommend going with a solution like this for your production solutions. If I go back to up stash and if I click on data browser, we can see that this database is clean at the moment.

So let's go back to postman and let's start a new conversation. I'm actually going to delete the session ID and let's change this message to hello. It's send this and we can now see in the response that we are using this up stash for Redis backed chat memory node and let's copy the session ID property as I do want to continue with this conversation going forward.

Also, if I refresh up stash, I can see this entry for our conversation and this is the same value as our session ID as well as the two messages. So the AI message and our human message. And this is also in sync with our view messages over here where we can see our session ID as well as our two messages.

The big benefit of this is that we no longer have to pass the history node as Flow-wise will automatically fetch the chat history from Redis and inject that into the conversation. Let me prove that to you. Let's say my name is Leon and now let's ask it, what is my name?

And indeed we do get our response back. But this time this was way easier to use. And if we refresh up stash, we can see that there were new entries added to this database as well.

Next, let's have a look at how we can interact with a rag chat flow. The API allows us to do a couple of very interesting things with rag. First, we can chat with our data using the API endpoint, but we can also use the endpoint to absurd data to the vector database.

In this flow, we are using pinecone. And if I go back to the pinecone console, we can see that this index is empty at the moment. We will also have a look at replacing the web scraper with a text file uploader so that I can show you how you can upload files using the API endpoint as well.

But first, this video was a lot of effort to put together and I will greatly appreciate your support. Hit the like button and please subscribe to my channel to support my work. Let's start with the basics.

Let's scrape information from this Lang chain documentation. And this URL is simply this page in the Lang chain documentation. We will have a look at upserting this using the API in a second, but let's upload it using this button for now.

We get this message saying that the upset was successful and back in our pinecone index, we can see one new entry containing the content for that page. Now let's query our data using Postman. First, I'm going to remove override config.

We will get back to that in a second. And let's also replace this session ID with a session ID for this RAG chat flow. So I'm just going to copy this value and then replace this with our RAG chat flow ID.

Let's also change this question to what is Lang chain? Let's run this. And we are getting the correct response back.

Great, so we are able to query our data. But now let's have a look at using the API to run the upsert as well. I'm actually going to clear pinecone so that we start with no data, then back in flow wise, let's click on upsert again.

And this time let's check show API. Let's click on curl and let's copy this URL. In Postman, I'm actually going to create a new tab.

It's placed in that API. Let's change the type to post in the headers. Let's add the content type property with a value of application JSON.

Then in the body, let's change this to raw. And actually for now, we don't have to enter anything in the body. Let's simply click send.

We get this response saying successfully upserted. And in the pinecone database, we can refresh this and we can see that the data was indeed upserted. Now let's take this one step further and let's add our data to a specific namespace.

Adding data to specific namespaces can be extremely important for real world applications. A practical example is an application that has multiple users. And let's say that users need to authenticate themselves in order to chat to your chat flows.

After authenticating, users might want to upload documents to their knowledge spaces. And when they ask questions about the data, only data that belong to them should be query. So you could create a namespace for each user in your system and then run your prediction against that specific namespace only.

Let's have a look at that. Again, I'm going to delete this data just so that we have a clean database again. Now, when we do the upsert, I'm going to add something to the body.

In here, let's add the override config property, which is an object. And when we switch back over to flow wise, we can see which properties we can set. On this pinecone node, we can set the pinecone index dynamically, the pinecone namespace, et cetera.

What I want to change is the pinecone namespace. So I'm going to copy this value, then back in Postman, let's add this value as a property. And then we can specify any value for the namespace that we want.

This could be something like the user's unique ID or email address, like Leon-data as an example. This could be anything you want. Now let's run this upsert.

Let's go back to pinecone. Let's refresh this. And we can see that the entry was created.

However, when we click on namespaces, we can now see a namespace for Leon data. Also back in this browser, we can filter by namespace. At the moment, we only have one, and we can see the data belonging to this namespace only.

And if we go back to Postman, and let's go back to the prediction API, if I now ask this question as is, we actually won't receive the correct answer. And indeed we get this response back saying, I'm not sure. And that is because we are not specifying the namespace for which this query should be run against.

So let's add the override config property. And let's also specify the pinecone namespace property, which is Leon data. Now watch what happens when we run this query again.

This time we do receive the correct response. And this is a fantastic way to isolate data to specific sessions or users. Of course, we can also use the override property to override the URL that we are scraping from.

So we can see that property by going to curl, then clicking on show input config. Here we can click on Cheerio web scraper, and this property is simply called URL. So what you could do in the absurd API is to simply add this URL property, and then we can provide a different URL endpoint to scrape from.

But let's move on to the final example. Let's replace this web scraper with a text file uploader, like so. This is a very simple example, and I'm simply going to upload a file that says that Leon lives in South Africa.

That's really it. But of course you can use PDF files and way more complicated data if you want. So let's save this chat flow and let's clear this pinecone index.

And let's have a look at how we can pass files to this API for the upsell. So back in Postman, we need to change a few things when dealing with files. When dealing with files, we need to change the content type to form data.

In Postman, that's quite easy. We can simply click on this form data radio button. But if you are using this API programmatically, you can set the content type to multi-part slash form data.

In the body, we need to pass in a property called files, and I'll simply change it from text to file. And now we can select a file from our local machine. We can still specify the pinecone namespace by adding pinecone namespace, and let's call it Leon data again.

Of course, this namespace is optional. And if you do remove it from the upsell, then just remember to remove it from the query as well. Let's test this by clicking on send.

This is saying the upsell was successful. And if we refresh pinecone, we can see this data was indeed loaded. So in our predict method, let's say, where does Leon live?

It's send this and we do get our response back. As a bonus, I do want to show you a few more very important features about APIs. We are able to secure these APIs using an API key, and we can also prevent people from abusing the APIs by setting a rate limit.

We can do that by clicking on API endpoint, then under curl to selecting an API key. At the moment, we only have one, and that is this default key, and we can also add a new API key. Alternatively, we can go back to the Flow Wise dashboard.

We can go to API keys. Here we can see that default key, or we can generate a new API key. Let's give it a name.

I'll call mine YouTube, and we can then copy or view that API key. So let's go back to that chat flow. Let's go to API endpoints.

Let's click on curl. This changes from no authorization to YouTube, and let's save this chat flow. Now watch what happens when I call this prediction endpoint again.

Now it's responding with a message saying unauthorized. Now in order to authorize our applications to call these endpoints, we can actually see an example of how to use this. In this example payload, and in header, we now need to set an authorization property.

So let's do this. Under headers, let's add authorization, and we can simply copy this bearer token as well as the API key. So it's everything from bearer up until that final quote, and we can paste that value in here.

And now when we call this endpoint, we now get our response back. So this is perfect for protecting our API routes. We can also set write limiting by closing this popup.

Let's go to settings, configuration, and under this write limiting tab, we can set how many messages should be allowed within a given timeframe. For instance, 200 messages for every 60 seconds. So 200 messages per minute.

If a limit is reached, we can also set the message that will be displayed back to the user, like slow down, try again in a minute. Just to demonstrate this, let's change the message limit to one message per minute. Let's save this, and in postman, let's execute this.

Let's immediately run it again, and we can see that message coming back. If you enjoyed this video, then you might enjoy this other Flow-wise API video, where we have a look at building a web UI for interacting with the Flow-wise chatbots.