Faster LLM Function Calling — Dynamic Routes

LLM function calling can be slow, particularly for AI agents. Using Semantic Router's dynamic routes, we can make this much faster and scale to thousands of tools and functions. Here we see how to use it with OpenAI's GPT-3.5 Turbo, but the library also supports Cohere and Llama.cpp for local deployments.

In semantic router there are two types of routes that can be chosen. Both routes belong to the Route object, the only difference between them is that static routes return a Route.name when chosen, whereas dynamic routes use an LLM call to produce parameter input values.

For example, a static route will tell us if a query is talking about mathematics by returning the route name (which could be "math" for example). A dynamic route can generate additional values, so it may decide a query is talking about maths, but it can also generate Python code that we can later execute to answer the user's query, this output may look like "math", "import math; output = math.sqrt(64).

⭐ GitHub Repo: https://github.com/aurelio-labs/semantic-router/

📌 Code: https://github.com/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb

🔥 Semantic Router Course: https://www.aurelio.ai/course/semantic-router

👋🏼 AI Consulting: https://aurelio.ai

👾 Discord: https://discord.gg/c5QtDB9RAP

Twitter: https://twitter.com/jamescalam LinkedIn: https://www.linkedin.com/in/jamescalam/

00:00 Fast LLM Function Calling 00:56 Semantic Router Setup for LLMs 02:20 Function Calling Schema 04:04 Dynamic Routes for Function Calling 05:51 How we can use Faster Agents ... https://www.youtube.com/watch?v=QsZm0XCysoQ