Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support tool calling. #581

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Support tool calling. #581

wants to merge 3 commits into from

Conversation

rectalogic
Copy link

This supports LLM tool calling. Plugins can register new tools (python callables that are introspected to generate JSON schema), and models are provided access to installed tools. Adds tool calling to the openai Chat model.

Bundles a default read_files tool.
Here is a sample brave_search tool in a plugin
https://github.com/rectalogic/llm-tools/blob/dev/src/llm_tools/brave_search.py

  • add register_tools hookspec to allow plugins to register tools
  • add @llm.Tool decorator that introspects a callables type signature and generates a JSON schema
  • expose installed tools to Models via new tools Model property
  • implement tool calling in the openai Model
  • register a default read_files tool

Note: this pins pytest_httpx until broken tests are fixed in
#580
Note: this requires openai>=1.40.0
Note: this drops 3.8 since a number of dependencies no longer
support it, and adds 3.13 support

* add register_tools hookspec to allow plugins to register tools
* add @llm.Tool decorator that introspects a callables type signature
  and generates a JSON schema
* expose installed tools to Models via new `tools` Model property
* implement tool calling in the openai Model
* register a default `read_files` tool

Note: this pins pytest_httpx until broken tests are fixed in
  simonw#580
Note: this requires openai>=1.40.0
Note: this drops 3.8 since a number of dependencies no longer
  support it, and adds 3.13 support
@rectalogic
Copy link
Author

Hmm, tool call responses can result in more tool calls, so I made openai loop.
e.g.

> llm chat --enable-tools
Chatting with gpt-4o-mini
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Search Brave for the query "hurricane milton" and write the first line of the esponse to /tmp/hurricane
Tool: brave_search(query=hurricane milton)
Tool: bash(command_line=echo 'Milton made landfall near Siesta Key, Florida, as a dangerous Category 3 hurricane before weakening as it cut through the state.' > /tmp/hurricane)
result
The first line of the search result for "hurricane milton" has been successfully written to `/tmp/hurricane`. The content states:

"Milton made landfall near Siesta Key, Florida, as a dangerous Category 3 hurricane before weakening as it cut through the state."

Introspecting callable type hints is the default fallback if no schema is provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant