Tool Calling Not Working

[dawenxi-007]

I am trying the tool calling function with Brave Search engine by following the simple instructions at https://github.com/meta-llama/llama-agentic-system#add-api-keys-for-tools.

Basically, I saved the api key inside an .env file by BRAVE_SEARCH_API_KEY=xxxxx run that at the CLI terminal. However, with the command mesop app/main.py, the search engine did not seem to work, giving the message like the knowledge is up to Dec. 2023.

Also tried PYTHONPATH=. mesop app/chat_with_custom_tools.py, didn't work either, it gave the following info:

INFO:     127.0.0.1:57624 - "POST /agentic_system/create HTTP/1.1" 200 OK
INFO:     127.0.0.1:57626 - "POST /agentic_system/session/create HTTP/1.1" 200 OK
INFO:     127.0.0.1:42830 - "POST /agentic_system/turn/create HTTP/1.1" 200 OK
role='user' content='Which team won NBA championship in 2024?'
role='assistant' content='' stop_reason=<StopReason.end_of_turn: 'end_of_turn'> tool_calls=[ToolCall(call_id='6ea013e3-8fa3-4758-aea8-d344d6051d5d', tool_name=<BuiltinTool.brave_search: 'brave_search'>, arguments={'query': 'NBA championship 2024 winner'})]
role='ipython' call_id='6ea013e3-8fa3-4758-aea8-d344d6051d5d' tool_name=<BuiltinTool.brave_search: 'brave_search'> content='{"query": null, "top_k": []}'
Assistant: The search results do not provide the answer to the question. I will try to search again with a different query.
INFO:     127.0.0.1:50940 - "POST /agentic_system/turn/create HTTP/1.1" 200 OK
role='user' content='What are the big news in 2024'
role='assistant' content='' stop_reason=<StopReason.end_of_turn: 'end_of_turn'> tool_calls=[ToolCall(call_id='8c19d850-fb20-4d17-b9be-ddeec447cc3f', tool_name=<BuiltinTool.brave_search: 'brave_search'>, arguments={'query': 'big news 2024'})]
role='ipython' call_id='8c19d850-fb20-4d17-b9be-ddeec447cc3f' tool_name=<BuiltinTool.brave_search: 'brave_search'> content='{"query": null, "top_k": []}'
Assistant: The search results do not provide the answer to the question. I will try to search again with a different query.
INFO:     127.0.0.1:54344 - "POST /agentic_system/turn/create HTTP/1.1" 200 OK
role='user' content='What are the big news in 2024'
role='assistant' content='' stop_reason=<StopReason.end_of_turn: 'end_of_turn'> tool_calls=[ToolCall(call_id='815ac2e6-f9aa-4fde-b1f7-16d08e140f36', tool_name=<BuiltinTool.brave_search: 'brave_search'>, arguments={'query': '2024 news'})]
role='ipython' call_id='815ac2e6-f9aa-4fde-b1f7-16d08e140f36' tool_name=<BuiltinTool.brave_search: 'brave_search'> content='{"query": null, "top_k": []}'
Assistant: The search results do not provide the answer to the question. I will try to search again with a different query.

Unfortunately, I am unable to provide any information on the news in 2024 as the search results do not provide any relevant information.

Do we have more docs on how to enable this?

[dawenxi-007]

Here is what is shown in the mesop WebUI.

[ashwinb]

I believe the instructions are outdated :/ We will fix that but the move to distributions and moving the agentic_system implementation to the llama-toolchain repository means that you need to do the following:

• you need to have .env available when starting the distribution server (via llama distribution start)

Then the mesop UI should work. Let me know if this works.

[dawenxi-007]

Thank @ashwinb for your quick response. Yes, I do have .env (content is BRAVE_SEARCH_API_KEY=xxxxx ) file under the llama-agentic-system folder before I launched the distribution (llama distribution start --name local-llama-8b --port 5000).

The mesop UI works, but the Brave Search Engine was not working, at least, it is not fetching the right results.

Let me know if there is anything I can try.

[ashwinb]

From which directory did you launch the llama distribution start command? Did that directory have the .env file?

[dawenxi-007]

(llama_stack_inference_demo) tao@r7625h100:~/local/llama-agentic-system$ ll
total 180
drwxrwxr-x 7 tao tao 4096 Aug 21 00:20 ./
drwxrwxr-x 4 tao tao 4096 Aug 20 22:23 ../
drwxrwxr-x 4 tao tao 4096 Aug 20 22:21 app/
-rw-rw-r-- 1 tao tao 3537 Aug 20 22:12 CODE_OF_CONDUCT.md
-rw-rw-r-- 1 tao tao 1339 Aug 20 22:12 CONTRIBUTING.md
-rw-rw-r-- 1 tao tao 87484 Aug 20 22:12 demo.png
drwxrwxr-x 2 tao tao 4096 Aug 20 22:12 docs/
-rw-rw-r-- 1 tao tao 53 Aug 20 23:39 .env
drwxrwxr-x 6 tao tao 4096 Aug 20 22:12 examples/
-rw-rw-r-- 1 tao tao 1062 Aug 20 22:12 .flake8
drwxrwxr-x 8 tao tao 4096 Aug 20 22:36 .git/
-rw-rw-r-- 1 tao tao 3092 Aug 20 22:12 .gitignore
-rw-rw-r-- 1 tao tao 7645 Aug 20 22:12 LICENSE
-rw-rw-r-- 1 tao tao 25 Aug 20 22:12 MANIFEST.in
-rw-rw-r-- 1 tao tao 1345 Aug 20 22:12 .pre-commit-config.yaml
-rw-rw-r-- 1 tao tao 87 Aug 20 22:12 pyproject.toml
-rw-rw-r-- 1 tao tao 18289 Aug 20 22:12 README.md
-rw-rw-r-- 1 tao tao 83 Aug 20 22:12 requirements.txt
drwxrwxr-x 2 tao tao 4096 Aug 20 22:12 tests/

[dawenxi-007]

@ashwinb Is it any prompt format that I have to follow in the mesop WebUI window for the tool calling function?

[ashwinb]

@dawenxi-007 I have verified that this is broken right now. For some reason, loading the keys from .env is not working. We will fix it.

[ashwinb]

PR has landed, but new packages still need to be deployed.