Smolagents

Contents

Smolagents#

In this notebook we will build two agents using smolagents.

Hints: To make the second agent work, you need to configure an SERPAPI_API_KEY environment variable. You can create such a key on https://serpapi.com/ - for free at the time of writing this.

from smolagents.agents import ToolCallingAgent
from smolagents import tool, LiteLLMModel
from smolagents.default_tools import GoogleSearchTool
import os

C:\Users\rober\miniforge3\envs\genai-gpu\Lib\site-packages\pydantic\_internal\_config.py:345: UserWarning: Valid config keys have changed in V2:
* 'fields' has been removed
  warnings.warn(message, UserWarning)

Under the hood, we use the litellm library to access a local or remote open-weight LLM.

model = LiteLLMModel(model_id="openai/llama3.2", 
                     api_base="http://localhost:11434/v1", # replace with remote open-ai compatible server if necessary
                     api_key="your-api-key")               # replace with API key if necessary

# this will work after the next release of smolagents > 0.1.3, as this PR was merged: https://github.com/huggingface/smolagents/pull/10
model = LiteLLMModel(model_id="openai/meta-llama/Llama-3.3-70B-Instruct", 
                     api_base="https://llm.scads.ai/v1", 
                 api_key=os.environ.get('SCADSAI_API_KEY'))

A custom agent#

First we demonstrate how to configure a custom function so that it can be called by the agent.

@tool
def read_arxiv_paper(arxiv_url:str)-> str:
    """Read the abstract of an arxiv-paper and return most important contents as markdown.
    
    Args:
        arxiv_url: url of the arxiv paper to read.
    """
    from agent_tools import get_arxiv_metadata
    arxiv_id = arxiv_url.split("/")[-1]
    metadata = get_arxiv_metadata(arxiv_id)
    title = metadata["title"]
    summary = metadata["summary"]
    authors = ", ".join(metadata["authors"])
    
    return f"""## {title}
By {authors}

{summary}
"""

agent = ToolCallingAgent(tools=[read_arxiv_paper], model=model)

agent.run("Read this paper and tell me its most important points: https://arxiv.org/abs/2301.00303v1")

╭──────────────────────────────────────────────────── New run ────────────────────────────────────────────────────╮
│                                                                                                                 │
│ Read this paper and tell me its most important points: https://arxiv.org/abs/2301.00303v1                       │
│                                                                                                                 │
╰─ LiteLLMModel - openai/meta-llama/Llama-3.3-70B-Instruct ───────────────────────────────────────────────────────╯

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'read_arxiv_paper' with arguments: {'arxiv_url': 'https://arxiv.org/abs/2301.00303v1'}            │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Observations: ## Rethinking with Retrieval: Faithful Large Language Model Inference
By Hangfeng He, Hongming Zhang, Dan Roth

Despite the success of large language models (LLMs) in various natural
language processing (NLP) tasks, the stored knowledge in these models may
inevitably be incomplete, out-of-date, or incorrect. This motivates the need to
utilize external knowledge to assist LLMs. Unfortunately, current methods for
incorporating external knowledge often require additional training or
fine-tuning, which can be costly and may not be feasible for LLMs. To address
this issue, we propose a novel post-processing approach, rethinking with
retrieval (RR), which retrieves relevant external knowledge based on the
decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting.
This lightweight approach does not require additional training or fine-tuning
and is not limited by the input length of LLMs. We evaluate the effectiveness
of RR through extensive experiments with GPT-3 on three complex reasoning
tasks: commonsense reasoning, temporal reasoning, and tabular reasoning. Our
results show that RR can produce more faithful explanations and improve the
performance of LLMs.

[Step 0: Duration 2.67 seconds| Input tokens: 1,280 | Output tokens: 35]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'final_answer' with arguments: {'answer': "The most important points of the paper 'Rethinking     │
│ with Retrieval: Faithful Large Language Model Inference' are: it proposes a novel post-processing approach      │
│ called rethinking with retrieval (RR) to utilize external knowledge to assist large language models (LLMs), RR  │
│ retrieves relevant external knowledge based on decomposed reasoning steps obtained from chain-of-thought (CoT)  │
│ prompting, and it does not require additional training or fine-tuning. The paper evaluates the effectiveness of │
│ RR through experiments with GPT-3 on three complex reasoning tasks and shows that RR can produce more faithful  │
│ explanations and improve the performance of LLMs."}                                                             │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Final answer: The most important points of the paper 'Rethinking with Retrieval: Faithful Large Language Model 
Inference' are: it proposes a novel post-processing approach called rethinking with retrieval (RR) to utilize 
external knowledge to assist large language models (LLMs), RR retrieves relevant external knowledge based on 
decomposed reasoning steps obtained from chain-of-thought (CoT) prompting, and it does not require additional 
training or fine-tuning. The paper evaluates the effectiveness of RR through experiments with GPT-3 on three 
complex reasoning tasks and shows that RR can produce more faithful explanations and improve the performance of 
LLMs.

[Step 1: Duration 5.86 seconds| Input tokens: 2,885 | Output tokens: 176]

"The most important points of the paper 'Rethinking with Retrieval: Faithful Large Language Model Inference' are: it proposes a novel post-processing approach called rethinking with retrieval (RR) to utilize external knowledge to assist large language models (LLMs), RR retrieves relevant external knowledge based on decomposed reasoning steps obtained from chain-of-thought (CoT) prompting, and it does not require additional training or fine-tuning. The paper evaluates the effectiveness of RR through experiments with GPT-3 on three complex reasoning tasks and shows that RR can produce more faithful explanations and improve the performance of LLMs."

A search agent#

Next, we use predefined tools, such as a tool for searching the internet.

agent = ToolCallingAgent(tools=[GoogleSearchTool()], model=model)

print(agent.run("What does ScaDS.AI do?"))

╭──────────────────────────────────────────────────── New run ────────────────────────────────────────────────────╮
│                                                                                                                 │
│ What does ScaDS.AI do?                                                                                          │
│                                                                                                                 │
╰─ LiteLLMModel - openai/meta-llama/Llama-3.3-70B-Instruct ───────────────────────────────────────────────────────╯

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 0 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'web_search' with arguments: {'filter_year': 2024, 'query': 'ScaDS.AI'}                           │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Observations: ## Search Results
0. [Events and Event Series - ScaDS.AI Dresden/Leipzig](https://scads.ai/event/)
Date published: Jul 15, 2024
Source: ScaDS.AI

ScaDS.AI Dresden/Leipzig regularly organizes interesting events and event series on Artificial Intelligence.

1. [Master's Seminar ScaDS.AI](https://scads.ai/education/masters-seminar-scads-ai/)
Date published: Aug 20, 2024
Source: ScaDS.AI

Participation in the “Master's Seminar ScaDS.AI” module enables students to familiarize themselves with current 
research topics at ScaDS.AI.

2. [ScaDS.AI and ZIH at SC24 in Atlanta, Georgia](https://scads.ai/sc24/)
Date published: Nov 22, 2024
Source: ScaDS.AI

From 17-.22.11.2024, ScaDS.AI Dresden/Leipzig & ZIH represented the Center for Interdisciplinary Digital Sciences 
(CIDS) at SC24 in Atlanta.

3. [ScaDS.AI Dresden/Leipzig](https://tu-dresden.de/cids/departments/scads-ai?set_language=en)
Date published: Nov 27, 2024
Source: TU Dresden

ScaDS.AI Dresden/Leipzig is a competence center for AI, Data Science, & Big Data with locations in Dresden and 
Leipzig.

4. [Summer School 2024](https://scads.ai/event/summer-schools/summer-school-2024/)
Date published: Jun 28, 2024
Source: ScaDS.AI

The 10th International Summer School on AI and Big Data will be hosted from 25.06 – 28.06.2024 by ScaDS.AI in 
Leipzig.

5. [General Assembly Meeting in March 2024](https://scads.ai/general-assembly-meeting-march-2024/)
Date published: Mar 19, 2024
Source: ScaDS.AI

ScaDS.AI Dresden/Leipzig organized the first General Assembly Meeting of 2024. More than 170 participants joined 
the event in Dresden.

6. [ScaDS.AI Dresden/Leipzig](https://www.linkedin.com/company/scads-ai)
Source: LinkedIn · ScaDS.AI Dresden/Leipzig

ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence) Dresden/Leipzig is a center for Data 
Science, Artificial Intelligence and Big Data ...

7. [Projects](https://scads.ai/research/architectures-scalability-security/projects/)
Date published: Jul 15, 2024
Source: ScaDS.AI

A center for Data Science, Artificial Intelligence and Big Data with locations in Dresden and Leipzig.

8. [NAIC: AI as a Study Coach (Podcast)](https://scads.ai/naic-ai-as-a-study-coach/)
Date published: Nov 8, 2024
Source: ScaDS.AI

The Junior Research Group NAIC of ScaDS.AI is investigating how chatbots can help students manage their academic 
life. NAIC stands for “Needs-Oriented AI ...

[Step 0: Duration 4.50 seconds| Input tokens: 1,311 | Output tokens: 28]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Calling tool: 'final_answer' with arguments: {'answer': "ScaDS.AI is a competence center for AI, Data Science,  │
│ & Big Data with locations in Dresden and Leipzig. It organizes events and event series on Artificial            │
│ Intelligence, offers educational programs such as master's seminars, and conducts research in areas like        │
│ scalable data analytics and artificial intelligence."}                                                          │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Final answer: ScaDS.AI is a competence center for AI, Data Science, & Big Data with locations in Dresden and 
Leipzig. It organizes events and event series on Artificial Intelligence, offers educational programs such as 
master's seminars, and conducts research in areas like scalable data analytics and artificial intelligence.

[Step 1: Duration 3.17 seconds| Input tokens: 3,353 | Output tokens: 101]

ScaDS.AI is a competence center for AI, Data Science, & Big Data with locations in Dresden and Leipzig. It organizes events and event series on Artificial Intelligence, offers educational programs such as master's seminars, and conducts research in areas like scalable data analytics and artificial intelligence.