agentboard

AgentBoard: AI Agent Visualization Toolkit

DeepNLP AgentBoard provides the toolkit of visualization of AI Agents Worflow, which helps visualize and monitor the AI Agents running loop and key entities such as chat messages, tools/functions, RAG(Retrieval Augmented Generation), workflow and raw data types including text, dict or json, image, audio, video, etc.

agentboard agent loop workflow

Key Features

APIs to log various data types for LLM calling and AI Agent development: text, dict or json, image, audio, video, etc.
AI Agent Key Entities Visualization: Visualize key entities of chat history messages, memories, tools/functions schema and input/output dict, etc.
Workflow of AI Agent Loop Running, Visualize the plan/act/react/reflect stage in the complete workflow chart. Can be modified and displayed for papers and technical reports.
Multi-Agents Support
Autonomous Agent Simulation Enviroment Support. Multiple build-in AI agent environment AI community, X(twitter) style mininal website for AI agents to interact (comment/like/follow/etc).
Chat Visualizer: A Chatbot Visualizer with multiple mainstream chat(WhatsApp, WeChat, ChatGPT, etc.) UI

You can install and import the ‘agentboard’ python package and use functions under a with block. See quickstart for install and run agentboard

DeepNLP AgentBoard Quickstart Docs

DeepNLP AgentBoard Python Full API Docs

AgentBoard Supported AI Agent Elements and Data Types

Functions	DataType	Description
ab.summary.messages	message	List of messages, json format [{“role”: “user”, “content”: “content_1”}, {“role”: “assistant”, “content”: “content_2”}]
ab.summary.tool	function	User defined functions, The schema of the functions which are passed to LLM API calling, Support OpenAI and Anthropic stype
ab.summary.agent_loop	str	User defined Agent Loop with various stages PLAN/ACT/REFLECT/etc
ab.summary.text	str	Text data, such as prompt, assistant responded text
ab.summary.dict	dict	Dict data, such as input request, output response, class dict
ab.summary.image	tensor	Support both torch.Tensor and tf.Tensor, torch.Tensor takes input shape [N, C, H, W], N: Batch Size, C: Channels, H: Height, W: Width; tf.Tensor, input shape [N, H, W, C], N: Batch Size, H: Height, W: Width, C: Channels.
ab.summary.audio	tensor	Support torch.Tensor data type. The input tensor shape [B, C, N], B for batch size, C for channel, N for samples.
ab.summary.video	tensor	Support torch.Tensor data type. The input tensor shape should match [T, H, W, C], T: Number of frames, H: Height, W: Width, C: Number of channels (usually 3 for RGB)

Installation

You can install agentboard through pip and start the Flask based web app using the command line agentboard. You can change ‘logdir’, ‘logfile’, ‘static’ and ‘port’ parameters to point to your folders. After installation, you can visit (http://127.0.0.1:5000) to see the web console of agentboard. There is build in log file to visualize multiple data types.

# installation
pip install agentboard

# cmd line start service
agentboard

# Change log dir and port agentboard --logdir=./log --logfile=xxx.log --static=./static --port=5000

AgentBoard Log Messages Chat Visualizer

Let’s start with an example of calling OpenAI LLM api with user input prompt.

    import agentboard as ab

    messages= []
    logdir="./log"
    with ab.summary.FileWriter(logdir=logdir) as writer:
        prompt = "Can you give me an example of python code usage of async await functions"
        messages.append({"role": "user", "content": prompt})

        ## Calling OpenAI Chat Completion API            
        #        completion = client.chat.completions.create(
        #            model="gpt-3.5-turbo",
        #            messages=[
        #                {"role": "system", "content": "You are a helpful assistant."},
        #                {"role": "user", "content": prompt}
        #            ]
        #        )

        response_message = {"role":"assistant","content":"Sure! Here's an example of Python code that uses async/await functions:\n\n```python\nimport asyncio\n\nasync def print_numbers():\n    for i in range(1, 6):\n        print(i)\n        await asyncio.sleep(1)\n\nasync def main():\n    task1 = asyncio.create_task(print_numbers())\n    task2 = asyncio.create_task(print_numbers())\n    await task1\n    await task2\n\nasyncio.run(main())\n```\n\nIn this example, we define an async function `print_numbers` that prints the numbers 1 to 5 with a one-second delay between each number using `await asyncio.sleep(1)`. We also define an async function `main` that creates two tasks using `asyncio.create_task` to run the `print_numbers` function concurrently. We then use `await` to wait for both tasks to complete.\n\nWhen we run the `main` function using `asyncio.run(main())`, the numbers will be printed concurrently by the two tasks."}

        messages.append(response_message)

        ab.summary.messages(name="OpenAI Chat History", data=messages, agent_name="assistant")

Then you can go visit the agentboard (http://127.0.0.1:5000/log/messages) to see the chat visualizer of the chat completion history.

Note the log writing –logdir and agentboard loading –logdir should match.

agentboard summary messages function

AgentBoard Agent Loop Visualization

Let’s start with an example of a basic asynchronously AI Agent Loop, consists of 3 stages: PLAN, ACT, REFLECT. And plot the stage and result on a workflow chart on agentboard. We can use the basic Asynchronous Agent Loop in the AutoAgent Package, you can also use agentboard with many other agent framework, such as AutoGen and LangChain.

To use agentboard with basic Async Agent Class, we need to first rewrtie the agent with logger in the desired place.

    cd /examples/async_agents/

    # write logs and static file, default to ./log and ./static
    python run_agentboard_autoagent.py

    # run agentboard and visualize agent loop
    agentboard --logdir=./log --static=./static --port=5000

visit the agentboard (http://127.0.0.1:5000/log/agent_loop)

agentboard agent loop workflow

AgentBoard to log Tools Function Calls

Let’s start with an example of calling OpenAI Tool Usage API with user defined functions get_weather().

    import agentboard as ab
    from agentboard.utils import function_to_schema

    def calling_bing_tools(keyword:str, limit:int) -> str:
        url="https://www.bing.com/search?q=%s&limit=%d" % (keyword, limit)
        return url

    with ab.summary.FileWriter(logdir="./log", static="./static") as writer:

        tools = [calling_bing_tools]
        tools_map = {tool.__name__:tool for tool in tools}
        tools_schema = [function_to_schema(tool) for tool in tools]

        ## Calling ChatGPT Tool Usage code omitted
        # Omitted
        # arguments = json.loads(tool_call['function']['arguments'])

        arguments = {"keyword": "agentboard document", "limit": 10}


        ab.summary.tool(name="Act RAG Tool Bing", data=[calling_bing_tools], agent_name="agent 2", process_id="ACT")
        ab.summary.dict(name="Act RAG Argument Input", data=[arguments], agent_name="agent 2", process_id="ACT")

agentboard tool function

AgentBoard to Display Image Tensor

Let’s log a random pytorch tensor with shape [8, 3, 400, 600] and display the batch image tensors on the agentboard.

import torch
import agentboard as ab

with ab.summary.FileWriter(logdir="./log", static="./static") as writer:

    input_image = torch.mul(torch.rand(8, 3, 400, 600), 255).to(torch.int64)
    ab.summary.image(name="Plan Input Image", data=input_image, agent_name="agent 1", process_id="plan")

agentboard image function

AgentBoard Display Audio Tensor

Let’s log a random pytorch tensor of audio with 2 channels, 16000 sample_rate lasting for 2 seconds.

import torch
import agentboard as ab
import math

with ab.summary.FileWriter(logdir="./log", static="./static") as writer:
    sample_rate = 16000  # 16 kHz
    duration_seconds = 2  # 2 seconds
    frequency = 440.0  # 440 Hz (A4 note)
    t = torch.linspace(0, duration_seconds, int(sample_rate * duration_seconds), dtype=torch.float32)
    waveform = (0.5 * torch.sin(2 * math.pi * frequency * t)).unsqueeze(0)  # Add channel dimension
    waveform = torch.unsqueeze(waveform, dim=0)
    ab.summary.audio(name="ASR Input Audio", data=waveform, agent_name="agent 1", process_id="asr")

agentboard audio function

AgentBoard Display Video Tensor

Let’s log a random pytorch tensor of video clip with 30 frames, 64x64 resolution, 3 color channels and use agentboard to visualize it.

import torch
import agentboard as ab

with ab.summary.FileWriter(logdir="./log", static="./static") as writer:
    T, H, W, C = 30, 64, 64, 3  # 30 frames, 64x64 resolution, 3 color channels
    video_tensor = torch.randint(0, 256, (T, H, W, C), dtype=torch.uint8)
    frame_rate = 24  # Frames per second
    ab.summary.video(name="Text2Video Output", data=video_tensor, agent_name="agent 1", 
        process_id="act", file_ext = ".mp4", frame_rate = 24, video_codecs = "mpeg4")

agentboard audio function

AI Services Reviews and Ratings

agentboard

AgentBoard: AI Agent Visualization Toolkit

Key Features

AgentBoard Supported AI Agent Elements and Data Types

Installation

AgentBoard Log Messages Chat Visualizer

AgentBoard Agent Loop Visualization

AgentBoard to log Tools Function Calls

AgentBoard to Display Image Tensor

AgentBoard Display Audio Tensor

AgentBoard Display Video Tensor

Agents Related Pipeline Workflow and Document

AI Services Reviews and Ratings

AI Agent

Chatbot

AI Image Generation

AI Video Generation

AI Education

Robotics

AI Tools

AI Widgets

Social

Self-Driving

Related Blogs