Capabilities

Tools

Tool factories bind every tool to an explicit workspace:

from libs import create_builtin_tools, create_read_only_tools

coding_tools = create_builtin_tools("/work/project")
review_tools = create_read_only_tools("/work/project")

The read-only set leaves out file writes and the shell but keeps web research, which suits explorer and reviewer agents.

• search collects up to 50 results across engines and opens with a breakdown by kind, site and recurring theme, so the model explains sources instead of pasting links.
• research(query, pages=5) searches, then reads the top pages in parallel and returns numbered sources to cite as [n].
• fetch_url turns pages and documents (PDF, DOCX, PPTX, …) into clean Markdown with code snippets, images and media, and transcribes audio and video. Pages that come back empty are rendered in the browser.

Loading tools on demand

Every tool schema is sent with every request, so put rarely needed tools in a ToolGroup. The agent gets a load_tools tool listing each group; a group joins the run when the model loads it or calls one of its tools by name.

from libs import Agent, ToolGroup, create_audio_tools

agent = Agent(
    name="assistant",
    tools=core_tools,
    deferred_tools=[ToolGroup("audio", "speak aloud and save speech audio", create_audio_tools(root))],
)

The CLI keeps files, shell, web search, workspace, knowledge-graph and skill tools always on, and defers audio, images, media, subagents, schedules, git, tool_builder and each MCP server. /tools lists them.

Authorization

authorize_tool runs before every call. Return True to allow, or False or a reason to deny; the model gets the reason and can recover.

def review_policy(call: ToolCall) -> bool | str:
    if call.name in {"write_file", "edit_file", "run_command"}:
        return "review agents are read-only"
    return True

agent = Agent(name="reviewer", tools=create_builtin_tools(), authorize_tool=review_policy)

Git

The deferred git group gives the agent git_status, git_diff, git_log, git_show, git_branch and git_commit for the workspace repository. They run git with an argument list, not a shell, and refuse refs and paths that start with -.

git_commit stages exactly the paths it is given and commits only them, leaving other staged or unstaged work alone. Push, reset and rebase are not included. plan and ask modes treat git_branch and git_commit as edits. In the SDK, create_git_tools(workspace) returns the same tools.

Tools the agent writes itself

When no tool fits, the agent can write one with create_tool(name, code, description, dependencies). It runs in a separate, time-limited Python subprocess with its own site-packages, so your environment is never changed. save_created_tool copies it into the workspace as a standalone script you can run with uv run. The CLI always asks before creating or running one.

Web research & citations

search, fetch_url and research tell the model to cite what it uses, and the agent checks the final reply: if the turn read web pages but the reply names none of them, a Sources list is appended with each page's title and URL (keeping research's [n] numbers):

Bun 1.2 adds built-in S3 support and a text lockfile.

Sources:
1. Bun 1.2 release notes — https://bun.sh/blog/bun-v1.2

Images found while reading a page are saved in a folder per page, .andromeda/assets/web/<site>-<page>-<hash>/, with a manifest.json listing the source page, its title, and each image's file, URL and caption, so they can be shown again without re-scraping.

Browser

The agent drives a real browser through Python Playwright. It reads each page as an accessibility snapshot whose elements carry refs, and acts on those refs instead of guessing selectors:

browser_open("news.ycombinator.com")
Page: Hacker News
Snapshot (act on elements by ref, e.g. browser_click(ref='e12')):
- link "new" [ref=e17] -> newest
- link "147 comments" [ref=e51] -> item?id=49823582
...
browser_click(ref="e51")

Tools: browser_open (always on), and in the on-demand browser group browser_snapshot, browser_click, browser_type, browser_select, browser_press, browser_back, browser_wait, browser_read, browser_screenshot and browser_close, plus the same wider set as the TypeScript edition: browser_search (DuckDuckGo, Google or Bing on a rendered results page that stays open), browser_tabs, browser_find, browser_scroll, browser_hover, browser_drag, browser_click_at, browser_fill_form, browser_file_upload, browser_drop, browser_resize, browser_emulate_media (dark mode, print), browser_handle_dialog, browser_evaluate (JavaScript), browser_run_code (a Python Playwright script), browser_console_messages, browser_network_requests and browser_network_request. Opening a workspace .html file serves it locally so its CSS and scripts load, which is handy for checking pages the agent builds.

Compared with the Playwright MCP server, the built-in browser sends far smaller snapshots, which matters for small local models, and only browser_open is loaded until the agent needs more. browser_run_code always asks first. Clicks, typing, selection and key presses are refused in plan mode and ask first in ask mode; the agent is also told to ask before buying, posting, sending or deleting anything.

browser:
  enabled: true
  engine: chromium     # chromium | firefox | webkit
  channel: ""          # or chrome, msedge to use an installed browser
  headless: true       # false shows the window
  idle_timeout: 900    # seconds before an idle browser closes

uv run playwright install chromium   # once (scripts/andromeda does this for you)

Desktop control

On macOS, Windows and Linux desktops the on-demand computer group sees and drives the desktop: computer_screenshot, computer_move, computer_click, computer_type, computer_press, computer_scroll and computer_open_app. The agent takes a screenshot and looks at it with view_image before acting, and opens apps with computer_open_app rather than shell commands.

Linux and Ubuntu

Linux needs a desktop session; on a server or over SSH the computer tools are left out. The session type picks the tools:

• Apps open by their menu name ("Files", "Firefox") through gtk-launch and the .desktop entry, or by command name.
• Keys: in computer_press, cmd means Ctrl on Linux and super is the Windows key; letters, digits and F1–F12 also work.
• Wayland without ydotool: xdotool can still drive apps running under XWayland, but not native Wayland windows.
• Display scaling: with fractional scaling, screenshot pixels and pointer coordinates can differ; 100% scaling keeps them matched.

Skills

A skill is a folder with a SKILL.md: frontmatter plus task instructions. Skills live under skills/ or .agents/skills/.

---
name: release-check
description: Verify a release candidate before publishing.
---

Read the changelog, run `{baseDir}/check.sh`, and summarize failures.

Only the frontmatter is read at startup. The model finds skills with find_skills(query) and reads one with load_skill(name). Each turn, up to two clearly matching skills are suggested. A skill written for Andromeda (agent: andromeda) is included outright when it ranks highly or when the request contains one of its triggers:

triggers: [website, web page, landing page, html, css]
agent: andromeda

Skills written for other agents often name tools like Bash or WebFetch; load_skill adds a note mapping each to its equivalent here. /skills [query] searches them in the CLI, and teach mode creates new ones.

Voice & audio

Speech output uses Kokoro-82M (default) or Qwen3-TTS; speech input uses Faster Whisper with voice activity detection. Everything runs locally, or TTS can stream from your own HTTP server.

Talk to Andromeda

• Press Ctrl+R or type /voice, or start with andromeda --listen. Pause to send; replies are spoken sentence by sentence as the model writes them.
• Esc stops a spoken reply; say "stop listening" to turn it off.
• On macOS, give your terminal microphone access under System Settings → Privacy & Security → Microphone.
• Ask "use kokoro" or "use qwen" to switch voices mid-session, if that provider is installed.

voice:
  provider: kokoro       # qwen | kokoro
  mode: local            # local | http
  kokoro:
    voice: af_heart
    language: a          # a = American English, b = British English, ...
  http:
    url: http://my-server.local:8880/v1/audio/speech
    api_style: openai
    fallback_after: 3    # seconds; speak locally if the server is slow
  listen:
    barge_in: false      # true: talk over replies to interrupt

Emojis and tags like [laughs] or [whispers] are acted out rather than read aloud. Qwen can clone a voice from 5–15 seconds of clean speech; only clone voices you have permission to use.

In your own code

from libs import Agent, SynthesizerVoice, VoiceConversation, play_audio

agent = Agent(name="assistant", model="gpt-oss:20b", voice=SynthesizerVoice())
reply = await agent.run("Say hello in one short sentence.")
await play_audio(await agent.speak(reply.output))

await VoiceConversation(agent).run()   # hands-free until "stop listening"

Voice providers follow Mastra's shape (agent.speak, agent.listen, get_speakers), and VoicePipeline chains speech-to-text, the agent and streamed speech. docs/audio.md covers custom providers, cloning, realtime speech and every setting.

Images

Andromeda generates and edits images, and can look at them with a vision-capable model to check its own edits. The vision model is served by the configured chat engine.

• Ask in plain language ("draw a lighthouse at dusk, watercolor", then "make the sky orange"), or use /image <prompt> and /image edit <change>.
• edit_image keeps the original, then asks the vision model whether the change is really there, so the agent can try again.
• view_image describes an image or answers a question about it. With Ollama, the default is qwen2.5vl:3b (ollama pull qwen2.5vl:3b). With Docker Model Runner, select a DMR model that supports image input; the example smollm2 and Nomic embedding models are text-only.

Images are saved as <timestamp>-<prompt>-AI-generated.png in ~/Pictures unless you set image.output_dir. The first generation downloads the model into model/image/. On a 32 GB Apple silicon Mac, 1024×1024 at 40 steps takes about 4.5 minutes; 512×512 at 12 steps takes under one.

from libs import ImageConfig, ImageGenerator

images = ImageGenerator(ImageConfig(provider="qwen"), workspace=".")
result = await images.generate('A neon shop sign that reads "ANDROMEDA", rainy night', seed=42)
edited = await images.generate("Change the background to a sunset beach", images=[result.path])

Videos

Short clips come from Wan 2.1 T2V 1.3B running locally through diffusers. Install it with uv sync --extra video; the first clip downloads about 17.6 GB into model/video/.

Ask for a clip ("a fox running through snow, low tracking shot"). generate_video runs as a deferred background task, so you can keep working; the agent tells you when the MP4 is ready. The default is 832×480, 33 frames at 16 fps (about two seconds); raise num_frames to 81 for about five.

video:
  output_dir: videos  
  width: 832
  height: 480
  num_frames: 33      # 4k+1; 81 is the model's maximum
  fps: 16
  steps: 30

Schedules

Ask Andromeda to do something on a schedule ("every weekday at 9, summarize my open PRs"), and it creates a durable cron schedule. Schedules live in SQLite, survive restarts, and their results appear in the conversation even when nobody is typing.

from libs import ScheduleService

schedules = ScheduleService(".andromeda/schedules.db", {"reporter": agent})
await schedules.create(
    id="daily-report",
    agent_id="reporter",
    cron="0 9 * * 1-5",
    timezone="Africa/Johannesburg",
    prompt="Prepare the morning report.",
)
await schedules.start()

Five-, six- and seven-field cron expressions and @hourly, @daily, @weekly, @monthly and @midnight are supported. Schedules can be paused, resumed, run now or deleted. The CLI stores them in the app's own .andromeda/ directory whatever the workspace; set ANDROMEDA_STATE_DIR to move it.