Agent S
Open-source GUI agent framework that lets an LLM use your computer like a human via an Agent-Computer Interface.
🛡️ AgentReady threat assessment
MAESTRO 7-layer threat model + OWASP AIVSS risk score for Agent S, derived from its capabilities.
These scores are auto-generated from public information (the agent's own listing, docs, and repository) using the canonical OWASP AIVSS formula and the MAESTRO framework — an estimate for guidance, not a penetration test, audit, or certification. See the scoring methodology. Are you the vendor? Factual corrections are free.
Overview
Agent S is an open-source framework for building and running “computer use” GUI agents that can autonomously operate a computer through an Agent-Computer Interface. It’s designed to let an LLM plan and execute multi-step tasks by observing the screen and taking actions (e.g., clicking, typing, navigating) on supported platforms (Linux, macOS, Windows). The project provides a CLI to run the agent, supports multiple model providers (including OpenAI, Anthropic, Gemini, OpenRouter, and vLLM), and recommends pairing a main LLM with a separate grounding model for UI understanding. Agent S also includes evaluation assets and reports results on established computer-use benchmarks such as OSWorld.
Key features
- computer use
- gui automation
- agent-computer interface
- grounding model
- osworld
- desktop workflows
- tool orchestration
- multi-step planning
Use cases
- Building computer-use agents that can operate desktop apps and websites through a GUI.
- Automating multi-step workflows on a real computer with an agentic CLI runner.
- Experimenting with grounding + planning model combinations for more reliable UI interaction.
- Evaluating computer-use agents on benchmarks like OSWorld and related environments.