The AI Agent Index

Documenting the technical and safety features of deployed agentic AI systems

SWE Agent

Basic information

Website: https://composio.dev/swe-kit/

Short description: SWE-Kit is a framework for building Software Engineering agents using Composio’s tooling ecosystem. SWE-Kit comes with examples agents including their SWE agent [source].

Intended uses: What does the developer say it’s for? General purpose software engineering.

Date(s) deployed: Announced November 26, 2024 [source]

Developer

Website: https://web.archive.org/web/20241216224755/https://composio.dev/

Legal name: Sampark Inc [source]

Entity type: Corporation [source]

Country (location of developer or first author’s first affiliation): Incorporation: Delaware, USA (SAMPARK INC, 7417658) [source]

Safety policies: What safety and/or responsibility policies are in place? None

System components

Backend model: What model(s) are used to power the system? Variable, but backends Claude by default [source]

Publicly available model specification: Is there formal documentation on the system’s intended uses and how it is designed to behave in them? None

Reasoning, planning, and memory implementation: How does the system ‘think’? There are three specialized agents for software engineering, code analysis, and code editing. The software engineering agent orchestrates the process [source].

Observation space: What is the system able to observe while ‘thinking’? It is able to work within the filesystem that it is deployed in [source].

Action space/tools: What direct actions can the system take? SWE agents are able to perform basic actions in its fileystem including running shell commands which can include running arbitrary code [source].

User interface: How do users interact with the system? A coding IDE and terminal.

Development cost and compute: What is known about the development costs? Unknown

Guardrails and oversight

Accessibility of components:

  • Weights: Are model parameters available? N/A; backends external model(s) via API
  • Data: Is data available? N/A; backends external model(s) via API
  • Code: Is code available? Available [source]
  • Scaffolding: Is system scaffolding available? Available [source]
  • Documentation: Is documentation available? Available [source]

Controls and guardrails: What notable methods are used to protect against harmful actions? “The SWE-agent runs in Docker by default for security and isolation. This sandboxes the agent’s operations, protecting against unintended consequences of arbitrary code execution.” [source]

Customer and usage restrictions: Are there know-your-customer measures or other restrictions on customers? None

Monitoring and shutdown procedures: Are there any notable methods or protocols that allow for the system to be shut down if it is observed to behave harmfully? None

Evaluation

Notable benchmark evaluations: 48.6% on SWE-Bench Verified [source]

Bespoke testing: None

Safety: Have safety evaluations been conducted by the developers? What were the results? None

Publicly reported external red-teaming or comparable auditing:

  • Personnel: Who were the red-teamers/auditors? None
  • Scope, scale, access, and methods: What access did red-teamers/auditors have and what actions did they take? None
  • Findings: What did the red-teamers/auditors conclude? None

Ecosystem information

Interoperability with other systems: What tools or integrations are available? None

Usage statistics and patterns: Are there any notable observations about usage? Composio has 4.3k forks and 14k stars [source]

Additional notes

None