The AI Agent Index

Documenting the technical and safety features of deployed agentic AI systems

Amazon Q Developer

Basic information

Website: https://web.archive.org/web/20241231162225/https://aws.amazon.com/q/

Short description: A general purpose coding agent [source]

Intended uses: What does the developer say it’s for? General purpose software development tasks

Date(s) deployed: November 28, 2023 [source]

Developer

Website: https://web.archive.org/web/20241231162225/https://aws.amazon.com/q/

Legal name: Amazon.com, Inc [source]

Entity type: Corporation [source]

Country (location of developer or first author’s first affiliation): Incorporation: Delaware, USA (AMAZON.COM INC (2620453)) [source]. HQ: Seattle [source]

Safety policies: What safety and/or responsibility policies are in place? Amazon’s Responsible AI policy [source]

System components

Backend model: What model(s) are used to power the system? Amazon (Titan models), Anthropic, AI21 Labs, Cohere, Meta, Mistral AI, Stability AI [source]

Publicly available model specification: Is there formal documentation on the system’s intended uses and how it is designed to behave in them? Available [source]

Reasoning, planning, and memory implementation: How does the system ‘think’? Unknown

Observation space: What is the system able to observe while ‘thinking’? System can access user’s input text, codebases, and data on AWS [source].

Action space/tools: What direct actions can the system take? System is able to write code, natural language responses, use AWS resources, and leverage third-party integrations [source] [source].

User interface: How do users interact with the system? Amazon Q can be used on code editors, like VS Code, or using command line interface [source]

Development cost and compute: What is known about the development costs? Unknown

Guardrails and oversight

Accessibility of components:

  • Weights: Are model parameters available? N/A; backends various models
  • Data: Is data available? N/A; backends various models
  • Code: Is code available? Closed source
  • Scaffolding: Is system scaffolding available? Closed source
  • Documentation: Is documentation available? Available [source]

Controls and guardrails: What notable methods are used to protect against harmful actions? Bedrock has the option of “returning control” to the developer by allowing them to decide whether to execute actions in their own applications [source]. There are also relevant AWS security measures [source].

Customer and usage restrictions: Are there know-your-customer measures or other restrictions on customers? None

Monitoring and shutdown procedures: Are there any notable methods or protocols that allow for the system to be shut down if it is observed to behave harmfully? Users can access AWS CloudTrail and Amazon Cloudwatch to monitor Amazon Q [source].

Evaluation

Notable benchmark evaluations: 55% on SWE-Bench verified [source]

Bespoke testing: None

Safety: Have safety evaluations been conducted by the developers? What were the results? None

Publicly reported external red-teaming or comparable auditing:

  • Personnel: Who were the red-teamers/auditors? None
  • Scope, scale, access, and methods: What access did red-teamers/auditors have and what actions did they take? None
  • Findings: What did the red-teamers/auditors conclude? None

Ecosystem information

Interoperability with other systems: What tools or integrations are available? Documentation discusses integrations [source].

Usage statistics and patterns: Are there any notable observations about usage? Unknown

Additional notes

None