Genie
Basic Information
Website: https://cosine.sh/genie
Short description: An AI software engineer
Intended uses: What does the developer state that the system is intended for?: Emulation of human software engineers [source]
Date(s) deployed: August 12, 2024 [source]
Developer
Website: https://cosine.sh/
Entity type: Private limited Company (UK) [source]. Corporation (US) [source]. US Corporation controls over 75 percent of the shares and voting rights of the UK Private limited Company [source]
Country (location of developer or first author's first affiliation): Incorporation: Delaware, USA (BUILDT INC. 7156765) [source]
Safety policies: What safety and/or responsibility policies are in place?: Unknown
System Components
Backend model(s): What model(s) are used to power the system?: Variable models which they fine-tuned on their data [source]
Public model specification: Is there formal documentation on the system’s intend...: None
Description of reasoning, planning, and memory implementation: How does the syst...: Proprietary [source], however, the system is trained on a dataset that resembles a software engineer's workflow [source].
Observation space: What is the system able to observe while 'thinking'?: GitHub access, workspace in which it can plan, write code, and run tests.
Action space/tools: What direct actions can the system take?: Write and execute code, debugging tools, GitHub access [source]
User interface: How do users interact with the system?: Can prompt with a freeform prompt, ticket, or link a GitHub issue; monitor the agent finding info, planning, writing code, and running tests [source]
Development cost and compute: What is known about the development costs?: Unknown
Guardrails & Oversight
Accessibility of components
Weights: Are model parameters available?: Closed source
Data: Is data available?: Closed source; however, Cosine claims that the data "perfectly emulates the cognitive processes, logic and workflow of human engineers. Our proprietary techniques generates data that represents perfect information lineage, incremental knowledge discovery, and step by step decision making" [source]
Code: Is code available?: Closed source
Documentation: Is documentation available?: Available [source]
Scaffolding: Is system scaffolding available?: Closed source
Controls and guardrails: What notable methods are used to protect against harmfu...: Unknown
Monitoring and shutdown procedures: Are there any notable methods or protocols t...: Unknown
Customer and usage restrictions: Are there know-your-customer measures or other ...: Currently limited to select users who can sign up via waitlist [source]
Evaluation
Notable benchmark evaluations (e.g., on SWE-Bench Verified): 30.08% on SWE-Bench [source]
Bespoke testing (e.g., demos): Demos [source]
Safety: Have safety evaluations been conducted by the developers? What were the ...: None
Publicly reported external red-teaming or comparable auditing
Personnel: Who were the red-teamers/auditors?: None
Scope, scale, access, and methods: What access did red-teamers/auditors have and...: None
Findings: What did the red-teamers/auditors conclude?: None