OS-Copilot
Basic Information
Website: https://os-copilot.github.io/
Short description: OS-Copilot is "a framework to build generalist agents capable of interfacing with comprehensive elements in an operating system (OS), including the web, code terminals, files, multimedia, and various third-party applications". FRIDAY is "a self-improving embodied agent for automating general computer tasks", built using the OS-Copilot framework [source]
Intended uses: What does the developer state that the system is intended for?: Potential use cases include operations requiring interactions with BASH, a Python interpreter, the keyboard/mouse, and an API. More details can be found in Appendix C [source]
Date(s) deployed: Paper arXived February 12, 2024 [source]
Developer
Website: https://os-copilot.github.io/
Legal name: Shanghai AI Laboratory (et al.) [source]
Entity type: Unknown
Country (location of developer or first author's first affiliation): China [source]
Safety policies: What safety and/or responsibility policies are in place?: Unknown
System Components
Backend model(s): What model(s) are used to power the system?: Variable for OS-Copilot. For FRIDAY, some backend models they have used include GPT-4 and GPT-4-Turbo plugins.
Public model specification: Is there formal documentation on the system’s intend...: None
Description of reasoning, planning, and memory implementation: How does the syst...: Implementation of OS-Copilot consists of three components: 1) A planner that "reasons over user requests and decomposes complex ones into simpler subtasks"; 2) A configurator, with working memory, that "takes a subtask from the planner and configures it to help the actor complete the subtask"; and 3) An actor that consists of an executor, proposing and executing appropriate actions, and a critic, assessing the outcomes of the executor's actions [source]
Observation space: What is the system able to observe while 'thinking'?: Common observation space is user input and outcome of agent actions.
Action space/tools: What direct actions can the system take?: The OS-Copilot framework "consolidates common practices for OS manipulation, including Python code interpreter (Significant-Gravitas, 2023), bash terminal, mouse/keyboard control (Cheng et al., 2024), and API calls (Qin et al., 2023)" [source]
User interface: How do users interact with the system?: There is a user interface to interact with OS-Copilot [source].
Development cost and compute: What is known about the development costs?: Unknown
Guardrails & Oversight
Accessibility of components
Weights: Are model parameters available?: N/A; backends various models
Data: Is data available?: N/A; backends various models
Code: Is code available?: Available [source]
Documentation: Is documentation available?: Available [source]
Scaffolding: Is system scaffolding available?: Available [source]
Controls and guardrails: What notable methods are used to protect against harmfu...: Unknown
Monitoring and shutdown procedures: Are there any notable methods or protocols t...: Unknown
Customer and usage restrictions: Are there know-your-customer measures or other ...: None
Evaluation
Notable benchmark evaluations (e.g., on SWE-Bench Verified): On "GAIA, a general AI assistants benchmark, FRIDAY outperforms previous methods by 35%" (40.86% on level 1, 20.13% on level 2, 6.12% on level 3 tasks). Additionally, on the SheetCopilot-20 dataset, FRIDAY passes 60% of the tasks with each task being performed only once [source]
Bespoke testing (e.g., demos): FRIDAY is essentially a demo for OS-Copilot [source].
Safety: Have safety evaluations been conducted by the developers? What were the ...: None
Publicly reported external red-teaming or comparable auditing
Personnel: Who were the red-teamers/auditors?: None
Scope, scale, access, and methods: What access did red-teamers/auditors have and...: None
Findings: What did the red-teamers/auditors conclude?: None
Ecosystem
Interoperability with other systems: What tools or integrations are available?: FRIDAY can work with other systems since it is equipped with tools for OS manipulation that include interacting with a Python code interpreter, bash terminal, mouse/keyboard control, and API calls.
Usage statistics and patterns: Are there any notable observations about usage?: The OS-Copilot github repository has 177 forks and 1.6k stars [source].
Other notes (if any): Some of the same developers have released a foundation model known as OS-ATLAS to serve as a backend for agents [source]