AutoGLM
Basic Information
Website: https://arxiv.org/abs/2411.00820v1
Short description: AutoGLM is an agent "for autonomous control of digital devices through Graphical User Interfaces (GUIs)". It focuses particularly on web browser and phone GUIs [source]
Intended uses: What does the developer state that the system is intended for?: The system is for operating GUIs, mainly across web browsing and Android environments.
Date(s) deployed: Arxiv submitted on October 28, 2023 [[source]](https://arxiv.org/abs/2411.00820v1)
Developer
Website: https://perma.cc/497J-2DZL
Legal name: Zhipu AI [source]
Entity type: Unknown
Country (location of developer or first author's first affiliation): China [source]
Safety policies: What safety and/or responsibility policies are in place?: Unknown
System Components
Backend model(s): What model(s) are used to power the system?: An internal version of ChatGLM3 (not the open source ChatGLM3) [source]
Public model specification: Is there formal documentation on the system’s intend...: None
Description of reasoning, planning, and memory implementation: How does the syst...: Unknown
Observation space: What is the system able to observe while 'thinking'?: The screen information displayed on a user's device.
Action space/tools: What direct actions can the system take?: Touch events (as on phone) and mouse/keyboard events (as in a web browser)
User interface: How do users interact with the system?: A Chrome extension where users write prompts to perform operations on the browser [source]. On a phone, users can provide prompts via text or audio to complete tasks [source]
Development cost and compute: What is known about the development costs?: Unknown
Guardrails & Oversight
Accessibility of components
Weights: Are model parameters available?: Unknown
Data: Is data available?: They use data from Android-Lab [source]
Code: Is code available?: Closed source
Documentation: Is documentation available?: Unavailable
Scaffolding: Is system scaffolding available?: Closed source
Controls and guardrails: What notable methods are used to protect against harmfu...: Unknown
Monitoring and shutdown procedures: Are there any notable methods or protocols t...: Unknown
Customer and usage restrictions: Are there know-your-customer measures or other ...: None
Evaluation
Notable benchmark evaluations (e.g., on SWE-Bench Verified): VAB-WebArena-Lite (55.2%), OpenTable (96.2%), AndroidLab (VAB-Mobile) (36.2%) [source]
Bespoke testing (e.g., demos): Videos [source], 89.7% success rate "on common tasks in popular Chinese APPs" [source]
Safety: Have safety evaluations been conducted by the developers? What were the ...: None
Publicly reported external red-teaming or comparable auditing
Personnel: Who were the red-teamers/auditors?: None
Scope, scale, access, and methods: What access did red-teamers/auditors have and...: None
Findings: What did the red-teamers/auditors conclude?: None
Ecosystem
Interoperability with other systems: What tools or integrations are available?: Chrome, Android [source], and browsers via chromium kernels
Usage statistics and patterns: Are there any notable observations about usage?: None
Other notes (if any): --