AutoGLM
Basic information
Website: https://arxiv.org/abs/2411.00820v1
Short description: AutoGLM is an agent “for autonomous control of digital devices through Graphical User Interfaces (GUIs)”. It focuses particularly on web browser and phone GUIs [source]
Intended uses: What does the developer say it’s for? The system is for operating GUIs, mainly across web browsing and Android environments.
Date(s) deployed: Arxiv submitted on October 28, 2023 [source]
Developer
Website: https://perma.cc/497J-2DZL
Legal name: Zhipu AI [source]
Entity type: Unknown
Country (location of developer or first author’s first affiliation): China [source]
Safety policies: What safety and/or responsibility policies are in place? Unknown
System components
Backend model: What model(s) are used to power the system? An internal version of ChatGLM3 (not the open source ChatGLM3) [source]
Publicly available model specification: Is there formal documentation on the system’s intended uses and how it is designed to behave in them? None
Reasoning, planning, and memory implementation: How does the system ‘think’? Unknown
Observation space: What is the system able to observe while ‘thinking’? The screen information displayed on a user’s device.
Action space/tools: What direct actions can the system take? Touch events (as on phone) and mouse/keyboard events (as in a web browser)
User interface: How do users interact with the system? A Chrome extension where users write prompts to perform operations on the browser [source]. On a phone, users can provide prompts via text or audio to complete tasks [source]
Development cost and compute: What is known about the development costs? Unknown
Guardrails and oversight
Accessibility of components:
- Weights: Are model parameters available? Unknown
- Data: Is data available? They use data from Android-Lab [source]
- Code: Is code available? Closed source
- Scaffolding: Is system scaffolding available? Closed source
- Documentation: Is documentation available? Unavailable
Controls and guardrails: What notable methods are used to protect against harmful actions? Unknown
Customer and usage restrictions: Are there know-your-customer measures or other restrictions on customers? None
Monitoring and shutdown procedures: Are there any notable methods or protocols that allow for the system to be shut down if it is observed to behave harmfully? Unknown
Evaluation
Notable benchmark evaluations: VAB-WebArena-Lite (55.2%), OpenTable (96.2%), AndroidLab (VAB-Mobile) (36.2%) [source]
Bespoke testing: Videos [source], 89.7% success rate “on common tasks in popular Chinese APPs” [source]
Safety: Have safety evaluations been conducted by the developers? What were the results? None
Publicly reported external red-teaming or comparable auditing:
- Personnel: Who were the red-teamers/auditors? None
- Scope, scale, access, and methods: What access did red-teamers/auditors have and what actions did they take? None
- Findings: What did the red-teamers/auditors conclude? None
Ecosystem information
Interoperability with other systems: What tools or integrations are available? Chrome, Android [source], and browsers via chromium kernels
Usage statistics and patterns: Are there any notable observations about usage? None
Additional notes
None