WebRL
Basic Information
Website: https://arxiv.org/abs/2411.02337v1
Short description: WebRL is a "reinforcement learning framework designed to train high-performance web agents using open LLMs." [source]
Intended uses: What does the developer state that the system is intended for?: Intended to be used to train agents that can accomplish tasks (described in natural language) on the internet through a web browser [source]
Date(s) deployed: Paper first put on arXiv on November 4th 2024 [source]
Developer
Website: https://github.com/THUDM/WebRL
Legal name: Tsinghua University (et al.) [source]
Entity type: Academic Institution and corporation.
Country (location of developer or first author's first affiliation): China [source]
Safety policies: What safety and/or responsibility policies are in place?: None
System Components
Backend model(s): What model(s) are used to power the system?: The authors release three agents trained using WebRL based on Llama-3-8b, Llama-3-70b, and GLM-4-9b [source]
Public model specification: Is there formal documentation on the system’s intend...: None
Description of reasoning, planning, and memory implementation: How does the syst...: Reinforcement learning is used to train the model to plan internally. That is there are no separate specialized planning modules [source]
Observation space: What is the system able to observe while 'thinking'?: HTML content of the current web page along with the history of previous actions [source]
Action space/tools: What direct actions can the system take?: In principle, WebRL can be used to train agents using any natural languages based action space. The authors use WebArena to train their released models, an environment with an "action space that emulates the keyboard and mouse operations available on web page" [source].
User interface: How do users interact with the system?: There is no publicly available UI. Users can download the models released by the authors and personally host them. The agents are designed to be interacted with using natural language descriptions of tasks, just like a regular chatbot.
Development cost and compute: What is known about the development costs?: Unknown
Guardrails & Oversight
Accessibility of components
Weights: Are model parameters available?: Authors release three open source agents created using the WebRL framework [source].
Data: Is data available?: Open source [source].
Code: Is code available?: Open source [source]
Documentation: Is documentation available?: Basic documentation on Github [source] and paper [source]
Scaffolding: Is system scaffolding available?: Public (see open source code).
Controls and guardrails: What notable methods are used to protect against harmfu...: None
Monitoring and shutdown procedures: Are there any notable methods or protocols t...: None
Customer and usage restrictions: Are there know-your-customer measures or other ...: None
Evaluation
Notable benchmark evaluations (e.g., on SWE-Bench Verified): Agent based on Llama-3-70b achieves 49.1% success rate on WebArena Lite [source].
Bespoke testing (e.g., demos): Paper contains various ablation studies [source].
Safety: Have safety evaluations been conducted by the developers? What were the ...: None
Publicly reported external red-teaming or comparable auditing
Personnel: Who were the red-teamers/auditors?: None
Scope, scale, access, and methods: What access did red-teamers/auditors have and...: None
Findings: What did the red-teamers/auditors conclude?: None