Runner H
Basic Information
Short description: Runner H is the flagship model for 'H Company's' Ai Agent development platform, "Studio."
Intended uses: What does the developer state that the system is intended for?: Runner H is a general-purpose agent designed to automate tasks on a filesystem and browser.
Date(s) deployed: Announced November 19, 2023. Currently in private beta [source]
Developer
System Components
Backend model(s): What model(s) are used to power the system?: Their agents are based on in-house models: H-VLM and H-LLM. Their H-VLM is a 3B parameter vision language model trained to extract information from and interact with GUIs and images. Meanwhile, they also introduce that H-LLM family of language models, one of which is 2B parameters [source].
Public model specification: Is there formal documentation on the system’s intend...: None
Description of reasoning, planning, and memory implementation: How does the syst...: Unknown
Observation space: What is the system able to observe while 'thinking'?: H-VLM is able to observe images, diagrams, and GUIs [source]
Action space/tools: What direct actions can the system take?: Runner H can interact with a browser. It is not clear what it can do beyond that.
User interface: How do users interact with the system?: Chat [source]
Development cost and compute: What is known about the development costs?: Unknown
Guardrails & Oversight
Accessibility of components
Weights: Are model parameters available?: Closed source
Data: Is data available?: Closed source
Code: Is code available?: Closed source
Documentation: Is documentation available?: Unavailable
Scaffolding: Is system scaffolding available?: Closed source
Controls and guardrails: What notable methods are used to protect against harmfu...: Unknown
Monitoring and shutdown procedures: Are there any notable methods or protocols t...: Unknown
Customer and usage restrictions: Are there know-your-customer measures or other ...: Currently in a private beta.
Evaluation
Notable benchmark evaluations (e.g., on SWE-Bench Verified): 67% on WebVoyage, 80% on Screenspot, and 65% average across several code and function calling benchmarks [source]
Bespoke testing (e.g., demos): Demos [source]
Safety: Have safety evaluations been conducted by the developers? What were the ...: None
Publicly reported external red-teaming or comparable auditing
Personnel: Who were the red-teamers/auditors?: None
Scope, scale, access, and methods: What access did red-teamers/auditors have and...: None
Findings: What did the red-teamers/auditors conclude?: None
Ecosystem
Interoperability with other systems: What tools or integrations are available?: H-VLM is designed to integrate dynamically with web user interfaces.
Usage statistics and patterns: Are there any notable observations about usage?: Unknown
Other notes (if any): --