Claude gains computer use

Anthropic releases a model that operates a computer by looking at the screen, moving a cursor and typing.

What happened and why it matters

Computer use let a general model drive arbitrary software rather than bespoke APIs. Early performance was slow and error-prone (~15% on OSWorld versus ~72% for humans), but it established the interface for general digital agents.

Sources

• Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku ↗ Anthropic · 22 Oct 2024