How to use Claude computer use demo tool

Here's a quick guide to setting up and using the Claude computer use demo

By Shivam Malani 2 min read
How to use Claude computer use demo tool

Claude's computer use tool is an experimental feature introduced by Anthropic that allows the AI assistant to interact with a computer interface much like a human would. By integrating this tool, developers can instruct Claude to perform tasks by simulating mouse movements, clicks, and keyboard inputs, enabling automation of complex, multi-step processes involving standard software applications.

Prerequisites

Before getting started, ensure that you have the following:

  • An Anthropic API key with access to Claude 3.5 Sonnet v2.
  • Docker installed on your machine.
  • Basic understanding of command-line operations.

Setting up Claude's computer use demo

Follow these steps to set up and use the Claude computer use demo:

  1. Clone the computer-use-demo repository from Anthropic's GitHub:
    git clone https://github.com/anthropics/anthropic-quickstarts.git
  2. Navigate to the computer-use-demo directory:
    cd anthropic-quickstarts/computer-use-demo
  3. Set your Anthropic API key as an environment variable:
    export ANTHROPIC_API_KEY=your_api_key_here
  4. Run the Docker command to start the demo:
docker run -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY -v $HOME/.anthropic:/home/computeruse/.anthropic -p 5900:5900 -p 8501:8501 -p 6080:6080 -p 8080:8080 -it ghcr.io/anthropics/anthropic-quickstarts:computer-use-demo-latest
  1. If you're on an x86 architecture, add the platform flag to the Docker command:
    --platform linux/amd64
  2. Once the Docker container is running, open your web browser and navigate to:
    http://localhost:8080/
  3. You should see the computer use tool interface, which includes a chat panel and a virtual desktop environment.

Using the Computer Use Tool

With the setup complete, you can now instruct Claude to perform tasks on the virtual desktop.

  1. In the chat panel, input a command for Claude to execute. For example: Open Firefox and navigate to "google.com". Search for "Anthropic press release". Extract the results as clean markdown.
  2. Claude will interpret your instruction and simulate the necessary mouse movements and keyboard inputs to complete the task.
  3. Monitor the virtual desktop to observe Claude's actions in real-time.
  4. Once the task is completed, Claude will provide the output in the chat panel.
0:00
/2:05

By following these steps, you try out the the power of Claude's computer use tool.

Artificial Intelligence

Shivam Malani