VisualWebArena benchmark for BrowserGym
This package provides browsergym.visualwebarena
, which is an unofficial port of the VisualWebArena benchmark for BrowserGym.
Note: the original VisualWebArena codebase has been slightly adapted to ensure compatibility.
Server installation
You have two options to setup your webarena instance:
We recommend option 2 as it allows you to easily customize the ports of each webarena domain, and offers a reset functionality that allwos browsergym to trigger a full instance reset remotely.
Setup
- Install the package
pip install browsergym-visualwebarena
- Download tokenizer resources
python -c "import nltk; nltk.download('punkt_tab')"
- Setup the URLs as environment variables. The ports for each domain here should correspond to those you used when setting up your webarena instance. Note also the
VWA_
prefix which is specific to browsergym.
BASE_URL=<YOUR_SERVER_URL_HERE>
export VWA_CLASSIFIEDS="$BASE_URL:8083"
export VWA_CLASSIFIEDS_RESET_TOKEN="4b61655535e7ed388f0d40a93600254c"
export VWA_SHOPPING="$BASE_URL:8082"
export VWA_REDDIT="$BASE_URL:8080"
export VWA_WIKIPEDIA="$BASE_URL:8081"
export VWA_HOMEPAGE="$BASE_URL:80"
export VWA_FULL_RESET="$BASE_URL:7565"
export VWA_FULL_RESET=""
- Setup an OpenAI API key
export OPENAI_API_KEY=...