openbmb/acebench

Python

17 environment variables found in .env.example

Set these on your host before deploying. Each platform has an “Environment Variables” section in the deploy flow.

DOCKER_IMAGE
GATEWAY_PORT
TMP_WORKSPACE
TASKS_SUBDIR
OUTPUT_SUBDIR
DEFAULT_MODEL
DEFAULT_PARALLEL
OPENROUTER_BASE_URL OpenRouter is used by default. You can also use any other OpenAI-compatible provider (e.g. DeepSeek, SiliconFlow): set OPENROUTER_BASE_URL…
OPENROUTER_API_KEY secret
JUDGE_BASE_URL
JUDGE_API_KEY secret
JUDGE_MODEL
MY_PROXY_API_KEY secret Using a Custom Model Endpoint
SEARCH_PROVIDER Lobster profile env keys (add values here for skills that need them)
SERPER_API_KEY secret
GEMINI_API_KEY secret
BRAVE_API_KEY secret

Best match Fly.io Anything Dockerized — requires the Fly CLI.

macOS

brew install flyctl && fly launch --from https://github.com/openbmb/acebench

Linux

curl -L https://fly.io/install.sh | sh && fly launch --from https://github.com/openbmb/acebench

Windows (PowerShell)

iwr https://fly.io/install.ps1 -useb | iex; fly launch --from https://github.com/openbmb/acebench

Or deploy to a different host