turboderp-org/exllamav2

Python

Best match Fly.io Anything Dockerized — requires the Fly CLI.

macOS

brew install flyctl && fly launch --from https://github.com/turboderp-org/exllamav2

Linux

curl -L https://fly.io/install.sh | sh && fly launch --from https://github.com/turboderp-org/exllamav2

Windows (PowerShell)

iwr https://fly.io/install.ps1 -useb | iex; fly launch --from https://github.com/turboderp-org/exllamav2

Or deploy to a different host