.env.example
Set these on your host before deploying. Each platform has an “Environment Variables” section in the deploy flow. (showing first 20, more exist in .env.example)
SPARK_MODEL_PATH
spark-moe runtime env (GX10)
SPARK_MEM_AVAILABLE_GIB
SPARK_OS_RESERVE_GIB
SPARK_RESIDENT_GIB
SPARK_SCRATCH_GIB
SPARK_KV_DTYPE
SPARK_MAX_CONTEXT
SPARK_KV_BUDGET_GIB
SPARK_MAX_NUM_SEQS
SPARK_QUALITY_TEMP
SPARK_BENCH_CONTEXT
SPARK_SGLANG_PARITY
SPARK_MIN_HOT_TIER_GIB
SPARK_HOT_TIER
SPARK_HOT_TIER_GIB
SPARK_EXPERT_BYTES_ESTIMATE
SPARK_KV_BLOCK_SIZE
SPARK_API_PORT
SPARK_GEMM_CUDA
SPARK_GEMM_MARLIN_TC
brew install flyctl && fly launch --from https://github.com/alfons-fhl/spark-moe
curl -L https://fly.io/install.sh | sh && fly launch --from https://github.com/alfons-fhl/spark-moe
iwr https://fly.io/install.ps1 -useb | iex; fly launch --from https://github.com/alfons-fhl/spark-moe