coliff/docsearch-scraper

Python

20 environment variables found in .env.example

Set these on your host before deploying. Each platform has an “Environment Variables” section in the deploy flow.

APPLICATION_ID
API_KEY secret
CF_ACCESS_CLIENT_ID WARNING! Please be aware that the scraper sends auth headers to every scraped site, so use `allowed_domains` to adjust the scope…
CF_ACCESS_CLIENT_SECRET secret
IAP_AUTH_CLIENT_ID WARNING! Please be aware that the scraper sends auth headers to every scraped site, so use `allowed_domains` to adjust the scope…
IAP_AUTH_SERVICE_ACCOUNT_JSON
CHROMEDRIVER_PATH
APPLICATION_ID_PROD
API_KEY_PROD secret
MONITORING_API_KEY secret
HELPSCOUT_APP_ID See https://developer.helpscout.com/mailbox-api/overview/authentication/
HELPSCOUT_APP_SECRET secret
NB_HITS_MAX
SLACK_HOOK See https://algolia.slack.com/services/B0MUUMGKH
PUBLIC_CONFIG_FOLDER
PRIVATE_CONFIG_FOLDER
BASE_INTERNAL_ENDPOINT See with Internal Dashboard team
INTERNAL_API_AUTH
DEPLOY_KEY secret
APPLICATION_ID_PROD_INTERNAL

Best match Fly.io Anything Dockerized — requires the Fly CLI.

macOS

brew install flyctl && fly launch --from https://github.com/coliff/docsearch-scraper

Linux

curl -L https://fly.io/install.sh | sh && fly launch --from https://github.com/coliff/docsearch-scraper

Windows (PowerShell)

iwr https://fly.io/install.ps1 -useb | iex; fly launch --from https://github.com/coliff/docsearch-scraper

Or deploy to a different host