EdgeLabBot

Who is visiting

Requests from Edge Lab identify themselves with this user agent:

EdgeLabBot/0.1 (+https://edgelab.ski/bot; hello@edgelab.ski)

What it reads

• Ski and boot makers’ own public product pages, for published specifications.
• Publicly published ski reviews, to write a short summary and link back to the original.
• Video metadata through the official YouTube Data API, not by scraping.

It does not log in anywhere, does not read retailer shop pages for prices, and does not take transcripts or audio.

How it behaves

• It reads robots.txt before fetching a page and skips anything it disallows for EdgeLabBot or for all crawlers. It honours Crawl-delay.
• It prefers feeds and sitemaps to crawling, and waits between requests to the same host.
• It runs on a schedule, not continuously, and backs off when a site is slow or returns errors.
• It does not follow links to other sites beyond the pages it was pointed at.

What we store

We keep as little of a page as the job needs:

• For specifications: the numbers, the page address and the date we read it.
• For reviews: our own short summary (600 characters at most), the link, the author and date, and at most one short attributed quote. Never the full text.
• A hash (fingerprint) of each page, so we can tell when it has changed without keeping it.

Opt out

To stop EdgeLabBot, add this to your robots.txt; it takes effect on the next run:

User-agent: EdgeLabBot Disallow: /

You can also write to hello@edgelab.ski with your site address and we will stop fetching your site and remove what we hold from it. Same address for any other question about the bot.