๐Ÿ“š General & Other

Sampling 50 Pages From a Crawl Without Fooling Yourself

A forty-thousand-page site needs a manual quality review. You can look at fifty. Which fifty decides what you conclude.

A forty-thousand-page site needs a manual quality review. You can look at fifty. Which fifty decides what you conclude.

The biased shortcut

Taking the first fifty rows of a crawl export samples whatever the crawler found first โ€” usually the top-level navigation, which is the best-maintained part of any site. Review those and the site looks excellent.

Random beats convenient

Shuffle, then take fifty. The sample now represents the site, including the neglected sections that never appear near the top of a crawl and are where the real problems live.

Sample within segments for a fair picture

Better still, sample separately from each template or section. A simple random sample of a site that is ninety per cent product pages will be almost entirely product pages, and tell you nothing about the other templates.

Try it: Shuffle Lines Randomly on SeoWolf's Notepad.