Website Crawler

Website URL

Site Name

Maximum pages (safety limit)

Maximum depth (0 = unlimited)

Delay between requests (ms)

Treat query-string URLs as separate pages

Include linked non-HTML resources in results

Respect robots.txt rules when accessible

0Discovered

0Analyzed

0Queued

0Failed

0Max depth

The crawler reads HTML directly into browser memory using fetch(); it does not require downloading pages. Run this tool from your own site for same-origin access, or let the crawler fallback to CORS proxies for cross-origin URLs. Link crawling cannot discover orphan pages that are not linked; sitemap URLs are also used as discovery seeds when available.

SEO Defaults & Page Metadata

Active Page Selection (Page-Specific Override) Page title Meta description

Author

Publisher

Canonical URL

Favicon URL

Open Graph image URL (Recommended: 1200x630)

X / Twitter image URL (optional)

Theme color

Robots meta policy

Structured Data

Schema type

Site Language

Schema description (optional)

Organization Logo URL

Social Profiles (sameAs URLs, one per line)

Include WebSite SearchAction

Search URL template (use {search_term_string})

Social Metadata

Open Graph title

Open Graph description

X / Twitter card style

X / Twitter handle

Crawl Results

URL	HTTP	Depth	Title	Description	Canonical	Index	H1	Links

Sitemap Pages

Path	Last modified	Change frequency	Priority

Previews

Open Graph / Facebook / LinkedIn Post Preview

X / Twitter Card Post Preview

Google Search Snippet Preview

Generated Files

robots.txt

sitemap.xml

seo-head.html

structured-data.json