Guide

Site architecture for SEO and AI search

Flat versus deep structure, hub and spoke linking, and how AI crawlers read your site, with a worked example and the common mistakes.

By Sunny Patel · Updated 2026-09-20

Site architecture is the structure a site uses to organise its pages and connect them with internal links. It covers the folder pattern in a URL, the hierarchy of category and hub pages sitting above individual content, and the specific links that let a visitor or a crawler move from one page to the next. Good architecture puts the pages that matter most within easy reach. Poor architecture buries them under layers of navigation that neither a visitor nor a crawler bothers to click through in full.

Flat versus deep

A flat architecture keeps most pages within two or three hops of the homepage by design, usually through hub pages that link directly to a wide set of children instead of forcing a visitor down through several intermediate category layers first. A deep architecture nests pages under subcategory after subcategory, so a specific product or article can sit five or six hops out even on a site that is not especially large.

Flat wins for most sites because both crawl attention and visitor patience fall off sharply after three or four hops. A deep hierarchy is not wrong on principle. It becomes a problem the moment nobody adds a shortcut back down to the content that actually matters as the hierarchy grows. Read the click depth guide for the exact mechanics of how that hop count gets measured and why depth beyond four gets flagged.

Hub and spoke as the standard pattern

A hub and spoke structure puts a well linked hub page above a group of related content, and links every important spoke directly from that hub instead of only from the page that happens to sit next to it in a strict hierarchy. A resources hub linking to its twenty best guides, or a category page linking to every product worth surfacing instead of only the five newest, both apply the same pattern. The hub absorbs the internal link equity a site has to distribute and passes a share of it straight down to whichever spokes it links to.

The pattern works because it decouples the folder hierarchy from the click path. A page can live several folders deep in the URL and still sit one hop from the homepage, as long as a hub links to it directly. That gap between where a page lives in the URL and how it is actually reached is the part most site owners get wrong.

URL folders are not the same thing as click depth

A URL such as example.com/shop/category/subcategory/product looks four levels deep, and many site owners assume the crawler experiences it that way too. It does not. Click depth is set by the shortest chain of real links a crawler can follow to reach that URL, counted from the homepage, not by how many slashes sit in the address bar. A product page with that exact URL can sit at click depth one if the homepage or a hub links to it directly, or at click depth six if the only path in runs through every intermediate category page in order.

Confusing the two leads to a common mistake: a site owner reorganises URLs to look flatter without changing a single link, and the crawl path stays exactly as deep as it was before. Fixing click depth means adding links, not renaming folders.

Internal link ownership

Every page on a site should have an owner responsible for its inbound links: the hub or template that is meant to keep it reachable. A related posts block, a category listing, and a footer section are all examples of a page taking ownership of the links pointing at a group of other pages. Ownership breaks silently during a redesign when a template changes and nobody checks what it used to link to. A related posts block quietly redesigned to show three items instead of ten can orphan every page that used to depend on it, and nothing about the change looks wrong until a crawl shows the gap. The internal links for AI guide covers how that graph gets built and audited in more detail.

How AI crawlers consume structure

An AI crawler moves through a site by reading the raw HTML it receives and following the real <a href> elements it finds there. It does not execute JavaScript, so a navigation menu built entirely from client-side script with no underlying href stays invisible to it, and a load more button that only appends links after a click event never gets triggered. Google has decades of accumulated URL history from sitemaps, backlinks, and prior crawls to fall back on when a site's own linking is weak. Most AI crawlers built in the last few years do not have that depth of history on a typical domain yet, which pushes them to depend more heavily on whatever link graph the site presents on the visit itself. Good site architecture matters more to a crawler with no fallback than to one that has built up decades of alternative discovery paths.

Mapping your own architecture

A crawl-based tool shows the real structure instead of the one implied by a URL pattern or a sitemap file. sitemap.digital builds the link graph from a live crawl and renders it three ways from the same data: a Tree view for the hierarchy a visitor experiences, a Flow view drawing the site's URL-hierarchy edges at rest and revealing a node's contextual links only when you hover it, and a List view sorted by click depth or inbound links for scanning the whole site at once. A free scan covers 20 pages, or 50 with an email unlock, and a site whose sitemap lists 150 pages or fewer is crawled to completion. Tree and List show every crawled page. Flow shows the 36 most connected pages on a large site, since a full graph past that size stops being readable as a diagram, and the underlying edge list is separately capped at 5,000 edges. They are three ways of looking at one crawl result, so switching between them costs nothing beyond a click. The visual sitemap generator is the same engine as a standalone tool for sites that only need this one check.

Worked example

The example below uses the placeholder domain example.com and stays under the 150-page sitemap ceiling that gets a site crawled to completion, so every verdict in it is a real structural read instead of a partial sample. Say a support section grows over two years from twenty articles to eighty, added one at a time under existing subcategories. Nobody updates the support hub's template to keep pace, so the hub still links to the same fifteen articles it always did. The other sixty-five articles are reachable only by drilling through a subcategory, then a topic page, then finally the article itself: three hops past the hub, five hops from the homepage.

A complete scan of example.com flags most of that new content as deep pages, and a handful of the oldest articles, whose only inbound links sat inside a related-articles block later removed during a template change, show up as orphans with no inbound link at all. The fix does not touch the eighty articles or their URLs. It adds a browsable index at the hub linking to every subcategory's most recent and most visited articles directly, and restores a related-articles block on each article page. A rescan afterwards shows the bulk of the section moving from depth five to depth two, and the orphaned articles clearing once their related-articles links come back online.

Common mistakes

• Treating URL folder depth as a proxy for click depth, then reorganising folders instead of adding links.
• Letting a hub template go stale as content grows, so it keeps linking to the same fixed set of children indefinitely.
• Building primary navigation entirely in client-side JavaScript with no href fallback, which hides it from any crawler that does not execute scripts.
• Redesigning a related-content block without checking which pages depended on it for their only inbound link.
• Relying on a sitemap.xml file to make up for a weak internal link graph. A sitemap helps a search engine find a URL. It does not build the link path an AI crawler actually follows, and does not substitute for a real link in from a reachable page.

See the full site audit for how structure findings like these combine with content and access checks into one ranked list of fixes.

Frequently asked questions