Site architecture for a small product: shallow, linked, and honest

A crawler does not see your directories. It sees pages and the links between them, and it walks the links.

• The home page and the pages linked from the navigation are crawled most often.
• Pages three clicks deep are crawled and recrawled less.
• Pages nobody links to are found only through the sitemap, and they rank worst.
• A page in a neat folder with no inbound links is still an orphan.

So the question for every page is not where does it live, it is which pages link to it and how far is it from the home page.

The rule is not a law about user patience. It is a description of how crawl frequency falls with depth.

A hub is where a section's crawl and ranking are distributed from. A list of links is a weak hub, because it ranks for nothing and gives Google no reason to prefer it.

• An overview worth reading on its own: what the subject is, the order to read the pages in, and who they are for.
• The links to the pages of the section, with descriptive anchor text, not "read more".
• A reason to exist as a search result: a hub can rank for the broad query while its pages rank for the specific ones.
• A link back up to the home page and across to sibling hubs, through the navigation or the breadcrumb.

Three to seven hubs is right for a small product. More than that and the navigation stops being a map.

Navigation belongs in HTML anchors with href attributes. A menu built from buttons and script is not a set of links to a crawler, whatever it looks like to a visitor.

<!-- Followed: an anchor with a real destination -->
<a href="/guides/">Guides</a>

<!-- Not a link: a crawler sees a button, and no destination -->
<button onclick="router.push('/guides/')">Guides</button>

• Anchors, with href, pointing at the final URL rather than through a redirect.
• Descriptive text inside the anchor, because that text describes the destination.
• The same navigation on mobile, in the HTML, even when it is behind a toggle.
• No link that only exists after a fetch, since content that appears only after JavaScript runs is seen late by Google and not at all by the answering AI crawlers.

A footer linking a hundred pages adds nothing to the crawl and passes little to each page. It feels like internal linking and is not.

Keep the footer for what it is good at: the sections, the legal pages, the contact, and the one or two pages every visitor eventually wants. Put the real internal linking in the body of pages, where a link has context and an anchor that means something.

A breadcrumb gives every page a link up to its hub and to the home page, marks the hierarchy for Google with BreadcrumbList markup, and replaces the URL line in some results.

• One trail per page, matching the site's structure rather than the URL if the two differ.
• The current page as the last item, not a link.
• Real anchors for the parents, in the HTML, on mobile too.
• Markup that matches what is visible on the page.

/                         home: what the product is, links to the hubs
/pricing                  one click deep
/guides/                  hub: an overview, then the guides by theme
  /guides/<theme>/        theme hub: an overview, then its guides
  /guides/<page>          the guide itself, linked from its theme hub
/tools/                   hub: the free tools, grouped by job
  /tools/<tool>           one tool, linked from the hub and from the guides that need it
/glossary/                hub: A to Z
  /glossary/<term>        one term, linked from every guide that uses it
/about  /privacy  /terms  footer, one click deep

Nothing in that map is more than three clicks from the home page, every page has a hub above it, and the cross links between guides, tools and terms are written in the body where they help a reader. Two hundred pages fit in it without a single archive or paginated list.

1. 1

   Crawl from the home page

   A crawler that starts at the home page and follows links reports the depth of every page it reaches. Anything it cannot reach is an orphan, whatever your sitemap says.

2. 2

   Compare the crawl with the sitemap

   URLs in the sitemap that the crawl never reached are the pages nothing links to.

3. 3

   Sort by inbound internal links

   The pages with the fewest are the ones to link from a hub or a related section.

4. 4

   Read the Crawl stats report

   If most fetches go to parameters, redirects or dead URLs, the architecture is spending crawl on pages you do not care about.

5. 5

   Click through as a visitor on a phone

   From the home page, reach your newest guide. If you cannot in three taps, neither can a crawler in three links.