Does a small site need to worry about crawl budget?

No. Crawl budget is a concern for sites with many thousands of URLs. On a site with a few hundred pages Google fetches everything it finds; what limits it is internal links and the sitemap, not budget.

Is structured data a ranking factor?

No. It makes a page eligible for a rich result and helps machines read what the page is about. A page with valid markup and nothing to say ranks for nothing; a good page without markup still ranks, and gets stars or breadcrumbs when the markup is added.

What is the one technical mistake that costs the most?

The same content at two addresses with no canonical: Google picks one, sometimes the wrong one, and the links and the clicks split between them. One canonical per page, one redirect for every other address, one address in the sitemap and the internal links.

Should I block AI crawlers in robots.txt?

Block the training crawlers if you want, they do not affect search. Never block the answering ones (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Bingbot, Googlebot): that is what makes a site invisible to assistants, and blocking a training crawler does not remove pages already collected.

Technical SEO for a small site, in the order that pays

Technical SEO is the machinery a search engine meets before it reads a word: whether a page can be crawled, which URL is the one, what the markup declares, what a move breaks. On a small site it is a short list, and most of it is done once. These guides take it one question at a time.

By Théophile Louvart, founder of Porteur · Updated 14 September 2026 · Markdown

What technical SEO covers on a product site

Three things. Whether a search engine can reach and read the page: robots.txt, the status code, the rendering, the speed. Which URL is the one for a piece of content: canonicals, redirects, www against non-www, trailing slashes, parameters. And what the page declares about itself in a form a machine reads: the title tag, the meta description, headings, structured data, hreflang, breadcrumbs.

What it does not cover is whether the page deserves to rank. A technically perfect page about nothing ranks for nothing. The technical side removes the reasons a good page is not shown; the content side is the guides on pages and content, and the reason it is shown at all.

On a site with a few hundred pages, crawl budget is not a concern, most statuses in Search Console are Google agreeing with you, and the technical work is a checklist done once, then a check after each change. Speed has its own theme here, because it is measured differently and fixed in different files.

One URL per page, and the ways that goes wrong

Most technical faults on a small site are the same page reachable at two addresses. http and https, www and the apex, with and without a trailing slash, with a tracking parameter, a print version, a paginated list, a filtered list. Google folds the duplicates into one canonical, sometimes the one you meant, sometimes not, and reports its choice under the canonical statuses of the Page indexing report.

The fix is always the same three moves: one canonical tag on every page pointing at the address you chose; a 301 from every other address that has no reason to exist; internal links and the sitemap that use only the chosen address. The canonical guide, the www and trailing slash guides, the redirect map and the status codes guide are that one rule applied to each case.

A move, of domain, of URL structure, of protocol, is the same rule at scale: a one-to-one redirect map, every old URL to the one new URL that replaces it, tested before the switch, kept for as long as possible, with the old sitemap left up until Google has recrawled the redirects. The migration guides walk each move in order, and the one lesson they share is to move one thing at a time so a drop can be attributed.

What the page declares, and what it earns

The title tag is the headline in the results page and the most rewritten element on the web: about 60 characters, the subject first, the site name last. The meta description earns the click and is rewritten by Google when it does not match the query. Headings outline the page for crawlers and screen readers alike. The on-page checklist holds the twelve elements in one list.

Structured data, as JSON-LD, makes a page eligible for a rich result and tells machines what the page is about; it is not a ranking factor. Article, Product, SoftwareApplication, Organization, BreadcrumbList and Review each have the fields Google documents as required and recommended, and the Rich Results Test says what a page is eligible for. Two types earn nothing any more: FAQ rich results are limited to government and health sites since 2023, and HowTo rich results were removed the same year; the markup still helps assistants read the page.

robots.txt decides what may be fetched, and since 2023 which AI crawlers may read the site for training and which for answering. The two are different tokens per vendor, and blocking the answering ones is what makes a site invisible to assistants. The guide names each token; the free tools test any path against any crawler and write the file.

The order to read the guides

First, the two checklists: the technical checklist for a small site, then the on-page checklist. They are the map; every other guide is one square of it.
Then the one-URL rule: canonical tags, the canonical status in Search Console, www against non-www, the trailing slash, the status codes, orphan pages and internal linking.
Then what the page declares: title tag length, meta description length, breadcrumbs, JSON-LD, the Rich Results Test, and one guide per structured data type you use.
Then robots.txt for AI crawlers, hreflang if the site has more than one language, subdomain against subdirectory when you add a blog or docs.
Only when you move something: the migration checklist, the domain change, the redirect map, changing the URL structure, HTTP to HTTPS. And the framework guides (Next.js, Shopify) for what the platform does and leaves to you.

How to check your own site in an afternoon

Run the on-page checker on the home page, the pricing page and one guide: title and description lengths, one h1, the canonical, indexability, the outline, the text present without JavaScript. Run the redirect checker on http://, on www, and on a URL with a trailing slash to see how many hops each takes; one is the answer. Run the sitemap checker to see that every listed URL answers 200 and that the file is declared in robots.txt. Run the structured data checker on a page of each kind you publish, and the robots.txt tester with the AI crawlers picked.

Then open Search Console's Page indexing report and read the reasons. Each one that names a page you wanted in the index has a guide here. That afternoon is the whole of technical SEO for most product sites; what remains is not to break it at the next deploy.

Where the free check fits

Each free tool checks one thing on one page. The check reads the whole site in one pass: the pages that redirect, the ones that answer nothing, the canonicals that disagree, the markup on every kind of page, what robots.txt does to each AI crawler, and the home page's speed on a phone, and writes three findings whole. The report, bought once, does the same for every page and every rival and says what to change first, with the file behind each page.

The guides, in the order to read them

61 guides on the theme, each on one question, each ending in your own figures.

Technical SEO checklist for a small siteGuide
Run these 20 technical SEO checks, in order. Each shows how to check it free and what fixed looks like for a site under 1,000 pages.
On-page SEO checklist: the twelve things on the pageGuide
The twelve page elements that move clicks and rankings, with examples you can copy to your own product pages today.
Canonical tags: what they do and the mistakes that cost rankingsGuide
What a canonical tag does, when to use one, the mistakes that cost rankings, and how to check and fix canonicals on a small site.
Alternate page with proper canonical tag: what Search Console meansGuide
What this Search Console status means, when to ignore it, when it hides the wrong page, and the exact checks and fixes to set the right canonical.
www or non-www: pick one, redirect the otherGuide
Pick www or non-www, there is no SEO gain either way. 301 redirect every path to the chosen host, set canonicals, and verify a Domain property.
Trailing slash: two URLs for one page unless you decideGuide
Google sees /page and /page/ as different. Pick one form, redirect the other, and keep links and your sitemap consistent. Here is how to do it.
HTTP status codes for SEO: what each one tells GoogleGuide
See how 2xx, 3xx, 4xx and 5xx codes change crawling and indexing, when to use 301 vs 302, handle soft 404s, 503 for maintenance, and how to check URLs.
Orphan pages: finding the pages nothing links toGuide
Find orphan pages on your site, why they happen, how to compare crawl, sitemap and Search Console, and what to do: link, merge, redirect or remove.
Internal linking for a small site: which pages link to whichGuide
Decide which pages rank. Build hubs, write clear anchors, fix orphans, and crawl your site to map links you control.
Title tag length: what fits, what Google rewrites, and how to write oneGuide
About 60 characters is a safe line. Learn pixels versus characters, why Google rewrites titles, and how to write titles that earn clicks.
Meta description length, with examples that earn the clickGuide
Keep meta descriptions around 155 characters on desktop and shorter on phones. Here is how to write snippets that earn clicks, with clear examples.
Breadcrumbs: the navigation, the markup, and what Google showsGuide
Add breadcrumb navigation that helps visitors, improves internal linking and gives Google a clean trail to show in results, with JSON-LD examples.
JSON-LD: how to write it, where to put it, how to test itGuide
See why Google recommends JSON-LD, where to place it, how to use one @graph, how to test with Google and schema.org, and what to fix.
The Rich Results Test: what it checks and what it does notGuide
Run Google’s Rich Results Test, read eligibility, warnings and errors, know its limits, and when to use schema.org’s validator and Search Console.
Organization schema: the fields that feed your knowledge panelGuide
Set up Organization schema to feed your logo, social profiles and knowledge panel. See the fields, a JSON-LD example, where to place it, and how to test.
Article schema: headline, images, dates and author, done rightGuide
Use Article, BlogPosting or NewsArticle schema the way Google expects. Get headlines, images, dates and the author right, and test it before release.
SoftwareApplication schema for a SaaS: an example that validatesGuide
Add SoftwareApplication schema that passes Google’s Rich Results Test. See the required fields, a clean SaaS example, and how to test and ship it.
Product schema: the fields Google reads and the ones it requiresGuide
Set up Product schema that earns snippets and merchant listings. See the required fields, the recommended ones and a full JSON-LD example.
Review schema: what is allowed, what is shown, what is self-servingGuide
What review schema is, when you can show stars, the self-serving rule, and how to mark up Product or SoftwareApplication reviews that validate.
FAQ schema in 2026: valid, readable by assistants, no rich resultGuide
FAQ rich results are gone for most sites, but valid FAQ schema still helps assistants read your answers. Here is the shape, rules and tests.
HowTo schema: what happened to it and what to do insteadGuide
Google removed HowTo rich results in 2023. Keep steps and headings, mark up as Article, and test. Here is what to keep and what to drop.
robots.txt for AI crawlers: GPTBot, ClaudeBot, PerplexityBot and what to allowGuide
Decide which AI crawlers to allow in robots.txt, why it matters, and copy‑paste examples for GPTBot, ClaudeBot, PerplexityBot, Google‑Extended and more.
hreflang implementation: head, sitemap or header, and the checksGuide
Add hreflang the right way, avoid errors and verify it works. Examples for head, sitemap and HTTP header, the rules, checks and what it won’t do.
Subdomain vs subdirectory: what Google says and what to chooseGuide
Google says both can work. Here is how to choose for blogs, docs, languages and new products, and how to move with clean redirects.
Faceted navigation SEO: which filters to index and how to hide the restGuide
Pick which filters should rank, hide the rest, and keep crawl under control. A practical plan for product sites with examples you can ship.
Website migration SEO checklist: before, during, afterGuide
Run a safe website migration: crawl and export, build and test one-to-one redirects, switch cleanly, submit sitemaps, use Search Console, and watch the data.
Changing your domain without losing search trafficGuide
Change domains without tanking your rankings. Map 301s one to one, use Change of Address, update canonicals and hreflang, and keep redirects for a year.
Build a redirect map: one old URL, one new URL, testedGuide
A practical redirect map: where to find old URLs, how to match them, store the table, apply 301s, and test every row for one hop and a 200.
Changing your URL structure: when it is worth it and how to do itGuide
When to change URLs for SEO, how to plan the map and redirects, how to keep equity, and what to expect in Search Console after the switch.
Moving from HTTP to HTTPS: the checklist that keeps rankingsGuide
Move from HTTP to HTTPS without losing rankings. Use one-hop 301s, update canonicals and sitemaps, fix mixed content, and verify the HTTPS property.
Next.js SEO: what the framework does for you and what it does notGuide
What Next.js gives you for SEO and what still needs your work: metadata, sitemap, robots, rendering, redirects, URLs, images, fonts and links.
Shopify SEO: what the platform fixes, what it leaves to youGuide
What Shopify handles for SEO, what you must change, and a practical checklist for a new store: URLs, canonicals, data, speed, and Merchant Center.
A second product: same domain, subdomain, or a new site?Guide
The three ways to launch a second product in search, what each costs, when a separate domain is right, and how to structure one site that holds both.
An SEO audit checklist for a small site: forty checks in orderGuide
Run these forty SEO checks in order on a small site. The tool to use, what a fail looks like, and what to fix next, in plain steps.
Astro SEO: fast by default, and the layout work that remainsGuide
Astro ships fast pages. This guide shows the SEO work you still need: metadata in a layout, canonicals, a sitemap, content collections and a clean pattern.
The best SEO Chrome extensions: what each shows in one clickGuide
The Chrome extensions that show titles, headings, canonicals, robots, schema and Web Vitals in one click, plus volumes on SERPs, and which two to install.
The best website crawlers for SEO: desktop, cloud, and free tiersGuide
Choose the right SEO crawler for a small site or a growing one. Screaming Frog, Sitebulb, Lumar, Oncrawl, and the suites’ audits, plus a first crawl plan.
Canonical tags in practice: the six situations you will meetGuide
Self-referencing canonicals, parameters, pagination, syndication and cross-domain, with the code for each and what to do when Google picks another URL.
Crawl waste on a small site: the fetches your real pages never getGuide
Crawl budget is not a small site's problem. Crawl waste is: parameters, filters, old redirects and dead URLs taking the fetches your pages need.
Docusaurus SEO: docs that rank and versions that do not fightGuide
Set up Docusaurus SEO the right way: front matter titles and descriptions, sitemap config, version canonicals, i18n, search, and llms.txt for docs.
Framer SEO: the settings, the staging trap, and what to checkGuide
Set up Framer SEO the right way: page settings, sitemap and robots, redirects, CMS fields, staging that stays noindex, speed, and Search Console checks.
Gatsby SEO: static HTML, the Head API, and what to watchGuide
Set up Gatsby SEO the right way: Head API for titles and canonicals, gatsby-plugin-sitemap, optimised images, and how to handle hydration and migrations.
Ghost SEO: a publishing platform that does most of it for youGuide
What Ghost does for SEO by default, what to tweak in themes and routes, how to use tags and members, and how to run a product site on Ghost.
Google ranking factors: what is confirmed, what is folkloreGuide
What Google has confirmed, what is plausible, and what is folklore. Then, the concrete fixes a product site can ship that move rankings.
How Google Search works: crawling, indexing, ranking, in plain wordsGuide
Crawling, indexing, ranking. What happens to one of your pages, what you can influence, how long it takes, and how to check it in Search Console.
How to add structured data to a site, whatever it is built withGuide
Where the JSON-LD goes on WordPress, Shopify, Webflow or a framework, how to generate it, how to test it, and the rules that keep it valid.
How to find and fix broken links on your siteGuide
Broken links come in three kinds and only one of them costs you traffic. Here is how to find each, what to fix, and the routine that keeps a site clean.
How to fix duplicate content, case by caseGuide
Google picks one URL and folds the rest. Here are the eight sources of duplicates on a small site and the right fix for each, with the checks.
Hugo SEO: what the templates must do, since nothing is automaticGuide
Hugo does nothing automatic for SEO. This guide shows the head partial, sitemap, robots, canonicals, taxonomies and images you must template.
JavaScript rendering: what Google sees, what the AI crawlers do notGuide
Google renders JavaScript in a second pass and the AI crawlers do not render it at all. Here is what breaks, how to test it, and what to change.
Nuxt SEO: useSeoMeta, the SEO modules, and the client-rendering trapGuide
Ship Nuxt pages that Google can crawl: server-rendered HTML, correct meta, a clean sitemap and robots, schema, fast images, and no client-only content.
SEO through a rebrand: the name, the domain, the pagesGuide
What a rebrand costs in search, the order to do it in, the brand pages to ship first, and how long the old name keeps sending people to you.
The SEO mistakes small product sites actually makeGuide
Twenty mistakes found on real small sites, grouped, each with the check that finds it in a minute and the fix that takes an afternoon.
Site architecture for a small product: shallow, linked, and honestGuide
A site is a graph of links, not a folder tree. Hubs that are real pages, navigation a crawler can follow, and a worked map for two hundred pages.
Squarespace SEO: the settings that exist, the ones that do notGuide
Set titles, descriptions, slugs and redirects in Squarespace, work around the non‑editable robots.txt, and use its sitemap, canonicals and schema well.
The 404 page: the status code, and what the visitor seesGuide
Why a 404 must return 404, how soft 404s pollute your index, what belongs on the page for visitors, and when a redirect is the better answer.
URL structure: the decisions you make onceGuide
Folders that mean something, slugs made of words, the variants that create duplicates, and a table of good and bad URLs for a product site.
Webflow SEO: full control, and the three ways designers waste itGuide
How to use Webflow’s SEO controls well: titles, descriptions, canonicals, robots, sitemap, redirects and hreflang. Plus the three traps to avoid.
Navigation: the links that decide what gets crawledGuide
Your menu decides which pages are found first and how deep the rest sit. What belongs in it, what belongs in the footer, and how to check yours.
Wix SEO in 2026: what the platform handles and what is left to youGuide
What Wix now does for SEO, what you still control, and how to set it up page by page. Clear steps, common limits, and a checklist you can ship.
WordPress SEO: what the CMS does, what the plugin does, what you doGuide
What WordPress handles, what an SEO plugin adds, and what you must fix: permalinks, archives, attachments, speed, schema and a ship-ready checklist.

The free tools

Each checks one thing on one page or one site, without an account, and says what it found in words.

On-page SEO checkerFree tool
Paste a URL and get the page read as a crawler reads it: title, description, canonical, indexability, headings, alt text, links, share tags, what to fix first.
Redirect checkerFree tool
Follow a URL's redirects hop by hop: each status and destination, the time each took, the kind of hop, meta and script redirects, one rule to replace it.
Sitemap checkerFree tool
Read a sitemap the way a crawler does: what it lists, how it is dated, whether robots.txt names it, and whether thirty of its addresses answer 200.
Structured data checkerFree tool
Read the JSON-LD on a page: every type, the rich result each can earn, the required and recommended fields it lacks, and a mended copy of each node.
Schema markup generatorFree tool
Build the JSON-LD for an organisation, a product, an article, a FAQ, a how-to or a breadcrumb from a form, required fields marked and every value checked.
robots.txt testerFree tool
Paste a site, paths and a crawler: what its robots.txt allows by the rules Google reads with, the rule that decided each, why it won, and the file mended.
robots.txt generatorFree tool
Build a robots.txt that says which AI crawlers may train on your pages and which may cite them, with the sitemap line and the paths closed to everyone.
hreflang checkerFree tool
Paste a URL and check its hreflang: the versions declared, whether the codes are ones Google reads, the self-reference, whether each version points back.
Broken link checkerFree tool
Every link on one page, up to 150, fetched eight at a time: the ones that 404 with the words they carry, the ones that redirect and where to, as a list.
Internal link checkerFree tool
Paste your home page and get up to sixty pages fetched and their links read: the orphans, the pages sitting deepest, the least and most linked, links per page.
Server response checkerFree tool
Paste a URL and time the server's answer: time to first byte, HTTP/2 or HTTP/3, brotli or gzip, Cache-Control, HSTS, the headers that decide when a page starts.
Title and meta description checkerFree tool
Paste a page or type a title and a description: pixel widths, character counts, desktop and phone previews, with the h1 and canonical beside them.

The words

What each term means for a small site, and what to do about it.

Canonical URLGlossary
A canonical URL names the original version of a page. Use it to handle parameters and duplicates, avoid split signals and keep the right page indexed.
301 redirectGlossary
A 301 redirect is a permanent move. Use it to send users and bots to a new URL, keep signals, avoid chains, and choose 308 when methods must persist.
302 redirectGlossary
A 302 redirect is temporary. Use it for short moves like maintenance or campaigns. See how Google treats it and when to pick 301 or 307.
410 GoneGlossary
410 Gone means you deleted a URL on purpose and it will not return. Here is when to use 410 vs 404 or 301, how Google treats it, and what to fix.
NoindexGlossary
Noindex tells search engines not to index a page. Use it for pages that should never rank, and avoid adding it to pages that earn.
Meta robots tagGlossary
The meta robots tag tells crawlers what to do with a page. See each directive, how to set it, how to test it, and the robots.txt trap.
X-Robots-TagGlossary
What X-Robots-Tag is, when to use it, how to set it on responses, and how to check it in Search Console without breaking indexing.
robots.txtGlossary
robots.txt tells crawlers which URLs they may fetch. See what it does not do, how to test it, what to put in it, and how to handle AI bots.
XML sitemapGlossary
What to put in sitemap.xml, how lastmod works, how to create, validate and submit your XML sitemap, and the traps to avoid.
Sitemap indexGlossary
A sitemap index lists your sitemap files. Use it when you split URLs by section or language, or pass one sitemap’s 50,000 URL or 50 MB limit.
Structured dataGlossary
Structured data is schema markup that names your page type. See why it changes results, how to add JSON-LD, and how to validate it.
Rich resultsGlossary
Rich results are Google results with more than a blue link. See types, how eligibility works, how to test them, and how to track clicks in Search Console.
Title tagGlossary
A title tag is the HTML title of a page. See how long it should be, why Google rewrites it, what to put in it, and how to fix weak titles.
Meta descriptionGlossary
What a meta description is, how long to make it, why Google rewrites it, how to check yours, and what to write anyway for higher CTR.
Heading tagsGlossary
Heading tags are h1 to h6 in HTML. See how to structure pages, help Google and screen readers, and write headings that win snippets.
hreflangGlossary
hreflang tells search engines the language and region for each page version. See when you need it, where to declare it, and how to avoid errors.
URL parametersGlossary
URL parameters are key=value after a question mark. Handle them to avoid duplicate pages, index bloat and wasted crawl, and to keep clean rankings.
PaginationGlossary
How to set up pagination for SEO after 2019: self-canonicals, real links, view-all for short lists, and infinite scroll with crawlable pages.
Duplicate contentGlossary
Learn what duplicate content is, how Google handles it, how to find it on a small site, and the exact controls to fix and prevent it.
CrawlingGlossary
Crawling is how Googlebot discovers and fetches your URLs. See what it fetched, fix slow or blocked areas, and know when crawl budget matters.
RenderingGlossary
Rendering is how Google executes your JavaScript and CSS after crawling HTML, which can delay indexing. Here is what to check and how to fix it.
JavaScript SEOGlossary
What JavaScript SEO means, how Googlebot renders, what breaks, how to fix links and content, and how to choose SSR or SSG so pages index fast.

Questions

Check my site, free

Paste your URL and the check reads the whole site in one pass, the redirects, the canonicals, the markup, the robots file and the speed, and writes three findings whole.

Free check, no card
Read-only, your own accounts
Readable by your agent