The Search Console API: pull your own search data

The API gives you rows, not pages. You pull up to 25,000 rows per request, and paginate until you cover your set. The interface caps exports far sooner.

• Every query and page pair you ask for, with clicks, impressions, CTR and position
• The same dimensions as Performance: query, page, country, device, search appearance and date
• Filters and regular expressions, so you can slice your site how you run it, for example only /docs/ or only mobile
• Automation: schedule pulls, compare periods, store history, and ship reports without logging in

You also get the URL Inspection API. You can check index coverage and canonical status at scale. Use it for a shortlist, not for a crawl.

You need a Google Cloud project and credentials. You also need access to the Search Console property you plan to query.

1. 1

   Create a project and enable the API

   In Google Cloud, create a project. In APIs and services, enable the Search Console API. This is under Google Search Console APIs as of 2026.

2. 2

   Choose OAuth or a service account

   For a script you run yourself, use OAuth client credentials. For a server or a scheduled job, use a service account.

3. 3

   Configure OAuth

   Set the OAuth consent screen. Create an OAuth client. For a local script, pick Desktop. For a web app, pick Web with your redirect URI.

4. 4

   Configure a service account

   Create a service account. Generate a key only if your runtime needs it. In Search Console, add the service account email as a user of the property.

5. 5

   Verify property scope

   Use a Domain property to cover all subdomains and protocols, or a URL-prefix property for one exact prefix like https://www.yourproduct.com/.

OAuth suits ad hoc use. The browser opens, you grant scopes, and you get a refresh token. A service account suits cron jobs and CI runners.

The Search Analytics method mirrors the Performance report. You send a date range, a list of dimensions, optional filters and a row limit. You page with startRow until you empty the set.

• Dimensions: any order of query, page, country, device, searchAppearance, date
• Metrics returned per row: clicks, impressions, ctr, position
• Filters: include or exclude with equals or contains, and regular expressions on some fields
• Row limit: up to 25,000 per request, use startRow for the next page
• Date range: up to 16 months of data, as in the UI

Example: you want every query and page pair for a section on mobile only. You set dimensions to ["query","page"], filter page contains "/guides/", device equals "mobile". You set rowLimit to 25000 and startRow to 0. You keep adding to startRow until the API returns no rows.

These show the request body and the call shape. They assume you have credentials and a client set up, and that your account can read the property.

# Python 3.x
# pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
from datetime import date, timedelta
from googleapiclient.discovery import build
from google.oauth2 import service_account

SCOPES = ["https://www.googleapis.com/auth/webmasters.readonly"]
KEY_FILE = "service-account.json"
SITE_URL = "https://yourproduct.com/"

# Use a recent window, accounting for the usual 2-3 day lag
end = date.today() - timedelta(days=3)
start = end - timedelta(days=27)

creds = service_account.Credentials.from_service_account_file(KEY_FILE, scopes=SCOPES)
service = build("searchconsole", "v1", credentials=creds)  # or "webmasters","v3" in older clients

body = {
  "startDate": str(start),
  "endDate": str(end),
  "dimensions": ["query", "page"],
  "dimensionFilterGroups": [{
    "filters": [
      {"dimension": "page", "operator": "contains", "expression": "/guides/"},
      {"dimension": "device", "operator": "equals", "expression": "mobile"}
    ]
  }],
  "rowLimit": 25000,
  "startRow": 0
}

rows = []
while True:
    resp = service.searchanalytics().query(siteUrl=SITE_URL, body=body).execute()
    batch = resp.get("rows", [])
    if not batch:
        break
    rows.extend(batch)
    body["startRow"] += len(batch)

for r in rows[:5]:
    dims = r.get("keys", [])
    print(dims, r["clicks"], r["impressions"], r["ctr"], r["position"])

// JavaScript (Node.js)
// npm i googleapis
const {google} = require('googleapis');

async function run() {
  const auth = new google.auth.GoogleAuth({
    keyFile: 'service-account.json',
    scopes: ['https://www.googleapis.com/auth/webmasters.readonly']
  });
  const client = await auth.getClient();
  const searchconsole = google.searchconsole({version: 'v1', auth: client}); // or webmasters v3 in older clients

  const siteUrl = 'https://yourproduct.com/';
  let startRow = 0;
  const all = [];

  // Use a recent window, accounting for the usual 2-3 day lag
  const now = new Date();
  const end = new Date(now.getFullYear(), now.getMonth(), now.getDate() - 3);
  const start = new Date(end.getFullYear(), end.getMonth(), end.getDate() - 27);
  const fmt = d => d.toISOString().slice(0,10);

  while (true) {
    const res = await searchconsole.searchanalytics.query({
      siteUrl,
      requestBody: {
        startDate: fmt(start),
        endDate: fmt(end),
        dimensions: ['query','page'],
        dimensionFilterGroups: [{
          filters: [
            {dimension: 'page', operator: 'contains', expression: '/guides/'},
            {dimension: 'device', operator: 'equals', expression: 'mobile'}
          ]
        }],
        rowLimit: 25000,
        startRow
      }
    });
    const rows = res.data.rows || [];
    if (!rows.length) break;
    all.push(...rows);
    startRow += rows.length;
  }

  console.log(all.slice(0, 5));
}

run().catch(console.error);

For a date trend, include date as a dimension. For a country split, include country. The API returns one row per unique keys array, for the window you sent.

The URL Inspection API returns the same checks you see in the URL Inspection tool for one URL: index status, last crawl, selected and user canonical, robots, and more. It is for diagnosis, not crawling a site.

• Quota: 2,000 calls a day per property, and 600 a minute
• Use cases: confirm a fix on /pricing, check canonical on /blog/api-guide, spot noindex on a test page before launch
• Shape: you send inspectionUrl and siteUrl, you get a structured report back

# Python: one URL Inspection call
report = service.urlInspection().index().inspect(body={
  "inspectionUrl": "https://yourproduct.com/guides/getting-started",
  "siteUrl": "https://yourproduct.com/"
}).execute()

status = report["inspectionResult"]["indexStatusResult"]
print(status.get("coverageState"), status.get("robotsTxtState"), status.get("pageFetchState"))

Since 2023 you can link a property to BigQuery. Search Console will write daily tables with the full row set for your site. You do not page, and you do not miss rows due to limits in the UI or API single-call caps.

1. 1

   Link BigQuery

   In Search Console settings, connect BigQuery and pick a project and dataset. Search Console creates tables for Search data by property and by URL.

2. 2

   Wait for daily loads

   Data lands each day with the usual lag. You query yesterday or earlier. Keep storage costs in mind as your site grows.

3. 3

   Query with SQL

   Use BigQuery SQL to build reports. For example, list queries with position between 11 and 20 on mobile for /docs/ in the last 28 days.

-- BigQuery: striking-distance in last 28 days
SELECT
  query,
  SUM(clicks) AS clicks,
  SUM(impressions) AS impressions,
  SAFE_DIVIDE(SUM(clicks), SUM(impressions)) AS ctr,
  AVG(position) AS avg_position
FROM `your_project.search_console.searchdata_url_impression`
WHERE date >= DATE_SUB(CURRENT_DATE(), INTERVAL 28 DAY)
  AND device = 'mobile'
  AND page LIKE 'https://yourproduct.com/docs/%'
GROUP BY query
HAVING avg_position BETWEEN 11 AND 20
ORDER BY impressions DESC
LIMIT 100;

Use the export if you run a content site or a store with many SKUs. For a small SaaS, the API is often enough, especially for focused weekly jobs.

These save you the clicks you repeat. Build them once. They then run on a schedule and post to Slack or email a report you can act on the same day.

• Weekly striking-distance list: queries at positions 11 to 20 with solid impressions. Add two internal links, refresh the title and meta, and watch them move.
• Pages shown and not clicked: pairs with impressions and a low CTR. Fix intent, title and intro. For example, "/guides/getting-started" shown for "api setup" yet with a low CTR.
• Query-to-page map: for each query, your top page. Use it to spot duplicate targets and split or merge. It stops two posts from fighting for "pricing tiers".

In each job, keep a 28 day window, mobile and desktop split, and country fit for your market. Output a CSV with a stable header so you can diff over time. A clean header lets you chart changes without edits to your code.

# Python: striking-distance from the API, last 28 days
from datetime import date, timedelta

end = date.today() - timedelta(days=3)
start = end - timedelta(days=27)

body = {
  "startDate": str(start),
  "endDate": str(end),
  "dimensions": ["query"],
  "rowLimit": 25000,
}
rows = service.searchanalytics().query(siteUrl=SITE_URL, body=body).execute().get("rows", [])

out = []
for r in rows:
    q = r["keys"][0]
    pos = r["position"]
    if 11 <= pos <= 20:
        out.append((q, r["impressions"], r["clicks"], r["ctr"], pos))

out.sort(key=lambda x: x[1], reverse=True)
for q, imp, clk, ctr, pos in out[:100]:
    print(q, imp, round(ctr,4), round(pos,1))

To list pages shown and not clicked, include page and query as dimensions. Filter to CTR below your norm. In a small SaaS, you often see intent mismatch on generic docs pages. Fix the h1 and intro to match the query that shows most.

// Node: pages with impressions and low CTR
// Use a recent window with the usual lag
const now = new Date();
const end = new Date(now.getFullYear(), now.getMonth(), now.getDate() - 3);
const start = new Date(end.getFullYear(), end.getMonth(), end.getDate() - 27);
const fmt = d => d.toISOString().slice(0,10);

const body = {
  startDate: fmt(start),
  endDate: fmt(end),
  dimensions: ['query','page'],
  rowLimit: 25000
};
const res = await searchconsole.searchanalytics.query({siteUrl, requestBody: body});
const rows = res.data.rows || [];
const low = rows.filter(r => r.ctr < 0.02); // typical threshold, not measured
low.sort((a,b) => b.impressions - a.impressions);
for (const r of low.slice(0, 50)) {
  const [q, p] = r.keys;
  console.log(`${p}\t${q}\t${r.impressions}\t${(r.ctr*100).toFixed(2)}%`);
}

For a query-to-page map, add both dimensions and reduce to the best page per query by clicks or impressions. Keep country if you serve multiple markets, so you do not map the US and the UK into one pick by mistake.

Use your URL structure to segment. Most product sites group pages cleanly: /pricing, /features/…, /docs/…, /blog/…. Filters let you send one query per area and keep results tidy.

• Page contains "/pricing" to track your money page every week
• Page contains "/guides/" to track content that assists, not sells
• Query regex like ^how\s to find guides to expand into a cluster
• Device equals mobile to match what users see on phones first

A clean run outputs a sheet where each section has its own tab. Example: a tab for /docs/ shows the top queries, their clicks, impressions and average position, over the last 28 days. You compare this week to the previous to spot drops fast.

The Sites and Sitemaps parts of the API let you script admin tasks. You can list verified properties, and submit a sitemap or a sitemap index by URL. This removes manual steps in a deploy.

• Submit https://yourproduct.com/sitemap.xml after a release
• Submit a docs-only sitemap when you publish a large batch at /docs/
• List properties so your script can pick the correct siteUrl for staging and prod

Know the sitemap rules. A sitemap can hold up to 50,000 URLs or 50 MB uncompressed. It can only list URLs on its own host, unless you set an approved cross-host setup. Google reads lastmod when it is consistently accurate. It ignores changefreq and priority. Submit the root sitemap or the index. The Sitemaps report will show Status, the date last read and the number of URLs discovered so you can confirm your script worked.

Start with one scheduled pull, one CSV, one Slack message. Do not build a dashboard until you trust the feed. Read the same slice in the UI to sanity check your code. They should match within the limits of anonymised queries and date lag.

• Authorisation errors: confirm the right property scope. For a subdomain, a URL-prefix property is not the same as a Domain property.
• Empty rows: widen your date range or remove a filter. Check you used the correct siteUrl with the trailing slash.
• Position surprises: average position is the mean of the highest position across impressions. It is not a rank tracker. Expect variance.
• Inspection limits: you have 2,000 calls per property per day and 600 a minute. Batch a shortlist and cache results for a day.
• BigQuery costs: export gives you every row. It is great for scale, but query and storage are not free in Cloud.

A fixed page looks like this: your script runs each morning, posts striking-distance queries for /pricing, and opens the tickets to add internal links from /features/billing and /guides/getting-started. You spend ten minutes, not two hours, to move work that week.