GuideMonitoring

SEO monitoring for a small site: what to watch and what to alert on

SEO monitoring for a small site: uptime every few minutes, indexing directives on every deploy, a technical scan every two weeks, Search Console monthly.

LaunchScalerPublished 10 min read

SEO monitoring is checking, on a fixed schedule, the handful of signals that decide whether Google and AI answer engines can reach, index and show your pages, and getting an alert when one of them changes. For a small site that means uptime and server errors every few minutes, indexing directives on every deploy, a technical scan every two weeks, and a Search Console review once a month.

Rankings are the last thing to move. By the time a position drops, the cause (a stray noindex, a robots.txt rule, a week of timeouts) has usually been live for days. The schedule below catches the cause first.

What is SEO monitoring, and what does it watch?

SEO monitoring watches the inputs to search visibility rather than the outcome. It checks that the server answers, that pages allow indexing, that robots.txt lets the right crawlers in, that pages stay fast, and that link previews and security settings still hold. Rank tracking only reports the outcome, after the damage is done.

The things that break on a small site rarely announce themselves. None of these failures makes the homepage look different to you:

What silently breaksA typical causeWhat it costs you
IndexabilityA staging noindex or X-Robots-Tag shipped to productionGoogle drops the page on its next crawl
Crawl accessA robots.txt rule from a plugin, a deploy or a CDN settingGoogle and AI search bots stop reading the pages
Canonicals and sitemapA template change points every canonical at the homepagePages are folded into one URL
Link previewsAn og:image path changes or the image starts returning 404Shares on Slack, X and LinkedIn show a blank card
SpeedA new third-party script adds main-thread workINP and LCP slip, visible in field data weeks later
SecurityA certificate fails to renew, or a header disappearsBrowsers show a warning page to visitors and crawlers alike
UptimeThe host, database or an API dependency failsVisitors bounce, and repeated 5xx responses lead Google to drop URLs
AI accessA firewall rule challenges bots, or content moves behind client-side JavaScriptChatGPT, Perplexity and Claude cannot read what they would cite

How often should you check each part of your SEO?

Match the frequency to how fast a failure hurts and how often the cause can change. Uptime can fail at any minute, so it needs continuous checks. Directives change when code ships, so check them on every deploy. Speed, previews and AI access drift slowly and suit a two-weekly audit, and Search Console totals are best read monthly.

What to checkHow oftenHow to check itAlert when
Uptime and 5xxEvery 1 to 5 minutesAn uptime monitor on the homepage and a health endpointTwo or three consecutive failures
noindex in the meta tag and X-Robots-Tag headerEvery deploy, plus weeklycurl -sI for the header, view-source for the tagIt appears on any page you want indexed
robots.txtEvery deploy, plus weeklyDiff the file against the last copyAny line changes
Canonical tagsEvery deployRead <link rel="canonical"> on key templatesIt points anywhere but the page itself or its intended canonical
XML sitemapEvery deployFetch it and count the URLsIt returns an error, or the count falls sharply
Technical scan: search, AI crawler access, security, speed, working flowsEvery two weeksA scheduled scan with a diff against the last runA check drops below its pass bar
Core Web Vitals trendEvery two weeksPageSpeed Insights or the CrUX APIA metric moves from Good to Needs improvement
Search Console clicks, impressions and indexed pagesMonthly, or after big changesPerformance and Page indexing reportsA sustained fall you cannot explain by season

The rest of this guide explains each tier and links to a full guide for each check.

What needs continuous checks: uptime and server errors

Uptime is the only check that needs to run every few minutes, because an outage hurts immediately and can start at any time. Point a monitor at your homepage and at a health endpoint that touches your database, and alert after two or three consecutive failures so a single slow response does not wake you up.

Google's crawler documentation explains why this belongs in an SEO schedule. Both 5xx and 429 responses make Google's crawlers "temporarily slow down with crawling," and Google's indexing pipeline "removes from the index URLs that persistently return a server error." A short outage costs you the visitors who arrived during it. A recurring one also costs you crawl rate and, eventually, indexed pages.

A plain uptime check has a blind spot: it passes on any 200 response, including a page that carries noindex or an app shell with no content in it. That is why the next two tiers exist. The difference is laid out in uptime monitoring vs SEO monitoring, and the Search Console side of server errors is in the server error (5xx) guide.

What should you check after every deploy?

After every deploy, check the four directives that a code change can flip without anyone noticing: noindex, robots.txt, the canonical tag and the sitemap. Each one lives in config or a template, so a single environment variable or layout edit changes it on every page at once.

Run these against production, not staging, right after the deploy finishes:

  1. Read the header: curl -sI https://yoursite.com/ | grep -i x-robots-tag. On a page you want indexed, this should print nothing, or a value without noindex.
  2. Read the meta tag: curl -s https://yoursite.com/pricing | grep -i 'name="robots"'. Repeat for one URL per template (home, a product page, a blog post).
  3. Fetch robots.txt: curl -s https://yoursite.com/robots.txt and compare it with the copy you saved last time. A Disallow: / under User-agent: * blocks every compliant crawler.
  4. Read the canonical on each template: curl -s https://yoursite.com/blog/some-post | grep -i 'rel="canonical"'. It should name that page's own clean URL.
  5. Fetch the sitemap and count entries: curl -s https://yoursite.com/sitemap.xml | grep -c '<loc>'. A sudden drop usually means a data source failed at build time.

Speed matters here because Google acts on what it finds at its next crawl. Its noindex documentation says that when Googlebot sees the rule it "will drop that page entirely from Google Search results, regardless of whether other sites link to it." robots.txt changes reach Google quickly too, since Google "generally caches the contents of robots.txt file for up to 24 hours."

The failure patterns behind these checks each have a guide: noindex shipped to production, SEO regressions after a deploy, monitoring robots.txt for changes and a staging site indexed by Google.

What should a two-weekly scan cover?

Every two weeks, re-run a technical scan of the site and compare it with the previous run. This catches slow drift that no deploy check covers: a speed regression building up in field data, a security header that disappeared, or AI crawlers that started getting challenged by your firewall.

Three areas need this cadence more than any other.

Core Web Vitals trend

Search Console's report is built on real Chrome users, and each metric covers "the last 28 days" at the 75th percentile. The CrUX data behind it is "a 28-day rolling average," updated daily. A regression that ships today only shows fully four weeks later, as new data replaces old. A lab run every two weeks shows it at once. The thresholds to hold are LCP at 2.5 seconds or less, INP at 200 milliseconds or less and CLS at 0.1 or less. The lag and how to beat it are covered in monitoring Core Web Vitals over time.

AI crawler access

AI search bots read robots.txt on their own schedule. OpenAI says "it can take ~24 hours from a site's robots.txt update for our systems to adjust," and Perplexity says "it may take up to 24 hours for our systems to reflect changes." Check that OAI-SearchBot, PerplexityBot and Claude-SearchBot are allowed, that your firewall does not return a challenge page to them, and that your main content is in the server HTML. Tracking AI visibility over time covers the answer side as well.

Certificates and security headers

Read the certificate expiry date and confirm HSTS and your other security headers are still sent. Checking SSL certificate expiry has the one-line openssl command.

What should you review in Search Console each month?

Once a month, open the Performance report and the Page indexing report. Google's own guidance is that you "might want to check your account around once every month, or when you make changes to the site's content, to make sure the data is stable." Monthly is enough because both reports lag behind your live site.

In the Performance report, compare the last 28 days with the previous period for clicks and impressions, then with the same period a year earlier to separate a real drop from a seasonal one. Appearances in AI Overviews and AI Mode are counted "within the 'Web' search type," and since August 31, 2026 the Generative AI performance report (Search) shows their impressions by page, so check it in the same review. A fall in clicks with steady impressions on informational queries, while the same pages gain AI feature impressions, is one pattern worth noting.

In the Page indexing report, watch the indexed count. A fall in indexed pages with no matching rise in errors usually means something is blocking pages that used to be indexed. The page indexing report guide explains every reason the report can give.

Search Console also emails you. Google says: "If new issues are found by Google on your site, you'll receive an email from Search Console alerting you." Those emails arrive after Google has crawled the problem, and they do not cover title changes, speed regressions on a single deploy or AI crawler access. What they catch and miss is in Search Console email alerts. When the monthly review does show a drop, work through the organic traffic drop diagnosis order.

What should trigger an alert, and what can wait?

Alert only on failures that cost you visitors or index coverage within days. Everything else belongs in the scheduled review. An alert that fires for every small score change gets muted within a month, and then the real one is missed.

SeverityConditionResponse time
Page someone nowSite or health endpoint down for 2 to 3 consecutive checksMinutes
Page someone nownoindex or Disallow: / on productionSame day
Page someone nowCertificate expired, invalid, or failing its chainSame day
Fix this weekCanonical pointing at the wrong URL on a templateDays
Fix this weekSitemap returning an error or missing most URLsDays
Fix this weekA check in the two-weekly audit falling below its pass barDays
Review monthlyCore Web Vitals trend, clicks, impressions, indexed countNext review

Send the page-someone-now alerts to a channel a person reads outside working hours. Search Console's own emails go only to people added to the property, so check who that is under Settings, Users and permissions.

Which tools cover which checks?

No single free tool covers every row above, so a small site usually combines three things: an uptime monitor, Search Console, and a scheduled audit that diffs technical checks between runs. Rank trackers and visual page-change monitors add little for a small site, because they report outcomes or pixel changes, not the directives underneath.

A visual change monitor will tell you the pricing copy changed. It will not notice an X-Robots-Tag: noindex header, because headers are not on the screen. Tools are compared by what they actually watch in the best SEO monitoring tools and website change monitoring tools.

When does a site need more than the routine schedule?

Two events need their own checklist on top of the routine: a migration (a new domain, platform or URL structure) and a redesign. Both change hundreds of URLs, templates and internal links at once, and the routine checks assume the URL set stays the same.

Before either, crawl and save every URL. After it, check redirects, titles and indexed counts weekly for at least a month. Follow the site migration SEO checklist for a move, and traffic drop after a redesign if the numbers have already fallen.

Put the two-weekly audit and uptime checks on autopilot

The deploy checks are commands you can script into your pipeline. The two-weekly scan and the uptime checks are the part people stop doing by hand after a month.

LaunchScaler Watch runs that part for one website at $29/mo. It re-scans your site every two weeks and emails a diff on every run, showing which checks moved, by how many points, and the evidence behind the shift. Between scans it runs uptime checks on your health endpoint and alerts you the moment it goes down, and it alerts you when a check drops below the published bar. It covers one domain per subscription, and you can cancel whenever you like. Start watching.

Questions, answered

What people ask about this

01

What is SEO monitoring?

  • SEO monitoring is checking, on a schedule, the signals that decide whether search engines and AI answer engines can reach, index and show your pages, and getting an alert when one of them changes. It covers uptime, indexing directives, speed, security, link previews and AI crawler access, not only rankings.
02

How often should a small site check its SEO?

  • Check uptime and server errors every few minutes, indexing directives such as noindex, robots.txt and canonicals after every deploy, and run a technical scan every two weeks. Google suggests looking at Search Console around once a month, or whenever you change the site.
03

Does Google Search Console alert me when something breaks?

  • Google says you'll receive an email from Search Console if it finds new issues on your site. Those emails come after Google has crawled the broken pages, so they tell you about a problem that is already affecting how Google sees you.
04

What is the difference between SEO monitoring and rank tracking?

  • Rank tracking records where your pages sit for chosen keywords. SEO monitoring watches the causes behind those positions, such as a page turning noindex, robots.txt blocking a folder or the server returning errors, so you can fix them before positions move.
05

What should trigger an SEO alert right away?

  • The site or its health endpoint going down, 5xx errors on pages you want indexed, a noindex appearing on a page that should rank, a robots.txt rule blocking the whole site or a key folder, and an expired or invalid certificate. Everything else can wait for the next scheduled review.

Sources

  1. Google Search Central: How to use Search Consoledevelopers.google.com
  2. Google Crawling Infrastructure: How HTTP status codes affect Google's crawlersdevelopers.google.com
  3. Google Search Central: Block Search indexing with noindexdevelopers.google.com
  4. Google Search Central: How Google interprets the robots.txt specificationdevelopers.google.com
  5. Search Console Help: Core Web Vitals reportsupport.google.com
  6. Chrome for Developers: CrUX APIdeveloper.chrome.com
  7. Search Console Help: Performance reportsupport.google.com
  8. Search Console Help: Generative AI performance report (Search)support.google.com
  9. Google Search Central: AI features and your websitedevelopers.google.com
  10. OpenAI: Overview of OpenAI crawlersdevelopers.openai.com
  11. Perplexity: Perplexity crawlersdocs.perplexity.ai

Newsletter

What launched, every Monday

Every Monday: what launched on LaunchScaler that week, what is launching next, and the reviews we published. Confirm your email once and you're in.

More from the blog

Guide

SEO regressions after a deploy: what a release silently breaks

A deploy can break SEO without breaking the page: noindex from staging, a changed canonical, content moved client-side, lost redirects, slower INP.

LaunchScalerSep 28, 2026 · 8 min read
Guide

Site migration SEO checklist: domain, platform or URL changes

A site migration keeps its rankings when every old URL 301s to its one new URL. The before, during and after checklist, built around the redirect map.

LaunchScalerSep 28, 2026 · 8 min read
Guide

SSL certificate expiry: check the date and catch problems early

Read an SSL certificate's expiry date with one openssl command, script the check, and fix the setups where auto-renewal quietly fails.

LaunchScalerSep 28, 2026 · 8 min read