ApollosWave/gsc-cli

First-party Google Search Console and technical SEO engine. Automate Googlebot indexing, audit Core Web Vitals, and capture Google AI Overviews in pure Ruby with zero dependencies. - ApollosWave/gs...

8 stars · 1 forks

README

GSC CLI. Zero dependencies. Pure search intelligence.

Zero runtime gems. Pure Ruby standard library (Net::HTTP, OpenSSL, JSON). Cold-boots in ~15ms with 371 test suites, 1,991 assertions, and 0 failures across Ruby 3.0 through 3.4. Extract first-party Google rankings, automate technical SEO audits, and manage Googlebot indexing without bloated SDK dependencies.

Real-World Production Impact: Ground Truth at Scale

Actual Google Search Console data from production domains audited, indexed, and scaled via gsc-cli.

Automated Indexation (+380% in 3 Weeks) AI Overview Capture (3.53K Impressions)

3,897 Pages Indexed (+380% Surge)Stalled at ~800 pages for months until automated sitemap inspection (gsc inspect-sitemap), soft-404 fixes, and priority queueing broke the crawl deadlock. 3,530+ AI Overview ImpressionsTracked and captured live visibility in Google's new Generative AI search features using citation gap analysis (gsc aio-hunter) and structure scoring (gsc cite-sim).

Why We Built This: Escaping the 40-Gem Tax Every software engineer, technical SEO, and developer-operator managing search presence faces the exact same architectural frustrations:

The 40-Gem Dependency Tax: The official Google API Ruby gems (google-apis-searchconsole_v1, google-apis-indexing_v3, googleauth, signet, faraday) pull in 40+ transitive gem dependencies. They add 3 to 5 seconds to cold-boot time, trigger Bundler version conflicts, inflate container image sizes, and introduce ongoing supply-chain vulnerability alerts. Paying $300 to $1,000+/Month for Sampled Guesswork: Third-party SEO suites charge hundreds of dollars per month to scrape search results with external proxy farms and estimate traffic using sampled third-party panels. Meanwhile, Google Search Console provides 100% first-party ground-truth data for your domain directly from production search logs—completely free. Google Search Console's Web UI Does Not Scale: Inspecting 50 URLs, isolating soft-404 indexation drops, or detecting cannibalization across multiple client properties requires hours of manual tab switching, filtering, and clicking. Google AI Overviews (AIO) Intercepting Search Clicks: Zero-click searches continue to expand. If technical content is not structured for citation in Gemini and AI Overviews, organic visibility drops even when ranking on Page 1. AI Coding Agents Cannot Click Web Buttons: Modern coding agents (Antigravity, Claude Code, Cursor, Cline) require deterministic, sub-50ms JSON over stdout to diagnose and fix search issues directly inside the local repository.

We built gsc-cli to automate Googlebot indexing and search diagnostics in < 50 milliseconds using pure standard-library Ruby, and open-sourced it 100% free under the MIT License so every builder, engineering team, and operator can run first-party search automation with zero dependency bloat.

📐 Core Engineering Principles

First-Party Ground Truth Over Guesswork: Third-party estimation suites rely on sampled clickstream models and external proxy scraping. Google Search Console stores the exact, 100% first-party click, impression, and position data directly from Google's search infrastructure—completely free. Zero-Dependency Purity (< 1ms Boot Time): Official Google API gems drag in 40+ dependency gems, slowing down boot times and creating dependency lock-in. GSC-CLI communicates directly with Google's bare-metal HTTP APIs using pure Ruby standard library (Net::HTTP, OpenSSL, JSON), executing in < 1 millisecond. Actionable Remediation Over Passive Error Dumps: Most diagnostic tools dump hundreds of error rows into a table without solutions. gsc-cli pairs forensic diagnostics with automated remediation—such as generating copy-paste redirect blocks for 14 server environments (Nginx, Caddy, Cloudflare, Next.js, Vercel, etc.) for soft-404 traps. Zero-Trust Local Execution (No Telemetry, No External AI Calls): Developer tools should respect machine resources and codebase privacy. gsc-cli runs 100% locally, requires zero external LLM API keys, makes zero calls to third-party AI servers, collects zero telemetry, and never reads or transmits private repository files.

⚡ The Contrarian Architecture: Why Pure-Ruby Beats 40-Gem SDKs Conventional wisdom says: "To build Google API integrations, you must install the official Google API gems (google-apis-searchconsole_v1, googleauth)." We reject that completely. Google's APIs are just standard, RFC-compliant HTTPS endpoints returning JSON. Forcing a developer to pull in 40+ transitive gem dependencies just to send a signed HTTP POST request is architectural malpractice:

It bloats container images. It inflates Docker deployment sizes. It creates endless bundle install version conflicts. It slows down sub-millisecond AI agent loops.

By implementing Google's JWT service account authentication in pure OpenSSL and streaming responses through native Net::HTTP with Gzip decompression, gsc-cli boots in < 1 millisecond and executes entire multi-step audit loops in under 50 milliseconds.

💎 Key Capabilities at a Glance

🤖 Google AI Overview (AIO) Hunter (gsc aio-hunter): Detects Google AI Overviews on SERPs, extracts cited sources, measures citation gaps, and generates snippet capture recipes. 🔮 AI Citation Simulator (gsc cite-sim): Tests how likely LLMs (Gemini, ChatGPT, Claude) are to cite your URL based on information density, fact ratios, and structural headings. 🏦 Agency Credential Vault (gsc vault): Local AES-256-GCM encrypted store managing 50+ service accounts with instant switching (gsc switch <domain>), strict POSIX permissions, and path-traversal isolation. 🚨 Soft-404 Diagnostic Engine with Instant Fixes (gsc soft-404 --fix nginx): Diagnoses soft-404 traps and outputs copy-paste redirect blocks for 14 server stacks (Nginx, Caddy, Cloudflare, SvelteKit, Next.js, Astro, Shopify, Vercel). 🎯 Tactical Striking Distance Playbook (gsc strike): Automatically detects high-impression Page 2 queries (Positions 4–20) paired with intent-matched title hook rewrites and internal link anchor recipes. 📉 Organic CTR Curve Simulator (gsc ctr-curve): Non-linear regression modeling expected CTR by position and calculating exact click upside for ranking leaps. ⚡ Ultra-Low Token Modes for AI Agents (--compact & --ndjson): Minified single-line JSON (--compact) saving 35–45% LLM context tokens, and newline-delimited streaming (--ndjson) for array data. 📱 Mobile vs. Desktop SERP Parity Auditor (gsc mobile-parity): Uncovers cross-device rank discrepancies, responsive suppression penalties, and mobile CTR leaks. 🩺 Zero-Gem Cold-Start Doctor (gsc doctor): Verifies 100% standard library purity, measures sub-50ms execution speed, and validates credential security. 📈 Real-Time Google Trends Engine (gsc trends): 5-year and 1-year search trajectory, velocity percentages, Unicode sparklines ( ▂▃▄▅▆▇█), and regional demand breakdowns with zero authentication. 🎯 Zero-Auth Keyword Planner (gsc planner): Instant seed expansion via Google Autocomplete with automated search intent classification (Informational, Commercial, Transactional). 💰 Keywords Everywhere Ingestion (gsc import clip & gsc ke): Ingest free keyword tables directly from clipboard (zero credits required) or connect a Keywords Everywhere API key (partner referral link that helps support continuous open-source maintenance of this project) for automated terminal lookups. 🚀 Instant Googlebot Re-Indexing (gsc index): Ping Google's Indexing API with URL_UPDATED or URL_DELETED for priority crawl queueing within seconds. ⚡ Multi-Engine IndexNow Protocol (gsc indexnow): Instantly submit pages and sitemaps across Microsoft Bing, Yandex, Seznam, and Naver simultaneously. 🖥️ Google SERP & Title Pixel Simulator (gsc serp): Simulate desktop (580px) and mobile (650px) Google SERP cards, calculate precise proportional pixel widths, and prevent truncation before publishing. 🛡️ Cannibalization & Decay Detection (gsc cannibalization, gsc decay): Spot internal URLs fighting for the same queries, and compare 28-day period-over-period traffic trends. 📊 Google Analytics 4 (GA4) Behavioral Link (gsc realtime, gsc ga4): Stream live active visitors and correlate SERP rankings with landing page bounce rates. 🕷️ Autonomous Site Audit & Broken Link Repair (gsc site-audit): Crawls all sitemap URLs, checks HTTP response codes for dead internal links (404/500/timeouts), audits missing image alts and heading defects, and exports an AI-actionable Markdown fix sprint. ⚡ Dual Lab & Field Core Web Vitals (gsc speed & gsc speed-correlate): Audits simulated Lighthouse Lab metrics (LCP, FCP, CLS, TBT, Speed Index) alongside official 28-day Chrome User Experience Report (CrUX) Real User Monitoring (RUM) field data (LCP, INP, CLS, FCP, TTFB) and correlates page speed directly with organic GSC search rankings. 🤖 AI Agent Native: Every single command supports --compact, --ndjson, and --json for instantaneous programmatic consumption by AI agents.

⚡ Architectural Comparison & Trade-Offs

Traditional Stack (Official SDK / SaaS) GSC-CLI Architecture

Manual Web UI Bottleneck: Clicking through 15 tabs in Google Search Console to inspect 50 URLs takes 45+ minutes. Instant 1-Command Batching: gsc inspect-sitemap sitemap.xml inspects all URLs with automatic quota pacing in seconds.

Sampled Third-Party Guesswork: Paying $300–$1,000+/mo ($3,600–$12,000/yr) for external proxy scrapers that model keyword volumes. 100% Google Ground Truth ($0): Raw impression, click, and position logs direct from Google Search Console.

40-Gem Dependency Tax: Bloating your Gemfile with Google SDK gems that add 3–5 seconds to cold boot. 0 Gem Dependencies: Pure Ruby standard library (OpenSSL, Net::HTTP, JSON) executing in < 1 millisecond.

Multi-Client Credential Chaos: Juggling loose JSON keys across folders and risking credential leaks. Agency Credential Vault (gsc vault): Local AES-256-GCM encrypted vault with instant domain switching (gsc switch).

Passive Error Reporting: Diagnostic tools tell you that you have 404s, but leave you to write the server rules. Active Automated Remediation: gsc soft-404 <url> --fix nginx synthesizes instant, copy-paste server blocks for 14 stacks.

Blind to AI Overviews: Unaware that Google Gemini / AI Overviews are intercepting 40% of zero-click searches. Google AIO Hunter & Citation Simulator: gsc aio-hunter identifies AI Overviews and extracts competitor citation recipes.

Token-Guzzling JSON in AI Agents: Feeding pretty-printed JSON into coding agents wastes 40% of your LLM context window. Ultra-Low Token Modes (--compact, --ndjson): Minified output saving 35–50% of tokens for Claude Code, Cursor, and Antigravity.

Untrusted Third-Party Scripts: Running opaque scripts that scan your repository or send files to external servers. Zero-Trust Local Execution: Completely isolated, deterministic UNIX tool. Never scans, reads, or transmits your private code.

⭐ Star the Repository If gsc-cli saved your team from 40 bloated gems or streamlined your search data pipeline: 👉 Star GSC CLI on GitHub to support zero-dependency open-source tooling.

📦 Quick Installation Option 1: Official RubyGem (Instant Global Install) gem install gsc-cli Option 2: 1-Line Standalone Installer (macOS & Linux) curl -fsSL https://raw.githubusercontent.com/ApollosWave/gsc-cli/main/install.sh | bash Option 3: In Your Gemfile (Bundler) gem 'gsc-cli' Ensure ~/.local/bin is in your shell PATH: echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshrc source ~/.zshrc Verify your installation: gsc version gsc doctor Option 4: Clone & Verify Test Suite Locally Verify all 371 test suites and 1,991 assertions on your own machine in seconds: git clone https://github.com/ApollosWave/gsc-cli.git cd gsc-cli rake test

=> 371 runs, 1991 assertions, 0 failures, 0 errors, 0 skips

⚡ 2-Minute Google Setup

🔑 Need help with GA4, Google PageSpeed, OpenPageRank, or Keywords Everywhere? Check out the complete Authentication & API Key Setup Guide (AUTH.md) for 30-second walkthroughs and zero-key features.

Step 1: Create a Google Cloud Service Account Key

Open Google Cloud Console. In APIs & Services > Library, enable:

Web Search Indexing API Google Search Console API (Optional for GA4): Google Analytics Data API

In IAM & Admin > Service Accounts, click Create Service Account (e.g. gsc-indexer). Click your new service account > Keys tab > Add Key > Create new key > JSON. Download the file.

Step 2: Run 1-Click Interactive Connect In your terminal, simply run: gsc connect gsc automatically searches your ~/Downloads folder for recently created Google Cloud service account keys, lets you confirm with Y, and saves it securely to ~/.config/gsc/service-account.json. Step 3: Add the Email to Google Search Console

Copy the service account email displayed in the terminal. Go to Google Search Console > Select your property > Settings > Users and permissions. Click Add User, paste the email, and set permission to Owner.

Verify access immediately: gsc domains

🧠 Deep-Dive Feature Walkthroughs 1. Core Search Performance: Top Queries, Pages & CTR (gsc top-queries, gsc top-pages, gsc performance) The Problem Google Search Console's web UI is sluggish, hides low-volume long-tail queries behind pagination limits, and makes cross-referencing brand vs. non-brand queries painful. Exporting CSVs to spreadsheets burns 20+ minutes every time you want to check yesterday's organic clicks. The Solution gsc-cli streams your exact Google Search Console ground-truth performance directly into your terminal in under 50 milliseconds. Filter by brand, segment devices, compare date ranges, and export clean JSON or CSV in one command:

Top 20 search queries with impressions, clicks, CTR, and average SERP position

Filter brand vs. non-brand queries automatically

Top indexed landing pages driving search traffic

Overall 30-day domain performance scorecard with device & country breakdown

Retrieve 100% of all queries via automated API pagination (no 1,000-row web limit)

2. Tactical Striking Distance Playbook & Organic CTR Curve (gsc strike & gsc ctr-curve) The Problem Most SEO reports simply dump a list of keywords without telling you what to write, where to link, or what the revenue payoff will be. Page 2 keywords (Positions 4–20) generate 80% of your search impressions but only 5% of your clicks. The Solution

gsc ctr-curve: Generates an empirical, non-linear CTR regression curve mapping your domain's real CTR by position against Google's global benchmarks, simulating the exact click gain of advancing to Top 3. gsc strike: Automatically synthesizes a tactical Page 2 attack plan with 3 SERP-safe hook titles (< 560px), heading recipes, and targeted internal link anchor suggestions:

Simulate organic CTR curve and click upside for Page 2 rankings

Generate tactical Page 2 striking distance playbook

Surface quick-win CTR underperformers (Top 10 ranking but low click-through)

Export playbook directly to CSV for copywriters & content teams

3. Instant Googlebot Priority Indexing & Live URL Inspection (gsc index, gsc inspect, gsc indexnow) The Problem Waiting days or weeks for Googlebot to discover new landing pages or updated documentation kills organic momentum. Meanwhile, checking if Google has indexed a URL or flagged a canonical error requires tedious manual clicking inside Search Console's URL Inspection tool. The Solution gsc-cli communicates directly with Google's Indexing API and live URL Inspection API to check status and force priority crawling in seconds:

Production turnaround: Stalled at ~800 pages with 4.5K unindexed for months, then surging to 3,897 indexed pages in 3 weeks (+380%) while clearing out 3,300+ unindexed backlog errors via automated inspection and indexing loops.

Query live Google indexing verdict, assigned canonical, and crawl timestamp

Ping Googlebot to prioritize crawling a newly published or updated URL

Batch inspect an entire XML sitemap with automatic quota pacing

Multi-Engine IndexNow: Instantly notify Bing, Yandex, Seznam, and Naver simultaneously

4. Cannibalization, Traffic Decay & Zombie Page Detection (gsc cannibalization, gsc decay, gsc zombies) The Problem Internal URLs competing for the same search intent (keyword cannibalization) split page equity, causing Google to oscillate rankings between pages. Meanwhile, stealth traffic decay and dead zero-click URLs quietly drain your Google crawl budget. The Solution gsc-cli diagnoses algorithmic cannibalization conflicts and period-over-period decay before traffic crashes:

Detect internal URLs fighting for the exact same queries

Period-over-period decay detection (28-day comparison: decaying vs surging queries)

Scan XML sitemap for 90-day zero-impression zombie pages draining crawl budget

5. Zero-Cost Clipboard Keyword Ingestion & Demand Trends (gsc import clip, gsc trends, gsc planner) The Problem Keyword research tools either lock data behind expensive API subscriptions or leave valuable volume data trapped in disconnected browser extensions. The Solution gsc-cli provides zero-cost clipboard ingestion and real-time demand tracking with zero API keys required:

1. Copy any keyword table from Keywords Everywhere or Google Ads in your browser (Cmd+C)

2. Run one command to parse volumes, sparklines, and cross-reference live GSC rankings:

Real-time Google Trends 5y/1y search trajectory and velocity percentage (zero-auth)

Google Autocomplete intent expander (Informational, Commercial, Transactional)

6. Detailed Off-Page + On-Page SEO Merger & Site Audit (gsc page & gsc site-audit) The Problem On-page SEO checkers (titles, headings, meta tags) live in browser extensions, completely disconnected from your real Google Search Console performance data. The Solution gsc page unites on-page DOM inspection with 90-day GSC search queries, clicks, and rankings in a single terminal view:

Audit any URL combining DOM inspection with GSC 90-day search performance

Deep internal link verification: tests HTTP status codes (200, 404, 500)

Crawl entire XML sitemaps to audit dead links, missing alts, and heading hierarchy

7. Google AI Overview (AIO) Opportunity Hunter & Citation Simulator (gsc aio-hunter & gsc cite-sim) The Problem Google AI Overviews (Gemini in search results) intercept high-intent queries before users ever reach blue links. If an AI Overview appears for your core keywords, your organic CTR can crater unless your site is cited inside the AI answer box. The Solution gsc aio-hunter scans live SERPs for AI Overviews, extracts cited sources, and measures your citation readiness:

Production impact: Capturing over 3,500 impressions directly inside Google Search Console's new Generative AI features (Beta) report.

Hunt AI Overview presence and extract cited competitor sources

Test how likely LLMs (Gemini, ChatGPT, Perplexity) are to cite your URL

Syn

source