SEO data is useful only when it leads to a decision. Google Search Console and Bing Webmaster Tools can show clicks, impressions, click-through rate, and average position, but I wanted a repeatable way to turn those numbers into work: pages to improve, articles to write, links to add, and old URLs to migrate.

That need led me to build the open-source GSC + Bing to Obsidian SEO Pipeline, a TypeScript command-line tool that collects search performance data and turns an Obsidian vault into a practical SEO knowledge base.

Why I built it

I manage more than one search property: a primary site, supporting content, and older properties that can still contain valuable search history. Looking at each dashboard separately makes it difficult to see the relationships between them.

The questions I wanted to answer were more operational:

  • Which non-branded queries already rank between positions 5 and 20?
  • Which pages receive many impressions but earn a low CTR?
  • Where does a blog post have enough relevance to link to a main page?
  • Are multiple pages competing for the same search intent?
  • Does a legacy property still rank for queries worth migrating?

I also wanted to keep the raw evidence. A generated recommendation should always be traceable to a query, page, date range, and set of metrics.

How the pipeline works

The pipeline has four main stages.

1. Read configured search properties

Each property is defined as a source with an ID, label, site URL, type, and optional brand-query pattern. The type can distinguish a main site, a blog, a legacy domain, or another custom source.

Sensitive values stay local. Credentials, API keys, vault paths, and real source files are excluded from Git, while safe example files document the expected setup.

2. Fetch daily search data

For Google, the CLI uses the Search Console API and requests several views of the same period: queries, pages, query-to-page pairs, countries, devices, and dated query-to-page rows. Requests are paginated in batches of 25,000 rows.

The tool pulls ranges one day at a time. This creates auditable daily snapshots before combining them into range and yearly datasets. Weighted average position and CTR are recalculated correctly during aggregation instead of averaging already averaged values.

The latest update adds Bing Webmaster Tools. Bing exposes a different API shape, so the adapter retrieves query, page, and page-to-query statistics, normalizes them into the same core columns, and marks every row with its search engine. Google and Bing remain in separate SEO/GSC/ and SEO/Bing/ folders so their data is comparable without being mixed accidentally.

3. Preserve raw CSV datasets

Every export includes its source, engine, dimensions, clicks, impressions, CTR, average position, and date range. Query-based datasets also get a non-branded counterpart generated from the source’s brand regex.

The Obsidian vault becomes both an archive and a workspace:

SEO/
  GSC/
    sources/<source-id>/raw/daily/
    sources/<source-id>/raw/ranges/
    sources/<source-id>/raw/yearly/
    sources/<source-id>/reports/
    sources/<source-id>/ideas/
    combined/
  Bing/
    sources/
    combined/

Keeping daily, range, and yearly levels lets me inspect a single date, study a campaign window, or build a pivot table across a complete year.

4. Turn metrics into actions

The reporting layer filters out branded searches where configured and looks for two useful signals:

  • high-impression query/page pairs with CTR below 2%;
  • queries with at least 50 impressions ranking from position 5 to 20.

From those signals, it writes Markdown reports and AI-ready idea files for quick wins, low-CTR pages, ranking opportunities, and possible new articles. Every idea includes the ranking page, why it matters, a suggested action, a priority, and the metrics behind it.

When several sources are analyzed together, the pipeline also looks for cross-site opportunities: blog-to-main internal links, legacy migration candidates, shared queries, and possible cannibalization. Entries use the source, query, and page as a stable key, so rerunning the tool updates existing ideas instead of endlessly duplicating them.

The commands I use

After the local configuration is ready, I can verify access without downloading data:

pnpm gsc:verify
pnpm bing:verify

I can then pull a recent period for every enabled source:

pnpm gsc:pull -- --all --last-days 30
pnpm bing:pull -- --all --last-days 30

Analysis does not need to call Google again. It can rebuild reports from the daily CSV snapshots already stored in Obsidian:

pnpm gsc:analyze -- --all --from 2026-06-01 --to 2026-06-30
pnpm gsc:rebuild

That separation matters. Collection, storage, and analysis can evolve independently, and changing a brand filter does not require another API download.

How the project evolved

I created the first working version on June 25, 2026. It already handled Google authentication, data collection, CSV exports, Markdown reports, local analysis, and combined source insights.

On June 30, I moved real source definitions into an ignored gsc-sources.json file, strengthened the example configuration, clarified setup instructions, and added the architecture image used in this post. Those changes made the repository safer to share and easier for someone else to understand.

On July 12, I expanded the pipeline beyond Google by integrating Bing Webmaster Tools. The update added separate verification and pull commands, a Bing-specific API adapter, engine-aware storage, and compatible outputs for the dimensions Bing provides. Empty files are intentionally retained when Bing has no matching rows or does not expose a Google-equivalent dimension; they form an audit trail rather than hiding a missing result.

What I learned

The most valuable part of this project is not the API connection. It is the data model around it.

Daily snapshots make rebuilds possible. Source metadata makes cross-property analysis useful. Separate raw and derived files preserve trust. Stable idea keys make repeated automation manageable. And Markdown output makes the result readable by me, searchable in Obsidian, and ready to use with AI tools without locking the workflow inside another dashboard.

The pipeline is still deliberately local and file-based. That keeps it transparent: I can open any CSV, inspect any recommendation, change the code, and rebuild the knowledge base. For my workflow, that is exactly the point.

The source code and setup guide are available on GitHub.