One authenticated GET, and any stack has a blog
The engine is the source of truth; your site is the consumer. Poll one endpoint with a bearer key, follow the next_since cursor, render the HTML and JSON-LD you get back, and delete whatever arrives unpublished.
Published 2026-09-02, reviewed 2026-09-04, by Rodd Azad, editor-in-chief. How we check facts.
Setup, end to end.
1Step 1
Ask us to register the site, or add it in the dashboard: registration mints the per-site pull key, shows it once, and stores only its SHA-256 digest.2Step 2
Put the engine origin, your site slug and the pull key in server-side environment variables, and keep them out of any browser bundle.3Step 3
Make the first call: GET /v1/publish/{site}/articles?limit=50 with an Authorization: Bearer header, and read the articles list and next_since cursor from the response.4Step 4
Persist next_since and send it back as ?since= on every later poll, so each request returns only what changed after the last one.5Step 5
Render each record: write body_html into your template, set the title and meta description, inject the jsonld block verbatim, and map canonical_path onto your own route.6Step 6
Render the supporting blocks the body deliberately omits: the faq array, the toc array, hero_image with its alt text and the internal_links list.7Step 7
Treat any record with status unpublished as a deletion: remove or 410 the page, drop it from your sitemap, and check /{site}/redirects for a 301 the engine recorded in its place.8Step 8
Proxy the derived feeds instead of generating them: /{site}/sitemap.xml, /{site}/rss.xml, /{site}/llms.txt, /{site}/robots.txt and /{site}/config.9Step 9
Serve the value from /{site}/indexnow-key at /<key>.txt on your own origin so IndexNow pings verify, then have us connect Search Console and submit the sitemap.10Step 10
Optionally register a push URL and secret so the engine POSTs each change to you, and verify the X-Thoth-Signature HMAC before trusting the body.11Step 11
Rotate the key whenever you need to by re-registering the site with a new one; the old digest is overwritten and the old key stops working immediately.
How does authentication work, and what is a key scoped to?
Every consuming site is a tenant with its own bearer key. The engine stores only the SHA-256 digest of that key, never the key itself, so it cannot be read back out of our database, and rotating it is a single re-registration.
Every read is a GET against /v1/publish/{site}/ with an Authorization: Bearer header. An unknown site slug answers 404. A missing or mismatched bearer answers 401. A key is scoped to one site and can never read another tenant's articles, which is the reason it is issued per site rather than per account.
Keep it server-side, in your environment or your secret manager, next to a database password. Nothing in this integration needs a key in the browser.
One behaviour to plan for: a paused site keeps serving everything it has already published. Pausing stops new articles being written and published. It does not withdraw what is already live, and a URL never starts answering 404 because someone pressed pause.
In one sentence
A pull key is stored only as a SHA-256 digest and is scoped to a single site, so one tenant's key can never read another tenant's articles.
How does paging with since and next_since work?
The feed is incremental, not a snapshot. GET /v1/publish/{site}/articles?limit=50 returns a list of articles and a next_since cursor, ordered oldest change first by updated_at then id, so two calls for the same page return the same rows rather than whatever the query planner chose that time.
The limit defaults to 50 and is clamped between 1 and 200. Store next_since, send it back as ?since= on the next poll, and the server returns only rows whose updated_at is later than the cursor. When nothing has changed you get an empty list and your own cursor back, so an idle poll costs one cheap request.
A page is widened rather than cut when several rows share an updated_at to the microsecond. Cutting inside a tied group would skip those rows forever once the cursor moved past them.
Articles with status scheduled are excluded. One waiting for its drip publishing slot is written and stored but is not live, so the single-article route 404s for it too.
How does a takedown reach my site?
The same feed carries removals. A retracted article comes back with status unpublished and, where one was recorded, a takedown_reason. Act on it: delete the page or return 410, and drop the URL from anything you generate.
Trust the status field on the record in front of you. It is stamped from the live row at the moment you fetch it, so a retraction reaches you as unpublished even though that article was stored as published on the day it first shipped. Read the field, rather than assuming a feed only ever adds and updates.
When a takedown is a consolidation rather than an editorial removal, the engine records a 301 from the retired path to the surviving one. Read the map from /{site}/redirects and serve it, so the old URL keeps its ranking signal instead of dead-ending.
Filter tombstones out of what you display; act on them when you sync. Those are two different code paths, and conflating them is the usual bug.
In one sentence
A tombstone is an article that arrives in a content feed with status unpublished, telling the consuming site to remove a page it has already published.
What does one article record contain?
Every record carries a schema_version and is built so you never have to generate the SEO furniture yourself.
- Body — body_html rendered and ready, plus body_markdown, excerpt, toc, word_count and reading_time_min.
- Meta — title, meta_description, slug, canonical_path, robots, hreflang, text direction, and complete og and twitter blocks.
- Schema — jsonld, authored by the engine and authoritative. Inject it verbatim rather than rebuilding it; see JSON-LD.
- Media — hero_image, cover_image and an images array, each entry carrying alt text.
- Blocks — an faq array of question and answer pairs, stripped out of the body so you cannot render them twice, plus internal_links, outbound_citations and recommended_products.
- Provenance — author, version, content_hash and first_party_grounded.
content_hash answers the question "did anything actually change?" between versions, which is what lets you skip a rebuild when a republish touched nothing. first_party_grounded tells you whether that article drew on your own product data rather than public sources.
What else does the same key open?
Articles are one endpoint of several, and the same bearer opens all of them.
- /{site}/sitemap.xml and /{site}/rss.xml — generated feeds you can proxy straight through.
- /{site}/llms.txt — the AI-crawler manifest; see llms.txt.
- /{site}/robots.txt — your robots policy, including the single toggle that flips GPTBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot between allow and disallow.
- /{site}/config — verification tags, analytics IDs, Open Graph defaults, locale and sitemap tuning, edited in the dashboard and applied by your site at request time with no redeploy.
- /{site}/redirects and /{site}/clusters — the 301 map, and the topic-cluster graph behind the internal links in your articles.
- /{site}/indexnow-key — serve this value at /<key>.txt on your origin so IndexNow pings verify against a file you host.
Proxying these is usually less work than generating them, and your sitemap then agrees with your feed by construction rather than by discipline.
One thing to check when you compare tools: an API is not always included in the price. Surfer's own integrations page states that its official API is “only available to active users with the API add-on or users on an Enterprise plan” (surferseo.com/integrations, checked 4 September 2026). Here the pull feed is how articles reach a site at all, so every registered site is issued a key and there is nothing to add on.
Should I poll, or be pushed?
Polling is the default and asks nothing of your infrastructure beyond an outbound request on a schedule. If you would rather not poll, register a push URL and a push secret, and the engine POSTs each publish, update and takedown to you as an event and an article.
Verify before you trust. X-Thoth-Signature is a hex HMAC-SHA256 of the exact request bytes, keyed by your secret. Recompute it over the raw body, compare in constant time, and read the event name from X-Thoth-Event. Delivery is at-least-once with exponential backoff and a dead-letter entry once the retry budget is spent, so make your handler idempotent on slug and version. The mechanics are set out under webhooks.
Nothing stops you doing both. Take the push as a nudge for instant updates, and run a nightly cursor pass to reconcile anything a failed delivery missed. The endpoint reference is in the docs.
What happened before my GET?
By the time an article is in your feed it has been through the whole pipeline. Topics come from measured demand: DataForSEO plus your own Search Console queries. Research reads the ranking pages, forums and, where it exists, your first-party data — product records, datasheets, support answers. A Claude model writes it. Then four checks run, none of them by the model that wrote it and each including at least one model from a different vendor: fact-check and novelty against the competing pages led by Gemini and GPT, plus an E-E-A-T critique and AI-tell detection, with up to three revision rounds. The model that writes is never the model that checks; that is cross-model validation.
Stated as measured, not promised: across 186 Standard-tier runs, 21% of drafts cleared every check on the first attempt with no revision, and 51% got there within the three revisions allowed; the rest never published. Pro-tier judges are stricter: 25% cleared every check across a small sample of 16 runs.
You pay per published article from a prepaid balance, with no subscription; a draft that fails the checks is not charged. Tiers are on pricing.
In one sentence
Nothing reaches the feed until it has passed judges that did not write it, each check including a model from a different vendor. A draft that fails is revised or dropped, not shipped.
When the pull feed is the wrong choice
The feed suits most stacks. Where it does not:
- You need changes live in seconds. A poll has a floor equal to its interval. Register a push URL and take the signed webhook, keeping a slower poll as the reconciler.
- You cannot store a secret server-side. The bearer key must never reach a browser. A purely client-rendered app has nowhere safe for it, and per-site isolation depends on that key staying private.
- You run WordPress. Use the plugin. It already implements the cursor, the tombstones, the media side-load and the Yoast and Rank Math mapping, and rewriting that in a theme's functions file is a bad afternoon.
- You want a hosted blog rather than a feed. This is a data contract: you render the HTML, own the routes and serve the pages. Nothing here hosts anything for you, by design — the articles must live on your domain to be worth having.
- You expect stock-level or real-time data. The feed carries finished articles and their metadata. It is not a product API, and an article's recommendations are only as current as its last refresh.
Two behaviours to design around rather than discover: a paused site keeps serving everything it has already published, and an article waiting for its drip slot is excluded from the feed and 404s on the single-article route, because to your site it does not exist yet.
Questions people ask
Do I need an SDK, or is plain HTTP enough?
Plain HTTP is enough. The feed is ordinary JSON over HTTPS with a bearer header, so curl, a cron script, a Go service or a PHP page can all consume it. There is a TypeScript package for Next.js sites and a plugin for WordPress, but both are conveniences over the same endpoints described here. If your stack is anything else, or you simply prefer to own the client, an HTTP call and a stored cursor are the whole of it.
How often should I poll the feed?
Hourly suits most sites, and a shorter interval alongside a cache purge is reasonable if you want changes to land sooner. Articles are drip-published on a steady cadence rather than dumped, so the feed moves gradually rather than in bursts. Because the cursor filters server-side, a poll with nothing new returns an empty list and your own cursor back, which is cheap enough to run often. If you want changes reflected in seconds instead, register the push webhook and keep a slower poll as a safety net.
What happens if my site is offline for a week?
Nothing is lost. The cursor is a timestamp, not a queue position, so when you come back you send the last next_since you stored and receive every article that changed while you were away, in the order the changes happened. Tombstones are part of that catch-up, so pages retracted during the outage are still reported and can still be removed. There is no expiry on a cursor and no separate backfill endpoint to call.
Can one site's key read another site's articles?
No. The key is bound to the site slug in the URL. The engine looks up that site, compares the SHA-256 of the bearer you presented against the digest stored on that specific row, and rejects a mismatch with 401 before any content is loaded. Presenting a valid key for site A against site B fails in exactly the same way as presenting no key at all. Per-site isolation is enforced at the auth layer, not by filtering results afterwards.
Do I have to render the HTML you send?
No, though it is the fastest path. Every record ships body_html already rendered and body_markdown alongside it, so you can pass the markdown through your own renderer, your own component library or your own sanitiser. The FAQ pairs and the table of contents are separate arrays rather than being baked into the body, precisely so you can style them yourself. The JSON-LD block is the one part worth injecting verbatim: it is authored to match the rendered article.
How do slug changes and canonical URLs work?
Each record carries canonical_path, which is the path the engine considers authoritative and the one it submits to search engines. Map that onto your route rather than deriving a path from the title. If a slug changes, the engine records a 301 from the old path to the new one and exposes it on the redirects endpoint, so you serve the redirect and the old URL keeps its ranking signal. Read that endpoint on the same schedule as the article feed.
Is the API included, or is it an add-on?
It is included. Every site registered on the engine gets its own pull key, and there is no tier on which the feed is switched off or charged for separately, because the feed is how articles reach a site in the first place. Billing is per published article from a prepaid balance, and reading the feed costs nothing on top of that, however often you poll. That is worth checking against alternatives: Surfer, for instance, states on its integrations page that its official API needs the API add-on or an Enterprise plan, checked 4 September 2026.
Read next
See what search engines and AI assistants find on your site
Free, and no account. Type your address and the check tells you what is missing and what we would write first.