integration · rest pull feed
One authenticated GET, and any stack has a blog
The engine is the source of truth; your site is the consumer. Poll one endpoint with a bearer key, follow the next_since cursor, render the HTML and JSON-LD you get back, and delete whatever arrives unpublished.
reviewed 2026-09-02 · by the IT Master editorial team · how we check facts
setup
- 01Ask us to register the site, or add it in the dashboard: registration mints the per-site pull key, shows it once, and stores only its SHA-256 digest.
- 02Put the engine origin, your site slug and the pull key in server-side environment variables, and keep them out of any browser bundle.
- 03Make the first call: GET /v1/publish/{site}/articles?limit=50 with an Authorization: Bearer header, and read the articles list and next_since cursor from the response.
- 04Persist next_since and send it back as ?since= on every later poll, so each request returns only what changed after the last one.
- 05Render each record: write body_html into your template, set the title and meta description, inject the jsonld block verbatim, and map canonical_path onto your own route.
- 06Render the supporting blocks the body deliberately omits: the faq array, the toc array, hero_image with its alt text and the internal_links list.
- 07Treat any record with status unpublished as a deletion: remove or 410 the page, drop it from your sitemap, and check /{site}/redirects for a 301 the engine recorded in its place.
- 08Proxy the derived feeds instead of generating them: /{site}/sitemap.xml, /{site}/rss.xml, /{site}/llms.txt, /{site}/robots.txt and /{site}/config.
- 09Serve the value from /{site}/indexnow-key at /<key>.txt on your own origin so IndexNow pings verify, then have us connect Search Console and submit the sitemap.
- 10Optionally register a push URL and secret so the engine POSTs each change to you, and verify the X-Thoth-Signature HMAC before trusting the body.
- 11Rotate the key whenever you need to by re-registering the site with a new one; the old digest is overwritten and the old key stops working immediately.
One key, one site, one GET
Every consuming site is a tenant with its own bearer key. The engine stores only the SHA-256 digest of that key, never the key itself, so it cannot be read back out of our database, and rotating it is a single re-registration.
Every read is a GET against /v1/publish/{site}/ with an Authorization: Bearer header. An unknown site slug answers 404. A missing or mismatched bearer answers 401. A key is scoped to one site and can never read another tenant's articles, which is the reason it is issued per site rather than per account.
Keep it server-side, in your environment or your secret manager, next to a database password. Nothing in this integration needs a key in the browser.
One behaviour to plan for: a paused site keeps serving everything it has already published. Pausing stops new articles being written and published. It does not withdraw what is already live, and a URL never starts answering 404 because someone pressed pause.
A pull key is stored only as a SHA-256 digest and is scoped to a single site, so one tenant's key can never read another tenant's articles.
Paging with since and next_since
The feed is incremental, not a snapshot. GET /v1/publish/{site}/articles?limit=50 returns a list of articles and a next_since cursor, ordered oldest change first by updated_at then id, so two calls for the same page return the same rows rather than whatever the query planner chose that time.
The limit defaults to 50 and is clamped between 1 and 200. Store next_since, send it back as ?since= on the next poll, and the server returns only rows whose updated_at is later than the cursor. When nothing has changed you get an empty list and your own cursor back, so an idle poll costs one cheap request.
A page is widened rather than cut when several rows share an updated_at to the microsecond. Cutting inside a tied group would skip those rows forever once the cursor moved past them.
Articles with status scheduled are excluded. One waiting for its drip publishing slot is written and stored but is not live, so the single-article route 404s for it too.
Tombstones, so a takedown actually lands
The same feed carries removals. A retracted article comes back with status unpublished and, where one was recorded, a takedown_reason. Act on it: delete the page or return 410, and drop the URL from anything you generate.
Trust the status field on the record in front of you. It is stamped from the live row at the moment you fetch it, so a retraction reaches you as unpublished even though that article was stored as published on the day it first shipped. Read the field, rather than assuming a feed only ever adds and updates.
When a takedown is a consolidation rather than an editorial removal, the engine records a 301 from the retired path to the surviving one. Read the map from /{site}/redirects and serve it, so the old URL keeps its ranking signal instead of dead-ending.
Filter tombstones out of what you display; act on them when you sync. Those are two different code paths, and conflating them is the usual bug.
A tombstone is an article that arrives in a content feed with status unpublished, telling the consuming site to remove a page it has already published.
What one article record contains
Every record carries a schema_version and is built so you never have to generate the SEO furniture yourself.
- Body — body_html rendered and ready, plus body_markdown, excerpt, toc, word_count and reading_time_min.
- Meta — title, meta_description, slug, canonical_path, robots, hreflang, text direction, and complete og and twitter blocks.
- Schema — jsonld, authored by the engine and authoritative. Inject it verbatim rather than rebuilding it; see JSON-LD.
- Media — hero_image, cover_image and an images array, each entry carrying alt text.
- Blocks — an faq array of question and answer pairs, stripped out of the body so you cannot render them twice, plus internal_links, outbound_citations and recommended_products.
- Provenance — author, version, content_hash and first_party_grounded.
content_hash answers the question "did anything actually change?" between versions, which is what lets you skip a rebuild when a republish touched nothing. first_party_grounded tells you whether that article drew on your own product data rather than public sources.
The rest of the SEO surface, same key
Articles are one endpoint of several, and the same bearer opens all of them.
- /{site}/sitemap.xml and /{site}/rss.xml — generated feeds you can proxy straight through.
- /{site}/llms.txt — the AI-crawler manifest; see llms.txt.
- /{site}/robots.txt — your robots policy, including the single toggle that flips GPTBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot between allow and disallow.
- /{site}/config — verification tags, analytics IDs, Open Graph defaults, locale and sitemap tuning, edited in the dashboard and applied by your site at request time with no redeploy.
- /{site}/redirects and /{site}/clusters — the 301 map, and the topic-cluster graph behind the internal links in your articles.
- /{site}/indexnow-key — serve this value at /<key>.txt on your origin so IndexNow pings verify against a file you host.
Proxying these is usually less work than generating them, and your sitemap then agrees with your feed by construction rather than by discipline.
Poll, or be pushed
Polling is the default and asks nothing of your infrastructure beyond an outbound request on a schedule. If you would rather not poll, register a push URL and a push secret, and the engine POSTs each publish, update and takedown to you as an event and an article.
Verify before you trust. X-Thoth-Signature is a hex HMAC-SHA256 of the exact request bytes, keyed by your secret. Recompute it over the raw body, compare in constant time, and read the event name from X-Thoth-Event. Delivery is at-least-once with exponential backoff and a dead-letter entry once the retry budget is spent, so make your handler idempotent on slug and version. The mechanics are set out under webhooks.
Nothing stops you doing both. Take the push as a nudge for instant updates, and run a nightly cursor pass to reconcile anything a failed delivery missed. The endpoint reference is in the docs.
What happened before your GET
By the time an article is in your feed it has been through the whole pipeline. Topics come from measured demand: DataForSEO plus your own Search Console queries. Research reads the ranking pages, forums and, where it exists, your first-party data — product records, datasheets, support answers. A Claude model writes it. Then judges from a different vendor, Gemini and GPT, run fact-check, novelty against the competing pages, E-E-A-T critique and AI-tell detection, with up to three revision rounds. The model that writes is never the model that checks; that is cross-model validation.
Stated as measured, not promised: across 186 Standard-tier runs, 51% of drafts passed every check first time; the rest were revised or rejected. Pro-tier judges are stricter: 25% first-pass, across a small sample of 16 runs.
You pay per published article from a prepaid balance, with no subscription; a draft that fails the checks is not charged at the full rate. Tiers are on pricing.
Nothing reaches the feed until judges from a different vendor than the writer have passed it. A draft that fails is revised or dropped, not shipped.
questions people ask
Do I need an SDK, or is plain HTTP enough?
Plain HTTP is enough. The feed is ordinary JSON over HTTPS with a bearer header, so curl, a cron script, a Go service or a PHP page can all consume it. There is a TypeScript package for Next.js sites and a plugin for WordPress, but both are conveniences over the same endpoints described here. If your stack is anything else, or you simply prefer to own the client, an HTTP call and a stored cursor are the whole of it.
How often should I poll the feed?
Hourly suits most sites, and a shorter interval alongside a cache purge is reasonable if you want changes to land sooner. Articles are drip-published on a steady cadence rather than dumped, so the feed moves gradually rather than in bursts. Because the cursor filters server-side, a poll with nothing new returns an empty list and your own cursor back, which is cheap enough to run often. If you want changes reflected in seconds instead, register the push webhook and keep a slower poll as a safety net.
What happens if my site is offline for a week?
Nothing is lost. The cursor is a timestamp, not a queue position, so when you come back you send the last next_since you stored and receive every article that changed while you were away, in the order the changes happened. Tombstones are part of that catch-up, so pages retracted during the outage are still reported and can still be removed. There is no expiry on a cursor and no separate backfill endpoint to call.
Can one site's key read another site's articles?
No. The key is bound to the site slug in the URL. The engine looks up that site, compares the SHA-256 of the bearer you presented against the digest stored on that specific row, and rejects a mismatch with 401 before any content is loaded. Presenting a valid key for site A against site B fails in exactly the same way as presenting no key at all. Per-site isolation is enforced at the auth layer, not by filtering results afterwards.
Do I have to render the HTML you send?
No, though it is the fastest path. Every record ships body_html already rendered and body_markdown alongside it, so you can pass the markdown through your own renderer, your own component library or your own sanitiser. The FAQ pairs and the table of contents are separate arrays rather than being baked into the body, precisely so you can style them yourself. The JSON-LD block is the one part worth injecting verbatim: it is authored to match the rendered article.
How do slug changes and canonical URLs work?
Each record carries canonical_path, which is the path the engine considers authoritative and the one it submits to search engines. Map that onto your route rather than deriving a path from the title. If a slug changes, the engine records a 301 from the old path to the new one and exposes it on the redirects endpoint, so you serve the redirect and the old URL keeps its ranking signal. Read that endpoint on the same schedule as the article feed.
related
See what search engines and AI assistants find on your site
Free, no account. Type your address and we show you what is missing and what we would write first.