Build Your Own AI Content Agent: 6 Parts Beyond the Prompt

If you are weighing a weekend spent building your own AI content agent, the honest estimate is that the drafting prompt is roughly the easy 10%. The other 90% is six subsystems — topic deduplication, link resolution, schema validation, scheduling, retry handling and cost ceilings — and each one needs a design decision you can defend, not a library import. The numbers make the case: a study of citation reliability across 53,090 URLs found that 3–13% of citation URLs generated by commercial LLMs and deep research agents are hallucinated, with 5–18% failing to resolve at all. Link checking alone is a permanent subsystem in anything you ship.

This is not a "don't build it" post — several builders cited below did exactly that and are happy with the result. It's an inventory: what each part does, what breaks when you skip it, and what it costs you.

The prompt is the small box in the diagram

The pattern has a canonical paper. Google's Hidden Technical Debt in Machine Learning Systems (Sculley et al., NIPS 2015) argued that the learning code is a small fraction of a real system, surrounded by glue and plumbing, and that such systems "may incur massive ongoing maintenance costs": "it is dangerous to think of these quick wins as coming for free" (paper).

Eleven years later, Anthropic's engineering team wrote almost the same sentence about agents: "the gap between prototype and production is often wider than anticipated," and — the line worth taping to your monitor — "agents are stateful and errors compound" (Anthropic engineering). A builder who automated his own SEO blog with a git-native pipeline put it from the other end: "The workflow is the moat. Not the model. Not the prompt." Start by asking "how to make bad content harder to ship" (Between the Prompts).

Six parts, then. For each: the naive version, what breaks, and the decision you have to make.

Part 1 — Topic deduplication: string matching is not memory

The naive version. Keep a published.jsonl of slugs and titles. Before generating, check whether the idea is already in it.

What breaks. "10 Ways to Speed Up Postgres" and "How to Make Postgres Faster: 10 Fixes" are two strings and one article, and an exact-match ledger waves the second one through. On a thirty-post blog with an agent running weekly, near-duplicates show up within months, because niche research keeps landing on the same high-volume keywords.

The design decision. Semantic dedup is the standard answer: embed each candidate idea, compare against everything written and planned, reject above a similarity threshold. The components are commodity. SemHash pairs Model2Vec embeddings with the Vicinity ANN backend and a user-set threshold (their example uses 0.9); its benchmark deduplicates 1.8 million text records in about 83 seconds on CPU. If your vectors live in Postgres, pgvector gives you cosine distance and states the fork clearly: "By default, pgvector performs exact nearest neighbor search, which provides perfect recall." Indexes trade that recall for speed, and at blog scale you'll never need them.

The hard part isn't the vector store. It's two judgement calls nobody can make for you.

What you embed. Embed the title alone and you catch rewordings but miss two posts with different titles and the same outline. Embed the whole draft and everything looks similar, because every post on your blog is about your product's world. The practical middle is target keyword + search intent + outline headings — roughly what the Between the Prompts pipeline does, refusing to create duplicate URLs where existing content covers the ground.

The threshold. There is no number to copy. Work comparing Levenshtein distance, cosine similarity and SBERT embeddings across 2,000 annotated title pairs from an 11-million-record corpus found string-similar titles that were semantically distinct, and reported no single optimal threshold (arXiv:2410.01141). Calibrate against your own back catalogue instead: run your existing posts through it pairwise and move the number until the flags match your judgement.

One honest caveat. The SEO stakes are smaller than the dedup discourse implies. Google's canonicalisation guidance contains no penalty language for duplicates; without a specified canonical, "Google will identify which version of the URL is objectively the best version to show to users in Search" (Google Search Central). John Mueller has pushed back on "keyword cannibalization" as a diagnosis, noting the label often masks thin content or weak internal linking (Search Engine Journal).

So the real cost of missing dedup isn't a penalty. It's paying to generate a post that competes with a better one you already have, then spending your own time noticing — a founder-time cost, which is the currency that decides this whole question.

Also decide: what happens on a collision (regenerate, or route to "update the existing post"?); whether the ledger includes planned-but-unwritten ideas; and how you allow a deliberate re-treatment a year later.

Part 2 — Link resolution: you can't ask the model to check the model

The naive version. Fire an HTTP GET at every URL in the draft; if it returns 200, ship it.

What breaks. Both halves. The links are wrong more often than you expect, and so is the check.

Start with the citations. The study behind this post's opening number found 3–13% of URLs hallucinated on the DRBench dataset (10 models, 53,090 URLs), defined precisely: "they have no record in the Wayback Machine and likely never existed." On ExpertQA (3 models, 168,021 URLs), the non-resolving rate was 8.22%. The finding most relevant to a research-then-write agent is counterintuitive: deep research agents hallucinated citations at 10.7% versus 4.8% for search-augmented LLMs. More autonomy has so far meant more need for an external check. (arXiv:2604.03173)

Independent work agrees on direction. GhostCite benchmarked 13 LLMs generating 375,440 citations, with hallucination rates from 14.23% to 94.93% by model. The sentence that kills the obvious shortcut: "LLMs perform poorly at citation validation, achieving only 38% average accuracy, even lower than random guessing" (arXiv:2602.06718). You cannot add a verifier prompt and call it done. Something has to make an actual HTTP request.

Now the check itself. A 200-or-bust verifier produces false failures across a large slice of the real web:

• Link rot is the baseline tax. Pew Research Center found 25% of webpages that existed at some point between 2013 and 2023 are no longer accessible, rising to 38% of pages that existed in 2013. Even 2023 pages showed 8% inaccessible (Pew Research Center, May 2024).
• Bot walls create false 404s. Cloudflare's 2025 Radar Year in Review reports AI crawlers were the most frequently fully disallowed user agents in robots.txt (Cloudflare Radar), and since 1 July 2025 new domains on Cloudflare block AI crawlers by default unless they pay (Content Independence Day). Your verifier's user-agent is a product decision now, not a header.
• The rest is plumbing. Servers that 405 on HEAD but answer GET. 403s and 429s from bot protection on pages that render fine in a browser (why automated checkers get 403s). Soft 404s returning 200. Redirect chains ending somewhere unrelated. Paywalls and JS-rendered pages.

The design decision. HEAD with a GET fallback. A status-code policy separating broken (404, 410, 500) from retry-later (429, 503). A soft-404 body heuristic. A timeout and concurrency budget across twenty-plus URLs. And the cheapest good idea in this post, borrowed from that same study: pair the HTTP request with a Wayback Machine lookup, and you can tell a hallucinated URL (no archive record) from a rotted one (archived, now unreachable).

The bigger fork is what happens when a link fails: drop the sentence, swap the citation, or fail the run and re-research? Each answer implies a different retry cost, which is why this part wires into Parts 5 and 6. It's worth doing — the same study's mitigation tool cut non-resolving URLs by 6–79×, to below 1% across all three models tested.

This is the step Magic Share treats as non-negotiable: no draft is saved until every cited URL resolves, and a verification report ships with each draft. Build it or buy it, but build the HTTP request, not the verifier prompt — and be a polite verifier while you're at it: identify yourself, respect robots.txt, cache results.

Part 3 — Schema validation: structured outputs guarantee shape, not length

The naive version. Turn on structured outputs, hand the model a JSON schema for your frontmatter, done.

What breaks. Constrained decoding does guarantee shape. OpenAI's Structured Outputs promises responses that "adhere to your supplied JSON Schema" (OpenAI docs); Anthropic's, GA for Claude 4.5 and later, promises "Always Valid… Type Safe… Reliable" (Claude docs).

Then read the unsupported lists, which are the interesting part. Anthropic's implementation does not support minLength/maxLength, minimum/maximum/multipleOf, recursive schemas or external $ref. OpenAI's has its own rules: root must be an object, every field must be required, additionalProperties: false is mandatory, and allOf, not and if/then/else are unsupported — plus a separate failure mode where a refusal field arrives instead of your schema.

Now look at what blog frontmatter needs enforced. Title under 60 characters. Meta description between 120 and 158. Slug kebab-case and unique. Target keyword present in the title. Every one is a length or semantic rule — exactly the category constrained decoding doesn't cover. So the rules carrying the value live in your validator with your repair loop, and every repair attempt is another paid model call.

And the numbers are yours to choose. Google is explicit that there's no hard limit: "While there's no limit on how long a <title> element can be, the title link is truncated in Google Search results as needed" (title link docs). Same for descriptions — no limit, truncated as needed, and Google only sometimes uses yours at all (snippet docs). Your validator is enforcing a house style. You pick the numbers, defend them and version them: a product decision hiding inside a JSON schema.

The downstream half. Astro content collections validate frontmatter with Zod at build time; a violating file produces "a helpful error during the build process" and the build stops (Astro docs). A schema miss in an auto-drafted post isn't a warning in a log — it's a red deploy from a run you slept through.

Also decide: where validation lives (model-side, your validator, the site build — realistically all three); how many repair attempts before the run fails; and whether a repair re-runs the whole draft or patches one field, since those differ by an order of magnitude in tokens. Real pipelines end up with a hand-rolled checklist rather than one elegant schema — the Between the Prompts gates are title present, description ≥80 characters, ≥650 words, no placeholder text, plus a hard "no research file, no publish" rule.

Part 4 — Scheduling: "06:00 UTC" is not one cron line

The naive version. A cron expression. 0 6 * * *. Ten seconds of work.

What breaks. The cron line really is ten seconds. The guarantees underneath it are not.

Punctuality. GitHub documents its own scheduler's limits: "The schedule event can be delayed during periods of high loads of GitHub Actions workflow runs. High load times include the start of every hour." On public repositories, "scheduled workflows are automatically disabled when no repository activity has occurred in 60 days" (GitHub docs). Community threads report typical drift of 20–60 minutes, with the usual workaround of scheduling off the top of the hour (community discussion). Vercel publishes precision as a plan feature — Hobby once per day at per-hour precision, Pro and Enterprise once per minute — noting bluntly that "Vercel cannot assure a timely cron job invocation" (Vercel cron docs).

Duration. This is the wall people hit first. Vercel Functions max out at 300 seconds on Hobby (hard), with 800s maximum on Pro and Enterprise; Vercel's own advice for longer work is to move to Vercel Workflows (function limits). AWS Lambda is a hard 900 seconds: "Code can run for up to 15 minutes in a single invocation" (Lambda quotas).

A five-to-nine-minute agent run on Vercel Hobby cron therefore fails twice over: the schedule can't fire more than daily, and the function dies at 300 seconds. That's the moment a weekend build becomes "now I need a queue."

The design decision. Plain GitHub Actions is a better answer than it gets credit for: Free includes 2,000 Actions minutes a month, and standard runners are free for public repositories (Actions billing). A daily nine-minute run is roughly 270 minutes a month — comfortably inside the free tier on a private repo. If you want durability instead, buying it is cheap: Inngest's Hobby tier is $0/month with 50,000 executions, wrapping steps in step.run() so "each step automatically handles retries, state, and failure recovery" (Inngest pricing). Trigger.dev's free tier gives $5 in monthly credits and 20 concurrent runs, and states that "tasks can run for as long as you need, with no timeouts" (Trigger.dev pricing).

Also decide: whether "due" means a fixed weekday or a spacing rule since the last publish; the catch-up policy when a run is missed (skip, or draft two?); an overlap lock so a delayed run and the next one don't both fire; and timezone handling, since cron is UTC and your cadence probably isn't.

Part 5 — Retry handling: retrying an agent is not retrying an HTTP call

The naive version. Wrap the run in a try/except with three attempts and exponential backoff.

What breaks. First, arithmetic. Inngest puts it cleanly: "If you have five steps with 99% reliability each, your overall success rate drops to 95%. With ten steps, you're at 90%" (Inngest). Extend that to a real content pipeline — ideas, dedup, research fan-out, outline, draft, metadata, link verification, image, commit, PR, notify, log — and twelve steps at 99% each is 0.99¹² ≈ 88.6% end-to-end. On a daily cadence that's three or four failed runs a month, roughly one a week, forever.

Second, determinism. "Agents are probabilistic. The same prompt can produce different responses across calls." A retry doesn't re-run the same computation; it runs a new one that may take a different path. And retries aren't free: "LLM calls are expensive. Re-running them on every retry doubles or triples your inference costs." What these workflows need is "exactly-once semantics for operations that cost money or have side effects."

Anthropic's production experience points the same way: stateful agents need "durable execution with error recovery" and checkpointing that lets a run resume rather than restart. Their early failure modes are instructive for a content agent — spawning dozens of subagents for simple queries, "endless web searching for nonexistent sources," and "preferring SEO-optimized content farms over authoritative academic sources."

The design decision. Three sub-decisions, all yours.

Backoff. OpenAI's guidance is to "automatically retry requests with a random exponential backoff" and "add random jitter to the delay," with limits enforced across RPM, RPD, TPM, TPD and IPM — whichever you hit first (rate limits). AWS supplies the numbers: with 100 competing clients, jittered exponential backoff more than halved total call counts, and "the return on implementation complexity of using jittered backoff is huge" (AWS Architecture Blog). Anthropic hands you a retry-after header on 429, and warns that "a rate of 60 requests per minute might be enforced as 1 request per second" (Claude rate limits).

Idempotency. The part people skip and regret. Stripe is the reference implementation: keys stored at least 24 hours, replays returning the original status code and body — even 500s are replayed — and mismatched parameters on the same key raising an error (Stripe docs). Your publish step is the side effect that matters. GitHub's create-pull-request endpoint returns 201, 403 or 422 ("Validation failed, or the endpoint has been spammed") and warns that creating content too quickly "may result in secondary rate limiting" (GitHub REST docs). A retry after a timeout you didn't observe is how you get two pull requests for one post.

A hard stop. OpenAI's Agents SDK raises MaxTurnsExceeded when max_turns is exceeded; otherwise the loop ends only when the model returns final output with no tool calls, or on an unhandled error. You can pass max_turns=None and disable the only thing between you and an infinite loop (Agents SDK docs).

Also decide: what counts as a resumable unit; which steps need idempotency keys; what a partial success means (draft written, PR failed — do you re-draft?); and how a silently failed 06:00 run reaches you before Friday.

Part 6 — Cost ceilings: the cap you want is per-run, and nobody sells it

The naive version. Set a spend limit in the provider dashboard.

What breaks. Those limits are monthly and organisation-level. Anthropic's spend caps are per calendar month by tier — Start $500, Build $1,000, Scale $200,000 — and "once you reach your tier's spend cap, API usage pauses until the next month" (Claude rate limits). OpenAI's usage tiers work similarly, from $100 to $200,000 a month (OpenAI rate limits). Neither saves you from one bad run.

And agent runs are where the multiplier bites: Anthropic measured agents using about 4× the tokens of chat and multi-agent systems about 15× chat, with token usage alone explaining 80% of the variance in browsing performance. Their conclusion warns anyone whose cost instinct is "use a cheaper model and run it more": "upgrading to Claude Sonnet 4 is a larger performance gain than doubling the token budget."

The scale of what can go wrong is public record. OpenClaw creator Peter Steinberger ran roughly 100 Codex agents for 30 days: $1.3 million in API spend, 603 billion tokens and 7.6 million requests (OpenAI covered the bill). The instructive detail is the follow-up: Steinberger said turning off "Fast Mode" alone "would cut costs by 70 percent" (The Decoder). One configuration flag was most of the bill — which is the whole argument for showing per-run cost before the run.

The design decision. For per-key ceilings the open-source pattern is an LLM gateway: LiteLLM supports max_budget with budget_duration per key and model_max_budget per model, reading spend from a Redis counter so enforcement is consistent across workers (LiteLLM docs). Then answer: ceiling per run, per day, or both? Abort at the ceiling, or downgrade to a cheaper model and finish? Do subagents inherit the parent's remaining budget?

What a post actually costs (an estimate)

Assume ~20 sources at roughly 2,500 tokens each — Anthropic's rule of thumb for an average 10 kB web page — so ~50k tokens of raw source text; a loop that re-sends context about four times across planning, research, drafting and verification, so ~200k input tokens; ~30k output tokens; and 20 web-search calls.

At current OpenAI list prices — GPT-5-mini at $0.25 per 1M input, $2.00 per 1M output, web search $10 per 1,000 calls on reasoning models:

• 200k input × $0.25/M = $0.05
• 30k output × $2.00/M = $0.06
• 20 searches at $10/1k = $0.20
• ≈ $0.31 per post

At current Anthropic list prices — Claude Sonnet 4.5 at $3 per 1M input, $15 per 1M output, web search $10 per 1,000 searches:

• 200k input × $3/M = $0.60
• 30k output × $15/M = $0.45
• 20 searches at $10/1k = $0.20
• ≈ $1.25 per post

These are estimates, not quotes — your context re-sends and source count will differ. But the shape holds, and it reframes the build-versus-buy question entirely: at four posts a month you are arguing over $1 to $5 of API spend. Inference isn't the cost. Your weekends are.

Two levers move real money at higher volume: Anthropic prompt-cache reads bill at 0.1× base input, and the Batch API takes 50% off both directions. One trap runs the other way — Claude 4.7 and later use a newer tokenizer that "produces approximately 30% more tokens for the same text," so a model upgrade can raise your per-post cost with no price change at all.

This is the shape of decision Magic Share's model picker and spend guardrails are built around: pick a model from GPT-5.4 Nano up to Claude Opus 5, see the per-run cost before the run starts, and let a monthly credit bank with per-run cost ceilings cap what any run can spend.

The seventh part: it is never finished

Model retirements are scheduled maintenance. Anthropic gives "at least 60 days' notice before model retirement for publicly released models," and the record is concrete: claude-sonnet-4-20250514 and claude-opus-4-20250514 were deprecated 14 April 2026 and retired 15 June 2026. "Requests to models past the retirement date will fail." Parameters move too — temperature, top_p and top_k are deprecated on Claude Opus 4.7 and later and return a 400 if set to a non-default value (model deprecations).

Scaffolding ages in the other direction. Nicolas Bustamante, building agents since GPT-3.5, describes deleting an entire embedding/RAG pipeline once long-context agents could grep and read files directly — "a year of engineering. Gone." A 310-line system prompt compressed to 104 lines on newer models; JSON validators and regex parsers became obsolete. He puts scaffolding half-life at "weeks to months" (hard lessons).

Evals and traces never get built on a weekend. Anthropic's approach starts small — about 20 queries and an LLM-as-judge scoring 0.0–1.0 on accuracy, citations, completeness, source quality and tool efficiency — while keeping humans in the loop, because human evaluation "caught edge cases, hallucinations and subtle biases automation missed." Bustamante is blunter: "Your eval is your moat." Observability follows the first bad run, when a log line fails to tell you which link broke or which subagent looped; Langfuse is the common open-source answer.

And the human stays. Stack Overflow's 2025 Developer Survey found only 3.1% of developers highly trust AI accuracy while 45.7% distrust it, and 66% name "almost right, but not quite" as their biggest frustration (survey results). Design for that review rather than around it — which is also the argument our post on what Google's scaled content abuse policy actually bans makes: the offence is volume without value, not authorship.

So: build or buy?

Build if the pipeline is the product, or if you want the education. Every builder cited here says the build improved their editorial thinking, not just their throughput. Bryan Chua's six-agent editorial pipeline recovered 2–3 hours per post on mechanical phases, and his summary is worth reading in full: "a pipeline that makes my thinking more rigorous and my publishing more consistent, in exchange for real setup cost, ongoing calibration, and the humility to accept that the output will always need me." His failure modes are the ones you'll meet — agents that couldn't arbitrate contradictions with each other, a critic agent that drifted into "skepticism for its own sake" (Bryan Chua). Karl-Johan Spiik's transcript-to-WordPress pipeline broke on a transcription quota, a server with no swap and a plugin blocking REST endpoints (karlex.fi).

Buy if the blog is the product. One consultant's build-versus-buy framework for AI SEO tooling draws the line at under 200 content operations a month (buy), 500+ (re-evaluate) and 2,000+ (custom almost always cheaper), with build costs of $25k–$40k and 5–15 hours a month of maintenance (nikoalho.fi). Treat the dollar figures as agency-scale estimates rather than audited data — but a founder publishing four to eight posts a month sits an order of magnitude below the build threshold on that framework's own terms.

And concede the strongest counter-argument. Three of the six parts are close to commodity: durable execution, retries and scheduling are a free tier away, and structured outputs are GA on both major providers. What is not commodity is your dedup threshold, your link-failure policy, your metadata conventions and the taste behind each. It's also why the sensible starting point, per Anthropic's guidance on building effective agents, is direct API calls rather than a framework: abstractions "can obscure the underlying prompts and responses, making them harder to debug," and you should "consider adding complexity only when it demonstrably improves outcomes."

FAQ

How long does it take to build your own AI content agent? The drafting prompt is an afternoon. A pipeline with dedup, link verification, schema validation, scheduling, retries and cost ceilings is realistically several weekends, plus upkeep — one published framework estimates 5–15 hours a month for a custom AI SEO build. Model retirements also arrive on someone else's calendar: Anthropic gives at least 60 days' notice.

Can I just ask the model to check its own citations? No. The GhostCite audit found LLMs achieve only 38% average accuracy at validating citations — worse than random guessing — while separate work measured 3–13% hallucinated URLs and 5–18% non-resolving across 53,090 citations. Verification has to be an HTTP request against the live URL, ideally paired with a Wayback Machine lookup.

What does one AI-written blog post cost in API spend? On the assumptions above — ~20 sources, ~200k input tokens, ~30k output tokens, 20 web searches — roughly $0.31 on GPT-5-mini and roughly $1.25 on Claude Sonnet 4.5 at current list prices. At four posts a month that's $1–$5.

Is GitHub Actions good enough to schedule a blog agent? For a small blog, often yes. Free accounts get 2,000 Actions minutes a month, so a daily nine-minute run (~270 minutes/month) fits. The caveats are documented: scheduled events can be delayed under load, especially at the top of the hour, and scheduled workflows on public repos are disabled automatically after 60 days without repository activity.

Will duplicate posts get my site penalised by Google? Google's canonicalisation documentation contains no penalty language for duplicates, and John Mueller has pushed back on "keyword cannibalization" as a diagnosis. The honest case for deduplication is economic and editorial: don't pay to generate a post that competes with a better one, and don't make readers choose between two near-identical articles.

The short version

Six parts: dedup, links, schema, scheduling, retries, cost. Each is a design decision — what you embed and at what threshold, what happens when a link dies, who owns the length conventions, what "due" means, where the ceiling sits — and none of them are answered by a better prompt. Build them if the building is the point. Otherwise spend the weekend on the blog itself.

If you'd rather have those six decisions already made, Magic Share drafts researched, link-verified posts on your schedule and holds them until you approve — merging the pull request stays your job. See the plans, or sign up and try it on your first three posts, free.