Specs, skills and memory: getting consistent Sitecore code from AI agents

Give an AI coding agent the same Sitecore ticket three times and you will get three components. Each will look reasonable in review, and no two will agree.

We measured this for our session at the Sitecore User Group Jaipur, running one ticket through Claude Code nine times while adding a written specification and then house skills to the repository. With the model and prompt held constant, the results tracked how precisely the repository wrote down the decisions the ticket left open, which has practical consequences for how Sitecore teams should set up their repositories for agents.

The problem is the question the ticket cannot answer

The ticket was the kind a business user raises every week. An extract:

We need to scaffold a card listing component for the site. The component will have a title and then 1 to many cards, a maximum of 8 cards are supported… Content authors can choose the sort order of cards.

The full ticket adds that every field must be editable in Page Builder, that the component hides when there are no cards, and that the Sitecore templates and renderings already exist.

As a business requirement, that is complete. Technically, it leaves the most important question open, because there are at least four ways to build a card listing in Sitecore:

• A rendering contents resolver returning the datasource’s children, which loses the wrapper’s own fields unless you write a custom resolver and deploy it to the CM.
• A multi-select field on the wrapper, listing the active cards in the author’s chosen order and serialised into the Layout Service response.
• A wrapper with child items, fetched with a query to Experience Edge.
• A wrapper containing a placeholder, with each card added as its own rendering.

All four are buildable, and choosing between them is a technical decision. The choice also changes what an author does, and “a maximum of 8 cards” is much harder to enforce when cards arrive through a placeholder.

With only the ticket and the 14 skills a Content SDK scaffold ships with, three runs produced 124, 108 and 119 lines of code, and three different production behaviours from that one sentence about eight cards. One run warned the author about the limit and then rendered a ninth card on the published page anyway. One truncated without a warning. One truncated and warned.

On the architecture the runs agreed, and all three were wrong. Each composed the cards through a placeholder, when the templates already built in Sitecore used a multi-select field. Two of the three stopped to ask which approach we wanted. The ticket did not contain the answer, so nothing running unattended could have got it right.

The agent was asked a question its input did not contain, and that is the normal condition of brownfield Sitecore work. Most AI spec tooling assumes the person who writes a specification builds from it straight away. On real programmes there is a gap in time and a gap in person: a specification phase or a refinement session, then weeks later a ticket picked up by somebody who was not in the room. Whatever picks up that ticket needs the answers to be in the repository.

How we tested it

We scaffolded a Sitecore Content SDK application and seeded it to look like a real estate: four card-shaped components, three conventions for reading rendering parameters, and a data-access migration nobody had finished. Then we ran the ticket in three stages of three runs: the repository as it stood, the same with a specification added, and the same again with house skills. Every run used the ticket verbatim, started from the same commit and ran in a fresh session. A clarifying question got one scripted reply, Use your judgement, because an unattended agent has nobody to ask.

We compared token usage on a weighted total (fresh input at 1.0, cache writes at 1.25, cache reads at 0.1, output at 5.0), a relative measure for comparing the stages, and inspected every implementation by hand. Nine runs gives a small comparison of outcomes, so treat the figures as indicative. A planned memory stage was dropped, because by the time the first two stages had run there was little left for memory to decide.

Three kinds of context, three different jobs

Specs, skills and memory map onto the questions a developer asks before starting work. What am I building, and why? Do I know how, or has someone already worked it out? Have I done this before? They are easy to blur together, and the results show why they are worth keeping apart.

A specification decides what gets built

We wrote the specification with OpenSpec: seven requirements and 14 WHEN/THEN scenarios, plus a design document and an architecture decision record for each significant choice. The requirement that did the most work reads:

The component SHALL take the cards it renders from the Cards multi-select field on the wrapper datasource item. It SHALL NOT issue a render-time query for the datasource’s children, and SHALL NOT accept cards through a placeholder.

The two SHALL NOTs are what make it useful, because they rule out the alternatives explicitly. With the specification in the repository and the ticket unchanged, all three runs read cards from the Cards field, none published a ninth card, all three handled the empty state the same way, and none asked a clarifying question. An agent that no longer needs to ask can run without somebody waiting to reply.

On tokens, the specification made no measurable difference. The spec runs used 5% more than the ticket-only runs, well inside the ±27% those runs varied by among themselves. The measurable price was about 90,000 tokens of cache writes per run, the specification being loaded once and read back cheaply. Writing it took about as much as one and a half ticket-only runs. The return on that comes through consistency and correctness, in review and rework avoided, which we did not measure.

A fair objection is that good skills make a separate spec layer redundant. We keep them apart. A specification can be run through your own review and approval steps, it leaves behind artefacts tied to one piece of work, and while you are turning a requirement into an architecture, a repository full of implementation skills is noise.

Skills decide how it gets built here

A good specification is portable. It says what has to be true, can be satisfied more than one way, and could be carried to another codebase. That portability showed: the spec said only that layout should use classes on the component’s own markup, and the three runs produced three different stylesheets.

Skills are the deliberately non-portable half: how we build things, in this repository. For the third stage we added two house skills, for component conventions and for Tailwind styling, plus a test harness. The ticket still said nothing about Tailwind, file layout or tests. Compared with the spec stage:

• Bespoke stylesheets fell from three runs of three to none.
• An adjacent .types.ts file went from none to all three.
• Passing unit tests went from none to all three, with 18, 22 and 22 tests.
• Passing rendering parameter styles through went from two runs to three.

Seven of our eight house rules were applied in every run. The process converged too. A turn is one step an agent takes on its own, such as a file read, an edit or a test run. The ticket-only runs took 62, 91 and 54 turns; the skills runs took 107, 110 and 111. The spread fell from 27.5% to 1.8%: three sessions that shared no context took the same path.

This stage did cost more, 82% more weighted tokens than the ticket-only runs. It also produced 133% more code, including types, a logic module and tests that no earlier stage wrote, and per line delivered it was the cheapest of the three stages. The extra cost was scope. If you want the tests, you were always going to pay for them.

A skill is a Markdown file with two required frontmatter fields:

The description is the triggering mechanism, the index entry the agent scans to decide whether to read further. Only frontmatter is always in context. The body loads when the description matches, so a repository can carry dozens of skills at the cost of their descriptions alone. A vague description means the skill never loads, however good its body.

Memory records what happened, which is why it goes stale

Memory answers the third question, and “before” should not stop at the project boundary. You may have solved the same problem six months ago for another client.

Claude Code has two built-in mechanisms. CLAUDE.md is committed and always loaded, which makes it work like a constitution, and every line in it is paid for on every task. A memory directory on your machine holds one fact per file behind a one-line index, read only when relevant. That is local, unreviewed and absent from every clone. A personal knowledge base is rich but on one machine; a committed file is shared but thin. Knowledge that is both team-wide and cross-project is the gap shared memory services such as mem0 sell into, on their own account through scope and semantic retrieval.

The difference that matters is this. Specs and skills are prescriptive: they say what should be true, so a wrong one is a bug you can find and fix. Memory is descriptive: it says what happened, and a wrong memory was usually true when it was written. “Prefer a multi-select over a query for card data” can hold for years. “We chose a multi-select on this project” expires with the project. Memory needs a review discipline of its own, and in a shared store one stale fact can mislead a whole team.

Precision predicts convergence

The most useful finding is one that held at every layer. What was written down precisely converged, and what was written loosely diverged.

The specification named the field cards come from, and three runs built the same thing. It said “layout classes” and nothing more, and three runs wrote three stylesheets. Our component skill said to export logic as a pure function without naming a file, and one run kept the logic in the component while two created a separate .logic.ts. Everything the skill named explicitly, all three runs agreed on.

Consistency comes from stating the decisions that matter exactly and ruling out the alternatives. The effect also shows up within a single stage, which makes it the easiest finding here to check for yourself.

Mistakes worth avoiding

We made most of these ourselves while building the demo.

• Installing skills nothing routes to. Vercel’s Next.js skills were available in six runs and never consulted. A Content SDK scaffold’s CLAUDE.md points to AGENTS.md, which points to Skills.md, which lists only the skills in .agents/skills/, and the Vercel skills lived elsewhere. Once we moved the Next.js skill there and listed it, every run read it. Loaded and found are different things.
• Letting a specification widen scope. The ticket said design system integration was a separate work item. All three spec runs wrote CSS anyway, because our layout requirement invited it, and no ticket-only run had. A spec that outlives its ticket also overrides it. The fix is a review question: does this requirement belong to this capability, or to work that was deliberately deferred?
• Encoding a convention you have not tested. Our skill said to detect editing with page.mode.isEditing. All six runs with a specification used page.mode.isNormal instead, reasoning that preview and the Design Library are modes too, and a component that wrongly hides cannot be clicked to fix. They were right, and the skill’s own rule that the spec takes precedence let them overrule it. An agent will usually apply a wrong convention faithfully, so the convention has to be right.
• Treating community skills as trusted and timeless. Skills are now installed with npx and pinned in a skills-lock.json. They are dependencies that shape what an agent does to your codebase, and they can be pinned to a framework version. The Content SDK moves to newer versions of Next.js frequently, and when the framework moves and the skill does not, the skill becomes confidently wrong with no compiler to catch it.
• Judging the approach on tokens. Per run, the specification made no measurable difference and the skills cost more. The case rests on consistency and correctness, so measure those.

Where to start

1. Write a skill the second time you explain something. Workflow skills such as commit conventions and branch naming are small, quick and immediately useful. Check for a published skill first, and fork one that nearly fits.
2. Specify any component with a real architectural choice in it. Use SHALL NOT to rule out the alternatives, and record the decision in an architecture decision record. It pays back every later time somebody touches that component.
3. Put house conventions in skills, precisely. Name the files, fields and methods involved. Write the description as the trigger it is.
4. Trace the routing. Follow the chain from CLAUDE.md or AGENTS.md to every skill you rely on. If nothing points at it, assume it will not be read.
5. Manage skills as dependencies. Pin them, read them before you adopt them, and re-check them whenever the Content SDK or Next.js is upgraded.
6. Separate lasting preferences from project facts in memory, and review what an agent has written before anyone else relies on it.
7. Run the same task more than once. Three runs of one ticket, before and after a change, show how consistent your repository makes the agent.

What this means for Sitecore teams

Less of the work is producing every line of implementation. More of it is intent, architecture, constraints and validating what comes back, plus a job few teams have budgeted for: maintaining the context itself, as skills drift, memories expire and specs are superseded.

That makes engineering judgement more important. Every result above rests on a decision somebody made about architecture, conventions or which memory is still true, and an agent applies those decisions consistently whether or not they are right. The upside is that a team’s Sitecore experience finally has somewhere to live other than people’s heads and review comments. It is worth being realistic about time, though. Capability gained does not automatically become time returned, and the work tends to expand to fill it.

Key takeaways

• Specs tell agents what to build. A precise specification removed every clarifying question and turned three implementations into one.
• Skills tell them how we build. Conventions that never appeared in the prompt were applied in every run, and the agents’ process converged along with their code.
• Memory tells them what we have learned, and it goes stale by nature, so it needs reviewing.
• Precision predicts convergence. Write down the decisions that matter exactly, and rule out the alternatives.
• Judgement holds it together. An agent is only as right as the decisions it is given.

This article draws on our talk Sitecore development with specs, skills and agent memory. The recording is on YouTube.