What is measured, what is cited, and what is not.

What we measured ourselves

Everything in this section is reproducible from a clone of this repository, offline, with no account and no key. If any of it does not reproduce, that is a bug and we want the issue.

• The environment runs. The word game is a real environment in this repository, not a mock behind the page. Three commands score a model with it:

  git clone https://git.karti.ai/PIG/PIG-Demo
  cd PIG-Demo && uv sync --all-packages
  uv run vf-eval wordle-five -n 8

• The word lists rebuild byte for byte. Both source dictionaries are committed beside the script that filters them, so uv run python envs/wordle_five/words/build_words.py produces the same 4,603 answers and 11,846 legal guesses on any machine.
• The two implementations of the rules agree. The grader exists twice — once in Python for the environment, once in TypeScript for the board in your browser. Continuous integration scores every ordered pair of words in the answer list through both, about 21.2 million feedback patterns, and compares a hash. A divergence fails the build rather than showing you one result and scoring another.
• The scores on a recorded run are re-derived, not recited. Where a demo ships a verifier, the page recomputes the reward in your browser from the recorded turns instead of printing a number stored in the file. If a trace is truncated, it renders as unverifiable — never as zero.

What we cite from elsewhere

Three claims on this site are not ours. They are linked so you can go and check them instead of taking them from us.

Every rollout on this site is recorded, not live

When you watch a model play here, you are watching a file. No model is called while you read this page. That is a choice, and these are the reasons for it:

• The argument is about determinism. A page whose model says something different on every reload cannot be used to make the case that the grader stays put. The demo has to behave like the thing it is describing.
• There is nowhere to hide a cherry-pick. Every run is a file in the repository carrying its model, its seed and the date it was captured. A bad run is in there being bad. A live demo lets you re-roll until it looks good and tell nobody.
• It works. No key, no rate limit, no bill, no dependence on a provider being up during the ten minutes you are looking at it.

The cost is real: you cannot make a model react to a word you chose. If that is what you want, run the environment on your own machine against your own endpoint. It is three commands and the code is above.

The reward editor re-scores. It does not train anything.

Moving a weight changes how the recorded attempts are scored, and the page recomputes and re-ranks them. That is the entire mechanism. Nothing is fine-tuned, no model is called, and nothing learns while you drag a slider.

What it is there to show is narrow and worth the space: two runs can swap places when you change what you are paying for. That is a fact about your definition of good, not about the models — and it is the reason the definition is worth an hour of your team’s attention before the training run is worth a GPU.

The other half — actually training against the reward you just wrote — takes GPUs and hours, and it does not belong in a browser tab. That is what a trainer like prime-rl is for, and the number at the top of the home page is what it produced.

The verticals are our proposals

Eleven of the twelve entries in the lineup describe environments that do not exist. They are labelled “Proposed by PIG-Demo” on every surface that shows them, and that label is the literal truth: we wrote them.

• They are not on Prime Intellect’s roadmap and we do not speak for Prime Intellect.
• No customer asked for any of them. No company appears anywhere on this site as a customer, a reference or a logo, because none is one.
• Where a page names a KPI, we are describing how that function usually measures itself. We are not reporting anyone’s numbers, and there are no numbers on those pages to report.

Two of them carry a standing caveat on their own page — the legal redline, where grading the drafting half collapses into an LLM judge, and semiconductor timing closure, which has the best reward on the page and is the one thing here we are telling you we are not going to build.

The word list is our own construction

We did not copy the original game’s word lists. Our answers are the intersection of two permissively-licensed sources — every five-letter headword in Wordnik word list (2021-07-29) (opens in a new tab) that also appears in SCOWL / wamerican (2020.12.07) (opens in a new tab), minus a short hand-written blocklist. Legal guesses are the whole five-letter Wordnik set.

Answers

4,603

Legal guesses

11,846

Original game’s answers

2,315

The list is built by a stated rule from sources anyone can fetch, rather than lifted from somebody’s editorial selection. That is the point of it. It also means every comparison to the original game is a comparison between two different games.

This environment is harder than the original game

Two things pull in opposite directions here, so we measured rather than argued. Our answer pool comes from a dictionary rule, so it keeps the regular plurals and past tenses the original game’s editor removed by hand: 32.9% of our answers end in a plain S and 6.1% in -ED. That is an exploitable regularity, and it makes our list easier. Our pool is also roughly twice the size, which makes it harder.

The net effect is the second one. We ran the same greedy solver over both pools, on 250 sampled words each:

About a seventh of a guess harder, and one worse in the worst case. Both rows are the same algorithm, which is the only fair comparison available — setting our greedy number against the published 3.4212 optimum for the original list would be comparing two different algorithms and calling the difference a property of the words.

The practical instruction is unchanged: do not compare a guess count from this site to your own statistics from the original game. It is a different game with a different shape.

What we have not measured

This is the list we would want if we were the ones being sold to.

• Any training result of our own. We have not trained a model against this environment. The before-and-after number on the home page is Prime Intellect’s, measured on their word list, and we did not rerun it on ours.
• Variance. The recorded runs are individual seeds, not samples. There is no confidence interval anywhere on this site and no claim that a given run is typical. A demonstration with one seed is a demonstration, not evidence.
• An optimum, or a human baseline, for our list. We do not know how few guesses perfect play needs on our answers, and we have not asked people to play it.
• Cost and latency. No number here is a price, a token count you should budget from, or a wall-clock comparison between models. The durations shown in a trace are what that recording measured, on that day, on that endpoint.
• Anything at all about the eleven proposals. Nothing in those pages has been run. Every reward, counterweight and KPI there is a design.
• A full review of the answer list. We block a short hand-written set of words from ever being the hidden answer. We have not audited the remaining 4,603 exhaustively, and a five-letter English dictionary contains words somebody will not want on a boardroom screen.