Judge, 1924–1939: our first full series

Every crossword in Judge magazine from 1924 to 1939, 531 puzzles, read from scans on archive.org into xd files. What went well, what was hard, and what we're changing for the next one.

Every Puzzle Project · 12 October 2026

Judge was a humor magazine, published weekly in New York since 1881. In the fall of 1924, with the country "Cross Word mad", it asked readers to send in humorous crosswords, paying $25 for each one it printed (later lowered to $10). The first ran on 1 November 1924. Judge became a monthly in 1932, printing two puzzles an issue. The last puzzles we have ran in January 1939, because that's where archive.org's scans end; the magazine itself went on until 1947. Readers wrote nearly all of the puzzles: 311 different names appear in the bylines that followed.

Until this month those puzzles existed only as page images. Now every one is an xd file, a plain-text crossword format, with its grid, clues, answers, title, constructor and date, and a link to the scan it came from. In xd they can sit in a research corpus beside a century of later puzzles, where they can be searched, counted and compared, and preserved for posterity.

Judge was the first full series the Every Puzzle Project took from start to finish, and we learned a lot. This is the story of how it went.

How a puzzle gets from a scan to xd

1. Findsearch archive.org's page text for crossword pages
2. ReadOCR built for this project: grid, clues, answer key
3. Agent reviewClaude checks the reading against crops of the scan
4. Proof pagea person checks what's left, with the scan beside it
5. Ledgerevery change, with who made it and what it was
6. xdone file per puzzle, rebuilt from the ledger

Teaching the OCR to read a crossword page

Off-the-shelf OCR reads a crossword page as a jumble: the grid's little numbers, lines from two clue columns run together, the ad next door. So the reader built for this project starts from what a crossword is.

Finding the grid

A grid is the one thing on a page made of a lattice of identical squares. Text, cartoons and ads don't form one. The reader looks for that lattice, which says where the grid is and how many rows and columns it has. It then straightens the grid, correcting the tilt and curl of a page photographed in a bound volume, and calls each square black or white. That last step is easier than it sounds. On the first test pages a white square was nearly all paper and a black one all ink (0–4% ink against 100%), with nothing in between.

Reading the answer key

The answer keys were harder. Judge printed them a week or more later, small, with hand-lettered capitals that fill each square. Letters that big hide the white squares the grid finder looks for, so keys get a second finder that looks instead for the points where the grid lines cross. Each square is then read on its own, in three different ways; when the readings disagree, the reader gives its best guess and marks it unsure.

To match a key to its puzzle, the pipeline compares black squares with the puzzles from the weeks before: the pattern is a fingerprint. Faint rules sometimes broke a key into pieces too small to match anything. For those, it tried growing a piece's grid to the size of each candidate puzzle, at every position where the piece's black squares fit, and kept the result only if the whole pattern then matched exactly. That found 42 more keys.

One surprise: some scanned copies had been solved in pencil by their owners, and a filled-in puzzle looks a lot like an answer key. The difference turned out to be simple. Puzzles have clue numbers in their squares, and printed keys never do.

Splitting the clues

Clue lists are set in narrow columns, with wrapped lines and often an ad alongside. The reader finds each column, stops lines from running into the ad copy next to it (one tobacco ad, "Judge men, my boy, as you judge Craven", kept turning up inside clues), and joins wrapped lines to the clue they belong to.

Then the grid helps again. A crossword's black squares decide its numbering, so the grid says exactly which Across and Down numbers must exist, and in what order. The clue lines are lined up against that list, which repairs misread numbers ("3." for "33."), dropped numbers and clues split in two. Where grid and clues still disagree is exactly where someone should look.

Checking the reading

Assembling the review packet

The most important step in checking is deciding what to show the checker. A puzzle's evidence is a page scan of several megapixels, plus an answer key from another issue. Handing all of it to Claude for every puzzle is what made our first reviews cost 60,000 to 90,000 tokens each. So before any model runs, code builds a packet for each puzzle, the way an editor would prepare proofs:

• The reading, as text. The answer key as rows of letters, with letters the reader wasn't sure of in lowercase and unreadable ones as dots. Then every clue beside its answer from the key, flagged wherever the pipeline did something or noticed something odd: a misreading it already fixed ("elean" to "clean"), a word that isn't English ("exelamatlon", "Jaliet"), a clue number it had to correct, a line it borrowed from next door.
• Whole-puzzle checks, already run: clue text missing, the same text on two clues, clue numbers that don't fit the grid, a key whose black squares disagree with the puzzle's. They cost nothing, and they catch most of the serious problems.
• Crops of the scan for everything flagged: each clue's printed line with a line above and below for context, a three-by-three patch of squares around each unsure letter, the title and the byline. Code chooses them, not the model, and packs them onto a few contact sheets.

## Answer key
Lowercase = the letter reader wasn't sure; '.' = unreadable; '#' = black square.
 8 G # U S E R S # E # A R B O R # S
 9 # o R # # . U K # A L E # # O N #
10 R # N O R S E # F # L A T H E # S

## Across
A1    LIBERAL     What kind of Seotchman would offer a penny for your thoughts?  [odd: Seotchman]
A38   INFER       How to make a deduction.  [spell:deduetion>deduction]
A53   .UK         This sounds like an exelamatlon, but it'e a bird.  [odd: exelamatlon]
A74   PINED       What Romeo did for Jaliet.  [odd: Jaliet]

Claude reads the text, decides what looks wrong, and checks each suspicion against its crop. It can ask for more (any clue, any square, any part of the page) at 40 to 400 tokens a crop. One rule keeps it honest: a correction to anything it never saw on a crop is recorded as unsure, so nothing gets fixed from the text alone. What comes back is a small file: corrections, misprints kept as printed, things confirmed as read, notes on what the tools got wrong, and a verdict, ready or what's left for a person. For No. 46 that was 15 clue fixes, the title, two answer-key letters and one misprint kept as printed ("plun.ber", a broken letter m).

That's how a review came down to about 17,000 tokens (the costs are in the table below). It also makes the work manageable: the reviewer's attention goes where there's evidence of a problem, and every decision it makes is tied to a piece of the scan.

What the person sees

The human reviewer gets a packet too, built for eyes instead of tokens: a proof page on the review site that shows the xd as it stands, every line beside its scan, and every change marked with who made it and what the OCR had read. In the answer key, the scan sits beside our letters, with changed squares outlined. A reviewer can lay our letters over the scan, or blink between the two, so a wrong letter jumps out.

Volunteers contribute by directing Claude through the same packets.

Every correction, from an agent or a person, goes into an append-only ledger with a name on it, and the xd files are rebuilt from it. Nothing reaches an xd file without showing on the review page, along with where it came from.

What went well

Agents did the reading; people made the judgement calls

Of the ledger's roughly 12,000 rows, about 11,200 were written by agents. Human reviewers made most of the rest: the project's maintainer made 489 edits by hand on 172 puzzles, and helpers on the review site made 46. That's the split we were hoping for. Claude is good at the patient part: comparing 90 clues and 200 squares with a scan, one by one. A person is needed for the questions with no right answer in the scan: is this a misprint to keep, or a broken letter to fix? Is that an extra black square, or a smudge?

It was also affordable, and how it got that way is a story in three steps. The measure that matters to us is how many puzzles a week Claude can check on a Claude Max 5x plan's weekly allowance, since that's what the reviews ran on.

Claude Fable 5.1 throughout. Puzzles a week: measured against the weekly Fable limit of a Max 5x plan, $100 a month (0.36% and 0.15% of that limit a puzzle). *The first row is estimated, not measured: scaled from its token count against the second row, the other review that reads mostly images. API cost: list price ($10 per million input tokens, $50 per million output), with no caching and about 2,000 tokens of output a puzzle.

1. Just ask Claude. The obvious way to transcribe a puzzle is to show Claude the page and the answer key and ask. Claude does remarkably well at it. Given the two whole pages for No. 46 (1928), it got all 289 squares and 110 of 112 clues exactly as our reviewed transcription has them (the other two differ by a period), and on New X Word Puzzle No. 1 (1927) it even caught a misprint in the printed key. But a puzzle costs about 78,000 tokens, there are no crops to show the evidence for each reading, and nothing says where to look. Someone still has to find the pages, too, in 458 issues, and match each puzzle to an answer key printed weeks later.

2. Add our own OCR. So we built OCR to find the puzzles and make the first draft, and had Claude check that draft against every image of the puzzle: the grid, the answer key, the page and a strip for every clue. That made the process reliable, but it barely moved the cost: Claude was still looking at everything.

3. Build the packet. The savings came from deciding what Claude looks at: the text first, then only the crops code chose for what looked wrong, with a reviewer agent trimmed to the few tools it needs. That brought a puzzle down to about 17,000 tokens, more than twice as many puzzles a week for the same allowance: about 650, or the whole of Judge in less than a week. Two separate reviews of the same puzzles agreed on 91% of their corrections.

We ran the reviews on the plan's allowance, not the API: the project has no source of funding. We did try the text-first review through the API, on 11 puzzles Fable had already reviewed. Opus came close: it found about 80% of Fable's fixes, at about $0.11 a puzzle. Sonnet was cheaper, at about $0.05, but found about 75%. Neither reliably kept printed misprints as printed. Pennies a puzzle sounds small, but we hope to check many thousands of puzzles, and at that scale it's thousands of dollars.

The tools learned from the reviewers

Every review ends with notes on what the tools got wrong: 1,599 of them for Judge. We group them across a batch, and each kind becomes either a fix in the code or a line in a short notes file the next batch of reviewers reads. The biggest win came early. The first ten puzzles a person reviewed needed 88 edits, and 52 of those were answer-key letters read as I. None had been flagged, because the OCR was sure of them. A new way of reading each square fixed all 52:

Two habits kept changes like that safe. We measured each change against the puzzles people had already corrected before merging it. One segmentation change that looked right on a handful of puzzles turned out to make nine of them worse. And the importer refuses a grid fix whose numbering no longer matches the printed clues. That caught real agent mistakes, including one on the last day.

Narrow pages are fast pages

The best review tool was the narrowest. On the last day, the remaining doubtful answer-key letters went on a single page: a crop of each square from the scan, the letter we had, keep or change. A person can settle one of those in seconds. Proofing a whole puzzle, once the page showed everything with its evidence, took about two minutes.

What was hard

1. Finding the puzzles came last, not first

Finding the pages sounds like the easy part. It wasn't. In 1925 and 1926 Judge printed the grid under a decorative title the OCR couldn't read, with the clues on the facing page. Neither page looked like a crossword on its own, and our first run found almost nothing from those two years.

Fixes like that kept coming. Some keys were printed under the next week's puzzle. Some were cut into pieces by faint rules. No. 156 was printed as No. 155. Some keys were in double issues, and some issues had a second puzzle. The puzzle numbers were our completeness check, and in the end every printed number was found. But 18 puzzles and about 70 answer keys turned up only in a final hunt for gaps, after most of the review was done. Each late find reopened puzzles that had already been checked: a newly found key meant letters nobody had looked at yet.

Help came from outside Judge, too. In 1925 newspapers reprinted its puzzles as "Cross Word Funnies, selected by Judge". Matching those reprints back to the magazine turned up puzzles that were in archive.org's scans but that our search had missed.

The lesson: settle everything the search turns up (missing numbers, keys that almost match a grid, duplicate scans) before any review starts. Next time, nothing goes to review until that list is empty.

2. One letter, all the way through

The answer keys were hand-lettered in a thin, light style, and the OCR kept reading T as I. (Also D as O, S as E, and the hand-lettered A as nothing at all.) The worst part was that it was sure of itself, so our low-confidence flags never fired.

The reading fix described above took care of most of it, but not all. On the last day, a rough check of every answer against a word list found about 60 more strong suspects, a dozen of them in puzzles already marked done. The check is simple: an answer that isn't a known crossword word, but becomes one when you change an often-confused letter, gets flagged. For the next series it runs from the first day, before anyone reviews anything.

Some things we found

Judge's puzzles are a window on what a humor magazine's readers found funny between the wars. Some clues the reviewers enjoyed:

• The wages of sin.HEADLINESNo. 325, January 1935. Ten years earlier, two 1925 puzzles had answered the same clue with CASH.
• Get a long little doggie.DACHSHUNDNo. 331, April 1935
• A sea-going gallstone.PEARLNo. 346, December 1935
• This is very shad.ROENo. 391, October 1937. (In 1936, "This is very shady." was ELM.)

Constructors played with the grid itself. In No. 4 (22 November 1924), the black squares spell the magazine's name, with HA HA in cross-hatching in two corners:

Two weeks later, No. 6 (by Alfred L. Pelham, of Cambridge, Mass.) was two face-shaped grids sharing one clue list: "Before Reading Judge" and "—and After". No square grid can hold that, so our xd stacks the two faces one above the other, and says so on the puzzle's review page.

Rules that weren't rules yet

Judge's puzzles come from the crossword's first years, while its conventions were still being worked out. Plenty would look familiar today: numbered squares, mostly symmetric grids, black squares rather than the open diagrams of the very first crosswords. (Though Judge's clue lists were headed "Horizontal" and "Vertical", not Across and Down.) Other habits fell by the wayside.

Unchecked squares. Today every letter in a crossword belongs to both an Across and a Down answer. In Judge thousands didn't: 5,781 squares in 399 of the 531 puzzles stand in only one word. In the 1920s that was the norm, and by the end of the run it was nearly gone.

Two-letter answers, 4,588 of them, in 454 puzzles, each with its own clue. Some are inventive: "These two letters will get you going in circles." is DZ (1934). No. 4, the JUDGE grid, even has one-letter answers, K and I, in the holes of its letters.

Made-up abbreviations. When a corner needed three unlikely letters, a constructor could simply declare them an abbreviation:

• Neurotic Railroads (abbr.).NRRJune 1925
• Persian Seamless Suspenders (abbr.).PSSSeptember 1925
• Plastered Business Men (abbr.).PBMOctober 1925
• Mocha Pouring Republicans (abbr.).MPRJanuary 1926

Circled squares. We didn't expect these so early. No. 6's second face has circled eyes, and in December 1936 a puzzle called "Dirty Work at the Crosswords" turned the grid into a whodunit: the letters in its circled squares were clues to "the culprit's identity", three words of 3, 7 and 8 letters. (THE FARMER'S DAUGHTER.) Our tools had nothing for circles, so we had to scramble. The review page gained a way to mark a square circled or shaded, shown with who marked it, and the marks go into the xd. Spotting circles automatically is still on the list.

What's next

For the next publications (GAMES magazine from 1977, then the Boston Globe, whose crossword goes back to 1917), the order changes:

1. Find everything first. Every gap in the numbering, near-miss answer key and duplicate scan is settled, with links to the evidence, before any review starts.
2. Check the answer words from day one, and give people the narrow key-squares page as the normal way to check keys.
3. One copy of the truth. The review page shows the xd itself, proofs record exactly which version they checked, and each puzzle's answer key is decided once and saved.
4. One command for the maintainer, which imports, rebuilds, checks and publishes, and stops when something looks surprising.

Most of the work on the next series should be people making quick judgement calls with the evidence in front of them, and Claude doing the reading.

How you can help

• Tell us about a puzzle. The catalog lists every publication known to have printed a crossword. If you've seen a puzzle in a paper or magazine we're missing, even on paper with no link, report it there. No technical skills needed.
• Hunt for Cross Word Funnies. If you have newspaper archive access, Judge's repo says what to look for.
• Check puzzles against the scans, by directing Claude through blitz, or by hand on the review site.
• Help complete Judge. archive.org's scans stop in January 1939, but Judge ran until 1947. It was revived in October 1953 as a weekly, with a crossword again. If you know where issues from 1939–1947, or from the 1950s revival, can be found, tell us in the catalog.
• Follow Judge's status, puzzle by puzzle, on its status page.

This is a research archive: an accurate record of puzzles as they were printed, each linked to its source scan. It isn't a place to play old puzzles. The scans shown here are from Judge issues published before 1931, which are in the public domain. Thanks to the Internet Archive for the scans, to Saul Pwanson's xd crossword corpus, and to David Steinberg's Pre-Shortzian Puzzle Project for showing what a group of highly motivated volunteers can do.