{"id":625,"date":"2026-11-07T11:41:23","date_gmt":"2026-11-07T19:41:23","guid":{"rendered":"https:\/\/jasonsla.de\/?p=625"},"modified":"2026-11-07T17:42:42","modified_gmt":"2026-11-08T01:42:42","slug":"small-ai-set-puzzle-part-1","status":"publish","type":"post","link":"https:\/\/jasonsla.de\/2026\/11\/07\/small-ai-set-puzzle-part-1\/","title":{"rendered":"Can a Small AI in My Garage Crack a Famous Card-Game Puzzle? (Part 1)"},"content":{"rendered":"\n

Part 1 of 5. Part 2 gives the AI a hint, part 3 swaps in an AI built for coding, part 4 measures how much of it was luck, and part 5 covers the cost, the lessons and what comes next.<\/em><\/p>\n\n\n\n

If you’ve ever played the card game SET, you already understand the maths problem in this post. In 2023 a Google DeepMind system made headlines by finding a new record on it, using a huge AI that wrote thousands of small computer programs. I wanted to know how far a much smaller AI could get, running overnight on two second-hand graphics cards in my garage.<\/p>\n\n\n\n

A card game with a hard question inside<\/h2>\n\n\n\n

SET is played with a deck of 81 cards. Each card has four features: the number of symbols (one, two or three), their colour, their shape and their filling. Three cards make a SET<\/strong> when every feature is either all the same or all different across the three. One “two of these and one of those” anywhere, and it isn’t a SET.<\/p>\n\n\n\n

\"Three
Every feature has to be all the same or all different. The right-hand trio fails on number and colour.<\/figcaption><\/figure>\n\n\n\n

Players race to spot SETs on the table. Mathematicians ask the opposite question: what’s the biggest pile of cards you can lay out that contains no SET at all?<\/strong><\/p>\n\n\n\n

For the real deck the answer is 20. It was proven decades ago, and the picture below shows one such pile. Lay out any 21 cards and a SET is guaranteed to be hiding in there somewhere.<\/p>\n\n\n\n

\"All
One of the largest possible piles from the real 81-card deck: 20 cards, no SET among them.<\/figcaption><\/figure>\n\n\n\n

Now imagine bigger decks. Give each card a fifth feature and there are 243 cards; with eight features there are 6,561. The question stays the same, but it gets very hard very fast. With five features the answer is 45, and with six it’s 112, both proven with a lot of work. Beyond that, nobody knows. For eight features the best pile anyone has found has 512 cards, and nobody can prove it’s the biggest possible. Mathematicians call these piles “cap sets”, and the problem has been studied for decades because it connects to deeper questions about patterns in numbers.<\/p>\n\n\n\n

The 512 was found in 2023 by FunSearch<\/a>, a Google DeepMind project. Their system ran enormous numbers of AI attempts, and even then it found 512 in only 4 of its 140 runs. In 2025 a team of researchers re-ran the experiment<\/a> with seven different commercial and open AI models. None of them reached 512. The best single run got 448, and the open 70-billion-parameter Llama model averaged 375.<\/p>\n\n\n\n

That study gave me a fair measuring stick. My server runs an open model from Google, Gemma 4, a 26-billion-parameter model, on the two 2016-era GPUs I wrote about in October<\/a>. It’s smaller than every model in their table, and it already has a day job running the house’s voice assistant. Could it hold its own overnight?<\/p>\n\n\n\n

The AI doesn’t play the game, it writes recipes<\/h2>\n\n\n\n

The clever part of FunSearch is that the AI never picks cards itself. Asking it to list 512 cards from a deck of 6,561 would be hopeless. Instead it writes a ranking rule<\/strong>: a short recipe that looks at a single card and gives it a score.<\/p>\n\n\n\n

A simple builder does the rest. It sorts the whole deck by the recipe’s scores, walks down the list from the top and keeps each card unless it would complete a SET with two cards already kept. The bigger the final pile, the better the recipe.<\/p>\n\n\n\n

\"A
The builder never changes. Only the order, which the AI's recipe decides, changes the result.<\/figcaption><\/figure>\n\n\n\n

It’s like packing a suitcase by working down a list of what matters most and skipping anything that won’t fit with what’s already in. The packing method never changes; all the cleverness lives in the order of the list.<\/p>\n\n\n\n

That turns maths into a cooking contest:<\/p>\n\n\n\n

\"A
Each turn of this loop takes a few seconds. The AI only ever writes recipes; it never scores them.<\/figcaption><\/figure>\n\n\n\n
    \n
  1. The cook.<\/strong> The AI is shown two of the best recipes found so far and asked to write a better one.<\/li>\n\n\n\n
  2. The kitchen.<\/strong> The new recipe is run in a sealed test kitchen.<\/li>\n\n\n\n
  3. The judge.<\/strong> A separate referee counts the pile and checks every possible trio for a SET.<\/li>\n\n\n\n
  4. The recipe book.<\/strong> Good recipes are kept and become examples for the next round. Weak ones fade away.<\/li>\n<\/ol>\n\n\n\n

    Round and round, about 1,700 times an hour. It’s evolution with recipes instead of animals. The AI provides the mutations, and the judge’s scores decide what survives.<\/p>\n\n\n\n

    Letting a stranger’s recipe into the house<\/h2>\n\n\n\n

    There’s a catch. Every recipe is a small computer program written by an AI, and I’m running thousands of them on a server in my house. Most will be harmless. But an AI can write anything, including code that tries to read files, reach the network or just loop forever.<\/p>\n\n\n\n

    So the test kitchen really is sealed. Each recipe runs in its own throwaway box that the operating system builds for it: no internet, not even to the server itself, no access to anyone’s files, a strict memory limit, and 60 seconds before it’s switched off. Before the first real run I tried to break out on purpose, with eleven hostile recipes. They tried to call home over the network, read the AI server’s file of access keys, look through my home folder, start a command prompt, swallow 4 GB of memory, run forever, and simply claim<\/em> a high score. All eleven were stopped.<\/p>\n\n\n\n

    That last one matters most. In the open-source version of FunSearch I started from, the recipe was briefly run inside the main program before being sent to the test kitchen, and the result was read back in a way that can itself run code. Those are two easy ways for a misbehaving recipe to slip past. In my version the kitchen hands back only the pile of cards, and a judge that the AI can’t touch does all the counting. The AI never grades its own homework.<\/strong><\/p>\n\n\n\n

    Sharing the server with the house<\/h2>\n\n\n\n

    The same AI runs the garage voice assistant and answers questions around the house, so the search mostly runs from 11:30 at night to 5:30 in the morning, when the logs showed nobody uses it. It never takes more than three of the model’s four lanes, so there’s always one free. A small watchdog keeps an eye out: if someone talks to the house AI, or a different model is needed, it freezes the search on the spot and lets it carry on once things are quiet again.<\/p>\n\n\n\n

    The first real test failed in an instructive way. This model “thinks out loud” before it answers, and it spent its entire word budget thinking and never wrote a recipe. With thinking switched off, it found the 20-card pile for the real SET deck in under two minutes.<\/p>\n\n\n\n

    Climbing the ladder<\/h2>\n\n\n\n

    I worked up through bigger decks, one or more nights each:<\/p>\n\n\n\n

    \"Bars
    The best pile from all the runs at each deck size, against the best known answer.<\/figcaption><\/figure>\n\n\n\n
      \n
    • Four features (81 cards):<\/strong> 20, the proven maximum, in under two minutes.<\/li>\n\n\n\n
    • Five features (243 cards):<\/strong> 40 out of a possible 45, within the first minute of both nights, then nothing better for two hours.<\/li>\n\n\n\n
    • Six features (729 cards):<\/strong> 84, then 96, then 84 again on three nights with almost the same settings. The 96 arrived 21 minutes before the night ended. Luck plays a big part in this kind of search.<\/li>\n\n\n\n
    • Seven features (2,187 cards):<\/strong> 182, against a best known of 236.<\/li>\n<\/ul>\n\n\n\n

      Eight features: the main event<\/h2>\n\n\n\n

      The study found that several short runs did better than one long one, so I split each night into three fresh two-hour starts and ran three nights: nine runs in total.<\/p>\n\n\n\n

      \"Nine
      Nine two-hour runs over three nights, against the 2025 study's results for the 70-billion-parameter Llama model and its best run overall.<\/figcaption><\/figure>\n\n\n\n

      Eight of the nine runs stopped at 364 or 366. One reached 390<\/strong>, which puts my small model’s best run above the big Llama model’s average of 375, and near the top of that model’s range. The nine runs averaged about 368, a little under that 375. So the small home AI landed right in the pack of the published models, in a few nights of borrowed GPU time.<\/p>\n\n\n\n

      Before believing the 390, I rebuilt that pile from scratch with a separate checking program, outside the whole search system, and tested all 390 cards. It holds: no SET anywhere.<\/p>\n\n\n\n

      The way each run unfolded says even more:<\/p>\n\n\n\n

      \"Best
      Each line is one run. Nearly all the progress happens early; after that, a run is a lottery ticket.<\/figcaption><\/figure>\n\n\n\n

      Every run sprinted to the 360s within the first 40 minutes or so, then sat on a plateau. The 390 run sat at 364 for over an hour before one recipe suddenly found a better idea.<\/p>\n\n\n\n

      What the winning recipe actually does<\/h2>\n\n\n\n

      Here’s the recipe behind the 390, exactly as the AI wrote it:<\/p>\n\n\n\n

      v_arr = np.array(v)\ncounts = np.bincount(v_arr, minlength=3)\np = counts \/ n\nline_prob = np.sum(p**3) + 6 * p[0] * p[1] * p[2]\nvariance = np.var(v_arr)\nreturn 1.0 \/ (line_prob + 1e-6) + variance<\/code><\/pre>\n\n\n\n
      In plain English: each of the eight features can take one of three values, call them 0, 1 and 2. The recipe counts how many of each value a card has, and favours cards whose mix of values makes them unlikely to complete a SET by chance, with a small bonus for cards that are spread out rather than bunched in the middle. It’s a sensible idea, and the AI even wrote its own comments explaining it.<\/p>\n\n\n\n
      But notice what the recipe never looks at. It counts how many<\/strong> of each value a card has, but never where<\/strong> they sit. To this recipe, cards reading 0-1-1 and 1-1-0 look identical. It’s like reading words by counting their letters without looking at the order: to it, “listen” and “silent” are the same word. Almost every top recipe in all nine runs worked this way.<\/p>\n\n\n\n
      That’s very likely why they all stalled in the 360s. The record-breaking 512 pile depends heavily on where<\/em> the values sit: in FunSearch’s winning recipe, what mattered was whether matching positions at opposite ends of the card held the same value. My AI never stumbled onto that idea on its own.<\/p>\n\n\n\n
      Next: give it a hint?<\/h2>\n\n\n\n
      So the obvious experiment is to nudge it, with one extra sentence in the instructions telling it to compare values at mirrored positions on the card. That raises a fair question: is it still the AI’s discovery if I point it at the answer? I’ll come back to that in part 2, which covers the hint, why it’s honest to try it only if you say so up front, and what happened when nine more runs went off with it.<\/p>\n\n\n\n
      All scores in this post were counted by the checker, never by the AI. The 390 pile was rebuilt and re-checked separately. Part 5 will include a small program you can run yourself to check the final piles.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"
      The card game SET hides a maths puzzle that made headlines when Google DeepMind’s AI set a new record on it. Part 1 of 5: how the search works (the AI writes recipes, a sealed kitchen runs them, a judge it can’t touch scores them), and how a small AI on two 2016 graphics cards did across nine overnight runs.<\/p>\n","protected":false},"author":1,"featured_media":618,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[],"class_list":["post-625","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-automation"],"_links":{"self":[{"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/posts\/625","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/comments?post=625"}],"version-history":[{"count":2,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/posts\/625\/revisions"}],"predecessor-version":[{"id":630,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/posts\/625\/revisions\/630"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/media\/618"}],"wp:attachment":[{"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/media?parent=625"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/categories?post=625"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jasonsla.de\/wp-json\/wp\/v2\/tags?post=625"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}