Home/Blog /Quality

The agent plays its own game before you do

A generated game that doesn't boot is worse than no game at all, because it costs you the ninety seconds it took to find out. So the build gets played automatically.

The worst outcome in a generative game studio isn't a mediocre game. It's a build that doesn't boot, because that costs the player the ninety seconds it took to find out, plus the confidence that the next one will work. Two of those in a row and the product is dead regardless of how good the third one would have been.

So no build reaches a person before it has been played. Not by a person: by the machine that made it.

The cheapest quality gate there is

Load the exported web build in a headless browser. Wait for the engine to signal that the main scene is running. Sample framerate for a few seconds while synthetic input drives the game. Collect everything the console says.

That's it. It's a shallow test by any serious QA standard, and it catches the overwhelming majority of what actually goes wrong, because generated games fail in a very particular way: not subtly, but totally. A missing resource path, a shader that won't compile on the web target, an autoload registered twice, an export preset that silently dropped a directory. These don't produce a slightly worse game. They produce a black screen.

Three signals

Does it start

The single highest-value bit of information in the pipeline. Web exports fail at load for reasons that never appear in a desktop run: WebAssembly memory limits, missing .pck files, a threading model the distribution's headers don't support. A build that boots has cleared most of the cliff.

The test needs a real "the game is running" signal (the main scene reporting ready) rather than "the page loaded". A canvas element exists long before the engine has done anything useful, and treating page load as success means passing every broken build in the class you most wanted to catch.

Does it hold framerate

Generated games have a characteristic performance failure: something in the scene is far more expensive than the code implies. A particle emitter with a rate three orders of magnitude too high. A light set to cast shadows in a scene with four hundred instances of it. Physics bodies where areas would do.

A framerate floor sampled over a few seconds catches those. It is not a performance benchmark and shouldn't pretend to be: it's a tripwire for the case where the game is technically correct and unplayable.

Does the console stay quiet

Uncaught exceptions, failed resource loads, shader compile warnings. Individually many are harmless; collectively they're the best available proxy for "this build is not in the state its author thinks it's in". The useful discipline is treating any uncaught error as a hard failure and everything else as a signal the agent gets to weigh.

Failures route to the agent, not the player

This is the design decision that matters more than the test itself. When a build fails its check, the failure goes back into the agent's context: the error text, the stack, the frame timings, and the agent fixes it and builds again. The person who asked for a game gets a game.

The alternative, surfacing "build failed: SHADER_COMPILE_ERROR" to someone who asked for a kart racer, is not a status update. It's an apology with extra steps, and it moves the debugging burden to the one participant who has no way to act on it.

There's a limit, obviously. Retry loops need a cap, and a genuine dead-end has to surface eventually: as plain language about what didn't work and what to try instead, not as a stack trace. But the default is: the machine cleans up after itself.

What automated play does not tell you

It does not tell you whether the game is fun. It cannot, and pretending otherwise is how you end up shipping technically flawless games nobody enjoys.

Automated play answers a narrower and more useful question: is this build in a state where a human's opinion would be worth collecting. That gate is worth automating precisely because it's mechanical. Fun is decided in the conversation afterwards, by someone actually playing, saying "the jump is floaty", and getting a new build a few seconds later.

Keeping those two things separate is what keeps the loop honest. The machine owns "does it work". You own "is it good". Neither is trying to do the other's job.

The economics of the ten-second check

Booting a build and watching it for ten seconds is close to free against the cost of generating the game in the first place. A rough accounting: the check adds single-digit seconds to a build measured in minutes, and it removes the failure mode that costs the most, a person opening a black screen and concluding the whole product is broken.

Almost every quality investment in generative software has this shape. The expensive part is producing the artifact; verifying it is a rounding error. Teams skip verification anyway, because the model sounded confident and the code looked right. The code always looks right. That's what the models are best at.

Frequently asked questions

How do you test an AI-generated game automatically?

Load the exported web build in a headless browser, wait for the engine to signal that the main scene is running, then sample framerate and collect console errors while synthetic input drives the game for a few seconds. Boot failure, a framerate floor and any uncaught exception are all hard failures. It is a shallow test by design: it catches the broken builds, which is the overwhelming majority of what goes wrong.

Can automated testing tell whether a game is fun?

No, and treating it as if it could is how teams ship technically correct games nobody enjoys. Automated play answers a narrower question: is this build in a state where a human's opinion would be worth collecting. Fun is decided in the iteration conversation afterwards, by someone actually playing it.

Keep reading

Related articles

Architecture

What an AI game agent actually needs (it isn't a bigger model)

Every model can write a game loop. Almost none of them can hand you something that runs. The gap between those two facts is the entire product.

11 min read
Platform

Why generated games belong in the browser

A game you have to install is a game nobody plays. The web build is the only target where "here, try this" is a link and not a favour.

10 min read