Procedural Generation and Immersion: Reading a Small Study

Start with the experience being evaluated

Procedural generation and immersion meet at a practical design question: how does a particular method of building levels shape the experience of playing them? A map can satisfy an exit-reachability rule while leaving open whether its encounters feel clear, repetitive or interesting.

Consider a proposed test in which every maze has a reachable exit. Passing that check establishes one property of the layout. It does not measure players’ experience of reaching it. That difference is the starting point for interpreting a comparison of generated and authored levels.

One game, two ways of building maps

Connor, Greig and Kruse’s 2017 study used Vektor, an abstract top-down shooter. Twenty participants were randomly assigned one version: human-designed maps or maps produced with a genetic algorithm. After a post-hoc exclusion of people playing less than an hour weekly, the analysis contained seven authored-map players and nine generated-map players. Experience was measured using a modified immersion questionnaire.

Each participant encountered only one version. When reading this kind of between-group comparison, ask how much variation could come from the people as well as the layouts. Random assignment helps establish the comparison; a small retained group still leaves considerable uncertainty about the size of a difference.

Preserve the disagreement between tests

Mean total immersion scores were 112.14 for authored maps and 94.56 for generated maps. The total-score ANOVA gave p=.0518, above the .05 threshold; the Kruskal-Wallis rank test gave p=.0443, below it. The tests therefore gave different threshold decisions.

Both results belong in the answer. Picking whichever p-value supports a preferred production method would hide that sensitivity. Neither result establishes a universal winner, and the ANOVA null result does not demonstrate equivalent experiences. Report the comparison and its uncertainty before assigning a broad explanation.

Inspect the encounters inside a valid layout

Imagine an invented generator that assembles rooms with a fixed number of exits. Two outputs might pass the same connectivity check, yet one places cover near a doorway and another leaves the player exposed. The generator’s rule and the resulting encounter are separate things to inspect.

A useful proposed follow-up would record which layouts people received and examine a defined property, such as cover placement, while keeping movement and enemy behavior stable. This hypothetical comparison would investigate that property specifically. It would not automatically explain every difference between content built by a person and content built by an algorithm.

Account for manual edits and scoring

Human placement of keys and enemies remained in the generated levels. The researchers also omitted the questionnaire’s final 1-to-10 immersion item, limiting comparison with other studies’ scores. Only one generator and one simple game were examined.

That hybrid pipeline makes an attribution question visible: which steps produced the layout, and which shaped the encounters inside it? A label attached to the map cannot answer both. Similarly, a questionnaire total needs its scoring definition attached before comparison with a total from a different instrument or study.

Ask what a claim about generation actually covers

Before accepting a claim about generated levels, look for the actual generation method, remaining manual edits, player sample and outcome measured. Ask whether the result concerns one session’s reported experience, successful navigation or a repeat-play decision. Those outcomes require their own evidence and should retain their own names.

This older paper was published in 2017. The linked manuscript, publisher page and author record describe the same study, without independent replication. For a proposed evaluation, decide which property of the content you want to understand, record how it was produced and choose a measurement that addresses that question.

Keep exploring

FOLLOW THE SOURCE

Sources & further reading

Have a correction? Tell our editorial team.

✦ NICE. YOU LEARNED SOMETHING.