Evil Evil Skinner Boxes

Evil Evil Skinner Boxes!

A major meme running the rounds in game designdom right now is the dangerous evil of Skinner boxes, and how we really need to stop making games that are just Skinner boxes.  We especially hate games like Pokemon Go or Clash of Clans.  These insidious creations encourage you to log on at least once a day for an extra reward of seeing your resource bar creep up a little each time you put energy into the game.  Their evil plan is clearly to addict players with this scientifically proven simple reward method which humans have no ability to resist, thus we will ultimately take our time energy and money against our will.  Clearly Supercell, Niantic, etc, are key players in the downfall of society!

I’m going to make a different claim.  I claim that a designer who puts effort into making sure their game is a better Skinner box is, almost objectively, just making a better game.  A lot of people are throwing around the term “Skinner box”.  What is a Skinner box?  In a nutshell, in the 1930s a scientist named BF Skinner did a series of experiments where he put rats in a box with a lever.  If the rat pulled the lever it got a little chunk of food.  There was a bunch of experiments Skinner performed with this setup, he wanted to see what was the ideal way to get Rats to pull levers obsessively as many times per minute as possible.  He tried a bunch of food distribution techniques, and ultimately found that best results came when initially rats got a treat every time, but after the first several tries a randomized delay would be put between lever pulls, so the rat may have to pull the lever 3 times before the treat would come, but the next pull it would come right away, and the pull after that he’d have to pull 6 times for the treat.  Skinner was looking for the ideal distribution of rewards per input to keep rats motivated to perform input.  It was later confirmed that, with some caveats, the ideal pattern for rats is also the ideal pattern for human motivation.

So clearly it’s pretty evil to put people in a giant metal box and make them press a lever for food.  But what really is a Skinner box game?  A big problem in the industry right now is that a lot of different people are talking about this, and there’s very little consistency on what a Skinner box game is, or why it’s a problem.  It’s relatively common to hear people throw phrases like this around “slot machines are bad because they’re just a Skinner box”.  I’m going to show that those types of claims are at best misguided, at worst irresponsible.  In order to do that, lets start with the root of the problem, a misunderstanding of what Skinner actually discovered.

Operant Conditioning

BF Skinner was one of the primary contributors to a field of science called “Behavioral Psychology”.  The idea behind Behaviorism is that we can learn a lot about what complex animals do just by scientifically observing their behavior.  They claimed you largely don’t have to look at how an animal is physically made up at all to understand it’s future actions, you can simply derive predictable information from it’s past actions.  Behaviorism is still one of the strongest fields of psychology today, because it has the innate advantage of actually being able to easily be measured.  It’s very difficult to measure how angry a person is, it’s very easy to measure how many times a person pressed a “nuke my enemy” button however.

There’s this common flawed assumption that Skinner was exploring a strange edge case, a glitch in the program that could be used to get people to do things.  Simply put, this is just inaccurate.  We’re biologically programmed to learn to do things that are beneficial to our survival, and operant conditioning is the method by which nature programs us.  If we need to eat coconuts to survive, and we know from accidentally discovering that smashing the big tree with a rock makes coconuts fall out of the tree, we keep doing it until we knock enough coconuts out of the tree either to be dissatisfied with the slow rate of progress, or until we’ve eaten so many coconuts there’s no reason to gather more.  All that Skinner did was basic mapping of this universal process present in all higher level living things.  The complexity of the action required nor the diversity of the reward earned does not invalidate the basic mechanic under which we learn to repeat an action.  So it’s still a skinner box if the action required to get a reward is dense and multi dimensional like “solve this complex calculus problem”, and the reward is abstract and immaterial like “the joy of learning”.  The only gap in the original model skinner developed was the same gap that applied to the science of behavioral psychology as a whole, he didn’t immediately predict the internal reward processes that are wrapped into the system as well.  We later discovered it’s possible to condition someone to an action by having them observe someone else performing the action to receive some reward.  And later still we discovered there are internal shortcuts both at the performing action side and the receiving reward side.  So our brain is able to give us rewards in the form of little dopamine dumps just for thinking about doing something we believe will produce positive results for us, and also, our brain rewards us in a similar way for doing things that don’t have any immediate benefit, but we associate with long term rewards.  Hence why the joy of learning is a totally valid reward for a skinner box.

All Games are Skinner Boxes

For a human being, every action we perform is roped into an operant conditioning loop somewhere.  We don’t interact with any system, ever, at all, that we don’t believe we’ll get some reward out of.  And we can do amazing mental gymnastics to convince ourselves that the things we’re doing are rewarding to us.  Every single game ever made is packed full of many many operant conditioning loops.  In fact, the best games are the ones so crafted that the operant conditioning loops are well scheduled and continue to be rewarding.  We also don’t really like games that satisfy us.  I mean, we enjoy playing them, and we might write a nice review about them, but we’ll usually just put them away after that and do not look at them again.  We’re far more enamored with the promise of something more.  Games which bring us back with hidden secrets (which reward you semi-randomly for taking time to explore and search more vigorously) and increasing ingame power for time invested are the games we go back to over and over again.  Skill building also works on this mechanism, we practice a difficult skill, each time we make an attempt we either succeed and are rewarded for our success, or fail, and miss the reward.

The Legend of Zelda executes the usage of operant conditioning brilliantly.  It’s a quilt of different rewards metered out based on the success of your performance (which is more or less random).  First you enjoy raw exploration, finding amazing and interesting looking scenes.  Then you enjoy defeating monsters.  Next you enjoy finding hidden objects.  After that you enjoy solving puzzles and the feeling of intelligence that gives you.  Finally you enjoy unlocking the next plot point.  Each of these enjoyable thing is a reward the game meters out to you on a reasonably deliberate schedule.  You never go too long without knowing there’s another reward coming, and because the rewards are varied in type and magnitude it takes a long time to grow bored of the rewards being metered out to you.

A Rose by any Other Name

There are many things we talk about which ultimately describe operant conditioning mechanisms.  Discussion of “pacing” is very common in game design and storytelling.  But pacing is just another word for scheduling out your rewards more effectively.  “Wow factor”, “Loot”, “Solving”, “Score”, are all various types of rewards.  We implicitly understand that if blowing up every monster results in the same score it’s not as entertaining if the reward is more varied.  In fact, as operant condition testing moved to humans one of the big discoveries is that we get sick of eating candies a lot faster than pigeons or rats do.  What researchers had to do was introduce novel rewards for continued participation in the conditioned activity to encourage.  Varying rewards is an integral part of implementing operant conditioning in humans.  This is the reason why slot machines sometimes distribute 5 coins, but sometimes distribute a million dollar win.  If you always only win 5 coins the reward becomes boring, and if you don’t win anything while you’re playing for the million it feels futile.

When most game developers and designers complain about skinner box games, they complain that the game gives you the same simple rewards again and again, keying into the varied schedule those games tend to work with.  This is actually not a good implementation of operant conditioning.  One of the tenants of operant conditioning is that the reward can not be one which becomes boring, nor can it be one which satisfies the subject (which is also a version of boredom).  Operant conditioning is all about how we get a subject to pursue the reward in the most vigorous way possible, and to do that optimally we want to provide varied, valuable, and ideally unpredictable rewards, not the same boring thing again and again.  Schedule is key to implementing a solid operant conditioning system, but rewards which hold their value are just as key.  This is why casinos use money or at least something that has a direct monetary equivalent, because people don’t really get bored of money, or ever have enough money.  Most of our games can’t just outright use money, and a player is going to get bored of your reward if it’s just another randomly generated sword slightly better than the last one.  A simplistic reward system ruins our operant system, it doesn’t make it more compulsive.

Building a better tomorrow

So should game designers build skinner boxes or should they not build skinner boxes?  Skinner was not describing nor observing a specific design pattern, Skinner was observing a fundamental property of how humans become interested in and attracted to a system.  Consequently he also found some of the fundamental requirements of addiction, but honestly, game designers put WAY more stock in that than it deserves.  A good game remains interesting as you continue to play it.  I’m suggesting, therefore, that game designers don’t specifically design games to be Skinner boxes, but rather that Skinnerism is used as a litmus test for game design.  If your game produces the same reward over and over without variance you’ve built a weak reward for your players time.  If your game goes hours at a time without rewarding a player you’re not scheduling your rewards well.  If your game SUPER heavily rewards the player at the beginning of the game, but has boring rewards at the end of the game it’s not going to keep people playing it.  If your game fails completely to reward players for starting to play, they’re not going to spend the time needed to master your game, let alone to get to the “good stuff” that comes later.  Game designers shouldn’t be specifically trying to build Skinner boxes, but you can use understanding of what makes a good Skinner box to test whether your game will attract, retain, and addict players.  And make no mistake, while it’s unethical to abuse addicted players for financial gain, you DEFINITELY should be making a game which players can get addicted to.

It’s bad to be a designer who isn’t aware that operant conditioning informs how players fundamentally form enjoyment of your game.  It’s bad to build a game and never think about whether it’s rewarding players on an appropriate schedule.  It’s bad to build a game which doesn’t do anything to keep the rewards it’s giving players compelling, interesting, and varied.  Games which we typically accuse of being “Skinner boxes” are usually worse Skinner boxes than games which we believe are just good games.  Minecraft rewards players in all kinds of cool and interesting ways on a nearly infinite schedule.  Minecraft player rewards span from punching your first tree block into wood, to defeating the ender dragon, to replicating Pokemon red entirely within the command block system.  The rewards are so diverse and scattered you can keep finding new rewards to prevent you from getting bored with them if you really choose to look.  Minecraft is a WAY WAY WAAAAAAY better Skinner box than slot machines are (Auro too).

There are bits of game design completely outside of the skinner box model.  The value of the understanding gained while learning about how gravity physics works by playing Kerbal Space Program is not within the Skinner box model. But the good feeling we get from understanding gravity physics we did not understand before is just another reward which Skinner’s box describes.  As game designers, we TOTALLY should be doing our best to provide value to players outside of the raw feels produced by Skinarrian rewards, but in all honesty, that strategic trick which allows you to crush your Civilization V opponents is probably nearly completely useless in real life.  The fact that it feels so great to master is AWESOME, but it’s completely and totally Skinarrian reward.  You’re not going to actually use that thing to beat the stock market or build the next great startup.  Well, probably not anyways.

Just Desserts

You gain nothing trying to get away from Skinner.  You make your games weaker by not testing them against the Skinner litmus test, and they have absolutely zero chance of being more educational or more meaningful in return.  Skinner didn’t just describe giving candy to rats, he described why we enjoy Citizen Kain and Beethoven’s Piano Sonata #14.  Those pieces of utter complete art reward our brain in varying ways on fairly randomized schedules even when we consume them over and over again.  The difficulty with building skinner boxes is not in tricking people to be more deeply addicted for no reason at all, but rather it’s even understanding how we make the next event in our game deeply and satisfyingly rewarding to the player in context to all the game we have already played so far.

Make Skinner boxes my friends, make amazing and spectacular skinner boxes.  Treat my brain, and surprise me as to when the next thrilling treat will come.  Don’t be afraid of Skinner boxes.  Embrace them and make them better than they ever have been before.


15 thoughts on “Evil Evil Skinner Boxes

  1. Dude, you can’t see a difference between ‘Myst’ and ‘Hit it Rich’? Are you seriously trying to tell people that there is no meaningful distinction between pushing a button and seeing a random animation while a random reward is being calculated and solving ‘Myst’?

    What the fuck are you on and can we send a sample to the FDA?

    1. No, there’s tons of meaningful distinctions, but most of them are not described within the context of operant conditioning. To the part of your brain a Skinner box describes, Myst looks very similar to Hit it Rich. There’s other parts of the brain which do other stuff which finds them totally different from eachother, but Myst is not providing a massively different pattern of dopamine dumps into your brain when compared with gambling games.

      The dopamine dump layer exists in all games. I’m arguing we don’t pretend it doesn’t.

      1. You are not advocating for us to pretend like it doesn’t exist, you are equating all activity to that of a Skinner Box, therefore eliminating any meaningful distinction.

        Categories are made by humans as tools to differentiate between things and make meaningful and useful observations.

        Your opinion is: all games do the same thing.

        Yes, they just “mask it” under layers and layers and layers of narratives, characters, tasks, music and art and use your emotions, memories and skills.

        Drugs are the same, they basically hit that same dopamine dump.

        Sex is the same.

        Wait a second? Everything enjoyable comes down to dopamine. Everything enjoyable is the same.

        What a useful observation… NOT

        This is only useful if you are a nihilist, but nihilists are useless to the rest of us.

      2. Almost all activities we do are partially operating on the operant conditioning dopamine dump layer. You can’t make a decent game that doesn’t operate on this layer, it can operate on other layers as well, but it’s not an option to have an enjoyable game that just doesn’t cause a brain dopamine interaction. It’s like trying to make a game that doesn’t require your eyes because games that require your eyes are addictive.

        I know game designers have decided to use the term “skinner box” to mean something that has nothing to do with what skinner boxes actually observed scientifically. I think it’s a thoughtless and slightly dangerous redefinition, but fine, if you really want you can have it.

        I’m advocating designers should not be stupid enough to conflate the redefined concept of a “skinner box” to mean it’s bad to make games that optimize operant conditioning. It’s no more bad to make a game that works better on an operant conditioning level than it is to improve the graphics of your game so they aren’t so visually repulsive. In both cases you’re adapting your game design to be less offensive to the fundamental biology of the people playing your game.

      3. You are a complete and utter idiot. Skinner games has a specific meaning, we use it to define specific kinds of games that are made to specifically target this type of behavior and exploit it to make you spend tons of cash on an activity.

        If you think distinctions and categories are useless because behaviors can be broken down to discover a similar component: Hitler loved dogs.

        Do you understand now?

        How don’t you understand that your point makes no sense.

        Eradicating the distinction helps no one. While making the distinction allows game designers to see what they can avoid to not make the equivalent of drug dealing or casino slots.

        Just because a documentary and reality TV share many concepts doesn’t mean I should throw away a base distinction to make you feel better for producing TRASH.

      4. Don’t you fundamentally understand that your argument doesn’t support your position, but actually the opposite.

        If something can be boiled down to something else, how would I make distinctions between things?

        Just saying that everything is fundamentally a thing doesn’t tell me anything useful about those things.

      5. The activity of creating games also operates on the same principle.

        How would I differentiate from games that abuse this for case and a creative activity like a hobby?

        Cocaine is a substance, caffeine is substance. How do I differentiate between the two when talking about negative side-effects?

        Do you have a replacement term? How do you propose you spread it?

        How many problems do you have to solve in order to solve the non-issue of perceived slight against a part of a field of science?

      6. It’s fundamentally not problematic to argue for redefinition of a term, especially when the initial colloquialism of that term is based on a misunderstanding. I’m not going and planting car bombs to kill game designers/critics I find using the termonology a way I don’t like. I’m just saying “hey, think about this”.

        I’ve already said, I’m willing to give up on the redefinition of “skinner box”. It’s not totally inappropriate the way game designers use it because there’s a coldness and sterility in the experimental setting which loosely is echoed in some of these games who seem to want to primarily design operant conditioning engines so they can add value extractors into them and exploit their most dedicated fans. But I DO want to dispel the totally inaccurate presumption which has arisen, since the popularization of “skinner box” in game design, that what makes what those game studios are doing evil is simply that they are designing games which use operant conditioning.

        All the good games you like also use operant conditioning, whether they were intentionally designed to or not. Operant conditioning is the sugar in your cake. It’s not evil to make the cake not too bland but not too sweet. Using operant conditioning well in a game is a sign of mastery, not maliciousness. Don’t you agree?

  2. I am a behavior scientist and I want to commend your writing here for pointing out the technical imprecision of how the term “Skinner box” is used in game design communities.

    1. You are a behavior scientist and you can’t see that the term Skinner box applied to games is functional for the needs of the domain?

      You don’t even have an alternative? And how do you suggest making the alternative wide-spread? This ‘skinner-box-but-don’t-confuse-it-with-skinners-research’ term would still need to be explained in context with Skinner’s research.

      Study human behavior more to understand how what you propose is ludicrous. Real-people in real conversations need to communicate effectively, what you propose is killing effectiveness of communication in favor of some perceived slight against Skinner experiments, Skinner’s research and their respective reputation.

      It isn’t already difficult to communicate, we have to make idiotic ‘PC-like’ compromises to satisfy self-proclaimed protectors of arbitrary terms.

      1. “Skinner box” is not an arbitrary term. It has been used for highly specific scientific communication for over 90 years. Non-science communities cannot simply use a pre-existing term to refer to whatever they want, especially not as technically incorrect and functionally imprecise as the way “Skinner box” is used in game design.

        I’ve suggested “gambling mechanics” or “slot machine design” before. These are easily understood and more directly comparable to how “Skinner box” is currently used in game design. Just say what you mean. If someone is specifically designing some interactive system to hook a player into an exploitative loop, then that’s designing a game around gambling mechanics. Systems built on these principles can easily be compared with slot machines (which, again, are for gambling). The majority of people will immediately know what a game designer might mean by “gambling” or “slot machine.” Why not use these terms? They are already in use, are already agreed upon to mean what we are referring to, and they don’t contaminate highly technical research terminology.

        This isn’t an attempt to police language or curb discussion on (bad) game design. This is just an attempt to get people on the same page. Saying “gambling mechanics” is incredibly clear and direct. Saying “Skinner box” is vague and a flagrantly ignorant misuse of a scientific term just to sound jargony.

      2. LOL. Are you still at this. No it’s not vague. Everyone else knows what they are talking about, you are just one random dude trying to conflate things.

        Gambling mechanics implies monetary reqard or spending. This is not what skinner box referes to.

        You are still an idiot.

      3. Calling it gambling or slot machines doesn’t make it apparent how this works.

        I am completely baffled by people who are able to spend time on a slot machine. I don’t find it in the least interesting.

        Saying it’s like a slot machine requires a certain familiarity with how slot machines actually work.

        Slow machines are also regulated and are legally considered gambling.

        Adopting that kind of terminology invites government regulation into something that may be playing on the same strings in the same way, but has a lot more baggage.

        If your proposal – after all the bashing I dished out – is ‘gambling’ and ‘slot machines’ you are helpless.

        You apparently can’t think how other people think, nor recognize that it is difficult to change established lingo (you aren’t cool dude) and ON TOP of that your proposal is to invite government regulation by conflating ‘skinner box’ with ‘gambling’.

        Great job! Is there a Darwin award for most retarded proposals blog?

      4. “Great job! Is there a Darwin award for most retarded proposals blog?”

        Are you like incapable of believing people can hold different opinions than you without imagining them as drooling nazi’s or something? Your argumentative style is extremely immature and unflattering.

        Even if I’m totally wrong, getting people thinking about this issue in a deeper way is good. Your wildly emotional knee jerk reactions show you’ve thought about this way less deeply than I have. The name calling is not helping your side in any way.

    2. “Non-science communities cannot simply use a pre-existing term to refer to whatever they want, especially not as technically incorrect and functionally imprecise as the way “Skinner box” is used in game design.”

      I mean, they can. It’s problematic, hence this article and debate, but this wouldn’t be the first time jargon usage crossed a line and definitions got muddy. It’s not like it’s morally wrong for “skinner box” to have two moderately disconnected definitions, but if game design communities agree I’d prefer they use something that’s less easily confuses people.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s