Evil Evil Skinner Boxes

A major meme running the rounds in game designdom right now is the dangerous evil of Skinner boxes, and how we really need to stop making games that are just Skinner boxes.  We especially hate games like Pokemon Go or Clash of Clans.  These insidious creations encourage you to log on at least once a day for an extra reward of seeing your resource bar creep up a little each time you put energy into the game.  Their evil plan is clearly to addict players with this scientifically proven simple reward method which humans have no ability to resist, thus we will ultimately take our time energy and money against our will.  Clearly Supercell, Niantic, etc, are key players in the downfall of society!

I’m going to make a different claim.  I claim that a designer who puts effort into making sure their game is a better Skinner box is, almost objectively, just making a better game.  A lot of people are throwing around the term “Skinner box”.  What is a Skinner box?  In a nutshell, in the 1930s a scientist named BF Skinner did a series of experiments where he put rats in a box with a lever.  If the rat pulled the lever it got a little chunk of food.  There was a bunch of experiments Skinner performed with this setup, he wanted to see what was the ideal way to get Rats to pull levers obsessively as many times per minute as possible.  He tried a bunch of food distribution techniques, and ultimately found that best results came when initially rats got a treat every time, but after the first several tries a randomized delay would be put between lever pulls, so the rat may have to pull the lever 3 times before the treat would come, but the next pull it would come right away, and the pull after that he’d have to pull 6 times for the treat.  Skinner was looking for the ideal distribution of rewards per input to keep rats motivated to perform input.  It was later confirmed that, with some caveats, the ideal pattern for rats is also the ideal pattern for human motivation.

So clearly it’s pretty evil to put people in a giant metal box and make them press a lever for food.  But what really is a Skinner box game?  A big problem in the industry right now is that a lot of different people are talking about this, and there’s very little consistency on what a Skinner box game is, or why it’s a problem.  It’s relatively common to hear people throw phrases like this around “slot machines are bad because they’re just a Skinner box”.  I’m going to show that those types of claims are at best misguided, at worst irresponsible.  In order to do that, lets start with the root of the problem, a misunderstanding of what Skinner actually discovered.

Operant Conditioning

BF Skinner was one of the primary contributors to a field of science called “Behavioral Psychology”.  The idea behind Behaviorism is that we can learn a lot about what complex animals do just by scientifically observing their behavior.  They claimed you largely don’t have to look at how an animal is physically made up at all to understand it’s future actions, you can simply derive predictable information from it’s past actions.  Behaviorism is still one of the strongest fields of psychology today, because it has the innate advantage of actually being able to easily be measured.  It’s very difficult to measure how angry a person is, it’s very easy to measure how many times a person pressed a “nuke my enemy” button however.

There’s this common flawed assumption that Skinner was exploring a strange edge case, a glitch in the program that could be used to get people to do things.  Simply put, this is just inaccurate.  We’re biologically programmed to learn to do things that are beneficial to our survival, and operant conditioning is the method by which nature programs us.  If we need to eat coconuts to survive, and we know from accidentally discovering that smashing the big tree with a rock makes coconuts fall out of the tree, we keep doing it until we knock enough coconuts out of the tree either to be dissatisfied with the slow rate of progress, or until we’ve eaten so many coconuts there’s no reason to gather more.  All that Skinner did was basic mapping of this universal process present in all higher level living things.  The complexity of the action required nor the diversity of the reward earned does not invalidate the basic mechanic under which we learn to repeat an action.  So it’s still a skinner box if the action required to get a reward is dense and multi dimensional like “solve this complex calculus problem”, and the reward is abstract and immaterial like “the joy of learning”.  The only gap in the original model skinner developed was the same gap that applied to the science of behavioral psychology as a whole, he didn’t immediately predict the internal reward processes that are wrapped into the system as well.  We later discovered it’s possible to condition someone to an action by having them observe someone else performing the action to receive some reward.  And later still we discovered there are internal shortcuts both at the performing action side and the receiving reward side.  So our brain is able to give us rewards in the form of little dopamine dumps just for thinking about doing something we believe will produce positive results for us, and also, our brain rewards us in a similar way for doing things that don’t have any immediate benefit, but we associate with long term rewards.  Hence why the joy of learning is a totally valid reward for a skinner box.

All Games are Skinner Boxes

For a human being, every action we perform is roped into an operant conditioning loop somewhere.  We don’t interact with any system, ever, at all, that we don’t believe we’ll get some reward out of.  And we can do amazing mental gymnastics to convince ourselves that the things we’re doing are rewarding to us.  Every single game ever made is packed full of many many operant conditioning loops.  In fact, the best games are the ones so crafted that the operant conditioning loops are well scheduled and continue to be rewarding.  We also don’t really like games that satisfy us.  I mean, we enjoy playing them, and we might write a nice review about them, but we’ll usually just put them away after that and do not look at them again.  We’re far more enamored with the promise of something more.  Games which bring us back with hidden secrets (which reward you semi-randomly for taking time to explore and search more vigorously) and increasing ingame power for time invested are the games we go back to over and over again.  Skill building also works on this mechanism, we practice a difficult skill, each time we make an attempt we either succeed and are rewarded for our success, or fail, and miss the reward.

The Legend of Zelda executes the usage of operant conditioning brilliantly.  It’s a quilt of different rewards metered out based on the success of your performance (which is more or less random).  First you enjoy raw exploration, finding amazing and interesting looking scenes.  Then you enjoy defeating monsters.  Next you enjoy finding hidden objects.  After that you enjoy solving puzzles and the feeling of intelligence that gives you.  Finally you enjoy unlocking the next plot point.  Each of these enjoyable thing is a reward the game meters out to you on a reasonably deliberate schedule.  You never go too long without knowing there’s another reward coming, and because the rewards are varied in type and magnitude it takes a long time to grow bored of the rewards being metered out to you.

A Rose by any Other Name

There are many things we talk about which ultimately describe operant conditioning mechanisms.  Discussion of “pacing” is very common in game design and storytelling.  But pacing is just another word for scheduling out your rewards more effectively.  “Wow factor”, “Loot”, “Solving”, “Score”, are all various types of rewards.  We implicitly understand that if blowing up every monster results in the same score it’s not as entertaining if the reward is more varied.  In fact, as operant condition testing moved to humans one of the big discoveries is that we get sick of eating candies a lot faster than pigeons or rats do.  What researchers had to do was introduce novel rewards for continued participation in the conditioned activity to encourage.  Varying rewards is an integral part of implementing operant conditioning in humans.  This is the reason why slot machines sometimes distribute 5 coins, but sometimes distribute a million dollar win.  If you always only win 5 coins the reward becomes boring, and if you don’t win anything while you’re playing for the million it feels futile.

When most game developers and designers complain about skinner box games, they complain that the game gives you the same simple rewards again and again, keying into the varied schedule those games tend to work with.  This is actually not a good implementation of operant conditioning.  One of the tenants of operant conditioning is that the reward can not be one which becomes boring, nor can it be one which satisfies the subject (which is also a version of boredom).  Operant conditioning is all about how we get a subject to pursue the reward in the most vigorous way possible, and to do that optimally we want to provide varied, valuable, and ideally unpredictable rewards, not the same boring thing again and again.  Schedule is key to implementing a solid operant conditioning system, but rewards which hold their value are just as key.  This is why casinos use money or at least something that has a direct monetary equivalent, because people don’t really get bored of money, or ever have enough money.  Most of our games can’t just outright use money, and a player is going to get bored of your reward if it’s just another randomly generated sword slightly better than the last one.  A simplistic reward system ruins our operant system, it doesn’t make it more compulsive.

Building a better tomorrow

So should game designers build skinner boxes or should they not build skinner boxes?  Skinner was not describing nor observing a specific design pattern, Skinner was observing a fundamental property of how humans become interested in and attracted to a system.  Consequently he also found some of the fundamental requirements of addiction, but honestly, game designers put WAY more stock in that than it deserves.  A good game remains interesting as you continue to play it.  I’m suggesting, therefore, that game designers don’t specifically design games to be Skinner boxes, but rather that Skinnerism is used as a litmus test for game design.  If your game produces the same reward over and over without variance you’ve built a weak reward for your players time.  If your game goes hours at a time without rewarding a player you’re not scheduling your rewards well.  If your game SUPER heavily rewards the player at the beginning of the game, but has boring rewards at the end of the game it’s not going to keep people playing it.  If your game fails completely to reward players for starting to play, they’re not going to spend the time needed to master your game, let alone to get to the “good stuff” that comes later.  Game designers shouldn’t be specifically trying to build Skinner boxes, but you can use understanding of what makes a good Skinner box to test whether your game will attract, retain, and addict players.  And make no mistake, while it’s unethical to abuse addicted players for financial gain, you DEFINITELY should be making a game which players can get addicted to.

It’s bad to be a designer who isn’t aware that operant conditioning informs how players fundamentally form enjoyment of your game.  It’s bad to build a game and never think about whether it’s rewarding players on an appropriate schedule.  It’s bad to build a game which doesn’t do anything to keep the rewards it’s giving players compelling, interesting, and varied.  Games which we typically accuse of being “Skinner boxes” are usually worse Skinner boxes than games which we believe are just good games.  Minecraft rewards players in all kinds of cool and interesting ways on a nearly infinite schedule.  Minecraft player rewards span from punching your first tree block into wood, to defeating the ender dragon, to replicating Pokemon red entirely within the command block system.  The rewards are so diverse and scattered you can keep finding new rewards to prevent you from getting bored with them if you really choose to look.  Minecraft is a WAY WAY WAAAAAAY better Skinner box than slot machines are (Auro too).

There are bits of game design completely outside of the skinner box model.  The value of the understanding gained while learning about how gravity physics works by playing Kerbal Space Program is not within the Skinner box model. But the good feeling we get from understanding gravity physics we did not understand before is just another reward which Skinner’s box describes.  As game designers, we TOTALLY should be doing our best to provide value to players outside of the raw feels produced by Skinarrian rewards, but in all honesty, that strategic trick which allows you to crush your Civilization V opponents is probably nearly completely useless in real life.  The fact that it feels so great to master is AWESOME, but it’s completely and totally Skinarrian reward.  You’re not going to actually use that thing to beat the stock market or build the next great startup.  Well, probably not anyways.

Just Desserts

You gain nothing trying to get away from Skinner.  You make your games weaker by not testing them against the Skinner litmus test, and they have absolutely zero chance of being more educational or more meaningful in return.  Skinner didn’t just describe giving candy to rats, he described why we enjoy Citizen Kain and Beethoven’s Piano Sonata #14.  Those pieces of utter complete art reward our brain in varying ways on fairly randomized schedules even when we consume them over and over again.  The difficulty with building skinner boxes is not in tricking people to be more deeply addicted for no reason at all, but rather it’s even understanding how we make the next event in our game deeply and satisfyingly rewarding to the player in context to all the game we have already played so far.

Make Skinner boxes my friends, make amazing and spectacular skinner boxes.  Treat my brain, and surprise me as to when the next thrilling treat will come.  Don’t be afraid of Skinner boxes.  Embrace them and make them better than they ever have been before.