The Production of Subject and Space in Video Games

My latest article has been published in GAME, the Italian Journal of Game Studies, along a number of very interesting other articles.

For GAME, go here.

For my article, go here.

Game Idea #47

Time for a new game idea! My new game idea is inspired by a discussion on the nature of images and language. Letters are often considered as two-dimensional graphical elements, not capable of three-dimensionality. On the other hand, they are seen as a different representative system, and not associated with systems of visual representation, although they use the same basic graphical elements such as lines, dots, planes and even volumes. Therefore I like the idea to make a text adventure whose world is visually constructed through letters and words, a world that maintains the illusion of depth and can be navigated, something that uses in its representation of its world the words that are associated with objects, rather than depicting the objects themselves.

Here comes..

This is not a Word

This is not a Word is a “text-world” adventure. Its world is visually represented by letters, words and sentences that are discernible as objects. The way in which these letter-objects are displayed, changes, as we approach such object. For example if we see a “tree” and approach it, we start to identify the many “branches”, and if we get closer we see its “leaves”. Or we see a “river” in a distance, and as we approach it, we see the “water” flow, and the “fish” in it.

kaligrafi

By eliminating from or adding to the world certain letters, the way we perceive the environment changes, since without certain letters, certain “objects” can’t exist.

The game makes use of color, font families and other aesthetic parameters of typography.

The title “This is not a Word” is intented to be a pun on “This is not a World“, since I want to draw attention to the “artifical” character of any type of representation. It is also a reference to Magritte’s series of paintings titled “This is not a Pipe”, which approached the question of representation from the “other end”, yet being about the relation between images and language.

The game will be available in several languages, including dead ones. Having the game in several languages is a funny aspect, because it somehow seems not to make sense at all, but then I believe that it makes a difference because we also would look at different visual styles of representation and not merely of objects that have a same shape regardless of what languages have been used to give them their shapes.  And I’m very intrigued by the idea to use dead languages, especialy cuneiform. I somehow like the idea to see people recognize letters and words of dead languages as objects rather than text, because it seems to show how the representative values of certain combinations of lines and dots can shift culturally and historically, questioning our ways of seeing and the way cultures maintain or “forget” about distinctions that decide whether graphical elements qualify as “text” or “visual representation”. A Quipu version that uses “real” threads would be also highly interesting to use.

If we lose all letters in the game, what would we see? The world “as it is”, or “nothing”?

Game Idea #46

After some time, I’ve come up with a new game idea. This time it is an experimental game about perspective… here it comes:

Game Idea #46

Cubism

This multiplayer online game starts by presenting an object or human according to rules of classic perspective. However, as more players join the game, the number of vantage points multiplies and starts to “deform” the object/human that the individual player sees. Stylewise, the object/human starts to turn into some sort of cubist painting. As players navigate around the object/human, the changes in the vantage points are reflected onto the representation of both object/human and environment, which turns gameplay into the shaping of an interactive cubist sculpture-world that morphs in real-time.

The game has two experimental goals: On one hand, it is a technological experiment, challenging the capacities of current game engines and graphics programmers; on the other hand it is an experiment on reception, challenging the ways in which players “view” 3D representations, and their notion of interaction.

Bonsai

“Narrative”, as it is perceived today, is the decision to chose one out of the many branches of a “branching story” and present it as the idealized “original work”. In a sense, it means to fix the outcomes at certain decision nodes in favor of a “best” story, and to restrict players to “play out” only one out of many possible scenarios: the one with the “halo”. We did this often when we were kids: We took our ball to the playground and kicked it around while we were imagining to dribble around the Maradona’s and Pele’s in our minds and scored in last minute that one goal that would bring us the World Cup and make us the hero of our nation. The ball is round, yes, but we never wanted it to be “too round”.

Narrowing down a possibility space to an idealized scenario is what we call “linearity” today. We’ve grown suspicious towards the word “narrative” because in today’s belief, a “narrative is linear”: In a “narrative” decision nodes are carefully decided and connected in a way that makes only one ideal scenario emerge. However, it seems to be a mistake to call this linearity for all the other scenarios that are left out wouldn’t have been less linear than the one that was favored. A better word to name this could have been essentialism: to refrain from embracing multiplicity and possibility and idealize singularity and determinism; to privilege one variation as the “original” and mark all other variations as being of a lesser nature. Isn’t it the artist himself who often defines his own artistic drive, practices and creativity on the grounds of this “original”? While he could have presented his work at many different stages and in a great number of variations, he picks one moment of what is a process and accepts all other moments to be rendered into “sketches” of the “original”. This is probably one of the most telling acts that makes visible the artist’s aleniation to his own labour in the light of the “original” as an instiution of art and a favored label of the industry.

Presenting only one stage out of the many stages in the creative process as the “finished” work, the “original”, has been for centuries a practice that is deeply rooted in our culture. Ironically, the game industry follows this practice and presents to us a certain development stage of a game as the “original”, as “that” game. But which one is “that” game really? The one that is presented as the “finished” one? We know that this can mean anything. What more evidence is then needed to lay bare the arbitrariness behind such decision? While on one hand the modding communities and the open source  trend open a door to reveal the arbitrariness behind the notion of the original, on the other hand we also seem to witness a kind of orgy of the original. The multitude of games within a game finds us in a fragmented way (for example in the form of increased sequelization), every “deviation” being presented to us as another “original” in the app stores. This can be traced back to the logic behind the game business, which can only make use of the “product” as a unique, seperate, stand-alone and “finished” object. The game designer has therefore no other choice to decide at some point which stage or variation of his work will be presented as “that” game. In fact, he has changed his creative processes as to make his labour to aspire to a product or a series of products: the “original”s that will be displayed on the front page of the app store as the “latest” hit.

Today’s art and popular culture criticism suffers, as it did in the past, from mistaking the restrictions of the “original” as an art institution and the modes of production in the culture industry for the nature of the artistic forms themselves. The traces of this confusion can be clearly seen in game studies: what it perceives today as the “problem of linearity” in the novel and in film, is merely the impact of industrial practices build around the notion of “originality”. Although it should be dropped as useless and misleading divide, the distinction between linearity and non-linearity somehow helps us to make visible the restrictions that have been historically put on modes of consumption in literature. What we call linearity should be seen in its historical context, as the preferred mode of consumption of the industries in the past. However, this is probably the only thing in which this distinction is helpful. Otherwise it just causes the responsibility of the institutions in the past in fostering “linearity” to be taken from their shoulders and being put on the shoulders of the “object”, “narrative” itself, now stripped from its historical context, and put as the scapegoat. Non-linearity, on the other hand, has turned a buzzword that presents the same institutions attempts to turn games into products and champion on “originality”, as progress, as an achievement.

The chance that bears itself today through the digitalization of games is that this may play a part in making the arbitrariness of the “original” visible, and allow narratives and storytellers to free themselves from the confines of the mode of production and consumption of the mainstream industries. Game researchers, however, need to revise those of their concepts that remain within the discource of “originality” if they want to play a pioneering role in this. A first step for this is to stop to see “linearity” as the nature of narratives and realize that linearity is rather a result of the mode of production of the culture industries and the way in which literature has become institutionalized. Narratives and games are the same thing, the “narrative” of the industry, however, is a “game” bereft of its possibility space, a bonsai, if you wish.

Rock-Paper-Scissors: A Linguistic Approach to Games

This essay is a sketch for a linguistics-based approach to games. It is “under construction” and will be frequently updated.

A Bit of Theory

Expression and Content

According to danish linguist Louis Hjelmslev, language has two layers:

  • Expression Plane
  • Content Plane

The expression plane is the plane of signifiers, whereas the content plane is the plane of signifieds. A sign (or meaning) is then the connotative sum of a signifier and a signified. [1]

When we play a game, what we “interact” with, and interpret, is the expression plane, which we perceive as a chain of signifiers. The expression plane must be always present in some material form, whereas the content plane is an organized accumulation of mental contepts, which are associated with the material forms of the expression plane according to some rules or conventions.

When we say that a sign is the connotative sum of expression and content, we mean with that that the expressive form alone is not the meaning. Meaning arises when the expressive form is associated with a mental concept. For example, the (X) button on a gamepad is not the sign itself, it is only a signifier. Before the signifier (X) can be read as a sign, it must be associated with a certain signified, something that it will stand for, such as “kick” or “shoot”.

The same counts for players. Their visible performance amounts to a chain of signifiers that can be associated with signifieds. Through this association, player action produces signs that can be read as meaningful action. In that sense, player itself is a sign, a meaning that should not be confused with the “real person” at play, who only serves as a signifier and doesn’t gain a meaning until her behavior stands for some content.

Substance and Form

In Hjelmslev’s model, both expression and content plane have a form and a substance. This gives us the following pairs:

  • Form of Expression
  • Substance of Expression
  • Substance of Content
  • Form of Content

Based on this distinction, we can say that every sign and the sign system that it is part of, consists of four related layers. The same can be said for signs in games, and games as sign systems.

Beyond the Sentence

While Hjelmslev’s was concerned with natural languages, Roland Barthes, and later Seymour Chatman, applied his distinctions to semiotic systems and discourses, such as narratives in the novel and in film. This move required to “expand” the linguistic model so that can it lend itself to the study of discourse, which presents an analytical unit that is larger than the sentence.

While in linguistics the largest unit of analysis are sentences (beyond the sentence, all that linguistics finds as an object of study, are just other sentences), discourse analyses and narrative theory studies units “beyond the sentence”, and works with categories that must “grow” accordingly: Actants and Events, rather than subject and predicate. Such growth in units also applies to modalities of time and space: story time versus discourse time, and the construction of fictional space through spatial signs.

Yet, the growth in categories doesn’t render Hjelmslev’s distinction between expression and content plane useless. His distinction applies to all semiotic systems, including games, as I try to show in this essay.

Rock-Paper-Scissors as Language

Based on the game Rock-Paper-Scissors, I briefly show how Hjelmslev’s (and Chatman’s “expanded”) model applies to games.

Form of Expression

The form of expression consists of the materialized forms that are utilized and recognized as the array of signifers of the language. In Rock-Paper-Scissors, this array of signifiers consists of the gestures |rock|, |paper| and |scissors|.

Substance of Expression

The substance of expression is the material substance that is utilized to shape the forms that are recognized as the array of signifiers of the language. In Rock-Scissors-Paper, these are the players hands.

Substance of Content

The substance of content is the catalogue of positive meanings that qualify as the signifieds of the language. In Rock-Paper-Scissors, these are semes like “rock”, “paper” and “scissors”.

Form of Content

The form of content is the arrangement of semes according to an internal system of operators. Through this internal system, semes leave their positive meanings behind and gain values, which allows them to be compared and exchanged. In Rock-Paper-Scissors, this is the logical form and its operators that holds up the intransitive relationships between the semes “rock”, “paper” and “scissors”. Indeed, as values that are determined by this underlying logical form, “rock”, “paper” and “scissors” can be exchanged, and they can also be compared in terms of whether one is superior to the other.

The Materiality of the Expression Plane

The content plane can be completely imaginary. One may invent a number of substances of content (a “Pac Man”, a “Pinky”, a “maze”, “dots” etc), and then give this content a form, that is, design a value system that defines how these substances stand in relation to each other. The expression plane however, must be material, because it is only through its material form through which the world of semes and their underlying value system (the way we have to evaluate their relationship, and interprete the resulting states) can be “expressed” to someone. Usually, a rulebook or a tutorial will be employed in order to describe the content plane. But playing (or observing players in action) serves the same purpose, because from the way a chain of signifiers is interpreted by others, one can infer the elements and structure of the content plane.

Articulation and Signification

During interpretation, we associate a chain of forms (signifiers) with a group of mental concepts ( signifieds). This produces a chain of signs (meaning). Roland Barthes describes this process like a flat line between two amorph masses of air and water suddenly taking on a particular shape. A certain order of the signifiers will change the “air pressure”, which in return will simultaneously divide and re-unite the “water mass” of mental concepts into patterns of waves. This wave pattern can be “read”.

An example for this can be given from Chess: When the board and chess figurines aren’t used for play, the line between air and water remains flat: only two unassociated masses of signifiers and signifiers are present. However, once we take the board out of the box and place the chess figurines onto it, the “air pressure” changes and discernible “waves” emerge: We can now “read” a game state. Each player move alters the “air pressure” and a new meaningful game state (a readable chain/composition of signs) emerges.

When we come back to Rock-Paper-Scissors we can describe the process as follows: When a player poses the gesture |rock|, she simultaneously mobilizes both planes together with both their forms and substances: Her hand is the substance that is used to “shape” the gesture |rock|and thereby it puts forward a form that is recognized as a valid signifier. This form, once posed, does not only “call” the seme “rock”, but also the potential value “rock”. Once the opposing player poses a counter-gesture (let’s say |paper|) and calls thereby calls the value “paper”, a productive articulation of signs takes place, which enables a “reading”: “Rock versus Paper. Paper wins.”

Lizard Spock

RPS Lizard Spock, a version expanded to include the signs Lizard and Spock to the existing sign system (or language) of RPS, alters both the array of signifiers and the array of signifieds. On the expression plane, new forms have been added (the gestures |lizard| and |spock|), whereas the substance of expression, hands, remains the same. On the content plane, things seem to become a bit more complex, especially when considering the form of content. In terms of substance, only the two semes “lizard” and “spock” have been added, but the truly significant change can be observed in the internal system that must integrate the added semes as replacable and comparable values.

A Classification of Rules

Rules of Substance

These rules specify the substances that can be used to “carry” the forms that are recognized as signfiers within the system. For example in professional sports, one may come across very strict rules in regard to existents, such as the weight, diameter, air pressure, and material of a football. However, kids would often use a tin can or any other object they see fit as a “football”. In Rock-Paper-Scissors, people often go by simply using their hands as the carriage/substance, but it could well be a pair of d3’s made of wood or diamonds.

Rules of Form

These rules specify the parametres and traits that a form must possess in order to be recognized as a valid signifier within the system. Forms may be subject to standards (as it is the case in professional chess), however, one can often see variations in style and theme (like in a Star Wars themed set of chess figures). It can be compared to font families, which are variations of a certain set of types with discernible traits. Quite often, style and theme variations may push the limits of recognition in regard to form. In Rock-Paper-Scissors, only three gestures with clearly discernible features are recognized as forms/signifiers.

Rules of Content

These rules delineate the range of existents and events in the game. For example the array of units in a RTS, or weapons in an FPS; or the array of actions, such as kick, punch or dodge, in a fighting game. In Rock-Paper-Scissors, the rules of content limit the number of existents to three (the existents “rock”, “paper” and “scissors”), and the actions to a single one: “attack”.

Rules of Value

The range of operators and the set of rules that specifies the arrangement of semes around these operators. These rules hold up a system of values against which relationships between content elements can be measured, and a state of affairs be expressed. In Rock-Paper-Scissors there is a single operator (“>”) which sets up an intransitive relationship between the semes “rock”, “scissors” and “paper”. When two gestures are posed against each other, they can be evaluated based on the value system (semes + operators or, if you wish, data + algorithms) they put into motion, and they express now a state of affairs that can be “read” to decide a winner.

Rules of Association

Rules that specify what form is to be associated with what content. In Rock-Paper-Scissors, the gesture |V| must be associated with the seme “scissors”. Many games are very strict in regard to association, since avoiding ambiguity seems to be a major concern in most games. But it is often possible that a signifier can express more than one signified, and that a signified can be expressed with more than one signifier. For example, many games feature a “joker” or “wild card”, which allows the player to associate this signifier with a seme of her desire. [Example for one signfied-many signifiers?]

Rules of Articulation

The rules in regard to how signifiers must be articulated and arranged in respect to time and space. These rules bring an order to the signifiers so that their arrangement in time and space (and changes in such arrangement) can be read as a state of affairs (and as a progression from state to state). These rules could be also called compositional rules. In its broadest sense, a standart game of football asks players to articulate their actions within the borders of a football field, and within the time frame of 90 minutes. Chess, on the other hand, provides different options for use of space and time.

The space in which articulation takes place must be seen twofold: On one hand, it is a format, that is, it provides a general frame in which articulation must take place, just like a sheet of paper or the frame of a painting, within which expressive forms (letters, words, lines, dots etc) must remain in order to be counted as part of the expression. On the other hand it also specifies rules for placement of signifiers within this format, that is, there may be measures that ensure that signifiers are placed according to a certain “reading line” or compositional aspect. For example, a chess pawn may be moved anywhere within the “format” of the chess board, yet it would need to be placed within the confines of a square as well, and not at a line at which two squares intersect, or there would arise ambiguity in regard to how to evaluate the relationship between signifiers, and it would be impossible to “read” the game state.

It is needless to say that a game may have neither a clearly defined format nor any rules in regard to placement. An ARG for example, may use multiple and diverse spaces (a city, GPRS, the internet) as parts of its format, and allow to place signifiers anywhere within these confines. Also, some games may include real life processes such as traffic, weather, stock exchange data etc. However, all these aren’t really a blurring of the line between games and real life: Anything that is recongnized within the format, even if this format is something in constant flux, is elevated into a ludic state, for its behavior-for-itself, is now recognized as a valid signifier, and stands in association with the signifieds of the ludic value system. Therefore, in design, non-intervention is an intervention: Leaving something as it is, is a design choice.

The distinction between the rules classes offered here is analytical. In reality, different classes of rules, and also rules within the same class, will “work together” in order to structure the shape of existents and events.

Computer Games

One of the things that a computer game does, is to display a signifier that is associated to elements in the content plane when it receives player input. For example if the (x) button on a console stands for the seme “kick”, upon receiving (x) as input, the program would call the (chain of) signifiers ( the relevant visuals and sounds), and display them through its output channels. Hence, a computer game is software that converts input signifiers into output signifiers, both of which are associated with the same signifieds. Output signifiers (what some would prefer to call feedback) actually confirm whether we’ve used the appropriate input signifiers or not. This is an important aspect of learning game controls.

Owners/producers of such “tranlation devices” (for example console developers) will ask game developers to stick with the confines of their translation machines, which means to game developers that they limit themselves to value systems that can be expressed through the set of input signifiers of the console (for example the range of buttons on their gamepad), and refrain from putting too much stress on the console’s capacity of displaying output signifiers (like its sound and graphic processing power, or its storage capacity). For example the content plane of a console fighting game must be associated with the limited set of expression elements on the gamepad (buttons etc).

However, as single button games clearly show, a single substance such as a button can host a variety of discernible signifiers/forms (which might be differentiated through button pressing duration, button pushing frequency, and different button states for example). Also complex combinations of mouse, keyboard and click&point interfaces allow for the content plane to be presented in an embedded manner, such as it is the case in The Sims.

Diversification and Institutionalization

What a person can do with already produced signs and the sign system as a whole is another question. Obviously, the first thing we “do” with them is to enjoy them, derrive fun and many other emotions from them. But there are also many ways to make such signs and sign systems subject to meta-languages and meta-games: re-producing them as part of fan art, commenting on them on blogs, making mods of them, or articulating them under a bigger frame of rules.

It is possible that a whole sign system may become subject to additional rules or conventions, which will often lead to a certain way of institutionalization of said sign system. Typically, a sports federation would frame an already existing game with additional rules, which may be motivated by concerns that weren’t essentially part of the sign system that has been framed, but will nevertheless have an impact on play styles. It is therefore important to consider games as ever-evolving sign systems that can diverge into many directions, depending on how they are framed by additional rules motivated by commerce, politics, culture etc. For example, certain types of institutionalization will introduce ways to articulate different agents into the greater frame, such as the ways in which fandom is being regulated and fan’s consumption of the sign system as spectacle is bound to certain rules. Supplying fans with seasonal tickets for a football team can be given as an example here.

But institutionalization also works on the level of player access to sign systems and gameplay. Business models for example, are a framework that specify the ways in which a person can access and maintain the state of “player”.

A Few Remarks

Learning to play a game can be likened to learn a language. A player gradually grasps the relationship between the substance and form of expression and content planes, and gains skills in both articulation (“writing”) and interpretation (“reading”).

Star Wars: Galactic Battlegrounds can be seen as a game that basically kept the form of content (the value system) of Age of Empires as it is, changing the expression plane almost entirely, and overwriting the array of semes of AoE (the substance of content) with its own array of semes (the existents and events of the Star Wars universe).

Game design documents could well be analysed based on the expression and content planes that they are a sketch of. In other words, they are also sketches of a sign system, a language.

We often witness non-productive instances of articulation in languages. This can be the case in Rock-Paper-Scissors, too. If both players pose the same gesture (lets say |rock| versus |rock|), no meaning, that is, “meaninglessness” is produced due to the value system underlying the semes, which denies a comparison between the same semes, and hence, doesn’t allow for meaning to emerge.

Transmediality in games becomes possible because of the flexibility of a sign system in employing varying materials as the substance of its expression plane. Indeed, one version of Rock-Paper-Scissors could be based on players hands as the substance to deliver its recongized forms, whereas another version would use cards or sounds as its substance.

While rules may specify aspects of a game in great detail, they may leave many things to be infered by the player. This means that players may fill in the gaps in regard to things which aren’t mentioned (bound to rules), just as readers would fill in such gaps according to reader-response theory. This is not simply the application of common-sense, but utilizing common-sense elements in accord to the ludic meaning system. Just because it is not mentioned that we cannot use knives to overcome the opposition, doesn’t mean we would use knives in a match of soccer.

Notes

[1] The tie between signifier and signified is essentially an arbitrary and socially motivated association, and not one of direct reference, equivalence or one of an inseparable nature. The latter view is known as the “referential” model of language. Referential models tend to confuse the signifier for the sign, and thereby assign substance to form, as if the signifier (say, a word) is the meaning itself. This renders the fact of an association between a signifier and a signified invisible, and causes people to think of language as an “immediate” experience, or as a “mirror” of reality that reflects it “as it is”.

However, the relation between a word (a signifier) and what it signifies (the mental image that it “calls”) is arbitrary: |Table|, |Tisch|, and |Masa| are all signifiers from different cultures that “call” into mind a mental image of the same object (that of a “table”). This clearly demonstrates the arbitrariness and conventional nature of the words, since there seems to be no reason other than social convention in regard to why one culture uses the signifier |Table| while another one uses the signifier |Tisch|. Hence, the signifier |Table| is only a form, not the sign/meaning itself. To linguistics, therefore, the signifier |Table| itself means nothing, but the sign ‘Table’, as the connotative sum of the signifier |Table| and the signified “table” (the mental image of a table), does.

The association of a signifer and a signified in order to produce signs/meaning is why in contrast to referential models, structuralist approaches claim that language is not merely a reflection of a natural world order, but that it plays a central role in the representation of contingency as a construct that can be experienced as “reality”. Language doesn’t reflect an already naturally categorized world, it categorizes the world and thereby constructs an experience of the “natural”. Dealing with the value system rather than words themselves is one of the goals of discourse analyses, and in particular, that of deconstruction, because it’s there where the “world”, or the logic underlying our interpretation of it, is constructed.

All this is just another way to say that language is an area of political struggle, for words are not simply an immediate expression of the world’s natural order, but that every word (every signifier) is subject to a constant cultural battle over which signifieds to associate it with. One example can be given from Ludology: The words |semiotics| and |narrative| have been associated with signifieds such as “insufficiency”, “dissonance”, “ornament”, “wrapper” etc over the past 15 years and have over time become positioned inferior to |interactivity| and |gameplay|, which are associated with values such as “essence”, “core” etc. Often such struggle over meaning may resort to referential uses of language like saying “a game is a game is a game”. The donimating type of associations between signifiers and signifieds are by no means harmless: A designer at Bioware has been subject to verbal aggression when she expressed that she would like to see games in which gameplay can be skipped, something which for the “core” gamers “violated” what they perceived as the “nature” of games.

Interaction and Time

[This article has also been published on gamasutra.]

Discourse Time and Objective Time

Player input is not part of a game’s ongoing discourse until it has been processed and given back through audio-visual or tactile output. In that sense, a player is bound to the narration of his own actions by the game discourse. Only when his decisions are given back to him  can he articulate these, and make sense of them in the context of the rest of the game universe that is being narrated to him.

Based on this understanding of the relation between the player’s activity and the game’s discourse, we may distinguish between two concurrent time frames during gameplay:

1) On one hand, we have the discourse time of the game, that is, the pace and order in which the game narrates the events that take place in the game world.

2) On the other hand, we have the player, living in the objective time of the real world, whose input, unless processed and given back, is not part of this ongoing game discourse. 

These two time frames are mediated through the video game medium: Input and output devices, “throughput”, memory, processors etc.

Putting Time Frames Against Each Other

In terms of perception, interpretation and reaction to screen events, players are subject to the pace of the discourse.

Games with increasing speed in the narration of events are an example in which designers put the discourse time of the game against the objective time of players. At some point the player will simply not be able to cope anymore with the speed in which the discourse narrates the events, and the player will have difficulties to respond to the narrated feedback of the choices he makes in his own objective time: the blocks in Tetris, or the chain of balls in Zuma will move to quick to cope with.

Such games are designed so that they arrive at a point at which the illusion of real-time starts to get distorted due to the impossibility of synchronization between the actions of the player in objective time, and the representation of events in discourse time. But the distortion will set in in small portions, hence being unnoticed for large parts of the game, and reach its peak just short before defeat, which, due to the immersion that has been achieved already, will feel like a climax to the action, and not the extreme incompatibility between the two time frames, that it actually has become.

Sometimes Time is On Our Side

While in a lot of arcade games designers prefer to put time frames againts each other, in many other games, designers allow the players to adjust the discourse time to their pace in objective time.

In games like The Sims, Railroad Tycoon or Sim City, we may slow down the pace of the discourse to a level that we feel is convenient to carry out our actions without feeling we miss out something from the ongoing discourse, or we are even allowed to bring all events, except our own actions as players, to a halt. In such a state of halt, the discourse would only narrate the player’s actions, and the rest of the game world would stand still.

On the other hand, the same type of games allow us to increase the pace of the discourse in order to quickly go through sequences of the game that we believe do not require any of our modifications: When all family members are put to bed, The Sims goes over to high speed mode, until one of the familiy members wakes up, or until we feel the need to modify something. In Railroad Tycoon it happens that during a recession there is nothing else to do than to wait until the economy gets back on track, so we just can “skip” this part in high speed, until we have enough funds to start carrying out operations again.

The Virtues of Delay

Whereas many games stick with a “real-time” representation of player input, we observe that even games that care to maintain this illusion, make effective use of delay. For example in The Sims, the orders we give to in-game characters under our control, are not carried out immediately, and it is often the case that due to orders given in quick succession, we will create a pipeline of these. In other words, my order to prepare breakfast  may started to be carried out by the in-game character minutes after I’ve given it, or I may cancel it before it is being carried out. Combined with the pace of the ongoing discourse about the actual events that take place in the household, this strategy of delay, creates grounds for interesting gameplay, forcing the player to constantly review previous decisions in the light of the actual situation. The game, again, puts effectively against each other the events that have been carried out in two different time frames.

Another very successful example, with much more implications in regard to the relation between the players objective time, discourse time and delay is Braid. Just like in The Sims, previous decisions become the subject of actual gameplay, but the depth of the re-writing of previous decisions growing immensely.

Conclusion: Beyond the Interaction Paradigm

The dominant convention in the game industry is to use the medium’s capacities in order to create a number of illusions: The illusion of immediacy, the illusion of agency, the illusion of real-time gameplay, the illusion of interaction. In other words: most games will be designed in a way that fosters the feeling that our actions as players happen here and now, that our use of controllers and interfaces in objective time feels identical to their audio-visual and tactile representations in the ongoing game discourse, that as players we forget that mediation takes places, and that we can immerse ourselves into the game with the help of all these.

In our current understanding about games, the word interaction stands for this type of experience.

However, I tried to show that there is more to it than just here and now. Thinking beyond the interaction paradigm that earns us game developers our daily bread may allow us to discover more about the possibilities of the video game medium.

Top 5 Member blogs @ Gamasutra

I’m happy that my latest blog on Gamasutra made into the top 5 list of standout member blogs of the week!

I wrote about Games That Can’t Be Won, and you can check out the other standout blogs of the week here.

Narrative Design and Audio-Visual Style in Games

These are the slides of a lecture on narrative design and audio-visual style that I presented last year in December at the METU Informatics Institute Game Aestethics course. I hope you enjoy it!

Game Narrativity and Interaction

[This article was first published on gamasutra.]

Introduction

This article presents a number of maps that help to frame the relationship between game narrativity and interaction. They define narrative layers and their elements in a game and tell us to what degree they are open to player manipulation.

You may use these maps to detect
 
a) the “open” and “closed” parts of your game (which of the narrative elements in your game are open to manipulation and which ones are not);

b) the focus of gameplay in your game (which narrative elements is the player asked to deal with most);

c.) which narrative layers and elements you may invest more into in order to increase the joy derived from your game.

Let us have a closer look at the maps now.

I. Main Map: Narrative Layers

Classical works on narratology distinguish between four interdependent layers that form a narrative.

The first two of these layers, Events and Actants, provide the content of the narrative.

The other two layers, Narration and Narrative Situation are regarded as the form of the narrative.

Shown as a map, it looks like this:

N/A

 
II. Detailed Maps: Narrative Form 

Narrative Form contains the first two narrative layers: Narrative Situation and Narration.

Narrative Situation

Narrative Situation addresses the broadest parameters. We usually deal with aspects of the narrative situation when we specify overall gameplay options or when we decide on a certain game mode: For example preferring a Capture the Flag mode over a Deathmatch mode are manipulations in regard to narrative situation. They are about specifying the broadest lignes of the narrative content and its narration.

Before we start to play a session of Rise of Nations or Civilization, we are allowed to specify the settings in the game (whether we’d like to play on a single continent or over a number of islands, which time frame we prefer etc), the number of existents (how many rival nations there will be, what the population limits will be etc) and their aspects (whether special abilities are allowed or not etc), the winning conditions (under what condition the conflict will be regarded as solved) etc. Moreover we can specify sight & sound settings and  change key assignments. All these operations are carried out on the narrative situation layer. Our choices will have an impact on narration and the content that will be narrated.

The bubble diagram below gives an overview of the layers of narrative form and their elements:

N/A


Narration

Narration, on the other hand is the audio-visual and tactile presentation of the fictional game universe, its existents (including the player’s presence), and the events that take place in it (including the narration of and “feedback” on the player’s actions and decisions).  

While interaction is an distinguishing aspect of video games and often leaves the impression that there is no narration since things are done by the player in real-time, players must still be informed about their own actions and the happenings in the game world. This is an elemental part of video game narration.

Video games make use of a huge arsenal of audio-visual and tactile narration style and techniques. Some of them are based on the style and techniques of previous media, whereas others may be regarded as being unique to video games. 

A game that allows us to modify the ways in which sight, sound and other types of information are presented to us can be regarded as being open to manipulation on the narration layer. Examples would be Need for Speed, in which we can switch between first-person and third-person perspective, or Counter-Strike, in which we can modify the angle of our perspective by moving the cursor. 

Statements

Statements are medium-dependent, that is, every medium has its unique possibilities and limitations in expression.

Narration consists of Statements which may be mediated and unmediated, that is, they may be expressed through the voice of a narrator or not. In many games we will find a mix of unmediated and mediated statements.

In cases in which the statements are mediated, they are voiced by or hint at a narrator whose speech addresses a narrattee.

Mediated and Unmediated Narration

A lot of games seem to be narrated eclusively through unmediated statements, that is, they make minimal or no use of a narrator. Instead of a verbal account of things, they go for a “simulation of events”. Indeed, most “formal” games (like for example Tetris) are of this kind.

The text adventure genre on the other hand, can be given as an example in which mediated narration is preferred. In such games, the presence of a narrator and a narrattee is often very easy to detect. The narrator would simply say things like “You [the narrattee] stand in front of a closed door.”

But we should always be prepared to find a mix of mediated and unmediated statements: In a game that went with unmediated statements for a long time, you suddenly may come accross a mediated statement (and vice versa). In Railroad Tycoon 2 for example, for most parts of the game, narration is based on the simulation of events and supported with frequent appearances of newspaper headlines etc. It is difficult to trace down a narrator in all this. Thus, the ongoing narration may be regarded as to be built around unmediated statements. Yet at certain moments, a narrator pops up and gives us an account of things: “The city council of Pittsburg offers you 200 K if you connect to their city before 1872. Do you accept?”; or “Good job. You’ve connected to Chicago. You can still earn a bronze if you connect to Denver before 1879” etc. Also the intro scene of a new game level welcomes us with the speech of another narrator; an old Southerner with a torn voice. He comments about our good work in the past level, and warns us about the challenge we will face in the next one.

Pro Evolution Soccer or FIFA, in which players can turn on or off in-game commentary, the choice is in regard to whether we want a match to be accompanied by a narrator, that is, the presense of flow of mediated statements.

Narrators and Narratees

Football Manager
is a good example of how complex it can become in regard to the presence of narrators. This game uses mediated statements throughout the communication that takes place on the message boards. Here, one of the narrators uses statements like “Your assistant informs you that…” whereas our assistant might have spoken directly to us. Another narrator uses the register of sports news portals. Yet another narrator is the match commentator that accompanies with his comments the simulated events that take place during a match.   

Games that allow us to turn off tutorials are often examples for at least one narrator in the story being eliminated. 

We must distinguish between pure narrators and character-narrators. Some narrators may not be involved into the events that take place, but they may be still all-knowing about them, while others may be actively involved into the ongoing events, but yet possess limited information on the situation. An example for the latter would be Max Payne, in which the protagonist is the narrator of the mystery that he tries to solve.

When a narrator is involved, his statements are always directed towards a narrattee, that is, someone whom the speech is addressing. The narrattee isn’t necessarily the player, but may be some other character in the game world. 

The Sims is an interesting example because it allows players to narrate the lives of their characters by adding character descriptions. That means that the player may add a narrator to the story. This narrator (which could be simply the player’s self) may pretend to know the truth about the otherwise only simulated characters and events, and once this narrator gives a recount, she has already addressed a narratee (which might simply be again the player’s self). 

While modern games give us a lot of freedom in modifying the parameters of narration, we must remember that just two decades ago this was rather rare.  For example the majority of classic arcade games do not allow for manipulations on the narration layer: In games such as Centipede, Pac-Man or Space Invaders the audio-visual style and the aestetic parameters of the statements that are produced through them are not manipulable. We have no option in regard to change whether the statements are mediated or unmediated. We have no chance to manipulate the number of narrators and the style and content of their recount, nor can we modify anything on the narrattee’s side.

The bubble diagram below shows us what questions may be asked in order to tell the openness of manipulation of narrative form.

N/A

III. Narrative Content: Actants

Actants are the characters and the settings that play a central role in the maintenance of the plot. Actants possess a degree of functionality in making the plot progress. Central to this are the actions they carry out, and how central these actions are in deciding the outcome of the narrated sequence of events. 

Degree of Importance to the Plot

A good example for the degree of importance to the plot is how we perceive our encounter with ordinary enemies and with bosses. Ordinary enemies are often just some “loot bags” to us, we often use words like “cannon fodder” when we speak of them and by doing so, we address their degree of importance to the plot. On the other hand,  the encounter with a boss is perceived as a milestone in the progress that we make. We consider this as a core event which also reveals that the boss type of actants have a degree of importance to the plot. Oviously, the actant with the highest degree of importance to the plot is our own actant.

The same counts for settings. In Rise of Nations for example, some environmental elements and existents may have no big importance to the plot, whereas resource areas are central to it.

Aspect

Aspect is in regard to the identity and qualities of the actants, such as their moods and traits. These may be detailed or shallow, depending on the game we play. Also the type of narration sets a limit on how much we know about the aspect of actants. For example in large parts of Duke Nukem we have not much knowledge about the physical appearance of Duke due to the first-person perspective that the narration is visually build around (until we come accross a mirror!).

Sometimes the aspect of actants may only become only visible through their actions. For example in Tetris, the player is an actant that becomes visible through his decisions in regard to how he moves and places blocks; whereas the “antagonist” becomes visible through his choices in regard to which block to send next. “It” challenges us by sending the blocks faster and faster, often denying us the block that we are in need of to clean the level. All these reveal aspect to a certain degree.

On the other hand, a lot of games provide us with a lot of information about the aspects of actants. In Football Manager for example, you may find information about the identities of managers and footballers, and the statements they release in the press tell us about their moods and traits. Some will always apologize when you fine them for unproper conduct whereas others will be very rejective and immediately demand to be put on the transfer list. Actually, the FM Editor provides a database that allows us to overwrite the aspect side of actants: You may change a soccer players identity, his mood (level of aggression), and his traits (likes to the chip the ball, or likes to play it around the keeper etc.) and all these will have an impact on how events will play out.

N/A

In terms of openness of the actants layer, The Sims is a remarkable example: The detailed character creation process in the game is first and foremost about creating characters that are important to the plot. But then, it is also about defining the details of their aspect. We have great freedom in shaping the identity and qualities of characters and settings. And we can simply put an end to their importance to the plot by removing families and homes from the neighborhood. Besides, the many expansion packs of the Sims series add a great number of settings and characters, and new options to alter their aspects. 

A lot of social games (Farmville, Cityville etc) are games with focus on the manipulation of settings. However, we may consider the farms (normally just a setting) as “characters”, because they become so central to the plot. On the other hand, the connectivity of these games due to the broader network that they are part of, ensures that players come across a great number of other players. Depending on how relations develop, some of the players we meet may turn into characters (because they become central to the plot, like a “worker” that agrees to work on your farm), and some remain just as “setting” (a bunch of avatars in the bar that say stupid things anyway).

We must add however, that a lot of games do not allow for change on the actants layer. In most games we use a given character and struggle against a given set of characters. We are often not allowed to change the aspect of these characters. More than that, in such games, the characters degree of importance to the plot is also often given. This is the case in most of the games in early video game history. For example in Donkey Kong, the array of actants, their degree of importance to the plot, and their aspects are fixed.

The questions we may ask about our game in regard to its openness to manipulation of the actants layer are shown in the bubble diagram below: 

N/A


IV. Narrative Content: Events

Events typically include the actions and happenings in the story. Whether an event is an action or a happening depends on agency, that is, the answer who carries out the action, and who is subject to it.

On the other hand, necessity marks how central an event is to the plot. An event that can be skipped or ommitted without making the narrative as whole collapse, is a sattelite, whereas an event that opens a gap in the logic and chronology when removed from the narrative, is a kernel.

Agency

Actions are the things done by actants. Happenings on the other hand are those things that happen to characters. The difference lies in agency. It must be understood that often one actants’s action equals to a happening for the actant subject to that action. For example in a multiplayer FPS, an action is when I shoot at someone, whereas for the player that is my target, the same thing is a happening.

In most single player games, players deal with the actions part, whereas the AI cares for the happenings. In Sim City for example, the actions are centered around the players decisions in regard to how to build the city, whereas the majority of other events comes in form of happenings: Traffic, smog, riots, disasters, crime rates, advisors speaking to us etc.

N/A

Necessity
 
Events are categorized based on the level of necessity they possess. Some events are core to the story, that is, if we’d leave them out, the experience feels incomplete, as if it were truncated. The narrative simply wouldn’t be able to maintain itself without that event. Certain quests or cutscenes are kernels, because when we skip them, we may not be able to obtain a key that is necessary to unlock another level, or we may miss an important information that may be later on necessary to solve the conflict. 

Sattelites, on the other hand, are complementary, but not necessary. For example we can skip quite a lot of quests or cutscenes because the events they contain are sattelites, not kernels. The narrative can live without them.  We may ask the following questions in regard to the events layer:

N/A

Historically, video games have been always open to manipulation on the events layer, even though to a limited degree: The archetype of video game interactivity has been the manipulation of actions. The player was given a minimum of agency by being allowed to decide on the actions carried out by an actant.

However, modern video games allow players for more than just carrying out actions. They may allow players to decide whether they want to be put through satellites (skipping cutscenes; choosing a save point to start from, being allowed to skip entire levels); they may allow to alter the freqrency of certain happenings (setting the traffic level in Need for Speed, or setting the frequency of train robberies in Railroad Tycoon 2), and they may allow to put constraints on certain actions (disabling or enabling special moves) etc.

V. Interaction Chart

The chart below present a summary of the questions we might ask about the narrative layers in a game and their degree of openness to interaction.

The questions are general, but they might be refined depending on the particular game that is being analysed.

N/A

As a designer, you may use this chart to ask testers and gameplayers about the experience they have with the game. The feedback these players provide to you may help you to assess the narrative aspects that they are happy or unhappy with in your game.

You may also use it in design meetings with other designers to discuss certain aspects of the game that you are developing together.

As game researchers you simply may use this chart to see how certain games deal with their narrative layers. You may also apply it in historical studies, trying to follow the evolution of certain platforms or genres in regard to their openness to manipulation of particular narrative layers and their elements.

Note:

The model of narrative layers presented in this article is based on the works of Seymour Chatman and Roland Barthes.

Anempathetic Game Design

Using games to raise critical awareness is an issue that finds more and more interest among scholars and game designers. Using games in a radically different way was an idea that has been also discussed during the ludology-narratology debate, in particular by Gonzalo Frasca.  Frasca would later develop a group of games that felt quite different from “conventional” games, among them September 12.  Ian Bogost was another designer and scholar going into this direction, and together with Frasca they were influential in creating Newsgaming as a form of critical play. Ever since we have seen games that have been radically different in their content and form when compared to mainstream games. And finally, in 2009, Mary Flanagan devoted a whole book on the matter, outlining the various perspectives that are out there and pointing at the possibilities in using games to raise critical social awareness.

In this article I want to explore whether we can borrow a term from audio-visual analyses in order to enrich the conceptual basis of critical play and radical game design. The term that I want to have a look at is anempathy.

Empathetic and Anempathetic

In his book Audio-Vision: Sound on Screen, french film critic and composer Michel Chion (1994) makes a distinction between two types of musical score in the cinema: Empathetic and anempathetic.

Empathetic Music

In Chion’s words, “music can directly express its participation in the feeling of the scene, by taking on the scene’s rhythm, tone, and phrasing; obviously such music participates in cultural codes for things like sadness, happiness, and movement. In this case we can speak of empathetic music, from the word empathy, the ability to feel the feelings of others.” (1994: 8) This is music “whose mode matches with the mood or rhythm of the action onscreen.” (1994: 222)

 
Anempathetic Music

On the other hand“, continues Chion, “music can also exhibit conspicous indifference to the situation, by progressing in a steady, undaunted, and ineluctable manner: the scene takes place against this very backdrop of ‘indifference’.” However, “this juxtaposition of scene with indifferent music has the effect of not freezing emotion but rather of intensifying it, by inscribing it on a cosmic background. I call this second kind of music anempathetic (with the privative a-). […] the frivolity and naivete reinforce the individual emotion of the character and the spectator, even as [this] music pretends not to notice it.” (1994: 8) This is sound “that seems to exhibit conspicous indifference to what is going on in the plot, creating a strong sense of the tragic.” (1994: 221)

The question I ask at this point is whether we cannot speak of rule design that functions in this way, i.e is it possible to speak of empathetic or anempathetic game rules and systems?

Empathetic and Anempathetic Game Design

I  call empathetic those games whose rule designs and systems are geared towards participation into established cultural codes rather than challenging these codes in a radical way. These are designs that frame their subject within “common-sense” categories. Such designs would be often culturally one-sided (or even biased), appealing to the sentiment of ‘mainstream’ lifestyles in their way of abstracting and representing the simulations of their worlds. And in quite some cases they would be built around concepts and values borrowed from reactionary rhethoric.

On the other hand, in anempathetic games we’d see the intention of breaking interpretative practices based on common-sense by using figurative devices in the form of game mechanics that exhibit a conspicous indifference against the ongoing plot, thereby not freezing our emotions, but rather intensifying them, which creates a strong sense of the tragic. We are forced into a different perspective which creates a rupture in common-sense. The position we took as a subject and felt ‘natural’ to us before we started to play the games, is suddenly exposed to ourselves: our initial inner stance feels awkward now, its ‘normality’ can no longer be maintained. We are forced to think differently, and if we should give up, we are faced with our own insincerity: we don’t want to think outside of the box. These are games that remind one of her conscience.

Conclusion

Command & Conquer: Generals – Iraq is empathetic; September 12 is anemphatetic.