Augmentology"...is a concise manual of reality for our digital age."

Mark Hancock,_Augmentology: Interfaccia Tra Due Mondi_

[Sponsored by The Ars Virtua Foundation/CADRE Laboratory for New Media]

This TED talk by Peter Molyneux:

…demos Milo, a hotly anticipated video game for Microsoft’s Kinect controller. Perceptive and impressionable like a real 11-year-old, the virtual boy watches, listens and learns — recognizing and responding to you.

The demonstration begins with an explanation of how Milo is constructed. A combination of the following three elements allow Milo to exist:

  1. A Kinect Camera
  2. Artificial Intelligence developed by Microsoft
  3. Emotional Artificial Intelligence built by Lionhead Studios.

Milo moves through a synthetic environment predicated on User-directed biofeedback/body gestures: no mechanical controllers are necessary. Unfortunately, Milo’s introductory learning curve [which is integral to the "game" leveling system] involves inherent gender bias: if you’re a girl, your initial game variable is a Butterfly whereas if your a boy, you’ll be presented with a Snail.

The demonstration goes on to illustrate how Milo’s face is comprehensively AI driven. His facial movements include blush response, nostril “flare” size [indicating stress], “body matching” [causing neuro-linguistically driven facial alterations] and responses to verbal cues. Peter then describes how Milo’s personality development is predicated on a Cause-and-Effect dynamic. This causality is showcased via 3 examples:

  1. The User can choose to direct Milo to squash a snail: if the User does it will effect “…how Milo develops”.  The specifics of the verbal stimulus employed including how the User vocalises [specific phrases and intonations] all contribute to a database that informs and effects future interactions.
  2. The User teaches Milo to skim stones over the surface of a river [skewed gender stereotyping is again evident here].
  3. The User choosing to clean Milo’s room: Milo’s recognition of the User’s beneficial intervention and verbal engagement promotes sustained developmental interaction based on [what Peter terms] “deep psychology”.

This “deep psychology” [or what is described in synthaptic terms as "augmentology"] encourages a User’s empathy loadings. This in turn allows such games to shift towards complex experientially-defined engagement. These games surpass the hollow reinforcement of contemporary Social Games such as Farmville: instead, the User “levels up” by knitting fictionalised engagement with personality/identity construction and personalised growth variables. The element of cloud-directed learning [coaching synthetic humans whose social and chronological development depends on "crowdsourced" input] creates enormous opportunities for instruction and feedback via these types of  “Reality Gaming” systems [highlighted here by Seth Priebatsch]:

The User named “showmeurcock” from Kentucky does not respond to the picture I send of the space heater next to my desk. The User named “whispers” from California rates a picture of my feet three out of five stars. The User “guest43723” from Germany sends a picture of a jar full of coins. I reply with a smiling emoticon and receive “Uu” in response.

A World of Photo is a geosocial multiplayer game during which:

…you ‘spin’ your phone, like spin the bottle, to select some random user, and then they take a picture and send it back to you. Once they do that, they can ‘spin’ and get a picture sent to them”.

Users involved in A World of Photo [AWOP] are a tightknit community where users’ attentions dilate and episodically contract along with fluctuations from their Android devices.

Although this background-running application rarely seems to have more than 100 users currently active, the game prods you towards constant connections with other users as it yanks you into a space of outright voyeurism.

This voyeuristic space is laid out on the screen through a map of the “thread” of users with whom you have connected. Few conversations carry past two messages: those that do weave scintillating life-tapestries glimpsed through a typically external visual representation. Few users send portrait pictures of themselves and instead expose their recipients to their contextual environment: the opposite seat of a subway car, a DJ Hero controller, someone drinking a beer and sitting on the floor in front of a TV. One photo displays imperceptible imagery on a television screen located in a dark room. Like the lives behind these cell phone cameras, this indistinguishable/unfocused image seems tantalizingly real, yet is ultimately unknowable. The game provides no discernible contact data or history. It is, however, possible to save the photos you receive. Once a user decides to stop replying to a textual message thread, that thread is over. It is conceivable (but unlikely) that two people would connect more deeply than the AWOP program intends without the compulsive motivation of biological and/or sexual gratification.

The possibilities of AWOP are subtly revealing in terms of a user’s constant awareness/presence. The game weaves randomized tangents from a global user-base. Like much collaborative software, AWOP emphasizes continuous threaded networks rather than merely linking individual lines of communication. Menus allow access to various statistics, including a user’s uploaded photo total, a user’s current image record and user rating system. Ratings are instrumental to the game element: User “Rob Zombie” rates my “Pretending to rock out” picture 5 stars and in return comments “Yeah!”. This comment prompts me to find objects in my surroundings that will rate highly based on user names. User “americansoldier” rates a similar picture of myself 2 out of 5 stars: as this is considered negative feedback, I am forced to lose a turn – to receive a photo from another user – and must fulfill a request positively in order to be rewarded. Thus, the game turns everyday life into an evaluation of personal experience that borders on the perverse. This may explain the missing ‘save’ functionality for a user’s sent photos.


Through peeking into the lives of others via AWOP, a user is left with reminders of spaces that exist outside the range of their mobile phone. These spaces overlay the objects that exist in the user’s “real life”/geophysicality, contrasting and contracting with[in] the corresponding layers constructed by AWOP’s present and potential social contacts. This augmentation does come at a price: such evaluation patterns (by the self and others) are, on some level, internalized. This internalization may contribute to a constricted reality sense that projects overarching importance to immediate (“real”) stimuli over the awareness of other possible environments. The gap between the two is likely where the user resides, conscious of their perceived and reinforced shortcomings. AWOP’s strongest hook is in harnessing the user’s desire to socially (and successfully) produce items for the community. Community approval becomes currency. This currency production produces struggles between internal and external systems of representation which are hashed out in lines of resolution via a personal digital assistant. If substantial narrative does not emerge, like music, “from the dimensions of ambient night” [Harry Partch, 1949, Genesis of a Music] then AWOP certainly allows the user access to its root: the personal, the spatial, and the physical.

<continued>

e) Information Deformation: akin to process centering, this Social Tesseraction involves a shift in the very definition of information:

Information may be defined as the characteristics of the output of a process, these being informative about the process and the input. This discipline independent definition may be applied to all domains, from physics to epistemology.”

These deformed systems of data are constantly in flux and available for perpetual revision. Examples include:

Users are able to simultaneously modify, update and adapt their input in real time:

This type of liminal practice results in a deformation of current information architectures. Although traditional information construction may be flexible over time, it still demands unitary data snapshots for knowledge formation. Deforming such data in real time acts to fundamentally alter meaning production. Socially structured input is the keystone of such a dynamic, perpetually fluctuating system:

Here the notion of Social Froth takes on a new level of importance: information becomes a constantly shifting construct with variable endpoints.  Rewiring information in such a way radically changes its cohesive nature. This in turn effects:

In this deformation system, facts can be reality-edited* in real time:

Information becomes pliable in ways that challenge the perceived authority of institutions. The concept of narrative deforms as:

Narrative progression repositions the representational towards the freeform [think: paidia as opposed to ludic]. An instance of this information deformation in action is troll play [or uncontrolled play]. A social example of troll play is found in the wiki Encyclopedia Dramatica which:

satirizes both encyclopedic topics and current events, especially those related to or relevant to internet culture. The wiki has been the subject of media attention given its focus on trolling and use of shock value, as well as its criticism of other Internet communities. It is also associated with the Internet subculture Anonymous.”

_Encyclopedia Dramatica_ – and the affiliated imageboard/meme propagation site 4chan - showcase the challenge faced by narrative frameworks.  Platforms like _Encyclopedia Dramatica_ encourage troll-based comedic intent. Users remix absurd, and sometimes taboo, content. In particular, invasion boards like _4chan_ utilize shock networking*: where social content attempts to subvert social codas through deliberate agitation. In comparison with established narrative conventions, platforms like _Encyclopedia Dramatica_ offer an experimental system which bypasses strict censorship and ethical constraints. These platforms cater for unfiltered interactions that operate via immediacy-of-response. They are highly idiosyncratic in execution and linguistic formation: censorship and moderation may be limited or non-existent. The output is propagative, with contributors encouraged to riff and rip-off, replace, and even delete content. Narrative is deformed beyond a sequential structure whereby the climax or pay-off event becomes the spectacle:

An example of such modification is Copypasta, which consists of repeatedly copying and pasting blocks of text designed to evoke a heightened emotional response:

A time-tested classic. This ending usually comes into play at the climax of a very troubling or exciting situation. Rather than resolve the story, one of the characters will abruptly say something to the effect of “I had Reese’s for breakfast.” At this point, the other character will completely forget about his/her worries and jump into the corresponding commercial dialogue, enamored by the peanut butter and chocolaty goodness that is Reese’s Puffs cereal. “It’s Reese’s… for breakfast!”"

_Copypasta_ derails notions of story or plot progression, resolution or dénouement. It embodies context-counteraction* and meme perpetuation. Dramatic intent shifts to reiterative moments containing affectivity spiking which ignores the rigors of institutionalized framing [think: morality, hierarchy or ownership]:

.

f) Attribution Modding involves an extension of Stewart Brand’s iconic phrase “Information wants to be free” to “Identity wants to be freeform”. This category describes users focused on mobilization rather than individual recognition.  The group Anonymous* projects attribution modding via collusive identity constructions. The collective’s title is based on the method _4chan_ uses to brand all contributors “Anonymous” by default:

As making a post without filling in the “Name” field causes posts to be attributed to “Anonymous”, general understanding on 4chan holds that Anonymous is not a single person but a collective (hive) of users.”

Anonymous is a social-tesseractivist group who perform raid actions [think: the immediate action to halt the abuse of Dusty The Cat and Project Chanology's DDoS attacks]:

The collective broadcasts non-attribution ideologies where members are viewed as units of a social mechanism with a deemphasis on individual identification. Attribution modding illustrates the rise of collective identity cognizance and the accompanying shift from expert-centric disciplines.

.

g) Decline of Silo Ghettos: as information deformation impacts knowledge formation, there’s an increasing need to provide social tesseractors with comprehensive dimensional engagement. This type of borderless interaction deforms monostreams into cross-channelled productions. Social tesseracts assist in addressing the somewhat restrictive walled garden approach to software and platform production [think: the frustration levels encountered whilst experiencing the locked door syndrome].

Google Wave is one system that removes such constraints and allows users to input directly into previously distinct arenas. Other instances of interoperable systems that require the reorientation of Information Silos:

  • augmented applications that encourage a pairing of geolocative and geophysical needs:

Information Deformation, Attribution Modding, and the Decline of Silo Ghettos are paradigm-shifting markers that highlight socially directed trends. One significant user-centric challenge involves ensuring a smooth migration into a Social Tesseracted future. Such transitions should lessen future shock and encourage a type of overlaid meta-comprehension which promotes the seamless recognition of synthetic conditions.

——

* Shock networking, reality editing, context-counteraction and Anonymous will be discussed in upcoming augmentology entries.

In _Social Tesseracting_: Part 1, we learnt that:

1. Dimensionality defines working concepts of reality.
2. Theoretically, dimensionality can also expand to define a spectrum of nascent social actions.
3. These particular social actions encompass communication trends defined by synthetic interactions.
4. Synthetic interactions create social froth that can be produced geophysically or geolocatively. Both connection types depend on relevant electronic gesturing:

5. This mix of synthetic interactions and electronic gesturing provokes a descriptive framework of this aggregated sodality. This framework is termed Social Tesseracting.
6. In order to adequately formulate Social Tesseracting, contemporary theorists need to extend “valid” reality definitions based currently on the endpoint of the geophysical.

In assessing the growing ethological importance of Social Tesseracting, the following markers demand examination:

a) Social White-Space: Just as with the convention of white space in graphic design, social tesseracts manifest in habituated actions performed routinely over a substantiated period [think: responding to smartphone emails during geophysical-based discourse]:

Social white space exists in synthetically mediated consciousness via overlaying reality clusters. These clusters may exist outside of the geoloaded end of the Reality-Virtuality Continuum [ie the locatable "real person"]. Conjunctive or intermediary areas of connectivity mediate this “primary” reality state [think: Information Shadowing, the Network Effect and Warnock's Dilemma]. Social white-space is currently effecting educative goals and is altering engagement within the workplace.

b)  Immediation: the instantaneous modification of remote events via the removal of geo-specific time lag. Immediation highlights the impact potential of synthetic connectors. Examples of Immediation in action:

c) Regenerative Comprehension: indicated by rapid shifts in the nature of content creation and absorption. A primary example is Twitter’s chronologically-reversed tweet reading order acting to modify awareness. Other examples include:

d) Process Centering: Social Tesseractions are marked by fluid, process-oriented engagement rather than rigid procedural structuring. Process centering prompts a re-evaluation of data formation and alters the entrenched importance of institutionalised categorisations. An emergent example of process centering is Google Wave. Google Wave uses an algorithmic variation of “operational transformations” [live concurrent editing] which occur through a process called transformation:

  • The server transforms the client’s request, resulting in the client manifesting the same transformed output.
  • The notion of concurrency is invariably important as it mimics geophysical conversational states.
  • Utilizing the server as a point of relay [when more than one client's output is involved] assists in providing scalability and reliability.
  • The playback feature allows the server to present the document as a stream of operations that have occurred thus far in a particular wave/state.

Transformation relies on continual modification via process centering. This accent on process acts to rewire the notion of documents as statically defined “objects” and [by proxy] any information contained within. This has enormous implications in regards to such institutionally-governed categories such as literacy, media, the professional/amateur divide, narrative, and information construction.

_Social Tesseracting_: Part 3 will expand on these indicators through examining: Information Deformation, Attribution Modding, and the Decline of Silo Ghettos.


Post Reenactment – “Don’t Kill Mr. Gandhi!”:

The 240 mile march across Second Life with my treadmill powered Gandhi ended on April 6th, 2008. At the conclusion of the walk, I had a great sense of accomplishment – feeling content and quite frankly looking forward to a much needed rest! I had completed a rather fascinating journey – both in the physical and conceptual sense. In the days that followed the end of the reenactment, this feeling of euphoria was slowly overtaken by a sense of melancholy – I found myself both saddened and conflicted for the march to be concluded.

One of the avatars who walked with me the most was named Gyovanna. She told me she was from Brazil. She had the endearing habit of referring to me as “Mr. Gandhi”. Several days after the conclusion of the walk, I received a message from Gyovanna pleading with me “Don’t Kill Mr. Gandhi!”. In the days after the reenactment, such sentiments were expressed to me by many others. I logged in once or twice as Gandhi after the march but it was just not the same. The layered factors necessary to conduct the reenactment were crucial to creating a symbiosis between the treadmill, me, and my avatar – these could not be replaced or duplicated by merely visiting SL in the traditional keyboard-based manner.

Over the course of the march I had been thinking about how best to document and, in a way, process the experience of the march into a series of artefacts and objects. As a way towards further exploring my connection with my SL avatar, MGandhi, I worked to extract the 3-D model from Second Life and process this information into several sculptural renditions – the first being an 8” 3-D print made using the rapid prototyping equipment at Eyebeam. Working with my intern, Lenny Correa, we extracted Gandhi using an open source application called OGLE developed, oddly enough, at the Eyebeam Open Labs.

17’ tall Gandhi Reproduction

As an Eyebeam resident artist, part of the award was to have an exhibition onsite at the end of the residency period. At some point during thinking about the exhibition and processing the 8” figure, I envisioned the creation of a monumental sized Gandhi figure. Most statues of Gandhi, including the one in NYC at Union Square, are roughly human scale. The thought of creating a larger than life representation of my Gandhi avatar seemed quite audacious and oddly appropriate. I settled upon using another open source application, Pepakura Designer, which is a popular papercraft program used by enthusiasts to create some rather amazing – albeit generally tabletop scale – reproductions of everything from anime figures to airplanes. I adapted this program to create a 17’ tall reproduction of my Gandhi avatar out of cardboard and hot glue. The entire process has been described and documented, in detail, on instructables.com – my first instructable which I see as an unorthodox but highly effective way to further disseminate the project. The resulting 17’ figure is a monumental physical representation of Gandi created from very simple materials. The figure was made to be the same height as Michelangelo’s David – a fitting conceptual connection to this iconic work of art history depicting the Biblical figure of David just before slaying Goliath.

Most recently, I have completed a second 3D print of the Gandhi figure that stands 12” tall. For this piece, I have treated the figure in genuine gold leaf – creating a rather contradictory sculptural representation of my avatar as a shining, fetishized object.

3D Print of the Gandhi Figure in Gold

I’ve also created a 6000+ frame stop action video created from screenshots recorded automatically every 60 seconds documenting the entirety of the reenactment in SL. The 9 minute long piece is compiled at 12 frames per second. I showed this piece at Eyebeam in an installation featuring the original treadmill; it includes an LCD panel laid upon the treadmill showing a looping segment shot from above, and in front of, my legs walking during the reenactment. For the exhibition, the re-installation of the treadmill, the 17’ Gandhi and the 8” Gandhi were complimented by six screen shots shown as large formatted prints. Also included was a mural size print of my recreation of the famous image of Gandhi making the salt at the beach at Dandhi.

The entirety of the Gandhi work is to be featured as part of the Guangzhou Triennial 2008, entitled Farewell to Post-Colonialism in China this fall. For this exhibition, the museum will be working with me to build a second 17’ Gandhi – a way of having a local connection to the creation of the works for the show. The first Gandhi was, btw, designed to allow for disassembly into parts for shipping – he is now on his way home to me in Reno in two very large cardboard containers, big and bulky but very light ;-) .

What pleases me most about the entirety of the reenactment experience, and ongoing exploration or processing of this Mixed Reality performance, is that through this work I have, for the first time, unified what have previously have been two different threads of my artistic practice. For years, my work in physical installation, sculpture and kinetic art has been separate from my work performing in computer games and online communities. The synthesis of the real and the virtual during the reenactment, and the resulting physical explorations post-reenactment have proven to be a revelation to my creative process that will surely inform my activities for years to come.