Transmedia Narratives, simulacra, simulation, fake and design fiction

Simulacra

Organize a fake holdup. Verify that your weapons are harmless, and take the most trustworthy hostage, so that no human life will be in danger (or one lapses into the criminal.) Demand a ransom, and make it so that the operation creates as much commotion as possible — in short, remain close to the “truth,” in order to test the reaction of the apparatus to a perfect simulacrum. You won’t be able to do it: the network of artificial signs will become inextricably mixed up with real elements (a policeman really will fire on sight; a client of the bank will faint and die of a heart attack; one will actually pay you the phony ransom). [p. 20]

This quote, taken from Jean Baudrillard’s “The Precession of Simulacra”, gives a very precise idea of the role that simulation can play in human societies and perception.

If, as suggested by Baudrillard, we were to simulate as closely as possible a fake holdup, we just would not be able to do it, because the people and, in general, the “machine”, the process through which people constantly interpret the reality they have around them, would not be able to distinguish the signals of what is real and fake/simulated.

“This is how all the holdups, airplane hijackings, etc. are now in some sense simulation holdups in that they are already inscribed in the decoding and orchestration rituals of the  media, anticipated in their presentation and their possible consequences. In short, where  they function as a group of signs dedicated exclusively to their recurrence as signs, and  no longer at all to their “real” end.”

We, as human beings, interpret what we perceive to be real by gathering a series of signals, of clues, from the context which surrounds us: gestures, patterns, things we recognize as meaningful in a certain way, objects, places and the context they suggest.

We, thus, use this series of clues coming from a variety of media (vision, sound, information expressed and communicated in different ways…) to form in our mind a description of what is real and what isn’t.

In this sense, the concept of Hyperrealism can help understand even further the ways in which we can imagine to use simulation to give credibility to a certain scenario, so that it is indistinguishable from truth.

Neuromancer

Neuromancer

In William Gibson’s “Neuromancer”,  a terrorist sect called The Panther Moderns takes advantage of the fuzzy boundary between the simulacra and the real to create chaos at the Sense/Net Corporation:

Nine Moderns, scattered along two hundred miles of the Sprawl, had simultaneously dialed MAX EMERG from pay phones.Nine different police departments and public security agencies were absorbing the information that an obscure subsect of militant Christian fundamentalists had just taken credit for having introduced clinical levels of an outlawed psychoactive agent known as Blue Nine into the ventilation system of the Sense/Net Pyramid. Blue Nine had been shown to produce acute paranoia and homicidal psychosis in eight-five percent of experimental subjects.

In the narrative, the Panther Moderns combine multiple media and modalities to stimulate as many perceptive modalities as possible to make the people in the Sense/Net building believe that a fundamentalists have infected the building with a powerful hallucinogenic drug, thus causing violence and horror.

Images, sounds, physical presence and video are only some of the techniques and media they use to achieve this:

  • 9 phone emergency calls trigger an emergency response by the police forces, who actually go to the building
  • showing video footage inside Sense/Net that triggers seizures in a certain percentage of employees
  • introducing images of contamination in the CCTV circuit
  • diffusing in the sound system of the building audio of a news segment dealing with a dangerous human growth hormone

By creating panic among the Sense/Net employees, The Panther Moderns simulate the effects of introducing Blue Nine into the ventilation system to the security forces

At the same time, the presence of the security forces reaffirms the employees’ belief that there are biological agents in the ventilation system.

It only required nine phone calls and five minutes of video feed.

Again, the characters are placed in a situation in which they fail to distinguish reality from simulation: all the signals and hints from the surrounding environment suggest a version of reality which, technically, is not true, but which, in perception, is a real as they can perceive it.

Transmedia Narratives

a transmedia story represents the integration of entertainment experiences across a range of different media platforms

Henry Jenkins, 2007

and:

Transmedia storytelling represents a process where integral elements of a fiction get dispersed systematically across multiple delivery channels for the purpose of creating a unified and coordinated entertainment experience. Ideally, each medium makes its own unique contribution to the unfolding of the story.

So, for example, in The Matrix franchise, key bits of information are conveyed through three live action films, a series of animated shorts, two collections of comic book stories, and several video games. There is no one single source or ur-text where one can turn to gain all of the information needed to comprehend the Matrix universe.

In 2007 the band Nine Inch Nails orchestrated a very interesting Tansmedia Narrative: some songs from their Year Zero album were thought to have been leaked as they were found on a lost flash drive in a hotel room, while the band was on tour. On that flash drive there were songs and recorded phone calls, which later turned out to be parts of a previously made-up online experience for Nine Inch Nails fans. From the recorded phone call mp3 fans decoded a URL of a website that gave them further clues to decipher.

From Transmedia Lab:

“In 2007, the band Nine Inch Nails (NIN) created an Alternative Reality Game (ARG) for the launch of their new album “Year Zero”, thanks to the agency 42 Entertainment.

This treasure hunt took place in an alternate reality, clues were given through texts on NIN T-shirts, singles of the new album left on USB keys, everything hidden in the toilets of concert venues, on websites or through secret phone numbers. All these elements helped players move ahead in the dark story of Year Zero: a world ravaged by an infinite war and an environmental catastrophe.

The goal of the project was to immerse fans in an experienced linked to the universe of the album.

The leader of the band, Trent Reznor, qualified this experience as a “new type of entertainment”. According to him, the combined effect of entertainmentword of mouth and engagement of the audience, made this ARG the perfect tool to promote this album. For more information on their transmedia experience see the case study of the agency 42 Entertainement here.”

Characteristics of Transmedia Storytelling Elements

From what we have seen so far we can say that Transmedia storytelling involves orchestrating multiple media and communication modalities to produce a Simulacrum, a simulation of a scenario or context which is able to immerse people in an overall experience, a “world” in which the Simulacrum is real.

Hence, it is of particular interest trying to understand the ways in which we can build the various elements which make a Transmedia Narrative.

Most elements in the following sections come from various posts on Henry Jenkins’ blog.

 

Spreadability vs Drillability

Spreadable media encourages horizontal ripples, accumulating eyeballs without necessarily encouraging more long-term engagement.

Drillable media typically engage far fewer people, but occupy more of their time and energies in a vertical descent into a text’s complexities.

In designing a Transmedia Narrative we might become interested in achieving a specific balance between spreadable and drillable media, and to distribute these two characteristics accordingly.

We might, for example, use more spreadable media experiences to suggest sharing on social networks, thus widening our audiences.

On the other side, we might use more drillable media to suggest immersion in a specific part of the overall experience, creating more lasting engagement and involvement.

On top of that, we can imagine creating links (more on that later) to suggest people to go back and forth between the two modalities, to capture them through spreadable media experiences and to make them go below the surface using more drillable ones.

 

Continuity vs. Multiplicity

Some transmedia franchises foster an ongoing coherence to a canon in order to ensure maximum plausibility among all extensions.

Others routinely use alternative versions of characters or parallel universe versions of their stories to reward mastery over the source material.

Both modalities are really useful.

Continuity suggests the possibility to describe a world which is coherent.

Marvel Comics represent a very successful example of continuity: the heroes of the Marvel Universe live in a coherent reality and many efforts are put in place to ensure that this happens. Each character possibly influences the development of their peers, in a whole, continuous, consequential reality that is crafted in order to ensure the opportunity to make explicit connections between one plotline to the other, possibly suggesting consumers to buy more comics to see stories unfold from different perspectives.

Spiderman and The Hulk

Spiderman and The Hulk

In Marvel’s continuity, one comic might see the Spider Man fight a certain villain and, in the end, receive the help of the Hulk to defeat him. While another comic might focus on the Hulk’s struggle to locate the same villain, from an entirely different point of view, to find him in the end while he is fighting the Spider Man.

The other modality, Multiplicity, is also very useful, to suggest the possibility of mastering the capability and the complexities of the characters involved in the narratives.

From the point of view of the Transmedia Storyteller, Multiplicity represents the opportunity to generate parallel narratives: “Wat if …?” universes; parallel or alternate realities and timelines.

These can leverage existing characters and settings and expand them through the power of multiplication. For example, through the “What if ….?” mechanism deeper insights can be created for existing characters, by exploring the opportunities that could have been brought on by making different choices at the times of the difficult decisions they made during the plots.

On the other side, Multiplicity is also a very powerful way to suggest engagement. For example the idea of fan-fiction and mash-ups which are found all over the publishing industry comes directly from this possibility: to suggest, through various schemes, consumers to become performers, and to create themselves the alternative stories.

 

Worldbuilding

Transmedia extensions, often not central to the core narrative, that give a richer description of the world in which the narrative plays out.

Both real-world and digital experiences can be used for this purpose, and it is often the case in which people are pushed to move back and forth from one domain to the other.

This modality often leads to the fan behavior of capturing and cataloguing many disparate elements.

Mentioning this modality, Janet Murray argues that stories will have to work for two or three kinds of viewers in parallel:

“the actively engaged real-time viewer who must find satisfaction in each single episode and the more reflective long-term audience who look for coherent patterns in the story as a whole (…) [and] the navigational viewer who takes pleasure in following the connections between different parts of the story and in discovering multiple arrangements of the same material.”

There are two main ways in which Worldbuinding is performed:

  • Negative Capabilities
  • Migratory Cues

Each is often combined with the other, to obtain “collaborating” push-pull effects that are able to help users traverse the stories through their transmedia elements.

About Negative Capabilities Geoffrey Long says:

When applied to storytelling, negative capability is the art of building strategic gaps into a narrative to evoke a delicious sense of “uncertainty, mystery or doubt” in the audience.

Simple references to people, places or events external to the current narrative provide hints to the history of the characters and the larger world in which the story takes place.

This empowers the audience to fill in the gaps in their own imaginations while leaving them curious to find out more.

In the TV serial “Columbo(“Il Tenente Colombo” in italian) the detective’s wife is never shown on screen throughout the many seasons. Nonetheless she is constantly mentioned throughout the episodes, and becomes a main character in its own right. The fact that se is never shown on screen, even on a single picture,  allows each member of the audience to create in their own mind a mental representation for her, using imagination to be more deeply engaged and involved in the world of the detective’s life, imagining his lifestyle, daily routine and habits, inferring them from the many clues disseminated throughout the episodes.

This fact would give a Transmedia Storyteller many hooks to enact a Migratory Cue.

For example, once the detective’s wife “absent” character was established, a website of her recipes and tips for housekeeping could have been launched and would probably have become a success.

Migratory Cue, thus, is the stimulus to change media, to follow one of the “hyperlinks” exposed by the transmedia narrative, and to engage in a different chunk of the world which it refers to and actively manages to build.

The letter in Matrix is a sample of a Migratory Cue – when used at the beginning of the second Matrix movie it exists as a hint to look for more information in Animatrix and in Enter the Matrix.

Yet the story functions even without audience members having experienced either the anime or the video game, as they can imagine their own answer to the question of where exactly that letter came from.

They retain the option to go and track it down, and their understanding (and enjoyment) of the story would be increased by their doing so.

Understand: any reference to external people, places or events as utilizing negative capability to craft potential migratory cues, and become actualized as migratory cues when those extensions become available.

 

Seriality

Seriality is an element which has been mentioned extensively in the previous sections.

Transmedia storytelling has taken the notion of breaking up a narrative arc into multiple discrete chunks or installments within a single medium, and instead has spread those desparate ideas and story chunks across multiple, different, disseminated media segments.

We might understand how serials work by falling back on a classic film studies distinction between story and plot.

The story refers to our mental construction of what happened which can be formed only after we have absorbed all of the available chunks of information.

The plot refers to the sequence through which those chunks of information have been made available to us.

A serial creates meaningful and compelling story chunks and then disperses the full story across multiple installments.

We can think of transmedia storytelling as a hyperbolic version of the serial, where the chunks of meaningful engaging story information have been dispersed not simply across multiple segments within the same medium, but rather across multiple media systems.

 

Subjectivity

The notion of different subjectivities participating to forming the overall narrative has, too, already been mentioned in the previous sections. For example while making the example of Marvel Comics and their continuity, in which the same story is viewed from different points of view.

Transmedia narratives often explore the central narrative through new eyes, such as secondary characters or third parties.

The diversity of perspective often leads fans to more greatly consider who is speaking and who they are speaking for.

In mainstream media productions (for example in the case of TV serials) different subjectivities are often pursued using backstories, mobisodes and webisodes.

 

Performance

We can define this modality as the ability of transmedia extensions to lead to fan produced performances that can become part of the transmedia narrative itself.

Some performances are invited by the creator while others are not.

Fans actively search for sites of potential performance.

In the notion of Performance, as enacted by Transmedia Storytelling, two definitions are of particular relevance:

  • Cultural Attractors: draw together a community of people who share common interests.
  • Cultural Activators: give that community something to do.

It is by combining these two that performative dimensions usually take place.

For example: by combining spreadable media, to gather people, and the combination of negative capabilities/migratory cues, people can be suggested to transition from “passive” to “active” conditions – represented by the platforms which are used to build the cultural attractors/activators – to engage the performative state.

Fan fiction, mash-ups, memes, collaborative encyclopedias about TV serials, flash mobs and more, are all examples of the performative modality.

 

Simulacra, once again

the Simulacra

the Simulacra

Thus, from what we have seen in the previous sections, we can imagine our initial view of the Simulacra as taking the form of a Transmedia Narrative, in the shape above.

The object of simulation, the dashed circle in the image above, is the concept of the world we wish to describe.

The object of our simulation does not exist, it is fake (the holdup, in Baudrillard’s example).

Through the simulation we wish to make it believable, by producing its manifestations in the world (the fake holdup in Baudrillard’s example).

From what we have seen, if we do it well enough, taking special care to preserve the credibility of the chunks of the transmedia narrative, and by following the principles according to which they can be created (as seen in the previous sections), we can imagine achieving a condition in which it is hard (if not impossible) for people to distinguish the fake from the real.

We might, at this point, continue our investigation, by examining the actual possibility to create new-real from fake.

Which is something that is incredibly fascinating, that has been extensively discussed (for example herehere, here, here, and in many, many more places) and that we commonly wo in our practice, as artists, scientists, philosophers, communication experts, designers, makers etc.

Fake is Real.

[NOTE: this is an excerpt from our first lesson at the 2013 Master of Exhibit & Public Design at "La Sapienza" University of Rome where we teach the ways in which it is possible to design new forms of engaging, interactive, performative communication]

A Course on Fabbing and Nanotechnologies at ISIA Design Florence

Art is Open Source will be holding this year’s course in Multi Platform Digital Design (Progettazione Digitale Multi-Piattaforma, in Italian) at ISIA Design in Florence.
This year’s course will be focused on the themes of Digital Fabrication, Nanotechnologies and on the shifts in paradigms for production, about which we will imagine, design and enact different scenarios for the near future.
ISIA course 2013

ISIA course 2013

Sites such as http://www.thingiverse.com/ and http://www.shapeways.com/ have become rather common. This fact suggests a radical shift in the ways the paradigms of productions are (and can be) perceived, and brings up a whole series of fundamental issues which will become the commonplace for the (near) future of design.
What is certain is that precursors of these processes, such as the guys from http://www.fluid-forms.com/, have changed the ways in which, now, we are able to perceive objects and the ways in which we design and produce them.
Fluid Forms is a good example for this. When starting out, back in 2005, the concept of a widely accessible role of the Creative Coder operating in a virtual Design Space to design objects which could be physically produced/sold was not something many people had a chance to deal with.
For example, Fluid Forms’ Design Spaces allowed coders/designers to create software-generated objects using a number of information sources, APIs and data.
For example you, given a latitude/longitude pair of geographical coordinates, you could Google Maps to find the streets and landscape in that location and use this geometrical information to produce the design of a jewel, or a clock, or of something else. The object was, then, put on sale and users could specify their own set of coordinates (e.g.: of a place that was particularly meaningful for them) and produce their own, generative object.
This has been a steady trend ever since, and, progressively, we are starting to find it rather easy to identify objects with software.
Objects can be produced directly starting from the 3D files that describe them, and this changes the whole scenarios and the supply chains that were traditionally found in design and production:
  • 3D files (and programs to generate them) are exchanged and used to produce the objects (through 3D printing, for example)
  • files can be freely modified, reproduced, copied, redistributed etc

This obviously determines a radical change in the ways in which we traditionally perceive intellectual property when we deal with physical objects.

Imagine a scenario:

  • Ikea enters the 3D Printing market
  • You go to the Ikea store, you buy your 3D Printer, your supply of base materials (usually resins of some sort) and you go back home
  • you connect to the Ikea website, browse the 3D printable products, choose one and download it
  • it is a 3D file
  • you feed it to the 3D Printer and there you have it: your fresh-printed brand new Ikea Ashtray
  • then you send the file to one of your friends, as she has a 3D Printer, as well
  • your friend takes the 3D file, opens it in an editor, makes some changes to it (“here, with this added curve it looks just wonderful!”) and prints it out
  • then she puts the file up on Thingiverse, where everyone may download it, print it, etc

Whose copyright (if any) is it? Ikea’s? The original purchaser’s? His friend’s? The downloaders’? And so on…

Nothing we’re used (yet) to thinking about when we speak about physical objects.

Tendentially: the factory comes home!  Meaning that production will progressively disperse, become disseminated across a number of different scenarios in which digital files will be distributed in peer-to-peer ways across a number of small production facilities (even single households) in which they will be use to produce objects, and where they will also be modified and redistributed.

And companies such as http://www.makerbot.com/http://objet.com/ , http://cubify.com/cube/ , and http://futurecnc.code.arc.cmu.edu/ (and a lot more, lately) are making wide efforts to making sure that this will happen.

The outcomes of this tendency are practically infinite. Here are some examples:
A 3D vending machine:
A 3D printed skull prosthesis:
A smart phone application that works as a 3D scanner:
A doodler, a 3D Pen:
A 3D printed home:
Obviously, these practices raise quite a few interesting issues for discussion, as well:
  • at ecological and environmental level:
    • where do the raw materials come from?
    • are they sustainable?
    • how do you dispose of them?
    • could the possibility to 3D print anything bring on phenomenons of over-production?
    • etc…
  • at social level:
    • will be become new forms of stay-home consuming (producing) machines?
    • will we ever get out of our house? :)
    • will we be the target of the strategies of global strategies that will see us really busy with bringing up printing/modifying/commenting businesses for operators, and progressively loose contact with traditional markets and the world outside?
  • at ethical level:
    • what if i 3D print a rifle? or a bomb?

Obviously, some of these issues are exaggerated, for the sake of clarity, and some of them have already (partial) answers. But we’ll learn about them along the way, during the course.

The advent of nano-technologies allows these processes to radicalize even more.
For example, the nanotech factory-in-a-box scenarios allow for the disappearance of the limitations found in current 3D printers about the materials which can be used to produce the objects (mostly resins of some sort) and their achievable quality: the possibility to assemble more complex molecules starting from simpler ones theoretically allows to produce objects in any material, and the nano-scale of the production would allow for unmatched precision and quality.
Or as with the possibility to produce objects that are “alive”, or that relate with the human body at some level (e.g.: organs, eyes, prosthesis…): as with anything that could be in such tight relationship with our bodies and identities, these scenarios provide both great hopes and fears for the future. (e.g.: imagine if i nano-printed a replacement for my diseased liver using a service which is able to remotely stop it from working if I’m not able to pay their monthly fee…).
Or at the level of being able to produce objects which can chemically, mechanically and organically activate themselves, even in “intelligent” ways, to take actions of some sort. (from t-shirts which spontaneously change color, up to hordes of nano-robots which are able to use the materials around them to build skyscrapers)
Or HIV-curing nano-robots, nano-energy mechanisms which produce energy using waste etcetera.

What will happen during the course:

We will learn about the scenarios of fabbing and nanotechnologies, and will prepare and use tools which will enable us to observe their processes, evolutions, events: we will establish a sort of observatory unto the state-of-the-arts and to catch the signals which will use to infer the future scenarios.
Then we will discuss and design scenarios for the near/far future. Be them far of near, scary or hopeful, positive or negative, we will make works of design fiction that will explore these scenarios.
And then we will create transmedia narratives for these projects.
Have we designed a service which people use for replacement organs for a monthly fee? A peer-to-peer market of 3D printed molecular cuisine? A peer-to-peer production system to produce nano-energy from waste? A 3D printing service to replicate our pets?
We will make them as true as possible by producing websites, apps, prototypes, fakes, installations, performances, street advertisements, guerrilla marketing campaigns and more.

Master UX | From the User Journey to a working prototype Part 1

This is my first lesson at the Master UX held at La Sapienza University of Rome.

In this lesson we analyze how to use the service concept formalized in the User Journey to create a working prototype.

In the process we use all the information emerged from the User Investigation, Analysis and Information Architecture phase.

This is Part 1. Part 2 will follow in the next days.

Redesigning Urban Communication in Trieste

a workshop in trieste

a workshop in trieste

We will be in Trieste from July 27th to August 2nd for “Insegna Trieste” a workshop to imagine and design the future of urban information and communication for the city.

Organized by the City Administration of the City of Trieste and by ISIA Design Florence, and with the collaboration of the Regional Administration of Friuli Venezia Giulia, the Faculty of Architecture of the University of Trieste, ISIA Urbino, IUAV University and the University of Nova Gorica the workshop will gather students to explore the city of Trieste from a new point of view, and trying to answer the question: “What is the information architecture of the city, as interpreted from a variety of cultures and points of view? How is it possible to define strategies and methodologies to design a system of signs, technologies and methodologies to represent it in meaningful ways? What is the future of urban signage?”

Urban signage is the street-interface for the city’s Information Architecture.

Information Architecture is a heterogeneous combination of different processes, including organization, categorization and the creation of navigation schemes in an Information System.

In the current scenario we can imagine to extend the set of techniques, methodologies and technologies which were classically used to design urban signage, and to include a wider, holistic, set of considerations into the discourse.

Ubiquitous technologies, tagging, augmented reality, urban screens and other technologies can be used to radically transform our experience of cities, as we navigate through streets, landmarks, businesses and opportunities for socialization, entertainment, culture, information and relation.

It is possible to embrace a holistic approach by taking into consideration the passage from Data, to Information, to Knowledge, to Wisdom.

from data to wisdom

from data to wisdom

Data, in city navigation, is composed by the positions of landmarks, streets and buildings, by the opening times of venues, by the presence of toilets and services. Or, in general, by discreet elements telling us what we know about the various things which are found in the city.

We can analyze elements of data to find relations among them. By interconnecting data in network of relations we can describe information. Information is a structure of data, a set of relationships among its elements, forming a network according to a certain strategy.

But we can go further. We can observe how data relates to other data to form information according to a series of recurring patterns. This is a valuable process, as it allows us to become aware of these patterns and to have a more useful point of view on reality, being able to understand information. This process is commonly defined as knowledge.

By understanding the principles according to which these patterns occur, we gain an even deeper understanding of reality, which we common refer to as wisdom.

Usually, urban signage refers to the domain of information. A series of scenarios and use cases are envisioned and, thus, the needs, desires and wishes of different types of users are evaluated to try to understand what information is more useful, and how it can be conveniently visualized onto signs, in accessible, usable ways: “A tourist is on the street in a certain city location, searching for the next landmark to visit.” Signage is designed in order to be visible, perceivable and understandable, allowing users to visually navigate the information domains of the city (mobility, environment, tourism, services, administration, regulations etcetera).

Now, specific technologies and methodologies can be used to assess the other two, valuable, scenarios of Information Architecture: Knowledge and Wisdom.

For example, in the touristic scenario, Knowledge could be represented by patterns of information, such as “the network of best landmarks for contemporary art lovers”, or “the accessible city”, or “the city’s venues, prioritized according to how my Facebook friends rate them”.

And Wisdom could be about understanding how people having similar cultures, contexts, tastes and needs have achieved their goals (for tourism, business, entertainment, sociality) in the city, and providing this as a service.

In our navigation through cities we mostly operate through visual queries.

What we see and experience in the city – and the way we interpret it according to context, culture and personal history – defines affordances for the use of public space.

Wether we operate in a task-based mode or wether something we see/hear/smell/touch in our environment synchs with our culture or sensibility of the moment, we perceive symbols, colors, shapes, and spatial layouts which suggest what is possible/impossible, suggested or advised against, promoted or prohibited in public space.

Affordances in public space are defined at visual, cultural, administrative and political levels, allowing us to experience a very complex set of stimulations coming directly from the urban environment.

Mobile, ubiquitous, devices – such as our smartphones – radically transform our experience of space. Though these devices we constantly experience a potentially infinite number of different, augmented, layers of information which fill the environment. Wether they come under the form of a call or a text message, through a geo-located social network element, via augmented reality, via the internet of things or via other digitally active object/space/architecture present in our surroundings, spaces transform in our perception thanks to the situated availability and perceivability of these elements of digital information.

Furthermore, these layers of information can be related to complex experiences, inventing entire new usage grammars to read, experience and perform public space.

This is what we will explore in the workshop.

The possibility to integrate Ubiquitous Publishing, Urban Sensing, Social Networking, Knowledge Ecosystems with Visual Design, Urban Design, Visual Thinking – and not forgetting to assess issues such as Digital Divide and Digital Inclusionto reinvent the ways in which data, information, knowledge and wisdom are created, disseminated and shared in public space.

We will explore the city and try to understand people’s diversity and needs, and, thus, the information domains which are more relevant to them, as well as their needs for visual navigation systems at global, local and contextual levels.

We will then explore the ways in which digital technologies and networks can be used to go one step further, making richer forms of information accessible and usable, and also making Knowledge and Wisdom accessible, participatory, engaging for citizens and other forms of city dwellers.

We will turn all these insights into a proposal for a new Information Architecture for the city of Trieste, and for its interface: a novel, innovative signage system, across visual design and Ubiquitous Publishing.

Stay tuned for a report after the workshop.

 

Augmented Reality: the Augmented City, communication and citizenship

On May 17th 2012, we took part  to prof. Marco Stancati’s course of Media Planning at La Sapienza University of Rome with a lecture on the scenarios offered by Augmented Reality to the creation of novel opportunities for communication and business.

HERE you can find some information about our lecture and the MediaPlanning course.

HERE you can download the slides we used for the lecture

(The slides are a lighter version of the ones we used in class, which were full of videos and hi-res images: please feel free to contact us should you want the original ones)

In the lecture, we started from a series of definitions about what, in our times, can be considered as “Augmented Reality”

Augmented Reality in the city

Augmented Reality in the city

In our definitions we chose to describe a wider form of the term, not limiting it to the set of applications to which we’ve all been accustomed to , and abandoning for a moment the vision of people happily strolling through cities with their smartphones raised in front of their faces.

Nonetheless we used classical examples of AR to introduce a possible evolution of what is/will be possible in our cities using ubiquitous technologies.

We focused on the idea of the Augmented City.

augmented city and its many voices

augmented city and its many voices

In this vision of the city, many subjects (individuals, organizations and, using sensors, also the city itself) add layers of digital information, in real-time. We can access and experience these  sets of information in multiple ways, and we can also use them to compose, dynamically, our personal vision of the city, by remixing, re-arranging, re-combining and mashing-up all the information layers which are available.

This is a very interesting situation for cities and their citizens, as it enables for the creation of entire new scenarios for communication, business and personal expression.

It also opens up possibilities which will probably have a high impact on the ways in which, for example, enterprises design their own products, and the ways in which they create the strategies according to which products and services are communicated, marketed, monitored.

We discussed this scenario below, among the many possible:

augmenting the voices on products

augmenting the voices on products

The physical packaging of products usually hosts information and messages which are created by a very limited number of voices (e.g.: the manufacturer, marketing team..).

In the drawing we see depicted a scenario which is becoming progressively more frequent: a multiplicity of subjects are now able to join the Brand in adding digital information to products and services, using Augmented Reality, QRCodes, computer vision, tagging (e.g.: RFID) and location based technologies. We call this Ubiquitous Publishing.

For example, in the Squatting Supermarkets project we used products’ packaging as visual reference for critical Augmented Reality experiences. In the performance, people could use their smartphone to “look at” products on a supermarket shell. When they did, a series of information became available:

  • a map, created using MIT’s Open Source Map, showing where the product and its components came from, where its materials came from, where it had been processed and assembled, and in which places it stopped during transportation; the product becomes a map of the planet highlighting all the places which it touched during its manufacturing and distribution processes;
  • a series of visualizations showing the product’s composition, the percentages of organics, chemicals, fat, aromas… all shown through interactive information visualizations.
visualizations in Squatting Supermarkets

visualizations in Squatting Supermarkets

One of the visualization was analyzed in deeper detail. On the top right of the previous image, is a timeline of the real-time conversations about the product: the timeline scrolls left and right and each colored block represents a conversation and its general sentiment (meaning: the sentiment which is most represented in the conversation); green, yellow and red code positive, mixed and negative sentiments.

So: while strolling through the aisles of a supermarket, you take a take a picture of your favorite product and you are able to see what people on social networks are saying about it, in real-time.

We observed this possibility (to publish real-time, user generated information using ubiquitous publishing techniques and accessible information visualizations) to describe an interesting loop which we are able to make.

We can imagine (and do) to harvest user generated information in real-time about our topics of interest (from blogs, websites, social networks and social media sites) and to publish them when/where they are most useful.

VersuS, the real-time lives of cities

VersuS, the real-time lives of cities

The image above shows the experiment we performed with the VersuS project during the city-wide riots taking place in Rome on October 15th 2011.

The 3D surface covering the map of the city of Rome shows the intensities of the social network conversations taking place during the protests and riots. The image is part of a real-time visualization through which we have been able to observe how social media conversations closely followed the evolution of the protest.

By using harvested conversations (from Facebook, Twitter, Flickr and Foursquare) we have been able to analyze what was being said and where, and we actually were able to demonstrate how a massive amount of useful information was being published by users: about violence, injuries, possible escape routes, missing people. All this information could have been actually collected and organized, and accessed by protesters through specifically designed interfaces to achieve important, pragmatic results such as avoiding being hurt, finding safe escape routes from the riots or find our friends who were lost in them.

We were able to design, for example, the simple Augmented Reality application for smartphones pictured below:

Augmented Reality for Riots

Augmented Reality for Riots

Augmented Reality for Riots

Augmented Reality for Riots

This experimental interface shows how a rioter could have visualized on the screen of the smartphone the degree of safety in the direction he/she was facing, as it could be inferred by the social media conversations with a geographic reference.

An immediate, easy to use tool to achieve important goals.

During the lesson we focused on how it would be possible to use these technological opportunities to conceive and enact innovative communication practices.

We described a couple of scenarios, which we can imagine being applied in different forms, ranging from the creation of scenarios for the public lives of cities and their citizens, to the needs of communicators for their work with enterprises and administrations, to the needs of marketing and advertising.

In synthesis, we imagined a novel, more extensive, definition for Augmented Reality, according to which a loop is formed among the digital and physical world.

In this definition of AR it is possible

  • to harvest user-generated (as well as database and sensor generated) real-time information about relevant places/topics/products/services,
  • to process it using techniques such as Natural Language Processing and Sentiment Analysis,
  • to publish it ubiquitously, where/when it is more useful, using interfaces and interaction schemes which ensure accessibility and usability (including smartphone apps, urban screens, wearable technologies, digital networked devices, information displays…) and
  • to provide ways according to which users are able to both contribute to the flow of information and to re-assemble and re-interpret it, creating additional points of view