Augmented Reality: the Augmented City, communication and citizenship

On May 17th 2012, we took part  to prof. Marco Stancati’s course of Media Planning at La Sapienza University of Rome with a lecture on the scenarios offered by Augmented Reality to the creation of novel opportunities for communication and business.

HERE you can find some information about our lecture and the MediaPlanning course.

HERE you can download the slides we used for the lecture

(The slides are a lighter version of the ones we used in class, which were full of videos and hi-res images: please feel free to contact us should you want the original ones)

In the lecture, we started from a series of definitions about what, in our times, can be considered as “Augmented Reality”

Augmented Reality in the city

Augmented Reality in the city

In our definitions we chose to describe a wider form of the term, not limiting it to the set of applications to which we’ve all been accustomed to , and abandoning for a moment the vision of people happily strolling through cities with their smartphones raised in front of their faces.

Nonetheless we used classical examples of AR to introduce a possible evolution of what is/will be possible in our cities using ubiquitous technologies.

We focused on the idea of the Augmented City.

augmented city and its many voices

augmented city and its many voices

In this vision of the city, many subjects (individuals, organizations and, using sensors, also the city itself) add layers of digital information, in real-time. We can access and experience these  sets of information in multiple ways, and we can also use them to compose, dynamically, our personal vision of the city, by remixing, re-arranging, re-combining and mashing-up all the information layers which are available.

This is a very interesting situation for cities and their citizens, as it enables for the creation of entire new scenarios for communication, business and personal expression.

It also opens up possibilities which will probably have a high impact on the ways in which, for example, enterprises design their own products, and the ways in which they create the strategies according to which products and services are communicated, marketed, monitored.

We discussed this scenario below, among the many possible:

augmenting the voices on products

augmenting the voices on products

The physical packaging of products usually hosts information and messages which are created by a very limited number of voices (e.g.: the manufacturer, marketing team..).

In the drawing we see depicted a scenario which is becoming progressively more frequent: a multiplicity of subjects are now able to join the Brand in adding digital information to products and services, using Augmented Reality, QRCodes, computer vision, tagging (e.g.: RFID) and location based technologies. We call this Ubiquitous Publishing.

For example, in the Squatting Supermarkets project we used products’ packaging as visual reference for critical Augmented Reality experiences. In the performance, people could use their smartphone to “look at” products on a supermarket shell. When they did, a series of information became available:

  • a map, created using MIT’s Open Source Map, showing where the product and its components came from, where its materials came from, where it had been processed and assembled, and in which places it stopped during transportation; the product becomes a map of the planet highlighting all the places which it touched during its manufacturing and distribution processes;
  • a series of visualizations showing the product’s composition, the percentages of organics, chemicals, fat, aromas… all shown through interactive information visualizations.
visualizations in Squatting Supermarkets

visualizations in Squatting Supermarkets

One of the visualization was analyzed in deeper detail. On the top right of the previous image, is a timeline of the real-time conversations about the product: the timeline scrolls left and right and each colored block represents a conversation and its general sentiment (meaning: the sentiment which is most represented in the conversation); green, yellow and red code positive, mixed and negative sentiments.

So: while strolling through the aisles of a supermarket, you take a take a picture of your favorite product and you are able to see what people on social networks are saying about it, in real-time.

We observed this possibility (to publish real-time, user generated information using ubiquitous publishing techniques and accessible information visualizations) to describe an interesting loop which we are able to make.

We can imagine (and do) to harvest user generated information in real-time about our topics of interest (from blogs, websites, social networks and social media sites) and to publish them when/where they are most useful.

VersuS, the real-time lives of cities

VersuS, the real-time lives of cities

The image above shows the experiment we performed with the VersuS project during the city-wide riots taking place in Rome on October 15th 2011.

The 3D surface covering the map of the city of Rome shows the intensities of the social network conversations taking place during the protests and riots. The image is part of a real-time visualization through which we have been able to observe how social media conversations closely followed the evolution of the protest.

By using harvested conversations (from Facebook, Twitter, Flickr and Foursquare) we have been able to analyze what was being said and where, and we actually were able to demonstrate how a massive amount of useful information was being published by users: about violence, injuries, possible escape routes, missing people. All this information could have been actually collected and organized, and accessed by protesters through specifically designed interfaces to achieve important, pragmatic results such as avoiding being hurt, finding safe escape routes from the riots or find our friends who were lost in them.

We were able to design, for example, the simple Augmented Reality application for smartphones pictured below:

Augmented Reality for Riots

Augmented Reality for Riots

Augmented Reality for Riots

Augmented Reality for Riots

This experimental interface shows how a rioter could have visualized on the screen of the smartphone the degree of safety in the direction he/she was facing, as it could be inferred by the social media conversations with a geographic reference.

An immediate, easy to use tool to achieve important goals.

During the lesson we focused on how it would be possible to use these technological opportunities to conceive and enact innovative communication practices.

We described a couple of scenarios, which we can imagine being applied in different forms, ranging from the creation of scenarios for the public lives of cities and their citizens, to the needs of communicators for their work with enterprises and administrations, to the needs of marketing and advertising.

In synthesis, we imagined a novel, more extensive, definition for Augmented Reality, according to which a loop is formed among the digital and physical world.

In this definition of AR it is possible

  • to harvest user-generated (as well as database and sensor generated) real-time information about relevant places/topics/products/services,
  • to process it using techniques such as Natural Language Processing and Sentiment Analysis,
  • to publish it ubiquitously, where/when it is more useful, using interfaces and interaction schemes which ensure accessibility and usability (including smartphone apps, urban screens, wearable technologies, digital networked devices, information displays…) and
  • to provide ways according to which users are able to both contribute to the flow of information and to re-assemble and re-interpret it, creating additional points of view


Wearing Emotions by FakePress, presented at the IV10 conference in London, July 2010

Wearing Emotions by FakePress, presented at the IV10 conference in London, July 2010 from salvatore iaconesi on Vimeo.

The video shows the presentation of the paper titled “Wearing Emotions: Physical representation and visualization of human emotions using wearable technologies” presented at the IV10 (Information Visualization 2010) conference at South Bank‘s college in London, on July 26th, 2010.

The paper and presentation describe a research process focused on the scientific research, design and implementation of wearable devices able to display human emotions – be them individual, group or global – on physical bodies.
The devices created in the process have been used to create 3 artistic performances as both proofs of concepts and as innovative forms of artistic and aesthetic expression: Talkers performance, OneAvatar and Conference Biofeedback.

the slides relative to the presentation can be found here:

the video can be found here:

or here

on Art is Open Source:

OneAvatar, wearable technologies connecting body and avatar

OneAvatar, wearable technologies connecting body and avatar

Reference links:

the IV10 conference website:

Art is Open Source:


Talkers Performance:


Conference Biofeedback:

Living CrossMedia @ Università La Sapienza in Rome

Living CrossMedia

Living CrossMedia


AOS & FakePress

insieme a

Università di Roma La Sapienza, Facoltà di Architettura Ludovico Quaroni, Dipartimento di Disegno Industriale

e con la presenza di

Assessorato alla Cultura e alla Comunicazione del Comune di Roma


Living CrossMedia

le opportunità dei nuovi media nei Centri Commerciali e nei nuovi spazi della cultura e dell’aggregazione

Martedì 1 Giugno 2010

Facoltà di Architettura Ludovico Quaroni

Università di Roma “La Sapienza”

via Gianturco 2, Roma

aula G11

dalle 9:30 alle 12:00

Marco Torresan, Direttore del Centro Commerciale “Porta di Roma”

Simona Pantò, Responsabile del Marketing del Centro Commerciale “Porta di Roma”

Alessandro Ferrante, Assessorato alla Cultura e Comunicazione del Comune di Roma

terranno una lecture e dibattito per discutere delle opportunità che, nello scenario contemporaneo, si aprono con la comunicazione e l’interazione multi e cross mediale nei nuovi spazi della socializzazione, dell’aggregazione e della cultura, a partire dalla loro esperienza.

Partendo dall’analisi della trasformazione degli spazi pubblici e commerciali nelle città del nuovo secolo, per arrivare alle suggestioni che vedono display, interazioni con dispositivi mobili ed ubiqui, ambienti interattivi, e nuove forme di comunicazione digitale arricchire la nostra esperienza della città, dei nostri rapporti con le persone, le aziende e gli spazi che percorriamo tutti i giorni.

L’intervento, propositivo anche nel ragionare sulle possibilità di interazione tra ricerca, arte, design e marketing, sarà seguito da un dibattito aperto a tutti gli intervenuti.

L’incontro sarà coordinato da Salvatore Iaconesi (Art is Open Source, FakePress e docente del corso di Sperimentazioni di Tecnologie e Comunicazioni Multimediali della Facoltà di Architettura Ludovico Quaroni dell’Unversità di Roma “La Sapienza”) e Oriana Persico (Art is Open Source, FakePress).


AOS & FakePress

together with

University of Rome La Sapienza, Faculty of Architecture Ludovico Quaroni, Department of Industrial Design

and with the presence of

Culture and Communication Council of the City of Rome


Living CrossMedia

the opportunities for new media in Commercial Centers and in the new spaces for culture and social aggregation

Tuesday, June 1st, 2010

Faculty of Architecture Ludovico Quaroni

University of Rome “La Sapienza”

via Gianturco 2, Rome

room G11

from 9:30 am to 12:00

Marco Torresan, Director of the Commercial Centre “Porta di Roma”

Simona Pantò, Marketing Responsible of the Commercial Centre “Porta di Roma”

Alessandro Ferrante, Culture and Communication Council of the City of Rome

will hold a lecture and debate to investigate the opportunities opening in the contemporary scenario through the adoption of multi and cross medial communication and interaction in the emerging spaces for socialization, culture and aggregation, starting from their experience.

Starting from the analysis of the current transformation of public and commercial spaces in the cities of the new century, and arriving to the suggestions coming from the wide availability of display technologies, of interactions with mobile and ubiquitous devices, of interactive and responsive environments, and of new forms of digital communication enriching our experience of the city, of our relationships with other people, companies and spaces.

The lecture will also explore the possible interactions among scientific research, art, design and marketing, and will be followed by an open dialogue to which everyone attending the event will be invited to contribute.

The event will be coordinated by Salvatore Iaconesi (Art is Open Source, FakePress and professor of Experiments in Multimedia Technologies and Communications at the Faculty of Architecture “Ludovico Quaroni”  at Rome’s University “La Sapienza”) and by Oriana Persico (Art is Open Source, FakePress).

Squatting Supermarkets / iSee at La Sapienza University in Rome

Squatting Supermarkets at La Sapienza in Rome

Squatting Supermarkets at La Sapienza in Rome

FakePress, Art is Open Source and The Hub Roma

in collaboration with

the “Multimedia Technologies and Communications Experimentations” course at the “Ludovico Quaroni” Faculty of Architecture at Rome’s University “La Sapienza”, Industrial Design Department


the “Management of Non profit Organizations” course at the Faculty of Economics at the “Tor Vergata” University

are happy to invite you to the lecture:

Squatting Supermarkets/iSee

Artistic fundamentals, eco-sustainability, market: from Shoptivism to the Active Consumer.

Who: Oriana Persico and Cary Hendrickson (AOS/FakePress), Dario Carrera and Ivan Fadini (The Hub Roma/Faculty of Economics, Tor Vergata), Ilaria Bassi, Vanessa D’Acquisto, Piergiorgio Malfa, Vittoria Mauro (research group at the “Management of Non Profit Organizations” course, Tor Vergata), Salvatore Iaconesi (visiting professor)

What: lecture/workshop

Where: Faculty of Architecture “Ludovico Quaroni”, via E.Gianturco 2 (Rome) – room G 11 [ MAP ]

When: May 4th, 2010 – from 9am to 12pm

A radical version of a marketplace, a point of sale in augmented reality, Squatting Supermarkets tells the tale of the evolution of our daily realities, entering the live and pulsating heart of consumism. Looking at products on the shelves, choosing, paying, debts, persuasion and seduction, relations with logos, messages and other people. Buying is an experience that fills our daily lives, built through images, suggestions and strategies that are so complex that they systematically evade the perception of the final user. Technologies can be used to create new spaces for action/communication, and to overlay them onto our ordinary reality, thus creating new action/communication spaces, allowing new possibilities for interaction and fruition: ubiquitous, accessible, emergent and polyphonic, emotional and relational. Squatting Supermarkets narrates this possibility: an augmented reality space that is technologically layered to everyday life, an interstitial marketplace living in squat on the physical and immaterial infrastructures.

Presented for the first time at the Piemonte Share Festival in 2009, Squatting Supermarkets has two souls: a site-specific installation and an innovative technology, iSee. The installation reproduces an interactive supermarket in which the widespread interconnection network defining products’ histories and stories can become explicit and accessible. The techniques of traceability and control are transformed into their ecosystemic, narrative and poetic versions: information is not only for corporations anymore. By overlaying and integrating codes coming from barcodes, tags, RFIDs, credit cads, financial tranactions.
Grabbing a product becomes an immersive experience with its story, and the possibility of writing a part of it: moving hands, drawing gestures, exposing our points of view and our emotions. Products animate, becoming a space for expression, a network of relations, a domain for possibility and opportunity. Distributed storytelling practices in which the hidden stories of producers and consumers come in contact. The access door to these domains are the logos. iSee, the technological heart of the installation is a mobile application based on image recognition and computer vision techniques.
Framing the product on the camera, an image processing algorithm identifies the logo, allowing the user to access an additional set of information coming from a plurality of sources. Narrative galaxies (open, emergent, multi-author) connected to p2p thematic social networks layered over products.
Logos become wikis, open communication infrastructures, distributed social networks, p2p ecosystems.
The Project is based on the analisys of the contemporary technological/economic context. On one side, environmentalism and eco-sustainability are issues that are acknowledged as being among the globally most important drivers in piloting the shopping choices of large parts of the population. On the other side, statistics demonstrate that progressively large numbers of individuals search for information on social networks befre purchasing commodities and services, and that the users of last-generation mobile devices are steadily growing by the numbers.
In all this, enterprises start feeling the need to confront a globally interconnected communication and relation system, emergent, polyphonic, in which the acquisition and the maintenance of a “good reputation” goes through complex social and cultural dynamics that are progressively more chaotic. The “Company” is not the sole center of communication and of the shaping of its corporate identity.

On may 4th 2010, from 9am to 12pm,  FakePress’ interaction design lab will analyze the possibilities for expression and interaction enabled by the shopping based narratives offered by Squatting Supermarkets under an artistic/performative point of view and by iSee under the perspectives of technology and infrastructures.

Oriana Persico will confront the themes related to the installation, introducing the artistic statement,

Cary Hendrickson will focus on the themes of eco-susteinability, social responsibility and on the governance of related processes;

The research group of the “Management of Non profit Organizations” course at the Tor Vergata University coordinated by Dario Carrera and Ivan Fadini will expose the first results coming from the research titled “Active Consumer: from Movements to Shopping Based Publishing“.

How are consumers sensible to the environment and to critical consumism? How do they relate to the opinions generated on social media and on brand reputation created through User Generated Content? How widespread are smartphones? How many individuals would be willing to use a mobile application such as iSee? Who could finance it? What are the possible business models?

The theoretical and artistic premises, the analisys on eco-sustainability and the market scenarios will be used by the students of the course at the Faculty of Architecture to create a series of project works focused on the evolutions of the iSee platform and on the creation of a site specific installation to be implemented in a public commercial space, integrating multiple skills and points of view and, most of all, creating multidisciplinary and inter-university collaboration paths among students and faculties to achieve common goals.

The project works will be presented together with the research during the Open Day at the end of the course.

Entrance to the lecture is free and open to all.

Additional information

Squatting Supermarkets/iSee is a co-production by FakePress/Art is Open Source. Special project at the Piemonte Share Festival 2009, winner of the “Zero Impact Technlogy” prize offered by the Environment Park in Turin. Squatting Supermarkets/iSee is part of the SMIR project under the artistic direction of The Sharing: (site available next week)


please use the Facebook Event to stay updated:!/event.php?eid=122966341053373

Holophonics & Design

HOLO - an AU plugin for holophonics & a social networking project

HOLO - an AU plugin for holophonics & a social networking project

FakePress and Ossigeno

in collaboration with

the course on “Multimedia Technologies and Communications Experimentations” at the Faculty of Architecture “Ludovico Quaroni” – La Sapienza, Università di Roma, Department of Industrial Design

are happy to invite you to the lecture:


Possible trajectories for the experience of holophonic listening
Who: Marco Fagotti (guest relator) and Salvatore Iaconesi (visiting professor)
What: lecture/workshop
Where: Faculty of Architecture “Ludovico Quaroni, Via E. Gianturco 2 (Rome, Italy) – room G 11
When: April 30th 2010 – 9am – 2pm

Holophony is a specific sound recording and diffusion technique  that reconstructs the experience of three-dimensional listening through the use of special microphones, algorithms and headphones. In practice, it allows the listener to perceive the origin of sounds coming from any point in space.

The possibility to use headphones as privileged fruition devices to hear music, sounds and voices, and the possibility to replicate analogue processes through digital algorithms seems an ideal territory to embrace 3D production, The implications concern not only technical and technological aspects, but a deep re-evaluation of the approach to music and to the knowledge of sound as a cognitive and communicative entity.

April 30th, from 9am to 2pm, at the Interaction Design laboratory held at the course on “Multimedia Technologies and Communications Experimentations” at the Faculty of Architecture “Ludovico Quaroni” – La Sapienza, Università di Roma, Department of Industrial Design by FakePress, will explore the territories of holophonic listening with Marco Fagotti, guest Lecturer for the event. Salvatore Iaconesi, visiting professor at the course, will illustrate the fundamental technologies and methodologies that can be used to design digital sound spatial positioning with the techniques of holophony.

The technologies will be used in the next few days to complete the production of an AU plugin (AU is a standard for plugin development used in most professional audio softwares) that will be distributed to enable for the synthetic creation of complete holophonic sound environments.
The research on Head Transfer Functions (those functions allowing to describe and reenact the diffusion of sounds from the environments to ur head and ears) will be presented.

Holophony will be discussed as a novel possibility for artistic and social interaction, with the ability to potentially fill the world with alternative, realistically spatialized soundscapes, imagined as an opportunity for augmented reality, for social and/or subjectivized reinvention of public/private spaces, on the possibilities for self-expression, knowledge sharing, entertainment and new sensorialities.

The students of the course will be invited to use the theories of holophonics to design innovative systems using these techniques in social, educational and entertainment scenarios: what is an holophonic social network like? Or a distributed, disseminated cinema? Or an holophonic tourist guide? Or a holophonic concert?

The results of their designs will be presented as project works for the end-of-year Open Day, in which all the projects will be presented, both at the University and in a selection of international events.

Please check the Facebook Event for up-t-date details on the event:!/event.php?eid=112871408751698&ref=mf