Augmented Reality Art: the Emotional Compass featured on a new book

The Emotional Compass featured in a chapter of the new “Augmented Reality Art” book from Springer, edited by Vladimir Geroimenko together with Mark Skwarek, Tamiko Thiel, Gregory L. Ulmer, John Craig Freeman, Conor McGarrigle, Patrick Lichty, Geoffrey Alan Rhodes, Jacob Garbe, Todd Margolis, Kim Vincs, Alison Bennett, John McCormick, Jordan Beth Vincent, Stephanie Hutchison, Ian Gwilt, Judson Wright, Nathan Shafer, Salvatore Iaconesi, Oriana Persico, Dragoş Gheorghiu and Livia Ştefan, Simona Lodi, Margaret Dolinsky, Damon Loren Baker.

view Augmented Reality Art on Springer

In the book our contribution is titled: “An Emotional Compass: Emotions on Social Networks and a new Experience of Cities

cite as:

Iaconesi, S. and Persico, O. (2014). “An Emotional Compass: Emotions on Social Networks and a new Experience of Cities” in Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium, part of the Springer Series on Cultural Computing, Geroimenko, Vladimir (Ed.). New York: Springer. ISBN 978-3-319-06202-0. http://www.springer.com/computer/hci/book/978-3-319-06202-0

Here is a short sample of the introduction of the chapter:

“The map is not the territory.” (Korzybski, 1933)

 

“The map is not the thing mapped.” (Bell, 1933)

 

“The tale is the map that is the territory.” (Gaiman, 2006)

 

“We say the map is different from the territory. But what is the territory? The territory never gets in at all. […] Always, the process of representation will filter it out so that the mental world is only maps of maps, ad infinitum.” (Bateson, 1972)

 

When we experience territories, we create stories. We model these stories using mental maps, referring to one person’s point of view perception of their own world, influenced by that person’s culture, background, mood and emotional state, instantaneous goals and objectives.

 

If we move along the streets of my city in a rush, trying to find a certain type of shop or building, our experience will be different than the one we would have had if we were searching for something else.

 

Focus will change. We will see certain things and not notice other ones which we would have noticed otherwise. Some things we will notice because they are familiar, common, or because associate them to our cultures, to memories and narratives. All this process continuously goes on as our feelings, emotions, objectives and daily activities change, creating the tactics according to which we traverse places and spaces, to do the things we do.

 

In the density of cities, this process happens for potentially millions of people at the same time. In his “the Image of the City” (Lynch, 1960), Lynch described cities as complex time-based media, symphonies produced by millions of people at the same time in their polyphonic way of acting, moving, interpreting, perceiving and transforming the ambient around themselves: a massive, emergent, real-time, dissonant and randomly harmonic, work of time-based art with millions of authors that change all the time.

 

In this, our mental maps – the personal representations of the city which we build in our minds to navigate them to fulfill our needs and desires – live a complex life as our perception joins into the great performance of the city.

 

Dissonance is the essence of the city itself, and represents its complexity, density and opportunities for interaction.

 

Augmented Reality Art

Augmented Reality Art

How to make a ubiquitous soundscape using augmented reality: Read/Write Reality, Ubiquitous Sound at Youbiquity!

AOS will be in Macerata ( May 2-6 2014 ) at the Youbiquity Festival for a workshop in which we will understand how to create an ubiquitous soundscape and installation, to create an immersive geography of sound.

“When you listen carefully to the soundscape it becomes quite miraculous.”

––R. Murray Schafer

From the Youbiquity website:

An immersive workshop whose objective is to create an Ubiquitous Soundscape: a sonic landscape which can be experienced using Augmented Reality, and which can be produced collaboratively, through sound sampling and audio representation of data and information.

 

Participants will learn how to design a specific Augmented Reality smartphone application (iOS and Android), on which to publish their Ubiquitous Soundscape, created through sound samples of any kind and the audio representation of data and information. All of this will form an immersive experience, in which it will be able to walk through the sounds disseminated across natural and urban spaces.

A result of the workshop will be the participation to the second volume of the Read Write Reality publications (you can find the first Read/Write Reality book on Lulu, which was about the creation of an Augmented Reality Movie), and a final show/exhibit/installation, ubiquitously distributed through the streets of beautiful Macerata.

Here is the Program and info for the Ubiquitous Sound workshop

To take part to the workshop you can contact: youbiquity.giorgio@gmail.com  +39 349 6441703

How do you create an ubiquitous Soundscape?

The Soundscape. The sound or combination of sounds which arises from an immersive environment.

This definition of soundscape comes from Canadian composer R. Murray Schafer, who identified three main elements of each place’s soundscapes: the Keynote Sounds, created by nature, geography and climate, and which live in the background of our conscious perception most of the time; the Sound Signals, which are the ones we consciously listen to; and the Soundmark, coming from landmark, which is the sound which is unique to an area.

Bernie Krause classified the elements of the soundscape according to their originating source: the Geophony of a place, generated by non-biological sources; the Biophony, as generated by non-human living beings; and the Anthrophony, generated by human beings.

Both of these definitions can be updated to try to engage the fact according to which entirely new dimensions of space have now entered our realms of perceptions.

Digital data, information and communication has become ubiquitously available and accessible, and everything we do generates data and information somewhere.

We have learned to use all these additional sources of information to transform the ways in which we communicate, work, collaborate, learn, express ourselves and our emotions, relate and consume. Ubiquitous information has entered our daily lives, blurring the boundaries between what is digital and physical, so much that it is progressively loosing sense to make the distinction in the first place.

In RWR UbiquitousSound we wish to address the phenomenology of the Ubiquitous Soundscape.

Our aim is to design a natural way to create and interact with digitally and ubiquitously produced sound in the environment.

As it happens for the biophony, geophony and anthrophony of places, we want to create an Infophony of space, in which we can walk through, orient, experience. We wish to describe and implement the parts of our soundscape which could be created through Ubiquitous Publishing techniques, from social networks, data sets, and from the digital information which we constantly produce from all the places in the world, through our daily lives. We want to make this information physical, evolving, emergent, experienceable, immersive, complex, just as the rest of the soundscape.

We want to create an explicit bridge between the physical and digital realms of our lives, through sound, allowing us to create information ubiquitously, and to experience it immersively.

What we will do

We will create an Augmented Reality application which will allow us to experience the immersive Ubiquitous Soundscape by wearing headphones.

We will create the application together, also co-designing its elements. The application will allow us to load sounds samples and sound-representations of datasets and information, and to map them to a physical space. Then headphones will be used to experience the soundscape in an immersive way: walking up to the sounds, away from them, being able to achieve a new form of sound orientation through the Ubiquitous Soundscape, in the physical world.

We will create our own Ubiquitous Soundscapes.

We will showcase them in a final performance though the streets of Macerata, and though an exhibit.

Who is this workshop for

Any artist, designer, hacker, architect or other who is interested in exploring the possibilities brought on by the opportunity to create ubiquitous sound experiences using samples, data and information.

Although many technologies will be used, no previous technological knowledge is required. The workshop is for everyone. Of course, people with additional technological expertise will be able to appreciate additional levels of detail.

What you need

Your laptop. All your smartphones (iOS or Android).

Optional: sound-related technologies (digital recorders, effects, controllers, software, microphones…).

Publication and digital distribution

Read/Write Reality Ubiquitous Sound will be also a digital publication about the results of the workshop, including as authors also all the participants.

Produced by AOS (Art is Open Source) in collaboration with Teatro Rebis, Youbiquity and Macerata Racconta, this publication will include the critical theoretical approaches used during the workshop, exercises, as well as the description of the techniques and tools used. A digital book for designers, artists, architects, hackers, communicators, ethnographers and developers wishing to expand their perspectives on ubiquitous publishing.

Real Time Cairo: the real-time digital life of the city of Cairo, Egypt

Real Time Cairo: the real-time digital life of the city of Cairo, Egypt

Launch the project by clicking HERE

Real Time Cairo screenshot

Real Time Cairo screenshot

To harvest all the data generated through social networks in a city.

Each day we use social networks such as Facebook, Twitter, Instagram, Foursquare as public spaces in which we express ourselves, our feelings and emotions, our perception of the spaces around us, our desires, wishes and expectations.

Or, such as in Cairo, Egypt, right now, to express our dissent, to enact our freedoms, and to inform of the difficult situations in which we find ourselves and with the communities we live in.

But social networks are not public spaces.

They are privately owned digital spaces whose strategies and interests have nothing to share with public space.

Through hundreds (if not thousands) of years of development of our cultures we have learned to form our expectation about what is public space, how it works and what to expect from it. We have developed shared ways by which we have collectively shaped our idea of what is a public space, and how to change its rules.

On social networks this does not apply.

A simple, unilateral, change in the terms of service of any of these networks/spaces can radically change the ownership and modalities according to which the information that we publish and share is used and leveraged, possibly giving rise to its exploitation, censorship, business usage.

But everything in these networks/spaces is designed to augment our understanding of them as public spaces and, thus, to apply or expectation of how public spaces work to them.

So,

  • on one side, we perceive a public space in which to express ourselves, expecting that it will function according to our understanding of the ways in which public spaces work;
  • on the other side we have private spaces whose objective is to mimic the ways in which public spaces work, so that people use them, increasing business
real time Cairo

real time Cairo

Just as we did with the VersuS and ConnectiCity projects, Real Time Cairo captures the life of the digital city and visualizes it on a map.

The intent is to create the availability of tools by which to re-appropriate information published on what we perceive to be our digital public spaces, and to make it available for visualization, aggregation, etc: to establish a novel source of Open Data in our cities by gathering all the conversations that take place in our digital public spaces, to allow people to use them to understand their cities and to imagine shared practices and methodologies to use this information, making it accessible and usable by everyone, not only by social network service providers.

The interface shows

  • the map, with the information popping up as soon as people in Cairo publish it using Facebook, Twitter, Instagram, Foursquare (information that is not geo-referenced using the networks’ tools is harvested as well, but it does not show on the map)
  • a tag cloud of the most used words in the last 80 minutes
  • some statistics showing the number of messages and users who published information in the latest 80 minutes, and in what languages they write
  • a timeline, allowing the comparison of the number of messages posted during each hour of the last few days

We are still developing elements of the project (more updates will come soon) including:

  • an API through which to download all of the dataset (or only parts of it, selected by date range, source, keywords etc)
  • a visualization to show the human geography/topography of the city

The source code will be released under GPL as soon as we are able to finish adding these first few pieces.

Stay tuned!

Real Time Cairo

Real Time Cairo

Cairo, Egypt, on Twitter, Facebook, Instagram

Cairo, Egypt, on Twitter, Facebook, Instagram

 

Leaf++ on Leonardo Electronic Almanac

Our text about Leaf++ has been just published on Not Here Not There Part 2, Leonardo Electronic Almanac, Volume 19 Issue 2.

LEAF++: Transformative Landscapes
+ Interview, Statement, Artwork [LINK]

by Salvatore Iaconesi, Luca Simeone, Oriana Persico, Cary Hendrickson

“Landscapes mark a certain terrain, delineating here from there and beyond: places and spaces of reference for our memories, our actions, helping us to place our being in specific moments. From private gardens to rolling hillsides in the country, landscapes represent overlapping visions of what is universal and at the same time personal. A multi-sensory experience we build and engage in anywhere, at any time. LEAF++ is a presentation and research project, an attempt at combining what we experience as individuals and, through the use of open source augmented reality (AR), adding on to the layers of what can be experienced by the wider community of users of a given site. The AR system combines computer vision techniques and allows for interactive experiences, educational contents and other information to be attached to leaves and other parts of plants: users contribute their own images, video, audio or other multi-media information to existing data available such as information about the plant’s habitat and biophysical characteristics. The LEAF++ project has been applied to not only identify Gilles Clément’s Third Landscape but also create a transformative view on urban space and foster experiential, analytical and aesthetic links between a diverse audience’s visions, perceptions and behaviors.

 

Full article is available for download as a pdf here.

Volume 19 Issue 2 of Leonardo Electronic Almanac (LEA) is published online as a free PDF but will also be rolled out as Amazon Print on Demand and will be available on iTunes, iPad, Kindle and other e-publishing outlets.

 

LEA Volume 19 Issue 2
Volume Editors: Lanfranco Aceti and Richard Rinehart
Editors: Ozden Sahin, Jonathan Munro and Catherine M. Weir

ISBN: 978-1-906897-23-9
ISSN: 1071-4391
Date of Publication: April 15, 2013
Number of Pages: 221

AOS at 18th International Conference on Urban Planning and Regional Development in the Information Society GeoMultimedia 2013

AOS will be at REAL CORP 2013, the 18th International Conference on Urban Planning and Regional Development in the Information Society GeoMultimedia 2013.

20-23 May 2013, Rome, Italy

House of Architecture
Piazza Manfredo Fanti, 47, 00185 Roma

Our contribution:

Interweaving the digital and analog lives of cities: urban sensing and user-generated cities

Abstract

A research process lasting from 2009 to 2012 has conceptualized, designed and implemented multiple tools and strategies to experiment novel forms of technologically-supported urban interaction. The goal of this process has been to understand the rituals which have started to shape contemporary citizens’ perception and performance of urban public and private spaces. An ethnographic approach has been used to gather insights about these emergent rituals, affecting the ways in which people have transformed the ways in which they work, learn, relate, consume, travel and entertain themselves in the city.

With the active collaboration of public administrations, organizations, citizen groups, tourist operators and research teams these practices have been enacted in the cities of Rome, Turin, Trieste, Cosenza, London, Berlin and Hong Kong for variable amounts of time. Engagement and results have been formally gathered, observed, processed and measured, allowing the research team to both explore the current scenario and envision new ones.

Real-time content harvesting from social networks, natural language analysis, geo-referencing/geo-coding/geo-parsing technologies, expert systems and ubiquitous technologies such as smartphones, custom electronic devices and conceptual consumer products have been employed to explore the ways in which people are and will be able to: perceive and understand their urban surroundings; access services and information; co-produce knowledge and distributed intelligence; collaborate in the creation of shared projects and city-governance practices; create and maintain peer-to-peer infrastructures for connectivity, commerce, services and culture.

This paper will present the initial analysis – including previous research taken into account in the fields of urban sensing, citizen science, urban planning, urban infrastructure management, urban environment perception and more –; the methodologies, both shared and project-specific, used to conceive, design, implement the prototypes and to measure their effects; the reports about each project in the aforementioned cities, including their usage on-the-field as well as elements of urban and digital ethnographic observation and user experience analysis; a description of a scenario for further research and for the production of service and product concepts, some of which are already in-progress, in the areas of the arts, culture, tourism and city administration.

What emerges is the opportunity to create multi-layered interactive landscapes in urban contexts which allow city dwellers to communicate, collaborate, govern their city, exchange knowledge and information, consume, entertain themselves, produce and distribute services.