Tag Archives: leaves

Leaf++ an augmented reality concert for leaves (the making of)

Posted on by
Reply

This video shows the setup of the Leaf++ augmented reality performance.

This is the second component of the Leaf++ project, a collaborative environment which uses computer vision and augmented reality to recognize and identify leaves, allowing to use them to share information and media (which can be added to them using a mobile application) and to use them in performative environments.

Leaf++ augmented reality concert for leaves

Leaf++ augmented reality concert for leaves

This video shows the second component. Here, leaves can be used on a lightbox to be recognized by a webcam and computer vision algorithm using the leaf descriptors created using the mobile application.

When leaves are recognized, their contours are used in a sond synthesizer algorithm and can be played live.

As you can see in different parts of the video, fingers moving the leaves around the lightbox are recognized and tracked as blobs, but they don’t generate any sound, which only uses the contours of the recognized leaves.

An interesting experiment we performed (a short video coming up soon) uses basic sine waveforms generated using the amplitudes and moments of the leaves contours (HU invariant moments): when you close your eyes and listen to the sounds generated by different types of leaves, it is possible to recognize them, and this is a really interesting area for research dedicated to people with disabilities who might use these techniques as a way to identify objects around them.

This part of the project is implemented using openframeworks and a video is coming up soon documenting a version for iPhone and Android

more info about Leaf++ on the website:

http://www.artisopensource.net/category/projects/leaf-plusplus/

Leaf++, a video of the first prototype

Posted on by
Reply

Leaf++ is an augmented reality and computer vision system which uses smartphones to recognize leaves found in the natural environment.

In this first promo video you will see some basic features of the system, embedded into a social network in which users share information on leaves: you can recognize a leaf using computer vision and write onto it.

 

People using Leaf++ on the a certain type of leaf will see the content written on it by previous users, thus enacting an ubiquitous social and knowledge network which is directly available in the natural environment.

Watch out for the next videos in which we will show an art performance in which leaves are used in an augmented reality concert using the shapes found in the natural environment to create live generative music and visuals.

Leaf++ will be officially presented at ISEA 2011 in Istanbul, and then at Mindtrek 2011, at DEOL 2011, and it has already been published on Parsons Journal for Information Mapping and on MediaDuemila.

Watch out for more presentations, publications and performances.

More info on

http://www.artisopensource.net

http://www.fakepress.it

Leaf++ at ISEA 2011, Istanbul

Posted on by
2

Together with FakePress Publishing we will be at ISEA 2011 in Istanbul to officially present for the first time Leaf++

Leaf++

Leaf++

A paper will be presented in the panel “Digitization of Biological Data”:

“Microscopic transformations: scientific visualization, biopower, and the arts by Roberta Buiani/ Design for Life by Meredith Walsh/ Leaf++ by Salvatore Iaconesi and Luca Simeone/ Mass Body Index: Bio-OS, a Biological Operating System by Mike Phillips, Birgitte Aga, Gianni Corino, Hannah Drayson, and Simon Lock/ Examining Issues of Body Image and Complex Regional Pain Syndrome within the Digital by Mark William Palmer”

And here is the abstract of the paper which I will be presenting together with Luca Simeone:

Natural interfaces and cross-medial technologies allow for the creation of new publishing paradigms in which the term “book” can be disarticulated and rearranged into unexpected forms, fostering new ways of interacting with data and information.
Leaf++ is the product of a research project that goes in this direction in which a prototypal interactive system involving computer vision, gestural interfaces, augmented reality technologies and cross medial systems to create a novel tool to experience botanical information about plants and their leaves.
In Leaf++ an interactive surface and a mobile application can be used to access information of a leaf. By placing the leaf on the interactive surface or by taking a picture of it using the mobile application, a computer vision system is able to recognize it (if it already has been added to a database) and to show available information sources including scientific classification and information, habitat information, world diffusion data, seasonality, curiosities, videos, images and stories regarding the leaf and the plant to which it belongs. Researchers and other forms of users are also allowed to add information to the system: by simply uploading texts, images, videos and geographical locations, they can contribute to the set of information available for the recognized leaf.
The overall interactive system comes out as a really significant experience, employable according to various usage scenarios that go from scientific research to education, to mobile and museum-based entertainment, which not only suggest possible effective uses for these ubiquitous, cross-medial technologies, but also enacts information access and knowledge sharing practices which are outstanding from the point of view of their usability, and of the cognitive approaches fostered by such direct, “augmented” methodologies.
Leaf++ is a cross-medial, augmented reality, multi-author, emergent, evolving and disseminated publication.