I was always facinated by We feel Fine by Jonathan Harris and Sep Kamvar.

The possibility to design using emotions has always intrigued me and, yet, it seemed like an unachievable task.

The notion of being able to identify emotions in an automatic (or assisted) way is quite a complicated activity, and it needs to enact complex classifications and profund analisys of cultures, contexts, noises, ironies.

FakePress - Publishing Emotions

FakePress - Publishing Emotions

Theory

Some theories come to our rescue while approaching the enormous feat of identifying and classifying emotions.

Plutchik‘s psychoevolutionary theory is based on ten postulates that identify basic emotions as entities derived from primordial behaviours, with actual, complex, emotions being represented as linear combinations of the basic ones, disposed in 2D and 3D geometries that highlight similarities and opposites among them.

Paul Ekman researched on models for identification and classification that have been very useful in the analisys of facial expression.  Our faces are one of the most evident places in which our emotions show, and Ekman’s research is very important from a cross-cultural point of view, as he identified a limited set of emotion/facial expression pairs that are virtually unmodified across all cultural extractions.

Magda B. Arnold provided an incredible surge in interest on the theories of psychology of emotions by researching them in the 1940s, when the behavioural perspectives were the main trend. Her research on the interrelations between emotions and the tendency to perform actions is somewhat outdated, but a milestone in the scientific history on these themes.

Nico Frijda focused, like Ekman, on the connections between facial expressions and emotions, in somewhat of a behavioural perspective in which emotions were the indicators of the readiness and tendency to perform actions.

Carroll IzardJeffrey Allan Gray, John B. Watson and Jaan Panksepp provided a somewhat more physical form of interpretation of emotions, creating their own schemes for classification originated from the idea of basic emotions as being hardwired into our bodies and primordial mind setups. Watson specifically dedicated several chapters of his books to the analisys of emotions as hereditary modes of response.

William James and Carl Lange produced a theory on basic emotions according to which within human beings, as a response to experiences in the world, the autonomic nervous system creates physiological events such as muscular tension, a rise in heart rate, perspiration, and dryness of the mouth. Emotions, then, are feelings which come about as a result of these physiological changes, rather than being their cause. James and Lange arrived at the theory independently. Lange specifically stated that vasomotor changes are emotions. A bodily connection with emotion and a classification scheme deriving from it.

William McDougall produced a theory intercnnecting emotions and basic instincts, in main opposition with Watson’s behaviourism. According to McDougall, behavior is not simply a response to a stimuli but is goal seeking and purposive. For McDougall, a person’s emotional core was stable and unimpacted by learning.

O. H. Mowrer presented some interesting results on the possibility of interpreting emotions through the lens of a learning process. His exaples were particularly effective on guilt and other forms of emotion that were easily imposed by cultural contexts.

Silvan Tomkins focused his research on understanding the level of neural activity connected to the activation of emotions. His Freudian derives were originated from the idea of the behavioural importance of the mechanisms generating from “lacking” something. His classification of nine pairs of affects depict this idea.

Oatley and Johnson-Laird assume in their theory, called by them “communicative theory of emotions” (Oatley & Jenkins, 1996, p. 254), a hierarchy of parallelly working processing instances, which work on asynchronously different tasks.  These instances are coordinated by a central control system (or operating system).  The control system contains a model of the entire system. This is a semantic theory of emotion that is particularly interesting for our purposes, as it is specifically designed to be implemented on a computer.

Ortony, Clore and Collins assume that emotions develop as a consequence of certain cognitions and interpretations. Their theory exclusively concentrates on the cognitive elicitors of emotions. They postulate that three aspects determine these cognitions: events, agents, and objects. The main objective of their research is to investigate the possibility to design a formal system or a computer that is able to draw conclusions about emotional episodes which are presented to it.

Roseman bases a theory on a model in which five cognitive dimensions determine whether an emotion arises and which one it is. The first dimension describes whether a person possesses a motivation to a desired situational state or a motivation away of an unwanted situational state. The dimension thus knows thus the states “positive” and “negative”. The second dimension describes whether the situation agrees with the motivational state of the person or not. The dimension thus knows thus the states “situation present” or “situation absent”. The third dimension describes whether an event is noticed as certain or only as a possibility. This dimension knows the conditions “certain” and “uncertain”. The fourth dimension describes whether a person perceives the event as deserved or undeserved, with the two states”deserved” and “undeserved”. The fifth dimension finally describes, from whom the event originates. This dimension knows the states “the circumstances”, “others” or “oneself”. From the combination of these five dimensions and their values a table can be arranged (Roseman, 1984), from which, according to Roseman, emotions can be predicted.

For Scherer five functionally defined subsystems are involved with emotional processes.  An information-processing subsystem evaluates the stimulus through perception, memory, forecast and evaluation of available information.  A supporting subsystem adjusts the internal condition through control of neuroendocrine, somatic and autonomous states.  A leading subsystem plans, prepares actions and selects between competitive motives.  An acting subsystem controls motor expression and visible behaviour.  A monitor subsystem finally controls the attention which is assigned to the present states and passes the resulting feedback on to the other subsystems. Scherer is especially interested in the information-processing subsystem.  According to his theory this subsystem is based on appraisals which Scherer calls stimulus evaluation checks (SEC).  The result of these SECs causes again changes in the other subsystems.

A very interesting approach comes from Aaron Sloman, according to whom emotions are not independent processes, but develop as emergent phenomenon from the interaction of the different subsystems of an intelligent system.  Therefore, no necessity exists for an own “emotion module”.  A look at psychological emotion theories leads Sloman to the conclusion:
“Disagreements about the nature of emotions can arise from failure to see how different concepts of emotionality depend on different architectural features, not all shared by all the animals studied.” (Sloman and Logan, 1998, p. 6)
Sloman’s approach intentionally disregards physiological accompaniments of emotions.  For him these are only peripheral phenomena: “They are peripheral because essentially similar emotional states, with similar social implications, could occur in alien organisms or machines lacking anything like our expression mechanisms.” (Sloman, 1992c, p. 20)

Dacher Keltner focuses on a social approach to emotions. In his own words: “My own studies have focused on the social functions of emotion, arguing that emotions enable individuals to respond adaptively to the problems and opportunities that define human social living. Based on this approach to emotion, I have documented the appeasement functions of embarrassment, the commitment enhancing properties of love and desire, and how awe motivates attachment to leaders and principles that transcend the self.” This is a really interesting perspective, as it accounts for the possibility to analyze emotions from a point of view that encompasses complex interactions among social contexts, cultural backgrounds, emergent dynamics.

Design

The visualization shown on FakePress’s homepage represents the beginning phase of a design process.

It uses Plutchik’s classification and applies by leveraging the availability of several realtime search engines that allow our systems to fetch information from a multitude of sources on social networks, blogs, websites. Services such as Collecta, OneRiot and the APIs offered by Twitter, Facebook, Friendfeed and Google .

This first implementation had two main goals in mind:

  • the possibility to design and implement a system that is able to analyze great amounts of data in realtime to search, identify and classify “emotional data”, meaning data that can be interpreted as the expression of an emotional state
  • the identification of representation schemes that are characterized by a high degree of expressivity, accessibility and synthesis

What we wanted was a “device” that was able to present low-doses of highly expressive information obtained by processing great amounts of information that were collected as being “emotion expression related”.

This first prototype does just that:

  • data is priodically collected using the aforementioned realtime services (every 2 minutes, the system collects the last 10 minutes of information, keeping a history of 40 minutes in its databases)
  • processing the data, before storing it (also to avoid problems related to privacy and intellectual property), by selecting the information regarded as “interesting” and “relevant” (e.g.: sentences with a definite structure, such as “i feel…”, and others), anonymizing it (removing references to actual sources) and, finally, storing it in a parametricized way (e.g.: the whole sentences making up messages are not stored; the list of words, relevancies and phrase structures are)
  • use Plutchik’s classification, applied using about 2000 synonims (and growing) to identify and classify emotional states
  • grouping data together with its identification indexes to aggregate it into the synthetic, graphic form shown on the visualization

Plutchik’s classification is far from satisfactory from our point of view. But it served two purposes: on one side it was easy enough to implement (in its necessarily approximated form for what concerns the need to filter input data); on the other side it offered useful insights for the deisgn of a global process that we will use for the next steps.

Next steps

In the next steps this concept will be expanded in the following directions:

  • implement a different classification methodology to achieve a more significant result: we will probabily use Keltner’s or Sloman’s methodologies to obtain results that assess social and anthropological issues
  • expand mechanisms and systems to include the possibility to analyze multiple cultural contexts, different languages and diverse communities or individuals
  • include in the system a generalization of the input layer, to be able to connect devices that collect biometric information to be used side by side with written language and other types of interaction to infer emotional states
  • create a multi-community structure, where individuals can join in to share and disseminate their emotional states
  • expand the set of visualizations and representations, to include the creation of emotional maps, body stimulations, visual compass-like visualizations

We will head toward a broadened definition of augmented reality in which “ordinary reality” is augmented with emotional layers acting as “interpretative layers” on the world we live in. What happens when i can share my emotions with people, associating them to specific places, times, objects, environments, bodies? What kind of comunication forms can arise? What kind of education processes can this practice stimulate? How can cultural production, knowledge sharing, education, creativity, innovation, entertainment, commerce, ethics, politics and, in general, science  and arts benefit from such practices.

Stay tuned and collaborate with us while we start this research process.

FakePress welcomes contributions and participation from interested researchers, artists, creatives, interaction designers, academics, entrepreneurs and experimenters wishing to join in the research process.

[fundamental contributions for this text come from this webpage ]