“TO DRAW A LINE YOU NEED A PEN, TO HOLD A PEN YOU NEED A HAND, BUT WHAT IS BEHIND THE HAND IS A DEBATABLE MATTER.”

ITALO CALVINO

What is an autobiography?

Are we writing an autobiography by leaving our traces on social networks, every day? What about the large quantities of other digital traces we leave behind in our lives?

What happens when non-human and algorithmic subjects/entities come into play, increasing the complexity of our interactions and influencing the process of construction and perception of the self?

We progressively expose ourselves more, every day, consciously and unconsciously, with consequences that are difficult to grasp: about time, identity, memory, rights.

For the Streaming Egos project we have decided tackle this topic and analyse the uncertain territories/boundaries of the “I” and “self” asking ourselves who (and/or what) is the author of our autobiographies in the hyper-connected and syncretic world we live in.

This article is an attempt to summarize and put together the elements emerged from the process.

Rather than conclusive answers, we found a series open questions, leading us to establish a new research field in which we will progressively try to explore the mutation of self narratives and, more precisely, the meanings and possible consequences of a contemporary, technologically mediated autobiography.

Writing an autobiography: from authorship to curatorship. That is, the narrative and processual nature of the Self

The Self is a puzzle that philosophy – and more recently psychology and cognitive sciences – has always dealt with.

Approaching this complex matter will lead us to frightening and fundamental questions concerning our existence, consciousness, the ways in which we perceive space and time, and how we understand, produce and transfer knowledge: Do “I” exist? Does the external world exist? What is the “subject”? What is the “object”? What is memory? Are my memories “true”?…

The list of questions is much longer and the puzzle is unsolvable in one single self-conclusive image. But with a few simple observations we can try to inspect them by watching our behaviors, which is a good starting point:

  • The self is a membrane: it acts simultaneously as the separation and meeting point between the subject and the outside world, and as such it allows us to establish relationships (between the I, the others, the world);
  • The self changes but also remains the same: for example, we can recognise ourselves in an old childhood photo as opposed to the adults we have become;
  • Despite the great complexity that the self raises as a philosophical object, we constantly and spontaneously “speak” about our self – everyday, since our childhood.

In his paper “The Self across Psychology: self-recognition, self-awareness and self-concept” (1), psychologist Jerome Bruner argues that the self is a matter of language. We can develop it because we have at our disposal a sort of “natural language for non artistic autobiography” (connective, temporal and causal signals etc…) we use to articulate a first “discourse” of the self, even episodic or contingent: these skills are developed around the first two to three years of our lives. But to give shape to an autobiography (under written or oral form) is a different process: we need to put the self in a larger context (which includes culture, our beliefs, our relations and us). We have to “tell a story” about ourselves, which means that we have a narrative problem to solve: not only create a story, but a story which makes sense (to us and possibly to others).

This is why, according to Bruner, we can consider the self as a narrative process, rather than an “object”, which allows us to create coherent narratives about our lives along space, time and cultures.

The self is strictly connected to memory processes and identity.

Searle and autobiographical memories

Searle and autobiographical memories

Not surprisingly, autobiographical memories are recognized as an important criterion of personal identity. Over the past two decades, as S. Smith and J. Whatson demonstrate in “Reading Autobiography. A Guide for Interpreting Life Narratives” (3), three terms have become central in autobiography:

  • performativity;
  • positionality;
  • relationality.

In theories of performativity, identity is seen as something “enacted and reiterated through cultural norms and discourses […] an effect of storytelling” (4). Positionality shifts the attention to the cultural and historical placement of the subject, and “subject positions” are viewed as “effects of social relations whose power is distributed unevenly and asymmetrically across difference” (5). Relationality refers to the idea that “the narrator’s story is often refracted through the stories of others” (6) and emphasizes the subject’s lack of autonomy.

All of these terms criticize the universal, stable and autonomous idea of individual, and shift the focus to the idea of the subject in process and in context.

We are never alone when we write our life story. Other people are always with us, with their presence, influences, relations, interactions, shaping not only our behaviours, but also what we remember, what we feel as relevant, important, worthwhile, changing the ways in which we express it, for whom, and the contracts we establish by expressing ourselves: what to show, what to hide, how to interpret it, how to shape it.

We don’t create our autobiography out of nothing. Rather, the story’s outline and plot are the result of numerous impulses and micro/macro events (conscious or unconscious), relations, power relations, one’s own memories and memories of others.

Our autobiographies, just like the self, are a process: the result of a constant remix. Rather than in “originality” authorship finds its basis in “composition”, in the continuous process of “sewing the pieces together” that the self operates in order to give shape to the outline and plot of our lives, turning them into narrative material with which we can mould identities. More than “authors” we are “curators” of our own story, which turns out to be a fragmented object formed by a mixture of elements and materials, acted by multiple subjects. The only seemingly compact definition of autobiography (“writing one self’s story”) shows us a polymorphic and recombinant nature, which nowadays intertwines with (and lives through) new ubiquitous, technologically mediated dimensions.

A new space exists in which we are confronted with unprecedented actors and materials: the software and the algorithmic matter. Most of the time their logic is opaque and unaccessible to us, from the ways in which algorithms watch and classify us; to the simple knowledge and perception of all the data we produce; to the algorithmic influence on our perceptions which comes about as software agents become able to shape our media environment around us, according to logics which are beyond our grasp and understandings.

Finding new paths to access, interact and play with it, is a new challenge for contemporary human beings and societies which directly affects the possibility to build, own and fully understand the processes behind the construction of the self, our identities, our intimate relation with time, our personal and collective memories.

GhostWriter and Algorithmic Autobiography: a non human author, a new literary genre

For the Streaming Egos project we decided to explore an evolutive tension.

On the one hand, we confronted with the emergence of new forms of writing and of non-human authors, which are already influencing our relations and the ways in which we can perceive/perform/build our self-narratives. On the other hand, we tried to deal with the opaque and hidden nature of these writings and authors, mainly the software agents, algorithms, artificial intelligences which fill many aspects of our ordinary experience, and the organisations which control them.

Whether we realize it or not, whether we want it or not, a number of subjects and entities continuously keep track of the digital traces we produce, constructing multiple versions of narrations of our lives, each with different focuses, parameters, points of view, perspectives.

These are, to all effects, biographies.

Even more: they are two times auto-biographies. Auto, because they are automatically collected, processed and composed. And auto, because we produce and express these bits of memory ourselves in our daily lives, through our ordinary performances, like entries in a ubiquitous diary.

If we can collect all of these bits, all these episodes, all of these digital traces in our ubiquitous diary, we can imagine to produce a novel form of autobiography. Currently, multiple algorithms do exactly this, collecting all of these bits about ourselves, classifying them, organizing them by time, topic, emotion, behavior, patterns, types, focuses and more.

These algorithms are the “ghostwriters” of our autobiographies. They already exist, as our ubiquitous diaries exist: they are just invisible to us.

We wanted to make them visible.

We wanted to create a new literary genre allowing this new form of writings to emerge, with al possible consequences: the Algorithmic Autobiography.

We needed a needed an non-human author author capable to do so.

GhostWriter

GhostWriter

How it works

GhostWriter embodies this new type of non-human author. It observes our digital traces, interprets them to extract the patterns of a story, and finally hands them back to us in a hierarchic structure under the form of proper publications: new types of books.

Imagine commissioning your autobiography to GhostWriter.

Doing this would be as simple as granting it with the possibility for a total invasion of your privacy: give it access to your email, bank account, social network profiles, mobile phone, your wearable devices, the network connected devices in your home, your blog or website and more. In short: to all the possible data you originate.

GhostWriter will constantly collect all of this data, analyse and process it by using a series of different techniques such as Machine Learning, Natural Language Analysis, Emotional Analysis, Network Analysis, to understand the topics, relations, emotions you express, the places you visit and the times you spend there, transforming the non-structured data flow coming from your daily life into a structured one: usable information to understand and describe you.

If we think about it, GhostWriter is not very different from its humans colleagues: it observes and analyses us; its identity remains hidden; the book it writes will bear our names, playfully preserving/revealing the ambiguous notion of “authorship” and the obscurity of algorithm.

The SMS: a model for autobiographical memory

How do humans create their autobiographies?

We had a concept, the tools and the technologies to create it. But to finalize the algorithm, and make the GhostWriter able to compose the plot of an autobiography, we needed a model for autobiographical memory. We started to search within several theories pertaining to the realm of psychology, cognitive psychology and neuroscience specifically addressing addressing this issue.

the SMS Model

the SMS Model

Among the many possible models, we adopted the SMS -Self Memory System: a model created by scholar Convey and Christopher W. Pleydell-Pearce (7), which interestingly describes autobiographical memories storage and processes in terms of information architecture. It was what we were seeking.

A reviewed version of the model is described in the article “The Self and Autobiographical Memory: Correspondence and Coherence”, published in 2004 by Martin A. Conway, Jefferson A. Singer and Angela Tagini and available online at this link.

In the new SMS model autobiographical memory is placed at the intersection of the Long Term Self and the Episodic Memory System (EMS), both in a complex and reciprocal relation with the Working Self (8):

“[…] memories represent information about progress in goal attainment and therefore have to reflect reality to at least some extent. On the other hand, the working self operates to make knowledge and memories that confirm and support current self-conceptions highly available and […] may also operate to distort and/or inhibit memories that undermine the current self” (9).

The EMS is the stream of memory as it emerges. It contains experience-near event specific sensory-perceptual-cognitive-affective details [..]. These memories are of comparatively short time-slices of experience (seconds, minutes, hours) and are formed at event boundaries when a change in goal-processing occurs (cf. Zacks et al., 2001). During the course of a day many episodic records will be formed by the working self and many of these will remain available to recall for short periods of time.” (10)

The Long Term Self is the combination of the Autobiographical Knowledge Base (KB) and the Conceptual Self (CS).

The KB is a set of information on what the self is, was and can be, categorized by authors into three broad areas:

  • the General Events, which are “categories of events linked across relatively brief time periods or organized by a shared theme, e.g., first-time experiences, academic meetings, and so forth”(11);
  • the Lifetime Periods, which are “large units that reflect particular overarching goals and activities, for example the early years of a marriage, graduate school, or a period of financial hardship”(12);
  • the Life-story Schema, which is the “individuals’ understanding of how the normative life story is constructed within our culture” (13).

The CS consists of “non-temporally specified conceptual self-structures. […] The units of the conceptual self are socially-constructed schemas and categories that help to define the self, other people, and typical interactions with others and the surrounding world.” (14). It is categorized into “Beliefs”, “Possible Self” and “Personal Scripts”, schemas and categories which are “drawn largely from the influences of familial and peer socialization, schooling, and religion, as well as the stories, fairy-tales, myths, and media influences that are constitutive of an individual’s particular culture” (15).

The CS interacts with the KB contributing to the organization of its hierarchical units and thematic grouping of lifetime periods and general events.

Here is how the authors explain the process of retrieval in the SMS model:

“When the Working Self initiates a retrieval model that accesses the autobiographical knowledge base of the long-term self, generative retrieval usually begins at the general event level (Haque & Conway, 2001). When these events intersect with lifetime periods and the life story schema, a temporal and goal sensitive framework can guide search through specific episodic events stored within the episodic memory system. The pattern of activation that synthesizes the structures of the autobiographical knowledge base with specific episodic images from the episodic memory system yields the specific autobiographical memories that reach awareness.” (16)

In GhostWriter the complex interactions and structures described by the SMS model are used  to schematize our digital traces:

  • by capturing and storing events as they occur in the EMS;
  • by searching for patterns in them, across topics and times, to discern the General Events;
  • by searching for temporal patterns and recurrences to describe the Lifetime Periods;
  • by searching for patterns and recurrences in it, to construct and evolve the Life Story Schema;
  • by looking for particular correlations and recurrences to highlight Beliefs;
  • by experimenting with promising or trending beliefs, and how they may interconnect and interrelate with each other, creating resonant configurations, to describe the Possible Selves;
  • and by juxtaposing both of these last two elements in recombinant ways to represent the Personal Script, and to follow its evolution through time.
GhostWriter, the Installation

GhostWriter, the Installation

The installation

An Algorithmic Autobiography generated by GhostWriter is composed by the three main structures of the autobiographical memory in the SMS, which here become three different interactive info-aestetic visualisations:

  • The EMS – Episodic Memory System;
  • The KB – Knowledge Base;
  • The CS – Conceptual Self;

In the installation the three info-aestetics are presented under the form of three actual books:

The books contain the info-aesthetic visualisations of the three modules of the autobiographical memory representation.

This is a detail of the visualisations:

As we can observe from the EMS data stream:

GostWriter: screenshot from the EMS

GostWriter: screenshot from the EMS

the algorithm does not simply re-post our data data (for example, an email or a tweet), but re-writes it according to its algorithmic logic and generating entirely new text. In effect, when an algorithm processes a text it does not read it as a “normal” human being would, but, rather, it disassembles it, eliminates certain words, changes some other ones, dissects it and recomposes it with different logics, in order to be able to analyse it: in the visualisations these “new texts” were shown, the way in which the algorithm processes them.

From a series of interactions and comments by the visitors at the Streaming Egos exhibit, we understood that this example made the concept of “non human writing” explicit and tangible and, consequentially, they were immediately open to recognising it as a new “literary genre”.

GhostWriter, the Diagrams

GhostWriter, the Diagrams

Three diagrams, printed on forex, explain how GhostWriter works:

  • The Model. A revisited version of Conway’s Self-Memory System, with the added element of the algorithmic interference produced by digital interactions on autobiographical memory.
  • Me/Me. A representation of the self and the technologically-mediated self, which coexist and influence each other.
  • Points of View. A description of how the data which is consciously and unconsciously produced in our digital interactions is processed and filtered, producing representations of us which vary depending on the point of view. The representation of my digital self which I can access is different from the ones my friends, a firm or other subjects and entities on the net may access. Algorithms are what determines the logic of representation.

Finally, hundreds of printed pages of the encrypted logs coming from the algorithm show the pure algorithmic, non-human and partially unaccessible nature of this autobiography.

The installation creates a new space for perception: in the concreteness of new cultural artifacts – interactive books we can now circulate, manipulate, criticise and understand – Algorithmic Autobiography becomes a new possible gateway to a technologically-mediated self.

GostWriter, Streaming Egos exhibit: discussing Algorithmic Autobiography with the audience

GostWriter, Streaming Egos exhibit: discussing Algorithmic Autobiography with the audience

(Here a photo gallery from the exhibit in Dusseldorf)

Capture All? Lifeloggings, Personal Informatics and autobiographical memory

In his paper “Autobiographical Memory, Behavior Change, Tecnology” (17) Artie Konrad, human-computer interaction and Ph.D. in Cognitive Psychology, investigates the influence of new technologies on autobiographical memories.

Two concepts of particular interest explore the tension between the self, technologically mediated memories and the imaginary of “total capture”: lifelogging and personal informatics.

Lifelogging is the process of tracking personal data generated by our own behavioral activities.

Here is how the author describes it:

“Imagine a future where we could capture everything that we see, touch, and do. We would forget nothing, and our fallible, natural memories would be replaced by prosthetic digital memories. We’d forget no names, no appointments, nothing that we had previously learned. We’d never get in arguments that question our actions because we’d have accurate records to review and resolve our disputes. This is the vision of lifelogging (also called “quantified self”, “life caching”, “self-tracking”, “auto analytics” etc.), which is an attempt to digitally capture every bit of information we touch, and every event we experience.” (18)

More focused on individual memory process, personal informatics is a step beyond:

“The terms Personal Informatics was first coined by Ian Li to describe a growing class of systems that were being developed to improve upon lifelogging (Li, Dey, & Forlizzi, 2010). While lifelogging attempts to capture as much information about one’s life as possible, personal informatics focuses on capturing personally relevant information with the express intention of retrieving this information later. In other words, this information is about and for the person. By capturing veridical records and facilitating subsequent retrieval of these records, personal informatics systems could be powerful tools to help reminiscence, reflection and behavior change.” (19)

The author categorizes personal informatics systems as:

  • Event-Specific Systems;
  • Technology Mediated Reminiscence;
  • Technology Mediated Reflection;
  • Extraction Systems.

Leaving critical reflections about this topics for the conclusion, what is interesting here is to reflect about the role and possible shapes of technologies specifically designed to support memory, trying to understand what are the more diffused and interesting technologies, how people currently use, perceive and are influenced by them.

Open questions and possible consequences

After the Streaming Egos exhibit, which was featured in Dusseldorf on January 16-17th 2016, we had in our hands a new, perturbing, object.

If we have demonstrated that non-human forms of writing are already in place influencing our perception and the way in which we construct the self, what (social, anthropological, political, legal) consequences can the existence and wide accessibility of an “Algorithmic Autobiography” have on people’s lives?

We want to conclude this article trying to expand this question.

Recombinant, human/not human identities: a possibilistic vision of the “autobiographical pact”

When we relate to an “autobiography” we accept to relate to “someone” which is telling us a “non-fiction”, “truthful” story about “his/her” life: this is what scholar Phlippe Lejeune defines as the “autobiographical pact” Which, he argues, is the condition for an autobiography to exist and to be considered as “valid”.

It means that we recognize this “someone” (the “author”) as a subject with a defined identity, and with precise responsibilities to us (the readers). In a word: we establish a “contract” (with social and even possible legal consequences).

In the context of the Algorithmic Autobiography, unprecedented types of subjects (authors) can come into play. A couple, a class of students, a group of friends, a collective of artists, a company, an institution and so on, could decide to feed the GhostWriter, collecting and using their data sources: the result would be an “autobiography” attributable to and directly written by the couple, the class, the group, the collective, the company and the institution itself, here respectively recognised as the “authors” and as single defined identities. At the same time, one single person with multiple digital identities could publish multiple autobiographies (multiple, coexisting, even contradictory versions of the self) theoretically without violating the pact.

What we see here is a shift from a concept of identity based on the compact vision of “individual” to a more fluid, polymorphic and recombinant structure: a “multividual”, as prof. Massimo Canevacci Ribeiro calls it.

Individual is at the very basis of societies, in particular western societies: our ID and all sort of contracts we are allowed to stipulate are based on it.

  • What happens when, starting from existence of a new literary genre and interacting with new types of cultural artifacts (books/publications in this case), we deal with new “multividual” authors?
  • What kind of new social interaction we can imagine (or need) to validate the autobiographical pact as described by Lejeune?
  • What are the consequences at psychological, anthropological, political and legal levels, on people and society?
  • What does a polymorphic, recombinant, multividual based ID look like? How can we design and validate it?
  • Following the same logic, can we imagine new type of “contract” based on multividuals? What do they look like? What are the consequences on property, work, marriage and so on?
  • What are the rights od a multividual?

Things can get even more complicated.

We are now able to disseminate the environment with sensors.

We are building Smart Cities through this possibility, as well as smart homes, smart rural spaces, and smart schools, workplaces, kitchens, hospitals, brothels and bodies. With the Internet of Things we are populating our houses as well as our imaginaries with new connected objects and services. We are effectively transforming all of these objects/processes/products/services into potentially sentient agents, into potentially new types of subjects. This means that not only new types of human subject come into play, but also non-humans ones: a apartment building, a square, a wood, a river, a fridge, our dog can now write their own autobiography and tell us their own life story, just like we do. The GhostWriter will not make any difference, because from its point of view there is really no difference: humans and not humans subjects are (or can easily become) data generators.

Unlike smart services, an autobiography is not something we just buy or consume: or better, the act of buying and consuming an autobiography culturally implies a reciprocal relation between the authors and the reader. Otherness is added to the equation, in potentially disruptive ways.

  • What are the consequences (psychological, anthropological, political, legal) of a non-human or interspecies autobiography?
  • Do non-human entities/subjects have rights?
  • If yes, what kind of rights?
  • What are the relations and roles formed through this further form of autobiography? After all, we would be the ones designing the sensors, writing the algorithms, establishing what is sensed and what is not, deciding what gets stored or is relevant, and more. What happens, from this point of view, when a new “book” comes out: “The Autobiography of a network-connected lawn”?

On top of that there is software and authorship.

An Algorithmic Autobiography is written by all this different types of (multividual, human/not human) authors as well as by the GhostWriter: an actual algorithm. This lead us to the controversial realm of robo-ethics:

  • Who is responsible (even legally) for our algorithmic autobiography? The software who writes it, us or both?
  • What tools do we have (or we can design) to discern responsibilities, attributions, implications, boundaries and their progressive mutations? And: do we want to design them?
  • Is a sort of “contract” needed between us and the GhostWriter ? If yes, what does it look like?

Questions of time: a continuous present

Our brain is not designed to store or remember everything. It is quite the opposite: we carefully select the memories we need and we want; we choose what to remember and what to forget, in complex ways; we craft our memory and we decide what is public, private, intimate, what to show or not to show in an autobiography. We need to forget and we have the right to be forgotten (or do we?).

Algorithmic Autobiography describes a continuous present in which we potentially access all our memories, all at the same time, constantly, and in which the algorithm selects them and passes them to us.

A hint of this is represented by the “Facebook memories”, which are periodically brought to our attention by the popular social network. They are an everyday, consistent example of this kind of process: what happens when I get my daily “Facebook Memory” which sadly corresponds to a painful remembrance which I really didn’t want to remember; so painful that I commit suicide after seeing it? Who is responsible? Did Facebook kill me? Could an algorithm be designed to kill me in this way? Do we need a contract for this? Can I do legal action? And so on. This type of issue and the model which it describes, brings up infinite critical questions.

This is of course problematic.

  • Are we able to bear this as human beings?
  • What are the risks of being exposed to our and others’ memories, constantly?
  • How does this affect our relation to time? To the perception of our past and possible futures?
  • What about the right to be forgotten, both from the legal and existential point of view?
  • How does a society get ready for this type of change? (since we don’t really seem ready yet)

Privacy, data ownership and possible balance: Ubiquitous Commons and Algorithmic Autobiography

We have described GhostWriter as a “total invasion of privacy”.

This is mostly because until now our relation to the data we produce is mediated by operators and platforms which own our data, because the algorithms are opaque to us, and because we don’t know what data is harvested from us, how it is processed and how it is used. On top of that we don’t really have any possibility to express and enforce how we’d wish this data to be captured, processed and used.

Using the metaphor of the “ubiquitous diary”: not only at the moment it is invisible to us; we don’t really own it and we largely don’t know we are writing it, who will be able to read it and for what purpose.

In our practice, we confront with these kind of issues through the Ubiquitous Commons project.

Ubiquitous Commons is a research project which tries to confront the current scenario by creating a technological, legal and cultural protocol/toolkit. The starting point of the research is the creation of a p2p infrastructure in which people can describe identities and relations among identities, creating high quality relational environments in which to express how data is used, to be able to technologically, legally and collaboratively enforce these expressions, by using the protocol/toolkit.

There are a number of projects trying to confront with these issues, each in its own ways and with its own philosophies, but we will refer here to the Ubiquitous Commons, because we’re familiar with it, and because we believe in its approach:

  • Is it possible to apply Ubiquitous Commons in the context of the Algorithmic Autobiography?
  • With what results?

This are all questions we want to explore in the near future.

Notes

  1. Bruner, J. (1997) “The Self across Psychology: self-recognition, self-awareness and self-concept”, in Annals of The New York Academy of Science, vol. 818
  2. Searle, J.R. 2004, Mind: A Brief Introduction, Oxford University Press, Oxford and New York.
  3. Smith, S. and Watson, J. 2010, Reading Autobiography. A Guide for Interpreting Life Narratives, University of Minnesota Press, Minneapolis and London.
  4. Ibidem
  5. Ibidem
  6. Ibidem
  7. Conway, Pleydell-Pearce, “The Construction of Autobiographical Memories in the Self-Memory System”, 2000
  8. The working self, often referred to as just the ‘self’, is a set of active personal goals or self-images organized into goal hierarchies. These personal goals and self-images work together to modify cognition and the resulting behavior so an individual can operate effectively in the world.” (from Wikipedia)
  9. Conway, M. A., Singer, J. A. & Tagini, A. (2004) The Self and Autobiographical Memory: Correspondence and Coherence. Social Cognition, 22(5), 491-529.
  10. Ibidem
  11. Ibidem
  12. Ibidem
  13. Ibidem
  14. Ibidem
  15. Ibidem
  16. Ibidem
  17. The paper is fully accessible at this link: http://www.artiekonrad.com/wp-content/uploads/2012/12/Artie-Konrad-Qualifying-Paper.pdf
  18. Ibidem
  19. Ibidem