This dissertation lies at the intersection of (1) artistic experimentation in altering human perception and consciousness through creation of digitally-based or-enhanced interactive art and (2) inquiry into the nature and implications of human- computer interactions (HCI).1 It examines known gaps between these two broad areas (involving user, information, and interface) and questions the validity of current ontological and methodological approaches to reducing these gaps. It utilizes a design approach, focusing on space-time aesthetics in an attempt to connect theory and practice through the framework of cybersemiotics. It reflects on the implications and impact on consciousness of such an integrative framework.

My own entry into this convergent, transdisciplinary conversation was my practice as technoetic artist2 and information designer developing interactive hybrid environments.3 More specifically, the initially uneven reception and limited success of my installation Mixing Realities, a large dynamic and interactive hybrid physical- digital installation begun in 2008, was the impetus for my delving into the theoretical frameworks available to conceptually ground and potentially enhance my artistic practice.

Mixing Realities was developed to focus on time perception as subject matter and content, and as such its design (form4) attempted to hypermediate time perception by “seeking to make the digital interface [and physical environment] ‘transparent’ [to the user]. In this sense, a transparent interface would be one that erases itself, so that the user is no longer aware of confronting a medium, but instead stands in an immediate relationship to the contents of that medium” (Bolter & Grusin, 2000, pp. 23-24). My modest intention was simply, as Roy Ascott states,“to start a dialogue, to set feelings and ideas in motion” (Ascott, 2007, p. 98) concerning time perception and physical embodiment. Nonetheless, observing how so few people actually engaged with the artwork instigated a sense of frustration and puzzlement about how or what people were actually experiencing.The artwork was designed to combine physical and digital elements in a visually enticing, intuitive, user-friendly environment. Cyberperception5 was explored through intentional communication loops between input, output, and observer, following the principles of second-order cybernetics. Materials were carefully selected, considering their form and function, in order to optimize the user’s experience. Nevertheless, it soon became clear that the initial connection between the user and the interface was not intuitively occurring.

Today most of our interaction with the world is mediated by digital technologies. As users, we must frequently interact with a new interface (a gadget, computer application, website, digital artwork, etc.), often experiencing frustration and disconnection from the interface similar to what I observed in Mixing Realities. I also frequently experience a similar dissociation as a user-designer of interactive hybrid environments. My research suggested that the user’s dissociation from the interface is related to a disconnect from an unnamed, but ever-present, third element: information. Users either lack knowledge (information) about the interface itself, or they are unprepared for or simply uninterested in receiving meaningful information from the digital data made available through their interaction with the interface. That recognition led me to an awareness that something more than optimal feedback processes, information design, and user experience guidelines and rules are needed to more fully integrate the user into the work as I originally intended and to more adequately conceptualize the interactive processes at work.

This research attempts to connect theory and practice, to understand the limitations of Mixing Realities, and to gain a better vantage point on human-computer interactions by employing a common method of analysis utilized by art historians, designers, architects, poets, and other creative individuals to evaluate practice through theory. It involves defining the subject matter and observing content and context behind the artwork, while focusing on form.Through this approach, it became possible to observe “Mixing Realities” as a whole while also focusing on the individual elements composing the artwork, their characteristics and relationships.

The creators and theorists of interactive hybrid environments utilize concepts from many fields, which do not necessarily exchange with or complement each other. HCI, cybernetics, information theory, semiotics, design, and new media are some of the fields which attempt to describe these environments, but they don’t necessarily agree on the subject matter, content, or context, much less on the same components or their characteristics and relationships. Such pseudo-transdisciplinarity revealed both ontological and methodological problems.As my research proceeded, I became especially aware of the frequent reliance on conceptualizations of the interface (1) as replicating and reinforcing physical perceptions of embodiment and also (2) as the sole mediator of the digital/physical experience.6

Ontologically, this thesis highlights the problem of the user-interface disconnect that I perceived in my artistic/design practice as well as the inadequate conceptualizations of human-computer (i.e., user-interface) interaction in the theoretical literature by systematically incorporating the third element—information— into the analysis and by introducing the concept of the meta-environment (see Figure 2) as the locus for describing and analyzing the triadic relationship among user, information, and interface.

Methodologically, the problem of the user-interface disconnect and inadequate conceptualizations of HCI is addressed through the analytical framework of cybersemiotics,7 a transdisciplinary theory of information, cognition, meaning, communication, and consciousness that integrates cybernetics and Peircean semiotic paradigms in a common framework (Brier, 2008), which allows for an integrative analysis of the elements in the meta-environment taking into consideration that information can be not only data but “jointly actualized meaning” (Brier, 2008, p. 20).

Cybersemiotics helped me expand the connection among the elements in the meta-environment by offering a common language for classification, description, and exchange among these elements where the following transdisciplinary aspects could be equally considered:

(1) physic-chemical aspects such as spatiality and temporality,

(2) biological and natural science aspects (atoms or bits),

(3) linguistic-cultural-social structuralist aspects related to subjective and/or objective sign interpretation and meaning creation.

(4) aspects of consciousness (qualia), the “phaneroscopic first person point of view” in embodied cognition and meaning creation from “one mental space to another” (Brier, 2008, p. 303).

This transdisciplinary, holistic analysis of the meta-environment highlighted the cognitive dissonance between human perceptions and computer-mediated processes while revealing the potential of the meta-environment to function as a complex adaptive system.