Return to archive
title
Exploring Spatial Perception through Performative 1:1 Extended Reality Models: Preliminary insights from Infra-thin Magick
Author
Paula Strunden
Abstract
Building on scenography, performance theory and findings from neurosciences, tacit knowing in architecture is understood here as embodied, embedded and enacted perceptual dimension of our built environment. Through art- and design-based research, tacitly knowing is examined as a form of practice and a new extended reality (XR) design tool is probed to exercise it. Since the atmospheric turn in architecture (Böhme 2017, McCormack 2014, Bille et al. 2015), it is well known that spatial perception is multi-sensory and that the interplay of our senses goes beyond the cross-fertilization of sight, touch, taste, smell and hearing. Nevertheless, architectural designers may have only touched the surface of what we might be able to feel regarding our spatial environments. Apart from the sensation of our movement and whether our environment is too hot or cold, our abilities to feel space physically remain challenging to represent and communicate through conventional architectural tools. This includes i.e. our sense of balance, our ability to feel time passing, our knowledge of which of our body parts are where without having to look at them, and our sense of gravity, orientation, and illumination. Some of these “always-there-but-never-felt” sensations can be revealed and physically experienced when entering a fully immersive virtual environment for the first time. As our brain takes a split second to adjust to the novel surroundings, it is at this moment that we can suddenly sense our senses at work. The XR case study “Infra-thin Magick”, exhibited as part of Speculative Fiction at the Academy of Fine Arts Vienna in 2022, explains how such unanticipated insights can be purposefully evoked by displacing and reassembling the components constituting our multimodal and synaesthetic spatial perception. Leaning on the design theoretician Thea Brejzek and Lawrence Wallan’s understanding of the “autonomous model”, this performative real-time and -scale XR model that oscillates between physical installation and virtual reality (VR) experience is employed as an operative tool for designing and analyzing spatial experiences beyond the known sensations of our built environment. First user-testing results are presented, and the premise of the autonomous model to co-create reality and allow architects to research through active participation, first-hand experience, discovery, and play are brought to light.
16 July 2022, 4:30 PM, Academy of Fine Arts Vienna, Exhibit Gallery, 1st Floor. The rectangular 2.7 x 5.4-meter room is darkened, the blinds are lowered, it is hot outside – 32 degrees – and the air conditioning is running at its max to cool down the building. The black curtains covering the entrance to the room are blowing in the artificial wind, cars are humming in the distance. The ceremony mistress opens the large wooden door, steps outside the room, walks towards the first visitor, takes her hand, and asks her to follow her with eyes closed. Back inside the room, a virtual reality headset is placed on the visitor’s head, before she is asked to re-open her eyes. The room is dark, silent, and almost cold. The rustling leaves of the tree outside dance across the wooden floor, moonlight flickers through the blinds. A raven lands on the windowsill and slowly starts to speak. It flies past the visitor, who sees, hears, and feels the feathers on her arm, while the first object gradually lowers from the ceiling. The visitor starts to walk towards the object, reaches out with her hands, and checks whether it really exists. Once she feels its texture and rigid shape, she turns around to lean backwards on her arms, slowly lowers her body, and sinks into the annular seat. The seat gives in to her weight, the velvet texture feels soft below her hands. She sees her hands, feels the velvet, and hears the rustling sound of the material below her fingers. She reaches out for the raven; it croaks upon being touched, and nestles against her. The moment she starts to relax, dropping her head deeper into the velvet cushion, the room slowly starts dissolving while she begins to float. 1 To view the full recording of the visitor’s experience, see: Paula Strunden, “RECORDING / Infra-thin Magick by Paula Strunden / EXHIBIT Gallery Academy of Fine Arts Vienna / 2022,” Vimeo, December 6, 2022, video, 10:54, https://vimeo.com/778401566.
This experience report describes the beginning of the extended reality (XR) ceremony, Infra-thin Magick, 2 Exhibited as part of Speculative Fiction, exhibition curated by Stephanie Damianitsch, July 9–October 16, 2022, Exhibit Gallery, Academy of Fine Arts Vienna. created within the framework of my design-led research project exploring tacit knowledge as an embodied, embedded, enacted, and extended perceptual dimension of the built environment. Tacit knowing is examined here as a form of practice, and a new XR design tool is probed to exercise it. Exploratory user-testing results are presented, and the premise of the ‘autonomous model’ to co-create reality enabling architects to research through active participation, first-hand experience, discovery, and play are brought to light.
The Autonomous Model
We no longer progress from model to reality, but from model to model while acknowledging that both models are, in fact, real. 3 Olafur Eliasson, “Models are real,” in Models, ed. Emily Abruzzo, Eric Ellingsen, and Jonathan D. Solomon (New York: Princeton Architectural Press, 2007), 20.
In their book, The Model as Performance, design practitioners and theoreticians Thea Brejzek and Lawrence Wallen introduce the ‘autonomous model’ as a new type of physical scale model, historically distinguishable from the iterative model and the representational model employed across architecture and stage design. 4 Thea Brejzek and Lawrence Wallen, The Model as Performance: Staging space in theatre and architecture (London: Bloomsbury, 2017), 2. With a shared background in theatre studies and stage direction, Brejzek and Wallen advocate an extended understanding of spatial design, which includes temporal activities employed in theatre, dance, and film, such as staging, performing, storytelling, and acting. Following this disciplinary blending, their book discusses over thirty case studies ranging from the Renaissance to the present that: (1) challenge the scale model’s role as a ‘second-order object awaiting its realisation’; and (2) acknowledge the model’s operative capacities in creating its own reality and ‘potential for cosmopoesis, or world-making’. 5 Ibid., 1 It is within this second perspective that the performative and autonomous capacity of the model lies. According to Brejzek and Wallen, the iterative (ad hoc/sketch/study) model, as well as the representational (presentation/communication/final) model, serve as preliminary stages or auxiliary means to describe reality, whereas the autonomous or performative model produces reality in itself. It is defined as an independently acting entity and an end unto itself that is ‘built to be autonomous’, and thus is neither a substitute nor a referent to an actual building or stage set. 6 Ibid., 2. By dissolving the dichotomy of model/reality, Brejzek and Wallen comb through history, looking for case studies that establish the autonomous model as an epistemic tool. Ranging from Giulio Camillo’s Theatre of Memory (1550), through Vladimir Tatlin’s Model of The Monument to the Third International (1919–20), to Caruso St John and Thomas Demand’s Nail House (2010), they position the autonomous model as ‘a cross-disciplinary global practice of spatial production that has operated from the Renaissance until today … fuelled by the designer’s desire for technological innovation and artistic expression’. 7 Ibid., 5.
Recalling the case studies I visited, I remember feeling disappointed at being unable to fully experience the 1:1 reconstruction of Le Corbusier’s Maison Dom-ino (2014) created by architect Valentin Bontjes van Beek for the 14thArchitectural Biennale in Venice, a full-scale wooden installation that could not be entered. Similarly, I was asked not to run my fingers over the blown-up paper texture of the 1:1 mock-up of the Nail House (2010), developed by architect Caruso St John and artist Thomas Demand. Both examples illustrate the limited agency of the audience in relation to autonomous models created for exhibition purposes and raise the question of whether these models can be understood as ‘practised space(s)’, as proclaimed by Brejzek and Wallen, given that they cannot be actively interacted with. 8 Ibid., 141. How can an audience that lacks the ability to act, make decisions, or take control recognise and acknowledge the agency of a model if it does not have agency itself?
To address this question and to add another dimension to Brejzek and Wallen’s definition of how autonomous models operate and where their performative potential lies, my research incorporates user-oriented practices from computer-game and experience design into their cross-disciplinary spatial practice. Here, the visitor is foregrounded as an active player who co-creates the model’s reality through interaction and first-hand experience – from inside-out and through first-person perspective. By focussing on the visitor’s involvement in the creation of the model, the traditional subject/object and viewer/creator dichotomies and the distinctions between actual/virtual and model/reality are further dissolved. Brejzek and Wallen’s decision to exclude interactive digital models from their discussion and leave out the virtual model’s potential for the production of reality is based on their observation that ‘the increasingly porous edge between physical and virtual spaces … is accelerating at a speed that from a typological point of view remains too fluid to be central to the argument’. 9 Ibid., 4. It is precisely this dynamically changing relationship of the virtual to the material world, this ‘too fluid to be central’ gap, which marks the starting point of my ongoing and open-ended research practice.
Approach and Technical Set-Up
At the intersection of architecture, immersive theatre, new media art, and game design, I develop artistic tools and instruments to reveal, uncover, and exhibit new insights into the underlying mechanisms of embodied and embedded spatial perception. My research uses mixed-reality techniques to hybridise physical and virtual spaces and investigate the convergence of their realms. Over the years, I have developed a series of experiential technical set-ups, which allow me to evaluate the effects of inhabiting non-linear interactive architectural constellations. I design, plan, and build full-scale physical models and craft props, superimposed with digital models drawn in the CAD software, Rhino 3D, and compiled in the game engine, Unity. The resulting hybrid installations are exhibited publicly and activated by one visitor at a time wearing a virtual reality (VR) headset. By attaching a small camera to the front of the headset and tracking the visitor’s hand and head position in real-time, their physical movements are translated into virtual interactions and gradually unveil the spatiotemporal narrative. Instead of working with high-cost haptic gloves or suits that simulate embodied spatial perception, I incorporate the physical room where the experience takes place, the visitor’s real body and real-time movements, the material objects at hand, and the invigilator who is present throughout the encounter. To emphasise the importance of the visitor’s bodily experience and its emotional engagement, I use the term ‘immersant’, coined by VR pioneer Char Davies, to describe the person participating in the installation. 10 Oliver Grau, Virtuelle Kunst in Geschichte und Gegenwart (Berlin: Reimer, 2000), 193; Char Davies, “Osmose: Notes on being in immersive virtual space,” in Digital Creativity: A Reader, ed. Colin Beardon and Lone Malmborg (Lisse: Swets & Zeitlinger Publishers, 2002), 101–10. This combination of analogue and digital techniques, which enables the immersant to naturally interact in a full-scale, real-time hybrid environment, I call the performative 1:1 Extended Reality Model (XRM).
To demonstrate the novelty and significance of constructing XRMs within the field of extended spatial production, this text has three interrelated agendas: (1) describing the design processes of XRMs through Infra-thin Magick; 11 Highlighted here as one of three case studies, next to Alison’s Room (part of Virtual CIAM Museum, curated by Dirk van den Heuvel, November 24–January 4, 2023, HNI Rotterdam) and Rhetorical Bodies (part of No Dancing Allowed, curated by Bogomir Doringer, June 22–November 20, 2022, Q21 Vienna). (2) evaluating the immersants’ experiences and reactions; (3) expanding upon existing theoretical positions on autonomous models and the evolving relationship between bodies, objects, and spaces in XR architecture.
By exploring the nature of human spatial perception through its own mechanism of operation, that is, the human body, I lean on design-driven, practice-led research methods ‘that situate creative practice as both a driver and outcome of the research process’. 12 Jillian G. Hamilton and Luke O Jaaniste, “Content, structure and orientations of the practice-led exegesis,” in Art.Media.Design: Writing Intersections, November 18–19, 2009 , ed. G. Melles (Melbourne: Swinburne University, 2009), 184. I engage in an interactive, exploratory creative process that involves the active engagement of my body, and fluid transition between various roles. These roles include designing an interactive model, which encompasses developing narratives, sketching storylines, writing voice-overs, creating virtual environments, and building physical props. Further, I produce large-scale drawings and prototype objects (Fig. 4.4), three-dimensionally experience the model first-hand, guide others through their experiences, train invigilators, and observe them performing. Simultaneously, I readjust the design of the XRM. The boundaries between myself as its maker, the immersant as its user, and the invigilator as its facilitator, are renegotiated every time the experience restarts, making all participants equally responsible for creating the model’s reality. To what extent, or at what moment, this dynamically changing model starts to create its own reality – identified by Brejzek and Wallen as a key characteristic of the autonomous model – is presented and discussed as follows.
Infra-thin Magick
After his death, the surrealist artist Marcel Duchamp left behind an envelope containing forty-six handwritten notes, each exploring his neologism of the ‘infra-thin’. Duchamp had revealed in an interview that his notion escaped any existing scientific definition and was impossible to explain, but he could give examples: 13 Francis Roberts, “Interview with Marcel Duchamp, ‘I Propose to Strain the Laws of Physics,’” Art News (December 1968), 62. infra-thin is the warmth of a seat that has just been left; infra-thin is when tobacco smoke also smells of the mouth which exhales it; infra-thin is the difference that exists between two forms that have been cast from the same mould. 14 Duchamp’s notes were posthumously published in French by Paul Matisse: Marcel Duchamp, Notes, ed. Paul Matisse (Boston: GK Hall, 1980). The following simplified translations by the author are based on Duchamp’s notes no. 4, 33, and 35, taken from 21 and 33. According to the art historian Hector Obalk, infra-thin describes a separation, a difference, or an interval between two things. It could be either ‘a difference that you can easily imagine, but that does not exist’, like the thickness of a shadow, or it could be ‘a distance that you cannot perceive, but only imagine’. 15 Hector Obalk, “The unfindable readymade,” Tout-Fait, The Marcel Duchamp Studies Online Journal (January 5, 2000), https://www.toutfait.com/the-unfindable-readymade/.
Expanding on Duchamp’s concept of a subjectively felt notion that eludes any scientific explanation and that can only be exemplified, I designed a family of hybrid artefacts that can both be manipulated in virtual space and interacted with physically (Figs. 4.7–9). These artefacts, entitled the Sinking Seat (1), the Magic Mirror (2), and the Seeing Orb (3), reconceptualise Duchamp’s infra-thin as the immeasurable gap between virtually-perceived and physically-experienced space; a threshold between different states characterised by a feeling of ambiguity or indeterminacy. Comparable to the ‘phenomenon of becoming aware of the fact that one is dreaming during ongoing sleep’, 16 Benjamin Baird, Sergio Mota-Rolim, and Martin A. Dresler, “The cognitive neuroscience of lucid dreaming,” in Neuroscience & Biobehavioral Reviews 100 (2019): 305, https://www.sciencedirect.com/science/article/pii/S0149763418303361. commonly known as lucid dreaming, I made the experience one of being fully immersed in a virtual environment, yet fully aware of one’s physical body and surroundings. Determined to explore this paradoxical feeling, I designed three infra-thin instruments that uncover the fleeting sensation I felt when working at the intersection of the realms to capture the highly subjective and ephemeral essence of this phenomenon and preserve it in some form to be shared with others. While VR is mainly associated with the possibility of escaping one’s body or leaving it behind in order to have an out-of-body experience, a series of projects from the 1990s led by female artists explore the phenomenon that ‘such environments can provide a new kind of “place” … in a paradoxical combination of the ephemerally immaterial with what is perceived and bodily felt to be real’. 17 Such as Placeholder: Landscape and Narrative in a Virtual Environment (1992) by Brenda Laurel and Rachel Strickland, Osmose (1995) by Char Davies, or Spectral Bodies (2000) by Catherine Richards; Char Davies, “Osmose,” in Multimedia: From Wagner to Virtual Reality, ed. Randall Packer and Ken Jordan (New York: W. W. Norton & Company, 2002), 294. The architectural theorist Karen A. Franck confirms that it is not the physical body that is left behind when entering VR, as ‘it is physical bodies that give us access to any world’. 18 Karen A. Franck, “When I Enter Virtual Reality, What Body will I leave Behind?” in “Architects in Cyberspace,” ed. Maggie Toy, special issue, Architectural Design 65, nos. 11–12 (1995): 20. While the body occupies both worlds simultaneously, it is their ability that is changing and enabling immersants to experience different kinds of being-in-the-world. 19 Ibid., 21. Like a toddler, immersants move slowly and carefully with their hands stretched out before them, gradually feeling their way into the newly-behaving space. The fact that they not only act in this space and interact with its objects, but actually enact themselves, introduces the additional element of discovery and play into the XRM. By wearing their body as a costume, seeing their shadow and the digital overlay of their hands moving in real-time, and doing and feeling real things while having extended (intensified, magnified, or multiplied) abilities, the thrilling sense of being-in-a-body-in-a-room is brought to the fore. It is not the motor and perceptual system per se, but the bodily interaction and entanglement with its environment, its embeddedness and situatedness in this dynamically changing space, that challenges the immersants’ inbuilt assumptions about the built environment.
Based on this idea, Infra-thin Magick is designed as a guided ritual sequence that takes the form of an XR ceremony, inviting one person at a time to experience the place that unfolds once the virtual realm can be felt with and through their body. Rather than experiencing the physical world within the virtual one, the XRM enables the immersants to experience the virtual physically. As demonstrated in the chapter’s opening anecdote, the brain seamlessly stitches together the virtual space experience with the underlying perception of the physical environment, turning the XRM into a tool to explore the nature and type of this very seam. Through weaving the irrational, surreal, and dreamlike space into the pillow’s texture, the feather’s timing, or the air’s temperature, I gradually learned to navigate between those realms. I became adept at adjusting the size of stitches, tightening them, or leaving little holes in between. These gaps evoke situations in which the experienced reality does not align with its expectation so that a void emerges between what is anticipated to happen and what actually occurs. The degree to which this void is subconsciously restitched by the immersant’s subjective spatial understanding, memories, imaginations, and speculations unveil the brain’s continuous endeavours to anticipate the nature and character of its surroundings, and thus actively co-create the XRM’s reality.
Here, the bodily lived experience is placed at the centre of the research by approaching virtual technologies from a phenomenological perspective. Following the writings of the philosopher Maurice Merleau-Ponty, cognition is understood to be ‘physically interactive, embedded in physical contexts, and manifested in physical bodies’. 20 Lawrence Shapiro and Shannon Spaulding, “Embodied Cognition”, in The Stanford Encyclopedia of Philosophy, ed. Edward N. Zalta, https://plato.stanford.edu/archives/win2021/entries/embodied-cognition/ (accessed November 20, 2022). Just as our perception of the actual world is mediated through our sensory apparatus, the virtual world is absorbed through our eyes, ears, mouth, nose, hands, and feet. Mel Slater’s decade-long research at the intersection of computer-science and psychology has proven that the emotional and affective responses to a fully-immersive bodily experience of a virtual environment equal that of a physical environment once a high level of immersion and presence is attained. While the quality of immersion depends on technological components, presence is the subjectively-felt experience of ‘being in the world’, 21 Mel Slater and Sylvia Wilbur, “A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments,” Presence: Teleoperators Virtual Environments 6, no. 6 (1997): 603–16; Bob G. Witmer and Michael J. Singer, “Measuring presence in virtual environments: a presence questionnaire,” Presence: Teleoperators Virtual Environments 7, no. 3 (1998): 225–40; John B. Walther and Malcolm R. Parks, “Cues filtered out, cues filtered in: computer-mediated communication and relationships,” in Handbook of Interpersonal Communication, ed. Mark L. Knapp and John A. Daly, 3rd edn (Thousand Oaks: Sage, 2002), 529–63. evoked by the interplay of spatial-, self-, and social-presence. 22 Catherine S. Oh et al., “A Systematic Review of Social Presence: Definition, Antecedents, and Implications,” Frontiers in Robotics and AI 5 (2018), https://www.frontiersin.org/articles/10.3389/frobt.2018.00114. While self-presence and social-presence are extensively discussed as embodied psychological experiences, 23 Ibid.; Julie R. Williamson et al., “Digital Proxemics: Designing Social and Collaborative Interaction in Virtual Environments,” in CHI Conference on Human Factors in Computing Systems, May 1–5, 2022, New Orleans (2022), https://doi.org/10.1145/3491102.3517594. spatial-presence is often limited to a mere audio-visual experience. This neglects the rich multisensory aspect arising from cross-modal correspondence, where all senses complement and reinforce each other, and thus ground us in the world. 24 Joy M. Malnar and Frank Vodvarka, Sensory Design (Minneapolis: University of Minnesota Press, 2004); Charles Spence, “Senses of place: architectural design for the multisensory mind,” Cognitive Research 5, no. 46 (2020), https://doi.org/10.1186%2Fs41235-020-00243-4; Alberto Gallace and Charles Spence, In Touch with the Future: The sense of touch from cognitive neuroscience to virtual reality (Oxford: Oxford University Press, 2014), 13. Recent neuroscience research has revealed that this complex interplay goes far beyond the cross-fertilisation of the five common senses. 25 Dustin Strokes, Mohan Matthen, and Stephen Biggs, “Sorting the Senses,” in Perception and its Modalities, ed. Strokes, Matthen, and Biggs (New York: Oxford University Press, 2015), 11. Yet, most other sensory modalities are taken for granted when experiencing the built environment, as they tend to remain stable. 26 Andy Clark, Surfing Uncertainty: Prediction, action, and the embodied mind (New York: Oxford University Press, 2015). Apart from the sensation of our movement (kinaesthesia), and whether our environment is too hot or cold (thermoception), our abilities to feel space physically remain challenging to communicate through conventional architectural tools. This includes our sense of balance (equilibrioception), our knowledge of which of our body parts is where (proprioception), our ability to feel time passing (chronoception), and, to name but a few, our sense of gravity, orientation, and illumination. Some of these always-there-but-never-felt sensations can be revealed and physically experienced when entering a fully immersive virtual environment. As our brain adjusts to the novel surroundings, we can suddenly notice our senses at work. This is where Infra-thin Magick comes into play, exposing the gap between our expectations and actual experience of spatial relations and revealing that their boundaries are impossible to stabilise or control due to their highly dynamic nature. Challenged to renegotiate our perception of distance, size, and orientation, every time we re-enter an XRM, its unpredictability leads to the creation of a unique reality for every individual immersed in it.
Transcending Dualities through Trust
In order to understand, inhabit, and evaluate space, it is crucial to recognise its temporal aspect. Space does not simply exist in time; it is of time. 27 Eliasson, “Models are real,” 19.
Between July and October 2022, over 150 immersants were introduced to the space between physical and virtual reality during three Infra-thin Magick sessions, each lasting three days. The immersants were asked to provide a written reflection or participate in an interview describing what they had experienced (Fig. 4.10). Preliminary analysis of their responses, coupled with personal reflections, confirm the key characteristic of the autonomous model to perform as an ‘active agent’ and thus become a ‘co-producer of reality’. 28 Brejzek & Wallen, The Model as Performance, 1; Eliasson, “Models are real,” 19. It is the XRM’s inherent ambiguity – its ability to be open to multiple interpretations and to operate in multiple ways – that qualifies it as an autonomous model, a characteristic discussed in this final section through the threefold act of ‘being here/there’, ‘letting go’, and ‘spending time’.
Beginning with the notion of ‘being here/there’, due to the real-time and full-scale nature of the XRM, the immersants’ movements, actions, and reactions shape the length and character of their experiences. Every participant behaves slightly differently, requiring the ceremony mistress to adopt a new approach in each instance. The ceremony mistress reports that she can sense whether the immersant trusts the environment to be real or not within seconds. Once this trust is established, the ceremony mistress characterises it as an ‘invisible pact’ between all involved, while immersants describe the XR objects as ‘magnetic’ or ‘alive’, suggesting that they exhibit qualities of autonomy or an existence beyond their perception, recalling the philosopher Jane Bennet’s concept of ‘thing-power’. 29 Notes from Guiding through Infra-thin Magick conversations with ceremony mistresses, Academy of Fine Arts Vienna, July 7–October 16, 2022; Jane Bennett, Vibrant matter: A Political Ecology of Things (Durham, NC: Duke University Press, 2010), 2. After taking off the headset, immersants reported the need to confirm that what they are seeing is actually there. This uncertainty is also reflected in the language used, including observations such as, ‘I came back after not having been away’, and ‘I travelled very far, in that room over here, right there’ . 30 Notes gathered from Infra-thin Magick Reflection Cards (Fig. 4.10), Academy of Fine Arts Vienna, July 7–October 16, 2022. In the XRM, the known frame of reference and the former distinction between inside/outside, here/there, are dismantled and overcome. The former description of a person’s physical location in the world relative to themselves and in relation to the boundaries of an object or space is transcended. The XRM not only allows the immersant to explore external space as if it were internal through a process of immersion and profound inner engagement, but to experience mental activities typically located within themselves, such as memories, imaginations, or dreams, as if they were external to their bodies, blurring the distinction between intangible places that exist within the mind and tangible places that have a physical existence and can thus be perceived through touch rather than mental visualisation. Unable to differentiate between the two, their experiences are intensified and reinforced until they start oscillating. This oscillation creates its own space that can be observed, expanded, and explored, confirming the XRM’s role as ‘co-producer of reality’ and recalling the indefinable notion of Duchamp’s infra-thin. The XRM thus not only blurs the traditional Cartesian dualism of body/mind, but offers a new perspective on the interconnectedness of the self and the environment, bringing the concept of ‘who I am’ and ‘where I am’ closer together. 31 UW Reality Lab, “Andreea Ion Cojocaru – Where Are You, Who Are You? The Thinning Thickness of The Real,” YouTube, May 11, 2021, video, 53:39, https://www.youtube.com/watch?v=6HInWCezqig.
Moving towards the concept of ‘letting go’, whether an XRM can be characterised as an autonomous model in Brejzek’s and Wallen’s sense depends on the immersants’ willingness to actively engage with and interact within the model, requiring them to be both active and passive in controlling interactivity through their body while being tracked, monitored, and controlled by the technological set-up and framework of the experience. Only when the immersants intuitively accept this dynamic dual role can the game of leading and being led by the spatiotemporal narrative begin. Incorporating affective responses in the design process of the XRM introduces a new dimension to the creation and experience of architectural models. It highlights that the actual use of the tool only works if the designer and immersant are willing to let the model ‘happen’ to themselves. In the same way, that trust requires an emotional openness and readiness; understanding novel spatial configurations created through experiencing an XRM requires a distinct emotionality or commitment. The ability to ‘let go’, or ‘give in’, and the desire to engage in this game, becomes a primary condition for understanding the model’s reality and drawing insights from it beyond the sheer novelty of immersion in a virtual world. Consequently, spatial understanding within or through an XRM is an active form of willingness to attend, engage, care, and trust. It is the immersant who moves slowly, who examines carefully, who touches, considers, and re-touches, who listens and responds to the objects encountered, who will start feeling the transformative effects of being immersed in between the realms.
This leads to the proposition of ‘spending time’: time to immerse yourself repeatedly and at length in different kinds of experiences, to test, try, and retry, to observe yourself moving through the realms as well as observing others in detail, to encounter glitches and errors, to drop, fail, and fall. The time spent making and inhabiting an XRM gradually allows the virtual space to sediment itself within the body. The XRM’s performative nature thus lies in its ability to create a lasting bodily impression, allowing for a better understanding of the medium through repeated and continual exposure. Moreover, this notion of embodied sedimentation of a fleeting sensation, which one is not-yet-able to verbalise, or a concept, which one is not-yet-able to grasp, explains why an altered space–body–object relationship can be fuelled through design-led research. Over time, interacting with the model aggregates a deeper understanding of its spatiotemporal logic. By literally stepping in and out of the XRM, the frame of reference shifts, and the boundaries between model and reality, subject and object, are renegotiated, so that being-in-space leads to space-explaining moments. Once it is no longer clear who is reacting to whom, a form of co-creation takes place that allows me, as the designer of the XRM, to step back, let go, and let the space happen to me, work on me, measure me. I allow myself to be infiltrated, influenced, swayed, and misled to develop a better understanding of how my body constructs space itself. Although I may rationally understand that the 3D-rendered room of Infra-thin Magick does not revolve around my head in real-time when interacting with the Seeing Orb (Fig. 4.7), my body applies its inherent logic to the phenomena occurring. When I hold my eyes between my hands and swing them from left to right, it is not the room but my embodied concept of spatiality that causes it to revolve around me, shaped by the physical principles of the natural world. Physically experiencing and experimentally proving that spatial environments are dynamically constructed inside and outside the body, mutually dependent and constantly evolving, creates a deep desire to explore this newly encountered realm further, and discover how the XRM uses the body as a site. From the shifting perspectives of a designer, immersant, and invigilator, it has become apparent that virtual space is not limited to what I see, hear, and feel, mediated through technological devices, but incorporates moreover what I expect, anticipate, and imagine seeing, hearing, and feeling, just before I actually do, just before reality becomes tangible. This makes the experience of the virtual an inherent aspect of spatial perception per se and a fundamental component for comprehending architecture.
To conclude, the outlined tripartite approach, which focuses on the notions of presence, trust, and engagement, generates a spatial experience perceived to be more real than the sum of its constituting elements. By adopting autonomous models as a framework for exploring XR applications in spatial research and practice, architects can move beyond the conventional use of virtual technologies mainly employed for representation purposes today. This shift allows architects to engage with the technology’s inherent narrative, and fictional and speculative potential, creating new possibilities for spatial research native to the medium. Once the model’s potential for cosmopoiesis, as proposed by Brejzek and Wallen, is extended to be found in the liminal, the gap, the intersection of our material and virtual world, materials/models/reality/subjects/objects can be reviewed on a flattened ontological hierarchy, and XRMs can furthermore be employed as operative tools for designing and analysing spatial experiences beyond the known sensations of the built environment.
Bibliography
- Baird, Benjamin, Sergio Mota-Rolim, and Martin A. Dresler. “The cognitive neuroscience of lucid dreaming.” Neuroscience & Biobehavioral Reviews 100 (2019): 305–23. https://www.sciencedirect.com/science/article/pii/S0149763418303361.
- Brejzek, Thea, and Lawrence Wallen. The Model as Performance: Staging Space in Theatre and Architecture. London: Bloomsbury, 2017.
- Clark, Andy. Surfing Uncertainty: Prediction, action, and the embodied mind. New York: Oxford University Press, 2015.
- Davies, Char. “Osmose.” In Multimedia: From Wagner to Virtual Reality, edited by Randall Packer and Ken Jordan, 293–300. New York: W. W. Norton & Company, 2002.
- Davies, Char. “Osmose: Notes on being in immersive virtual space.” In Digital Creativity: A Reader, edited by Colin Beardon and Lone Malmborg, 101–10. Lisse: Swets & Zeitlinger Publishers, 2002.
- Duchamp, Marcel. Notes. Edited by Paul Matisse. Boston: GK Hall, 1980.
- Eliasson, Olafur. “Models are real.” In Models, edited by Emily Abruzzo, Eric Ellingsen, and Jonathan D. Solomon, 18–25. New York: Princeton Architectural Press, 2007.
- Franck, Karen A. “When I Enter Virtual Reality, What Body will I leave Behind?” In “Architects in Cyberspace,” edited by Maggie Toy. Special issue, Architectural Design 65, nos. 11–12 (1995): 20–23.
- Gallace, Alberto, and Charles Spence. In Touch with the Future: The sense of touch from cognitive neuroscience to virtual reality. Oxford: Oxford University Press, 2014.
- Grau, Oliver. Virtuelle Kunst in Geschichte und Gegenwart. Berlin: Reimer, 2000.
- Hamilton, Jillian G., and Luke O Jaaniste. “Content, structure and orientations of the practice-led exegesis.” In Art.Media.Design: Writing Intersections, November 18–19, 2009, edited by G. Melles, 184–95. Melbourne: Swinburne University, 2009.
- Malnar, Joy M., and Frank Vodvarka. Sensory Design. Minneapolis: University of Minnesota Press, 2004.
- Obalk, Hector. “The unfindable readymade.” Tout-Fait, The Marcel Duchamp Studies Online Journal (January 5, 2000). https://www.toutfait.com/the-unfindable-readymade/.
- Oh, Catherine S., et al. “A Systematic Review of Social Presence: Definition, Antecedents, and Implications.” Frontiers in Robotics and AI 5 (2018). https://www.frontiersin.org/articles/10.3389/frobt.2018.00114.
- Roberts, Francis. “Interview with Marcel Duchamp, ‘I Propose to Strain the Laws of Physics.’” Art News. December 1968.
- Shapiro, Lawrence, and Shannon Spaulding. “Embodied Cognition.” In The Stanford Encyclopedia of Philosophy, edited by Edward N. Zalta. Accessed November 20, 2022. https://plato.stanford.edu/archives/win2021/entries/embodied-cognition/.
- Slater, Mel, and Sylvia Wilbur. “A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments.” Presence: Teleoperators Virtual Environments 6, no. 6 (1997): 603–16.
- Spence, Charles. “Senses of place: architectural design for the multisensory mind.” Cognitive Research 5 (2020). https://doi.org/10.1186%2Fs41235-020-00243-4.
- Strokes, Dustin, Mohan Matthen, and Stephen Biggs. “Sorting the Senses.” In Perception and its Modalities, ed. Strokes, Matthen, and Biggs, 1–22. New York: Oxford University Press, 2015.
- Walther, John B., and Malcolm R. Parks. “Cues filtered out, cues filtered in: computer-mediated communication and relationships.” In Handbook of Interpersonal Communication, edited by Mark L. Knapp and John A. Daly, 3rd edn., 529–63. Thousand Oaks: Sage, 2002.
- Williamson, Julie R., et al. “Digital Proxemics: Designing Social and Collaborative Interaction in Virtual Environments.” In CHI Conference on Human Factors in Computing Systems, May 1–5, 2022, New Orleans. 2022. https://doi.org/10.1145/3491102.35175941–12.
- Witmer, Bob G., and Michael J. Singer. “Measuring presence in virtual environments: a presence questionnaire.” Presence: Teleoperators Virtual Environments 7, no. 3 (1998): 225–40.
- To view the full recording of the visitor’s experience, see: Paula Strunden, “RECORDING / Infra-thin Magick by Paula Strunden / EXHIBIT Gallery Academy of Fine Arts Vienna / 2022,” Vimeo, December 6, 2022, video, 10:54, https://vimeo.com/778401566.
- Exhibited as part of Speculative Fiction, exhibition curated by Stephanie Damianitsch, July 9–October 16, 2022, Exhibit Gallery, Academy of Fine Arts Vienna.
- Olafur Eliasson, “Models are real,” in Models, ed. Emily Abruzzo, Eric Ellingsen, and Jonathan D. Solomon (New York: Princeton Architectural Press, 2007), 20.
- Thea Brejzek and Lawrence Wallen, The Model as Performance: Staging space in theatre and architecture (London: Bloomsbury, 2017), 2.
- Ibid., 1
- Ibid., 2.
- Ibid., 5.
- Ibid., 141.
- Ibid., 4.
- Oliver Grau, Virtuelle Kunst in Geschichte und Gegenwart (Berlin: Reimer, 2000), 193; Char Davies, “Osmose: Notes on being in immersive virtual space,” in Digital Creativity: A Reader, ed. Colin Beardon and Lone Malmborg (Lisse: Swets & Zeitlinger Publishers, 2002), 101–10.
- Highlighted here as one of three case studies, next to Alison’s Room (part of Virtual CIAM Museum, curated by Dirk van den Heuvel, November 24–January 4, 2023, HNI Rotterdam) and Rhetorical Bodies (part of No Dancing Allowed, curated by Bogomir Doringer, June 22–November 20, 2022, Q21 Vienna).
- Jillian G. Hamilton and Luke O Jaaniste, “Content, structure and orientations of the practice-led exegesis,” in Art.Media.Design: Writing Intersections, November 18–19, 2009 , ed. G. Melles (Melbourne: Swinburne University, 2009), 184.
- Francis Roberts, “Interview with Marcel Duchamp, ‘I Propose to Strain the Laws of Physics,’” Art News (December 1968), 62.
- Duchamp’s notes were posthumously published in French by Paul Matisse: Marcel Duchamp, Notes, ed. Paul Matisse (Boston: GK Hall, 1980). The following simplified translations by the author are based on Duchamp’s notes no. 4, 33, and 35, taken from 21 and 33.
- Hector Obalk, “The unfindable readymade,” Tout-Fait, The Marcel Duchamp Studies Online Journal (January 5, 2000), https://www.toutfait.com/the-unfindable-readymade/.
- Benjamin Baird, Sergio Mota-Rolim, and Martin A. Dresler, “The cognitive neuroscience of lucid dreaming,” in Neuroscience & Biobehavioral Reviews 100 (2019): 305, https://www.sciencedirect.com/science/article/pii/S0149763418303361.
- Such as Placeholder: Landscape and Narrative in a Virtual Environment (1992) by Brenda Laurel and Rachel Strickland, Osmose (1995) by Char Davies, or Spectral Bodies (2000) by Catherine Richards; Char Davies, “Osmose,” in Multimedia: From Wagner to Virtual Reality, ed. Randall Packer and Ken Jordan (New York: W. W. Norton & Company, 2002), 294.
- Karen A. Franck, “When I Enter Virtual Reality, What Body will I leave Behind?” in “Architects in Cyberspace,” ed. Maggie Toy, special issue, Architectural Design 65, nos. 11–12 (1995): 20.
- Ibid., 21.
- Lawrence Shapiro and Shannon Spaulding, “Embodied Cognition”, in The Stanford Encyclopedia of Philosophy, ed. Edward N. Zalta, https://plato.stanford.edu/archives/win2021/entries/embodied-cognition/ (accessed November 20, 2022).
- Mel Slater and Sylvia Wilbur, “A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments,” Presence: Teleoperators Virtual Environments 6, no. 6 (1997): 603–16; Bob G. Witmer and Michael J. Singer, “Measuring presence in virtual environments: a presence questionnaire,” Presence: Teleoperators Virtual Environments 7, no. 3 (1998): 225–40; John B. Walther and Malcolm R. Parks, “Cues filtered out, cues filtered in: computer-mediated communication and relationships,” in Handbook of Interpersonal Communication, ed. Mark L. Knapp and John A. Daly, 3rd edn (Thousand Oaks: Sage, 2002), 529–63.
- Catherine S. Oh et al., “A Systematic Review of Social Presence: Definition, Antecedents, and Implications,” Frontiers in Robotics and AI 5 (2018), https://www.frontiersin.org/articles/10.3389/frobt.2018.00114.
- Ibid.; Julie R. Williamson et al., “Digital Proxemics: Designing Social and Collaborative Interaction in Virtual Environments,” in CHI Conference on Human Factors in Computing Systems, May 1–5, 2022, New Orleans (2022), https://doi.org/10.1145/3491102.3517594.
- Joy M. Malnar and Frank Vodvarka, Sensory Design (Minneapolis: University of Minnesota Press, 2004); Charles Spence, “Senses of place: architectural design for the multisensory mind,” Cognitive Research 5, no. 46 (2020), https://doi.org/10.1186%2Fs41235-020-00243-4; Alberto Gallace and Charles Spence, In Touch with the Future: The sense of touch from cognitive neuroscience to virtual reality (Oxford: Oxford University Press, 2014), 13.
- Dustin Strokes, Mohan Matthen, and Stephen Biggs, “Sorting the Senses,” in Perception and its Modalities, ed. Strokes, Matthen, and Biggs (New York: Oxford University Press, 2015), 11.
- Andy Clark, Surfing Uncertainty: Prediction, action, and the embodied mind (New York: Oxford University Press, 2015).
- Eliasson, “Models are real,” 19.
- Brejzek & Wallen, The Model as Performance, 1; Eliasson, “Models are real,” 19.
- Notes from Guiding through Infra-thin Magick conversations with ceremony mistresses, Academy of Fine Arts Vienna, July 7–October 16, 2022; Jane Bennett, Vibrant matter: A Political Ecology of Things (Durham, NC: Duke University Press, 2010), 2.
- Notes gathered from Infra-thin Magick Reflection Cards (Fig. 4.10), Academy of Fine Arts Vienna, July 7–October 16, 2022.
- UW Reality Lab, “Andreea Ion Cojocaru – Where Are You, Who Are You? The Thinning Thickness of The Real,” YouTube, May 11, 2021, video, 53:39, https://www.youtube.com/watch?v=6HInWCezqig.