Looking back at people looking forward

In 1995, Heath, Luff, & Sellen lamented the uptake of video conferencing indicating that it had not at the time reached its promise. But looking back at this projection, the ubiquity of video systems for social and work communication can be seen. And subsequently, research has gone about understanding it further in a variety of HCI paradigms (CHI2010, CSCW2010, CHI2018). So, for my research, making projections on the use of VR for music collaboration, it might be that findings and insights do not reach fruition, either, in a timely fashion, or in the domain of interest that they were investigated in, or ever! Though this could be touching on a form of hindsight bias.

Going back to the article that speculated on the unobtained promise of video conferencing technologies, Heath Luff, and Sellen (1995), provide a piece of insight that can still be placed into perspective on design interventions for collaboration:

It becomes increasingly apparent, when you examine work and collaboration in more conventional environments, that the inflexible and restrictive views characteristic of even the most sophisticated media spaces, provide impoverished settings in which to work together. This is not to suggest that media space research should simply attempt to ‘replace’ co-present working environments, such ambitions are way beyond our current thinking and capabilities. Rather, we can learn a great deal concerning the requirements for the virtual office by considering how people work together and collaborate in more conventional settings. A more rigorous understanding of more conventional collaborative work, can not only provide resources with which to recognise how, in building technologies we are (inadvertently) changing the ways in which people work together, but also with ways in which demarcate what needs to be supported and what can be left to one side (at least for time being). Such understanding might also help us deploy these advanced technologies.

The bold section highlights the nub of what I’m interested in; for VR music collaboration systems. I break this down into how I’ve tackled framing collaboration in my research:

  • conventional collaborative work – ethnographies of current and developing practice. Even if you pitch a radical agenda of VR workspace, basic features of the domain of interest need to be understood for their contextual and technical practices.
  • building technology is changing practice – observing the impact of design interventions on how people collaborate in media production. Not only does a technology suggest new ways of working, it can enforce them! Observing and understanding this in domain-specific ways is important.
  • what needs to be supported – basic interactional requirements, we have to be able to make sense of each other, and the work, together, in an efficient manner.
  • what can be left to one side – the exact models and metaphors of how work is constructed in reality, in VR we can create work setups and perspectives that cannot exist in reality. For instance, shared spatial perspectives i.e. seeing the same thing from the same perspective is impossible in reality as we have to occupy a separate physical space. In repositioning basic features of spatial collaboration, the effects need to be understood in terms of interaction and domain requirement. But the value is in finding new ways of doing things not possible in face to face collaboration.

Overall, the key theme that should be taken away is that of humans’ need to communicate and collaborate. In this sense, any research that looks to make collaboration easier is provisioning for basic human understanding. That is quite nice to be a part of.

Lessons learned in VR dev

Following is some pastoral advice gained from doing a project in a new field when the brief is quite open. As with all advice, it depends on your personality!

Define the concept as simply as possible – if communication of the underlying concept isn’t clear, how will the implementation not be?

If its a good idea, follow it – When populating a design space with early concepts, tangents and ideas abound. These may diverge significantly from the original concept you thought of, but in the creative process this is perhaps the nature of ideas. As when balancing all the elements, hidden parameters and approaches appear. These are things you couldn’t perceive in your original constructs and perhaps hold a grain of something truly novel. If you don’t have a strict brief, let go and see where it leads.

Domain knowledge – when coming from a specialist field, such as audio, be wary of the perceived knowledge in users. Your domain knowledge and intellectual predispositions will guide your design space decisions. If you are not careful your ability to communicate to a wider audience will be doomed from the start, due to relying on existing interface metaphors that do not communicate effectively to new users. But if you focus the application to specific domains, these nuances can make it through a design process and be of use to the field more generally.

Deciphering Space in Virtual Reality

Space, and its perception, is fundamental to navigation through the world. Whether through sight, sound or touch we build mental images of how we relate to the world around us. So, to better understand the possibilities and pitfalls of design for virtual reality interfaces a thorough understanding of spatial cognition is advised. The following article is a literature review, many sources have been copied as is so beware of plagiarism if you use these words. Plus why rephrase something that is already very clear?

William Gibson Visions of future worlds

In Neuromancer, Gibson envisioned highly complex structures like a digital hallucination, overwhelming and alienating. Gibson coined certain phrases that describe certain interactions with game systems, how a space is ‘consensually hallucinated’ in a ‘non-space of the mind’.  Given user centered developments in HCI and transparent natural user interfaces (NUI), graphic representations now would conceal this Gibson’s complexity and to visualise the underling systems no standard geometry and dimensional space could accurately grasp this vision. Why would we want to see everything? As  Aarseth (2007) describes in Allegories of Space, Gibson’s vision is highly perceptive, bleak and ironic but this irony is missed by many futurists that it inspired. Additionally in this essay, Aarseth laid out how structurally different, games are as a new medium. In this capacity they must engage a player through new structures of interaction, by breaking with familiar forms of representation. But to what end does this enhance experience, and on a subtler point what should be the goal of these new mediums? How can they address shortcomings in previous experiences, and the consequences of their success; such as addiction and excessive emphasis on violent interactions.  Aarseth goes on to define spatiality in games: 

"The defining element in computer games is spatiality. Computer games are essentially concerned with spatial representation and negotiation, and therefore the classification of a computer game can be based on how it represents or, perhaps, implements space. More than time (which in most games can be stopped), more than actions, events and goals (which are tediously similar from game to game) and unquestionably more than characterization (which is usually nonexistent), games celebrate and explore spatial representation as a central motif and raison d’être. In terms of playability, themes, tasks, subgenres and dramatic structure, nothing much has changed over the last two decades. The innovation takes place in spatial representation, and the genre’s more slowly evolving complexity in the other areas (such as physical simulation) can be seen as a result of the increasing complexity of the spatial representation." 

– Aarseth, E. (2007). Allegories of space.   Space Time Play , 44-47.

Together this proposes how games can be used as mediums of experience in themselves, what knowledge can they convey? This then poses the question of how we define space, beyond a physical abstraction of geometry. What makes a social space? How do we generate regions of space for different purposes. How do computer mediated spaces differ?

“Rilke wrote: ‘These trees are magnificent, but even more magnificent is the sublime and moving space between them, as though with their growth it too increased.”

– Bachelard,   Poetics of Space,   Beaccon, 1958, p. 201

In the Poetics of Space Bachelard is, in part, inviting designers to re-consider how users’ experience by encouraging reference to formative, intimate spaces rather than formal abstractions. The quotation of a quotation above is useful as it prompts us to consider game spaces on two fronts – poetically (‘the sublime’: what does this space invoke? ), and kinetically (‘moving space’: reminding us that, above all, we are concerned with dynamics and locomotion ). Both these aspects are useful, as they allow us to consider and assess qualities of virtual-spaces that side-step ‘newer must be better perspectives’.

Ecological Perspective

[space is] neither a mere ‘frame’, after the fashion of the frame of a painting, nor a form or container of a virtually neutral kind, designed simply to receive whatever is poured into it. Space is a social morphology: it is to lived experience what form itself is to the living organism, and just as intimately bound up with function and structure. 

– Lefebvre, The Production of Space

This poetic quote summarises the nature of human relationship to space, a view that was built upon by James J. Gibson. James Gibson’s theory of direct perception states that the environment contains all of the information needed to specify its properties. Hence,perceiving these properties is a matter of detecting the information available in the environment.  This concept is significant precisely because it allows James Gibson to locate the moving, perceiving body at the heart of meaningful perceptual experience; an experience which emerges in the dynamical structures that cross the body and the world.  His theory of environmental affordances shows that perception is more than a means of passively representing the intrinsic physical organization of objects. Perception is inherently active and exploratory, we seek out alterations in the vast flow of information enveloping it. These alterations are detected when the perceiver moves through the environment and probes it with a pair of glancing eyes.  Locomotion opens up new possibilities for the pick-up of information specific to the perceiver’s environment. This information can then be used to guide subsequent movements, as in a perception-action loop (the two being inseparable). As David Morris puts it, “we do not … perceive naked properties of the environment, rather we perceive what the environment affords to our bodies, what we can do with, or in the environment.” The theory of affordances demonstrates that the structures of information are intrinsically meaningful for perceptually-guided action.  Affordances are not subjective valuations superimposed on sensations (as theories of perception typically presuppose). Rather, the affordances of the environment are directly perceived as structural information in the environment. 

– Braun, The Structures of Perception: An ecological perspective, 2008

Of all the terms we use to describe the world we inhabit, [space] is the most abstract, the most empty, the most detached from the realities of life and experience. Biologists say that living organisms inhabit environments   not space, and whatever else they may be, human beings are certainly organisms.

– Tim Ingold, Against Space: Place, Knowledge, Movement

Acoustic Spaces

“perception must be understood as a relationship between environmentally available information and the capacities,sensitivities, and interests of a perceiver.” 

– Eric Clarke, Ways of Listening: An Ecological Approach to the Perception of Musical Meaning

To use ecological perception theory to determine musical meaning, Clarke identifies properties of musical sounds that “afford” certain meanings. Affordance is a term coined by Gibson to describe the purposes or uses of various objects as communicated by stimulus. As an example, a chair affords sitting,taming lions, and knocking down bad guys. Clarke uses this concept to show how listeners can glean meaning from a musical performance. Note that he either stipulates a specific listener and a specific performance, or gives ranges of possible meanings dependent upon the context of the performance and the listener.

Interesting Links not discussed in detail:

Leap Motion Blog post about VR OS

Harrison and Dourish’s Work on Collabritive spaces