Data Visualisation

More trawling results! Here is a few interesting data visualisation techniques and representations. Finding these quite stimulating for the idea of how a human would have to navigate extremely complex data structures. By simplifying the complexity through visual algorithms a form of order can emerge that is almost tangible and readily usable. Hope to integrate some of this work as an influence to the virtual reality interface in the project.  

The example above is particularly mathematics heavy once you inspect the linked paper, imagine this one will just have to be used as influence rather than direct transcribing…  The visual designer’s interpretation and implementation for this project is phenomenal, after inspecting the paper and watching again I am in awe.

This rhizome navigation example reminds me alot of minority report style interfaces with circular information modules utilising layered detail and selectable depth. Unfortunately the source code for the project is no longer available, but in principle a similar node architecture could be mocked up using Prefuse or Processing, trouble is migrating this into Unity. Music is proper hootenanny aswell!


Expression vs Engagement

In the video above Eran Egozy of Harmonix Music, makers of guitar hero, describe how they have always pushed towards a goal of everyone being able to enjoy the expression of music. Egozy presents previous works, and describes nature of game interaction in musical games. The key takeaway of this presentation is the concept of balancing expression and engagement in musical games. Through the iterative development within their games various approaches have been tested in the field, and this presentation serves as a review of their strengths and weaknesses in the context of expression. Egozy poses the question of whether expression can exist in a game environment at all, as no agency is given for creative deviation from the mind of the user. Contrary to this statement Ergozy’s mission statement is to provide levels of musical engagement to anyone, musically proficient or not. Many of the systems deployed work through sophisticated audio and visual feedback that allows development of a game dynamic. In some game platforms levels of virtuosity have been allowed but have received mixed success. In other games no expression was present but large levels of player engagement and flow were observed. In their most recent game development, Fantasia Music Evolved, a balance is said to be struck between game engagement and expressive affordances. By including interactive music content and unique musical variations this game excels in both game and music contexts, while simultaneously supporting one of the most important concepts in game immersion, narcissism, as exemplified by the lasting level of inclusion in the game. An example of the game being played is shown below, albeit a little impaired…

A note on future posts, and a wee taster

For the next few months this blog will serve as container for some research updates. The imaginative category tag for all these pieces will be "Research Updates". 

The research project is in the field of immersive audio environments, meaning many areas are due to be reported on. In general the flavour will be quite scientific but at times some more esoteric artistic approaches will make an appearance. 

As a taster of what to come here’s some videos of influences and general stimulation!

Interactive architecture

Light Piano by   , a light machine instrument that provides a novel interaction space.

Irregular Polyhedron Study #1 – Vertex, edge and volume, found at creative applications

Augmented Audio

In this rather text heavy sprawling post I’ll be briefly covering the present and future possibilities of augmented reality audio (ARA for the purposes of speed). For the uninitiated the augmented audio website provides a comprehensive overview:

“Augmented Audio is defined as live audio being modified and/or enhanced by computer-generated sensory input. By introduction of another layer of information or media on top of an input feed, it is possible to produce highly enriched audio feedback. The concept belongs in the Augmented Reality (AR) family, but has started to distinguish itself as an important independent application area due to recent media attention. New innovative implementations have shown that the concept is applicable in many different contexts, including tourism, location-oriented communication and information sharing, gaming and entertainment and even as a distinctive new interaction technique.”

Also this website provided a link the audio gaming company RjDj (Reality Jockey Ltd). This company was an early developer in the area pushing beyond the confines of the research based approach to fully fledged consumer applications. Their apps won critical acclaim, and are still developing new concepts in audio gaming and general entertainment. Their new situ app is particularly useful for playlist junkies such as myself, whereby it learns your activity cycle’s and related music choices to automatically recommends music. Like a automated, situational, related artist search. Seeing as I’ve spent hours doing this on Spotify its very welcome. Semantic music analysis straight in your lugs, and its free!

To pick up on a article in designing sound (fantastic sound design blog), in which they covered Audio within VR environments. The article mentioned the 3Dception toolset for game design. Surround sound has existed within the gaming community for a long time. But, where 3Dception makes its mark is the ease by which a developer can integrate binaural audio into their workflow with tools for the Unity and Wwise platforms. So, where does fit with ARA? Glad you asked. They are currently developing toolsets for NativeSDK, Pure data (and libpd…hence android and iOS) and Max/MSP. This then allows for construction of augmented environments using sensor data from the respective platform, ta da, augmented reality! This coupled with their interest in integrating procedural audio could make some amazing possibilities for dynamic reactive audio environments.

(Additionally from the article on designing sound I was pleased to find this chap in the comments. Frank the terrorist is a homemade dummy head used for binaural recording, and it works pretty well, the recent thunder storm clip is very detailed)

So now we have some ARA options, what else are we going to need? In steps the intelligent headset with its uninspiring promotional videos and prohibitively high cost except for affluent early adopters. Not that prohibitively high cost has stopped many things in audio geekery. This being said dedicated hardware for ARA is a good idea, as positioning of sensors is not optimal when they all stored within a handset. By adapting headphones with accurate positional indicators the HRTF algorithms can happily match head position to aural environment. Also indluded in the headset is GPS, compass, gyro and accelerometer. One key element missing, in my humble opinion, that is accurate stereo microphones so that augmented audio can be layered on top of existing real world audio. Though quoted as a ARA solution I think it fundamentally misses the mark. In truth this is just a well optimised binaural solution.

Also Dolby recently launched their Atmos mobile venture inside the new Amazon kindle. This lab note is a great technical description and well worth a look. But whether this technology can be adapted for ARA remains to be seen. Dolby haven’t yet created any tools beyond that of professional services. Though this maintains brand integrity it does stifle the development of the technology in the public sphere. This has the benefit of separating professional products, such as Hollywood films, from other content. But innovation in a medium does not always come from the establishment. So maybe it would be worth Dolby developing accessible production tools to allow smaller ventures to endorse and enhance the technology in a organic sense. A Dolby mobile API maybe?


 To finish, if you are still reading, I would like to point you in the direction of a fantastic tech blog praxtime. In the article highlighted a series of possible wearable augmented reality solutions are presented. These predictions and concepts are based on the trends and possibilities in wearable tech. 

Thanks for reading.


Interference: Journal of Audio Culture

Courtesy of UCLA

Listen too much you have trouble listening? Working in the audio field can often sap you of your basic ability to listen, as you constantly have to produce to deadlines, assess quality and generally conform a product to external conventions. But the most fundamental requirement of sound to a human is to inform them of a position in space, as before verbal communication the auditory system prevented you from getting eaten by bigger predators. This being said the importance of communication cannot be overlooked, as it is tied with our gradual evolutionary supremacy.

In steps Interference. A peer reviewed journal, supported by Trinity College Dublin, that is entirely free access. They describe themselves as follows:

“Interference is an open access forum on the role of sound in cultural practices, providing a trans-disciplinary platform for the presentation of research and practice in areas such as acoustic ecology, sensory anthropology, sonic arts, musicology, technology studies and philosophy. The journal seeks to balance its content between scholarly writing, accounts of creative practice, and an active engagement with current research topics in audio culture.”

Of special note is one article in the current issue Leandra Lambert called “Experienced Sonic Fictions“, which I shall very superficially contextualise for this article. Throughout the introduction of the piece Lambert mentions the founders in the field that established the ‘deep listening’ and ‘sonic awareness’ disciplines, which can be approximated to a form of listening meditation. Proceeding this she describes the process of free-form sound walks, and the associated imagery that is stimulated. As by letting her imagination guide her through these walks the stimulation is less and less guided by any conscious purpose, and in reaction the ideas and concepts imagined become more lucid and fantastic. Though rather random and quite time consuming it does reaffirm the idea that we need to listen to our environments and not try to block them out or classify them to swiftly. Though this capacity for ordering reality is essential in modern life, for a sound designer the ability to stop and actually listen to a scene for all its richness is worth remembering. In many respects it is reminiscent of the John Cage works on silence and of how evocative the absence of direct stimulus is, paradox or contradiction I’m not sure?

Coming back to the opening gambit, though sound walks may not be for you, the idea that to truly assess and recreate a sound scape one must remember how to listen is a very important skill. How you choose to do this can come in many forms, as with all creative processes, but it is a important principle as audio technology reproduction methods approach the means to reproduce true soundscapes to a mass market.

Another journal of note is that of SoundEffects, also open access and very stimulating.

Roli Seaboard Grand

Just in case you haven’t seen it, its awesome. Part of the new breed of expressive musical controllers that allow new dimensions of expression in electronic music. The company based in Dalston, London are pushing forward with beautifully designed products that don’t compromise on key functionality. Go visit their products page for a better look!

The continuous pitch control mechanism allows a performer to introduce musical “systematic deviations”. These deviations are the things missing from standard electronic music controllers unless all kinds of other controllerism is exploited. This simple yet elegant approach can be used to faithfully recreate original acoustic instruments or explore new depths by providing tactile micro control to an array of synthesis parameters. Plus its got squishy keys which is always nice.