Discrete REconfigurable Aural Matrix or DREAM interface is a research on spatial sonification conducted between 2006 and 2010. Its primary focus was the assessment of 6×4 anterior speaker array’s ability to portray geometric shapes using exclusively sonification and spatialization with the ultimate goal of designing an assistive tool for visually impaired as well as to enhance existing data monitoring paradigms with a robust sonification component. The pilot received funding from the Virginia Tech‘s College of Liberal Arts and Human Sciences ($32,000) and an additional funding from the Institute for Society, Culture, and Environment ($20,000).

Users had an opportunity to listen to the layered (layers A, B, and C, were combined to generate composite image D) sonification of geometric shapes and were given a Nintendo Wiimote with a visible cursor to paint what they heard onto mosquito netting hung on top of the speakers, so as to provide a consistent projection surface, as well as limit users’ ability to cheat by observing vibrations of speaker membranes. Rendered shapes were compared to that of reference images generated by computer to assess their accuracy. Based on results collected through a user study, 73% users were capable of accurately perceiving geometric shapes, 78% shape location, and 56% shape size.

The second phase focused on the display of up to four concurrent shapes in order to portray an imagery (e.g. a sunset over mountains with a lake), setting stage for more complex scenes and a new expressive paradigm, the aural painting. Each shape used a spectrally complementing sound that is psychoacoustically linked to the context it represented (e.g. sound of water for the water surface). The ensuing system designed in MaxMSP rendered 288 channels of audio in real-time with <20ms latency. Based on the similarity assessment of user data, users’ accuracy varied between 86.01% and 88.54%, with the biggest drawback being the time required for comprehension. In addition, users were capable of interpreting the overall image context and have found the overall expirience soothing, mainly due to choice of relaxing sounds found in nature.

Research Publications:

• Bukvic, I. and *Ji-Sun Kim. (2010). “PERCEPTION AND INTERPRETATION OF CONCURRENT AURAL SHAPES USING DREAM INTERFACE.” International Computer Music Conference, Stony Brook, New York.
• Bukvic, I., D. Gracanin and F. Quek. (2008). “INVESTIGATING ARTISTIC POTENTIAL OF THE DREAM INTERFACE: THE AURAL PAINTING.” International Computer Music Conference, Belfast, United Kingdom.