Optical Soundsystem is a speculative design prototype for an augmented reality (AR) application which would allow you to "see" music within your environment. With it, you’d identify a speaker in your home and generate a real-time visualisation of the sounds you're listening to; allowing you to experience your music not just aurally but also visually and spatially. It would look really pretty, too.

'We wanted to develop something that anyone, anywhere, would be able to experience. Devices aside, augmented reality and virtual reality are inherently accessible technologies – so we just did our best to embrace that.'

Optical Soundsystem would show a user visualisations of the waves and frequencies emitted from a single or multi-speaker sound system setup in an impressive display of colorful formations. The sound waves would bounce off and wrap around objects in your apartment and you’d be able to customise what kind of style you’d like to see.

The visuals would be achieved by identifying the speakers and their orientation with object detection and pose estimation using a modern camera-phone, as well as configuring each speaker to synchronise audio coming from it. Simultaneously, a three-dimensional (3D) model of the surroundings are constructed using LiDAR. The visual 'sound waves' are a 3D representation of the music as it moves around and interacts with the surroundings.

Although Optical Soundsystem would integrate a new visual device into our environment, the idea was founded on the desire to expand the users’ perception of a space. It would be about adding a new sense (vision) to something we’re all familiar with – the experience of listening to music at home – in the most tactile and spatial way possible.