Kandinsky Sonified

Wassily Kandinsky would wanna play with this. Greatly inspired by his aesthetics and synaesthesia on abstract colors/shapes, this is a designing trial to combine the graphics with the acoustics in an interactive way.

Each shape has a timbre, and each color is mapped to a pitch frequency. The volumn of the shape determines the loudness (and certain effects behind). The position determines the panning (audio image).

The interface is pretty intuitive. You can draw squares, lines, and circles by first selecting the corresponding shapes in the toolbar. The color picker is designed to generate a random color with each click. And the eraser can erase a shape by clicking on it. Further interactions include

• Scroll mouse wheel to adjust the playback speed. You can even go reverse!
• Right click switches the sweeping axis.
• When no tool is selected (click again to de-select one), left click on the canvas moves the play head to the mouse position (You can potentially DJ via this).

Codes can be accessed here. Make sure to install the latest ChucK and ChuGL versions. Run the code via chuck kandinsky.ck.

Stories & Details

I love my way towards the final design! (final as for deadline, not final as a design). After the mechanism was established at the milestone, I spent quite some time refactoring my code, puting it into multiple files using the brand new import system. More tweaks happened in visual hints and timbre selection.

Architecture:

• constant.ck: storing constant values across files.
• draw.ck: toolbar design, tool selection, drawing mechanism.
• mouse.ck: mouse class.
• play.ck: timbre design, acoustic system.
• shapes.ck: shapes, playing mechanism.
• kandinsky.ck: main file.

Thanks Kunwoo and Andrew for their support!