2024-01
This interactive piece allows spectators to engage directly with Kandinsky's works through gesture interaction technology. By doing so, they can activate sounds intricately tied to specific elements within the paintings, thus resonating with the synesthetic theories and artistic ethos of the early 20th century.
This installation explores the interplay of technology with the essence of artistic spirit. It serves as a modern interpretation of cross-media expression, enriching sensory experiences and offering new perspectives.
Accepted for the 2024 HarvardXR Conference.
Co-author: Jiaying Fu, Hongxin Fu, Tiange Zhou*
We utilize Leap Motion to track the hand movements of users and determine if their hands are within any predefined shape areas. When a match is identified, corresponding visual effects are triggered along with sound output.
This is the first time I have seriously engaged with new media art. Using TouchDesigner and Max/MSP has been intriguing, as these tools don’t require extensive coding but still embody programming concepts. I’m pleased to discover that the skills I acquired in college can be used to create artistic works, not just technical projects. This project is still in its early stages, and there is much room for improvement. For instance, the sound currently utilizes simple chords and matrix music, which could be developed into something more sophisticated.
Nevertheless, this has been a great beginning for me to explore the intersection of art and technology.
We utilize TouchDesigner to recreate Kandinsky’s painting, bringing it to life with dynamic animations such as rotation. When an audience’s hand is detected within the area of a specific shape, that shape will scale up to twice its size.
The foundational layer of our background music is generated by MusicGen. And We employ Max/MSP to produce distinct sounds corresponding to different shapes.
、