IBM Cloud Instrument
Powered by IBM Cloud™; the project aimed to create the new aesthetics of “connectedness” using multimedia and immersive experiences that offer the interconnected world of Internet of Things (IoT) and Artificial Intelligence (AI). The Internet of Musical Things (IoMust) represents a coalescence of objects that share a collective “connectedness” in natural and human-made worlds. As an interactive artpiece, the installation embodies movement, changing based on real-time data input.
INTERACTIVE AUDIO VISUAL INSTALLATION
The enablers of the Internet of Musical Things (IoMust) concept are powerful IoT technologies like IBM Watson supercomputer cluster, motion sensors, WiFi-enabled micro-controllers, LED displays, and back-end servers.
The spacial 3D-layer mapping framework and tools where connecting link to hand gestures and sound parameters. This gesture-instrument mapping system was later expanded to more audio-visual systems of manipulating input data data streams among diverse media systems like Notch and Touch Designer; heterogeneous interactive performance environments.
Through the complex network of technologies, we invite our audience to perform by controlling parameters of the analog synthesizer Moog Slim Phatty and observe the performance through audio-reactive visuals in real-time.
FOR NERDS BY NERD
Another framework supporting sensor-based interactive performance is Wekinator and Leap Motion which seeks to extend music language toward gesture and visual languages, with a focus on analyzing expressive content in gesture and movement, and generating expressive outputs.
In particular, the sensor inputs of Leap Motion are fed to Deep Learning based AI and Machine Learning (ML) algorithms that are integrated in the IBM Cloud. Node-Red framework in the IBM Cloud allows us to create a connected network of art objects and human-computer interfaces, and multimedia frameworks such as Max/MSP, Notch, TouchDesigner, and custom software written in Python.
Project Tags
Companies
IBM
- Technology
