mesh deformation using audio data

Demo Here

  1. Loads .ply mesh into WebGL

  2. Get frequency & time domain data from audio stream

  3. Load audio data into GPU and deform mesh in vertex shader

Deformation Modes

  • Time domain - displace along y axis

  • Frequency domain - displace along mesh normal

Postprocessing

  • Kaleidoscope

  • RGB Shift

  • Dot Matrix

  • Impulse Detection