Jun 2019

Beyond the Senses EP release

What is a Virtual Concert?

This was the first big show we did using the new Wave Studio tech stack I designed. This includes:

  • Xsens Mocap Suit
  • ManusVR gloves
  • ARKit Face Tracking
  • Pioneer CDJs and Mixer
  • Bunch of projectors and TVs
  • An bunch of wires
  • The Wave crunching all this data in realtime in Unity

We Stream this out to multiple online video streaming services, like Twitch, Youtube, Facebook, Periscope etc…

But also stream this into the Wave VR and desktop clients where you can go and watch the show on your computer at home and see all the animation and audio data coming through

We’ve built a custom Camera Man build that can do realtime HD renders and crossfading, without the VR or desktop controls. We can use an XBox controller to fly the drone cam! Or have prebuilt camera animations.

And obviously we’re triggering all the visuals Live over MIDI (at least the stuff that’s not audioreactive)

Here’s the stream (skip to 20 min in):

The cool thing about this tech is that it’s all realtime, so you could, say, create a music video pretty easily! Here we had our Artist Daniel Sierra Live VJing, and the venerable Strangeloop was flying the Drone cam! It’s the future of digital collaboration! Of course then the guys at Strangeloop Studios did a sick edit and threw in some more goodies.

Behind the scenes

It was very hot, LA in the middle of the summer and we have this tiny office. And when we have all the GPUs and sound system running there isn’t enough power left for the AC to actually work. img img img img Here I am looking intensely at some very important performance information probably relating to bits per second. img img img Aaron Lemke is in the zone manning the cameraman build. Note the 2 keyboards for double hacking.

img We got this cad model of the actual stage setup REZZ would use in her tour! So we put it into Unity and hooked up lights and video walls! I hope somebody goes to her show and thinks, for some reason this looks familiar….

some early avatar tests img

img img

img attendee collision bounds

img img img realtime twitch interaction via Chatty moth technology

img img


“This is literally the most insane thing I’ve ever seen in my entire life” - REZZ