Beyond the Senses EP release
What is a Virtual Concert?
This was the first big show we did using the new Wave Studio tech stack I designed. This includes:
- Xsens Mocap Suit
- ManusVR gloves
- ARKit Face Tracking
- Pioneer CDJs and Mixer
- Bunch of projectors and TVs
- An bunch of wires
- The Wave crunching all this data in realtime in Unity
We Stream this out to multiple online video streaming services, like Twitch, Youtube, Facebook, Periscope etc…
But also stream this into the Wave VR and desktop clients where you can go and watch the show on your computer at home and see all the animation and audio data coming through
We’ve built a custom Camera Man build that can do realtime HD renders and crossfading, without the VR or desktop controls. We can use an XBox controller to fly the drone cam! Or have prebuilt camera animations.
And obviously we’re triggering all the visuals Live over MIDI (at least the stuff that’s not audioreactive)
Here’s the stream (skip to 20 min in):
The cool thing about this tech is that it’s all realtime, so you could, say, create a music video pretty easily! Here we had our Artist Daniel Sierra Live VJing, and the venerable Strangeloop was flying the Drone cam! It’s the future of digital collaboration! Of course then the guys at Strangeloop Studios did a sick edit and threw in some more goodies.
Behind the scenes
It was very hot, LA in the middle of the summer and we have this tiny office. And when we have all the GPUs and sound system running there isn’t enough power left for the AC to actually work. Here I am looking intensely at some very important performance information probably relating to bits per second. Aaron Lemke is in the zone manning the cameraman build. Note the 2 keyboards for double hacking.
We got this cad model of the actual stage setup REZZ would use in her tour! So we put it into Unity and hooked up lights and video walls! I hope somebody goes to her show and thinks, for some reason this looks familiar….
some early avatar tests
attendee collision bounds
realtime twitch interaction via Chatty moth technology
“This is literally the most insane thing I’ve ever seen in my entire life” - REZZ