What could Apple’s Vision Pro mixed-reality headset mean for music production?

It feels far-fetched, doesn’t it – mixed-reality goggles being useful to music producers and DJs? But Apple wasted no time at WWDC23 in showing that you can slap on the Vision Pro headset and start scratching virtual vinyl with an augmented-reality version of Algoriddim’s djay app.

The dust has settled since WWDC, but with the Vision Pro’s “infinite screen real estate,” gestural input systems and “advanced audio experiences,” as Apple says, it feels like more music production opportunities in spatial computing are on the horizon.

READ MORE: What does AI mastering mean for artists, engineers, and music?

Ready Studio One

Candidly, creating music in a lonely mixed-reality bubble initially feels like it paints a dismal, dystopian picture. Collaborating with your peers becomes a completely different experience, and wearing that headset for long production sessions with headphones seems uncomfortable. And will this thing really be able to run massive projects with chains of plugins and virtual instruments? Let’s not even think about that price tag just yet.

Let’s face it, though, many producers often do create in isolation, whether in a bedroom or a studio. And mixed-reality music-making apps aren’t new – SynthVR gives you a modular rig in a virtual space, while Tribe XR gives you access to emulations of high-end Pioneer DJ gear to hone and show off your DJ skills with other users. This month, Korg officially released Gadget VR, a virtual 360-degree studio environment complete with synths, drum machines, MIDI piano rolls, and other arrangement features.

Apple’s clearly not shying away from more hands-on experiences with Logic Pro, either. Last month, it announced Logic Pro for iPad, which is almost a complete port of the desktop DAW to iPadOS. One might wager that, if Vision Pro is a success, the brand will look to ‘spatialise’ its flagship creator apps, Logic Pro and Final Cut Pro.

But whatever Vision Pro can provide, the question is, ‘Who actually cares?’ Who’s going to drop $3,499 on a headset to get a few extra screens and some quirky music apps? Not me. Maybe not you. But maybe Jesse Ray Ernster.

Mixing Reality

With production and engineering credits for the likes of Doja Cat, Kanye West, and Burna Boy, Jesse has spent endless hours mixing and recording in music studios. And you can bet that he’s found plenty of workflow papercuts that a Vision Pro, or another XR headset, may well remedy.

“Immediately, this can replace the need for a display monitor in the room, which has always caused significant issues for me,” he says. “I’m either experiencing a bizarre reflection from the monitor in front of me, or I have a sore neck from looking at it across the room or down on the floor; I just can’t seem to win!”

Though Jesse’s home workspace is fairly snug, he says that he’s “excited for a reality where I can sit down to mix from the comfort of my living room couch, mix in a simulation of a million-dollar space, and get great results and mix approvals while doing it.

“No more upright chair, sore back, and picky sweet spot between the speakers in my flawed room. Rather, a consistent true north reference source that is easily accessible and comfortable for creating with.”

Virtual monitoring environments are already available, too. Waves’ Abbey Road Studio 3, for example, is a plugin that puts you in – you guessed it – a virtual representation of Abbey Road’s Studio Three, where you can mix in the same ‘room’ that Pink Floyd, The Beatles, Amy Winehouse and Kanye West have been produced. Meanwhile, dearVR can put you in a variety of virtual studio spaces to test your mixes.

With head-tracking and a more tactile experience with these plugins, XR headsets could completely change up your mixing workflow. This will be especially true when mixing in immersive formats, such as Dolby Atmos and Sony 360 Reality Audio.

On this point, Jesse adds that he uses Logic Pro to produce all of his Dolby Atmos mixes.

“Between the [Vision Pro] and the forward trajectory of Logic and spatial audio, it will be incredible to have creative writing sessions in virtual studios with virtual interactive instruments, equipment, consoles, and so on.”

Still, Jesse isn’t expecting to transport his entire mixing process into VR. “[This is] not necessarily a replacement for the real experiences in real rooms, but certainly inspiring to think about.”

Meta music-making

Metaverse maestro VNCCII is already inspired by the potential of mixed-reality applications in music production. Conducted by Australia’s Sami Tauber, the VNCCII project has been a means of exploring the vast possibilities in an ever-unfolding virtual landscape – including building the ‘Holoship’ recording studio.

Naturally, VNCCII has much to gain from the launch of a mass-appeal XR headset.

“The Apple Vision Pro and Vision OS launch is super exciting for the VNCCII project and positive for the entire XR/VR/AR ecosystem,” Tauber says. “It unlocks a cornucopia of opportunities for creators and developers.

“I would love to see real-time music production sessions and mixed-reality concerts with the Apple Vision Pro headset,” continues Tauber. “Imagine stepping inside VNCCII’s Holoship music studio environment and providing real-time feedback or co-producing with VNCCII!”

Maybe mixed reality music-making won’t be so isolated after all?

However, the real challenge, Tauber says, is ensuring that audio developers are swift to adapt to Vision Pro and tailor apps to the headset. “Developers must create augmented reality and virtual reality apps specific to the headset and they must apply for the Apple Vision Pro developer kit.

“I already use Apple’s ARKit for animating the face mocap of my VNCCII avatar in Unreal Engine with the Apple iPhone’s 52 blend shapes for LiveLink. Remarkably, Apple will update ARKit and RealityKit to suit the XR headset – ARkit is a renowned tool in the XR space that enables world tracking, image anchoring, scene reconstruction, plane estimation, and hand tracking to build immersive experiences.”

Tauber adds that generative AI for music and mixed-reality apps will be “super-compelling content” for the Vision Pro. Spatial audio with Dolby Atmos will also play a crucial role in audiovisual experiences – Tauber stresses that the audio format will form the “nucleus” of mixed reality.

Tribal headset

VNCCII’s utopia of shared virtual experiences is already underway in one corner of the mixed-reality universe. Tribe XR is currently the leading brand in VR DJing, offering budding disc jockeys the chance to practice with the most sought-after decks and mixers in the industry. With accurate gesture tracking, users can move the faders and pots of a simulated DJ mixer and turntables to blend tracks as in real life. You can even invite friends to DJ and dance along with you.

https://www.youtube.com/watch?v=pqbPdnBELqA

In the words of co-founder Tom Impallomeni, Tribe XR is “on a mission to turn aspiring DJs and performers into professionals – essentially, give beginners all the tools, the virtual studio access and community access to teachers, workshops and so on.

“Then, we get them to the point where they can actually perform in the real world.”

Currently, Tribe XR is available on Meta Quest, Oculus Rift, Pico, Viveport and Steam VR platforms. That means it’s available right now – you certainly don’t have to wait for the Vision Pro to drop to start mixing tracks. Better yet, you can pick up headsets like the Meta Quest 2 for a tenth of the price of the Vision Pro’s projected price.

So why is Apple’s mixed-reality venture important for companies like Tribe XR?

“It’s a great signal that Apple is seeing the broader spatial computing market as not just games, but music technology, learning and other use cases,” says Impallomeni.

“Many people use virtual CDJs in Tribe XR to learn because they can’t afford $6,000 to $10,000 for two to four decks and a mixer. So Tribe XR is an entry point for them to learn on this equipment.”

Even still, at $3,499, the Vision Pro combined with Tribe XR is still more affordable than a venue-standard DJing setup. Two Pioneer DJ CDJ-3000 players and a DJM-A9 mixer will set you back around $7,800. And in the virtual space of Tribe XR, you don’t have to rent out a studio or worry about upsetting the neighbours with blistering jump-up drum ‘n’ bass at 1 am.

With a Pioneer DJ partnership, Tribe XR’s arsenal of mixers and players is abundant, too. Want to swap from a DJM-A9 to a V10 six-channel mixer? Easy. Want another two CDJ-3000s in the cockpit? Done.

“It’s not a toy,” continues Impallomeni. “This will truly teach you DJing… We didn’t build Tribe XR as a replacement for the CDJ. We built Tribe XR to democratise access to this equipment.”

Tribe XR might sound like it’s serving a small, niche market, but Impallomeni cites rumours of Meta selling over 20 million Quest headsets. With Quest 3 arriving later this year, Impallomeni expects this audience to grow – and the Tribe platform to grow alongside it.

“We expect the market to grow quickly. At WWDC, Apple presented a clear vision for spatial computing. We believe that spatial computing offers huge potential to Tribe XR and other learning platforms… Apple is now a legitimate contender and this will increase the pressure on Meta to innovate. So now we’re looking at Meta and thinking, ‘Okay, it’s time to step up the quality of your products.’ And that’s exactly what the market needs. We need the big players investing heavily in the space because that’s what will ultimately enable spatial computing to reach mass-market scale.”

Tribe XR’s potential audience and demographic may grow, but will die-hard DJs be willing to adapt to new technologies?

“The DJ market doesn’t move particularly fast,” says the Tribe XR co-founder. “It took a long time for DJs to move from vinyl to digital DJing, and for CDJs to no longer need CDs. There has been plenty of scepticism towards new technologies, but we’re finding that many DJs eventually see the value in new technologies, and ultimately become really creative with these technologies.

“The latest CDJs, mixers and controllers can do incredible things; you can essentially produce music on the fly. New technologies like stem separation add to the creative armoury of a DJ. We’re bullish about the potential of this technology when combined with spatial computing. Once DJs see the creative opportunities, we believe they will really embrace it.”

The Vision

Apple’s headset isn’t going to immediately change how we listen to or create music. It’s unlikely that producers will scramble to get their hands on the first version when it drops. The price alone is alienating for most of us – VNCCII agrees that the “monstrous” price tag will “automatically price out mass accessibility to the broader population.” She hopes, however, that the price will become more accessible over time; “like any new fancy toy, it charges a premium for accessibility.”

Still, the potential of the Vision Pro is palpable, with tech that has surpassed many enthusiasts’ expectations. “I’m impressed with the high-performance Apple M2/R1 chips and the precise head and hand tracking, which powers real-time 3D generated avatars for immersive presence and connection,” continues VNCCII.

Impallomeni adds that Vision Pro’s tracking capabilities are a boon for developers: “Eye tracking will enable developers to simplify the UX. Users can look at inputs or tracks, then pinch, grab and manipulate those controls and sounds… DJing is an excellent showcase for hand tracking within these headsets.”

Even with its impressive feature set, the Vision Pro will be a tough sell. Will Rick Rubin and Mark Ronson really be donning mixed-reality headsets in two years? Sceptics are already bemused by spatial audio mixes – spatial computing probably won’t take off so quickly either. But, as Impallomeni said, it may well be worth embracing mixed reality, rather than shrugging it off. Perhaps we’ll find these means of expanding our creativity aren’t so far-fetched after all.

Learn more about music technology trends. Find out more about the Vision Pro at apple.com.