Virtual reality: How will it change the way you make music?

Making music in the modern age generally requires a lot of intense screentime, but we’re all prone to distractions as we find our music-making workflows getting stale. However, what if you could just ditch the keyboard and mouse completely and dive right inside your DAW with nothing to remind you of the mundanity of the real world?

The implications of virtual reality and its related technology on music-making are profound – the total immersion in the creative process and the positional accuracy of Ambisonics will undoubtedly result in greater focus, while the growing popularity of VR media will require greater technical understanding of mixing for the virtual world.

Let’s take a look then, at how this technology has developed to its current level of sophistication, how it works and how, potentially, the game is being changed forever for the composers of the future.

Music-making and VR

As sound engineers and music producers, there are numerous ways that the development of virtual reality software is going to impact on us. On the one hand, there’s a growing number of game-engine VR apps aimed squarely at music creators and producers, while there are also new technologies and techniques we’ll need to master in order to record and work with audio for cinematic VR.

On the music-making front, a lot of the current crop of apps are little more than elaborate toys – VR drum kits are common, as are sound-to-graphics visualisations that place you within a visual wonderland that’s generated from the music you’re listening to (the hit VR game Electronauts for example). There are a few notable exceptions, though, which hint at the tantalising possibilities VR offers to music-making.

One of the best of these has to be LyraVR, which places you in a virtual environment with a palette of samples, sounds and instruments along with various controllers – a keyboard, drum pads, and the like. These tools allow you to create and record hits, notes and chords, which appear as colourful gem-like objects that hang in the air around you, and that you can move around in space to change their characteristics.

Sequences are built by drawing links between these objects, with a visible time pulse running through the linkages. It’s an absorbing and unique music-making experience, although at this stage, the lack of DAW integration may deter many professional music-makers, who’ll miss their professional synths and sample libraries.

However, The Music Room from Melbourne-based developer Chroma Coda can fulfil this wish: it provides a collection of instruments that you play in VR, and which act as MIDI controllers that can be used in conjunction with any DAW, or with a built-in, 8-track version of Bitwig Studio. You can play drum kits that map velocity and virtual strike position to articulations within suitably equipped drum samplers such as BFD3; a Jean-Michel Jarre-like Laser Harp played by striking and strumming beams of light; a set of glowing neon tubes that you stroke and caress to make notes and chords; and more besides.

Loki Davison is the director of Chroma Coda, and tells us that The Music Room was born out of the desire to “explore how VR could be useful for MPE and continuous pitch-style control for Bitwig. I’m mostly focused on guitar and bass but I’ve always been interested in having some of the expressive elements of the double bass for electronic sounds while also being easier to play in tune. That was the inspiration for all the note bending and slide instruments in The Music Room. The controller vibrates when you’re in tune, so you can feel the frets.”

This MPE support allows for much greater expressive control than standard MIDI controllers and is a perfect match to The Music Room’s very cool collection of instrument controllers. The room itself can take the form of a selection of virtual spaces – studios, practice rooms and live venues – intended to inspire your performance and creativity. It’s quite a pricey app compared to most VR software, but is in-keeping with what you’d expect to pay for a decent DAW plug-in. “VR means that as an instrument maker I have different constraints,” Loki tells us. “VR gives you other ways to interact with synthesis or effects. This translates into giving the musician new options.”

If you prefer to play decks rather than instruments, then Tribe XR is going to be of interest. Its virtual CDJ decks and mixer are conventional enough, as is the way it works with your library of tracks and tunes, and in this sense alone it’s a very inexpensive and accessible way to play with such hardware (if you already have the VR hardware, that is). It provides access to various learning tools, too, such as video and interactive tutorials, community support, and even one-on-one live instruction from professional DJs.

But DJ’ing is much more fun as a social activity than a solitary one, and this is where Tribe XR becomes really fascinating: you can invite friends to collaborative sessions, with their virtual-world avatars appearing in your virtual room, or you and your avatar can join them in their own sessions. Basically, you can get your mates around to your virtual pad for a bit of a party! Tribe XR also allows live streaming with built-in support for Twitch, and budding superstar DJs can try to grow their following by booking a slot on the Tribe XR Radio Show.

All of these apps offer innovative takes on conventional ideas, but Drops: Rhythm Garden takes things to a whole new level of abstraction. The app does away with instruments, sequencers and controllers altogether, and instead allows you to create fascinating, complex rhythms and grooves by creating objects and structures onto which drops of liquid are dripped. The materials and shapes you use in your structures influence the pitches and sounds produced when a drop strikes a surface, while the size, number and drip-rate of the drops determine amplitude and timing. Drops can dribble off one object to then drip onto another, creating even more syncopated loveliness. It’s an enchanting, organic and deeply engrossing way to create music.

None of the apps mentioned (with the arguable exception of The Music Room) are going to let you build an entire, polished, finished track. Yet they amply demonstrate the potential of VR music-making and reveal a new world of creative tools limited only by the imagination. But there remains the sense that many of these ideas need to be honed and polished by people who really know about digital music-making. That is, the Steinbergs and Native Instruments of this world.

These big-name developers are very tight-lipped about future VR plans, but it stands to reason that these companies will undoubtedly be researching and developing VR (or VR-related) products. It may be a while still before we start seeing the fruits of their labours, as there are significant hurdles to clear, not least how to combine the CPU-intensive task of audio processing and mixing with the similarly processor-intensive task of generating a coherent interactive virtual world. But these issues aren’t insurmountable, and should music-making in virtual reality start to become a new norm, we may find ourselves in a future where an HMD will be as important in the studio as a DAW is today.

A long road

The lack of integration between DAWs and VR-based music-making apps is one of the biggest challenges to working with such tools: the fantastical world you may be inhabiting to play physically impossible instruments isn’t much use if you can’t easily integrate your creations with the rest of your studio. Apps such as LyraVR allow you to save your creations as audio files that you can import into a DAW, which works but isn’t the smoothest or simplest of workflows.

The 8-track version of Bitwig Studio built into The Music Room shows a different solution, allowing you to open and work with Bitwig’s windows from within the VR environment, where they appear as panels that hover in space around you. Still, this feels a bit like the instrument is hosting the DAW rather than the DAW hosting the instrument – what’s really needed is a DAW designed specifically for VR and/or AR, and with a plug-in format that supports the creation of innovative VR/AR control panels and GUIs.

Another drawback to making music in VR is the need to wear headphones while you’re within the virtual environment. On the one hand, working in headphones for extended periods can become extremely tiring on the ears, and when that happens, it’s difficult to make critical decisions with confidence. On the other hand, you may be using VR tools to make elements for music that will be distributed in a conventional stereo or surround format, so may want to monitor a static stereo mix of your music even while working on parts within the virtual domain. As a result of these – and other – challenges, making music in VR can be challenging if working alone.

Find out more about recording and mixing with VR in our guide.