Where Does Virtual Reality Really Belong In Video Production?
VR is already being used in film and television production, and it is here to stay. But where exactly does it fit in?
A few years ago I wrote an article about using VR in film production, and I thought it was time I updated that. With a little insight and careful thought, here are my suggestions for making VR thrive in the Production pipeline.
Let’s dive back into the discussion from the last time. In my first article, I wrote about how the PlayStation’s VR headset was picking up steam, and how smartphones could be used in rudimentary devices, like the Google Cardboard, to turn them into makeshift VR headsets. Since then, things have come a long way in a relatively short time.
For one, Google Cardboard is now just an oddity of the past, a bizarre footnote in the development of modern VR systems. Don’t get me wrong, you can still buy these! In fact, they don’t only come in corrugated heavy paper products anymore. You can get them in all sorts of colors, styles, materials, and even ones that look like the old stereoscopic image viewers from the 1800s.
However, it is a discontinued product line. So while they may still be available for purchase, and the app will still work (for now), they aren’t likely to release any new updates for it. Nowadays, the Google Cardboard is more like the 1980s ViewMaster toys, but for the 2010s.
Let’s look at some of the newer headsets that have been coming out.
Instead of paper boxes, we now have legit mainstream VR headsets, like in the movies.
Sony PlayStation’s original VR headset just got a huge upgrade to the PlayStation VR2 for the PS5. This new headset boasts some impressive capabilities, including pupil tracking, to increase the resolution only where your eyes can see it, an ultra-wide view, haptic feedback, and a very high-resolution display.
You also have the Valve Index VR Kit (where’s Portal 3, Valve?), one of the more highly rated headsets. It’s such a successful device that, as of writing this, rumors are saying the Index 2 may be dropping sometime this year!
There’s also the HTC Vive Pro 2, and the HP Reverb G2. Both are well rated although they have different pros and cons. You can read more about all these headsets here.
However, the big name in VR headsets right now is the Meta Quest, yes the company that owns Facebook, has released both the Meta Quest Pro and the Meta Quest 2. Formerly known as the Oculus Quest, the new Meta Quest headsets hit 2 key demographics. The Pro is by no means cheap, but also not nearly the most expensive, and yet it can cater to the Professional and High-End crowd as well as any of them, if not better. While the Quest 2 reigns as the top VR headset so far, with a price tag that makes it an easy buy for most people curious about VR. I purchased one myself, and while it does have its limitations, it is a very good product at an amazing price that’s hard to pass up.
There are a few things I suggest when it comes to picking out a VR headset. To start, what do you want it for? Do you need something for gaming or game development? Or are you looking for something portable and versatile? Or maybe you just need the highest quality money can buy. Consider your needs before buying one, to avoid buyer’s remorse. Personally, I find that the Quest 2 is very versatile. It not only has its own onboard OS which can store and run a large selection of programs, but it can connect wired or wirelessly to your PC (as long as it meets certain requirements) and let you access even more programs, often ones that require more processing. Plus the fact that you can take it anywhere and use it without setting up a bunch of sensors gives it a big boost in my book.
Second, consider not just the hardware, but the software. The PlayStation VR2 has access to an exclusive library of content, which adds a lot of value for the gaming crowd who already loves titles like Horizon: Forbidden West. Meta has given the Quest series a large library of apps, both stand-alone for the headset and pc-tethered. The one drawback here is… Facebook. A lot of money and attention has gone into the development of Zuckerberg’s dream child, the Metaverse. But it has not been nearly as well received as they had hoped. In fact, it is less of a feature and more of a drawback at this point. While it doesn’t require you to use it (yet), the fact that they are putting so much effort into this failing endeavor makes you wonder if they shouldn’t be putting that money elsewhere.
But what does this mean for Production? Simply that the options are growing, and there is a range of products that will fit almost any budget you need. However, I don’t think we’re going to be walking into theaters, putting on a headset, and instantly being transported into another interactive world. Most of the content you’d think of like this wouldn’t even be real VR. They would be 360 videos. Simply put, true VR requires the user to be able to change their perspective and interact with objects. 360 video (live-action, animated, or otherwise) is a passive experience, like going to the movies, where the user can only look around them, and see in stereoscopic 3D. They would have no way of moving through the scene or lending the protagonist a helping hand. Sure you need the same headset, or any smartphone with the right sensors, to do this too, but it isn’t taking any real advantage of the technology in the Production pipeline.
Besides, stereoscopic 3D in theaters is falling by the wayside. While it is still out there, and still making money, it isn’t the spectacle it once was. When Avatar (the blue people, not the magic Tai Chi people) first hit theaters, the 3D was otherworldly, immersive, and like nothing, anyone had seen. By the time Avatar 2 came out last year, we’d already seen it, done it, and many of us have moved on. The stereoscopic aspect aside, there isn’t much to be gained from 360 videos for movie production. All it really does is add more cost and complexity while limiting what the creator can do to progress the narrative. Most of the 3D movies produced now don’t even use real 3D, captured on set. Instead, they use post-3D effects, where a team recreates the scenes of a movie but only generates the z-dept data, which is then applied like a filter on top of the movie. Sort of.
In its current state, it appears VR is not well suited for the movie-going audience, at least on the front end. It will grow and gain popularity in interactive media, but when it comes to film production, where does it fit? The answer is a little surprising actually.
Where doesn’t it fit?
In 3D animation, it is all too easy to move the camera around in a 360 environment. It is exactly that ability that gives VR an advantage in almost all aspects of filmmaking, from pre-vis to post-work.
When it comes to design, using an interactive 3D system gives you a whole new take on how to build and design everything from costumes to entire worlds. 3D software, like Blender and Maya, can let you access VR cameras, which allow you to walk through a scene, or interact with an object with your hands.
Personally, I used my Quest 2 recently to sculpt a 3D model of a character. While I wasn’t yet able to take the model to the result I wanted entirely in VR, I was able to very quickly and effectively build a character 80% of the way there, and then easily take it into Blender and finish it there. What normally would have taken me a day to finish I did in about an hour.
I also connected my headset to Blender and was able to explore a pre-visualization of a set. While it is still not without limitations and bugs, the potential was obvious. I could far more easily and naturally see angles, scale, and potential camera moves just by looking around me.
On-set production can even use VR, depending on the project. Virtual Sets, like those used on The Mandalorian, are becoming more popular, and the ability to move a character or camera around in 3D space on the fly is going to be essential.
Post-work will see the biggest advantage. The ability to share files remotely, while working collaboratively, in the same virtual environment, will revolutionize how people work and interact, especially post-pandemic. Imagine going into an office, pulling up the latest design of the big bad monster for your production, and being able to see every detail like it is standing right there, and then providing detailed feedback at that same moment to another artist, who can make those changes while you are watching. All without getting out of your pajamas.
In short, the flexibility and versatility Virtual Reality can provide to a creative team working on a production of any size is not something to be ignored. While not every team will need to use this now, it may not be that long before we can’t imagine a time when we couldn’t just step into a scene and start making artistic decisions simply by putting on a pair of glasses. In my original post, I joked about how I looked forward to taking a lunch break on the surface of Mars while in my office. Well, I can say that, as of today, been there, done that.