By popular demand, this blog is a summary from Session 1 of my Masterclass in Virtual Production held at the Cannes Film Festival, as part of the Cannes NEXT program, May 19. 2022
Created & written by Cinematographer Jannicke Mikkelsen, FNF
Instagram @jm_fnf ✉️ jm@jmfnf.com
Virtual Production is an extremely wide umbrella term. In its simplicity, anything in the film industry that has been converted to digital can be considered Virtual Production. Referred to as VP, 'Virtual Production' is a recently coined phrase and buzzword that poorly describes with any accuracy what it actually is. Virtual Production can be assumed to be Metaverse, gaming, XR, VR, AR or NFTs. VP can also be video projection behind actors. One aspect of VP is basically just an adaption from analogue to digital workflow in the film industry, whilst another aspect of VP is the introduction of an entirely new work force entering the film industry; New people popping up on film productions with whom the rest of the film crew have no clue what do. Let me explain:
Digital World = Virtual World
In my 15 years of working in the film industry, I feel lucky to be part of the rapid change in technology that make film productions possible. Back when I was a film student I felt I had to lie and pretend I knew how to shoot analogue 16mm/35mm film. I didn't. At that time it was just assumed that if you were in the film industry you had worked on industry standard film cameras shooting film. This in turn meant you knew how to correctly expose the silver emulsion on celluloid film, but worse of all, it was assumed you knew how to load film, and every model of film magazine required its own sequence of threading and care. Trust me, when you are an inexperienced film student there is no bigger fear in life than working blind while loading or unloading a reel of exposed film from magazine to film lab can. Later, I’ve understood that the phenomena of having only experienced digital technology has a name: - Millennial.
Enter two decades later, and now we are all talking about virtual film productions, and the process is equally as mystifying to film creators today as celluloid film was to me back when I started.
First I'd like to thank Marché Du Film - Festival De Cannes & Cannes NEXT @mdf_cannes and EPIC & Unreal Engine for inviting me as a Virtual Production expert to host a 3+ hour Masterclass and attend the Virtual Production panel discussion.
I'd also like to thank Film Soho who hosted the Cannes NEXT Virtual Production Masterclass at their fully kitted out pop-up virtual studio located in the V-Studios tent at Le Grand Hotel Cannes gardens next to the Tom Cruise Top Gun Maverick Exhibit. Film Soho is a group of innovative brands that encompass the entire film and TV ecosystem. Under the Film Soho umbrella is V-Studios who will allow filmmakers to place actors in virtual environments without ever having to leave Central London. Disguise, n-cam, CVP Group & ZOAN who are all leading innovative companies in virtual production and metaverse technology.
What is Virtual Production?
At the Cannes 2022 Masterclass on Virtual Production we focused on Virtual Studio productions in the Unreal Engine. We were situated in the fantastic V-Studios tent with a state-of-the-art Virtual Production stage featuring an LED backdrop and a dessert rock film set to match the digital backdrop. This allowed me to create an interactive hands-on workshop within the masterclass itself.
A shift happened in the film industry round about 2007 - 2008 when digital cameras and digital rushes became a serious contender in film production. Take note that this also corresponds to the exact same time of the global economic crisis at that time and the film industry had to adapt rapidly to survive. At this time I was a student and digital was simply there, available to explore and didn't cost me a penny to shoot. Jump ahead to today's situation and at the tale end of a global pandemic 2020 - 2022. Virtual Production allows filmmakers to explore their scripts, and combined with digital cameras on cellphones, everyone now has a virtual film studio in their pocket.
Virtual Production (VP) is a loosely defined term for any digital workflow or digital creation in the production of a movie or series, and VP is neither restricted to the creation of fiction nor documentary. Digital workflow and ‘cloud’ can also be defined as virtual production, and most film productions already use digital assets (VFX) which have now been re-branded under the umbrella of VP.
The reason for the rebranding of digital technology is in part due to the rise of highly effective game engines driving the future of image technology.
Gaming technology from the gaming industry is infiltrating film production, simply because: It Makes Sense.
The most obvious challenge we face in Virtual Production is the merging of two worlds: the real world & the virtual world. The responsibility asked of me as the Cinematographer is to coordinate both worlds and know these worlds cinematic possibilities and limitations in order for the production to be able to seamlessly merge both worlds. Beyond the role of a VP Supervisor, the Cinematographer is in charge of the artistic interpretation and cinematic sensitivity of which the script demands, and most importantly of all knowing how to steer the film crew in a unified direction to achieve the artistic vision on time and on budget.
Session 1 of this VP masterclass can be defined as 'Virtual Production Lite'.
All you need to know about Virtual Production in 45minutes
This is the skinny on Virtual Production. What is it and who needs it? What kind of virtual production should you go for and how to decipher what virtual production elements are right for your production?
One contributing factor as to why the definition of virtual production and digital technology is as confusing as it is, is due to what we in the tech industry comically refer to as 'Moores Law'. Gone are the days when you could learn your craft as an apprentice working with celluloid film to hone your skill until you became a Master. Digital is forever evolving and the rules of the game is constantly changing with the improvement of technology. If you’re not on top of your game continually learning and updating your knowledge, your apprentice will swiftly surpass you on the tech-game. With this in mind, here begs the next question: What drives a film production when introducing virtual production? Technology or art? In virtual production it is easy to allow the virtual tail to wag the artistic dog.
When you enter the world of virtual production there are two main capture method categories of virtual film production. These two main methods are:
Virtual Shoot & Virtual Studio
Out of utter industry confusion over what virtual production(VP) actually is; I’ve chosen to define and divide VP into two main categories to reflect the location of where the virtual production takes place:
Virtual Shoot is when your film location takes place inside of the game engine and you use virtual cameras to shoot your production.
Virtual Studio is when you are on located in a studio with an LED screen as your backdrop and shooting your production with a regular camera with optical lenses.
I’ve experienced the role of a cinematographer to become increasingly demanding with the rise of virtual production. I see the increasing benefit of having two cinematographers collaborate on the same film production, where the one is the traditional DoP and the second role is a supporting cinematographer who is head of the VP department: AKA the Virtual Cinematographer. This is only one of many new roles entering film productions in the near future as they adapt and convert to virtual production.
Virtual Studio - it's the one where you use a virtual backdrop.
Short recap of the use of movie backdrops. All films shot in a studio use backdrops, whether it is a photographic backdrop, a painted diorama canvas, or a chroma key blue/green screen. Rear projection was one way of creating video action behind an actor, which later got replaced by the blue screen and then the green screen. Now we have LED screens and photorealistic volumetric universes built in game engines that get projected on to LED backdrops, simulating the same technique from the days of rear projection. The main difference is we can now move the camera that we are shooting the movie scene with - and the background perspective projected on to the LED backdrop moves with the point of view of the camera.
Our virtual backdrops locations are built in game engines to create a 3D model of the movies environment where we can move anywhere within that location. These are volumetric locations and can also be referred to as metaverse locations. It is also interesting to note here that these metaverse locations can be made into NFTs and sold, traded, and most noteworthy, rented out to other film production as a virtual landscape.
You can now own a virtual film location as an NFT and charge rent for future film productions who wish to shoot at your virtual location.
The stage in front of the LED screen is like a regular traditional film set smartly staged to blend seamlessly in with the backdrop. The scene is shot with a regular movie camera kitted out with tracking technology which links the physical camera with the virtual world projected on the LED backdrop. What is actually happening is that your real world camera is now a digital twin with the virtual camera which exists inside of the game engine. That virtual camera image is the one you see on the LED backdrop and your real world camera is the one that is capturing the action filming the image on the backdrop behind the actors. -Remember I said this first session, session 1, is VP Lite. We can get in to all of the details in the upcoming sessions.