When I wrote the cover story for the June issue of American Cinematographer magazine on the challenges and methodologies that the camera team led by Dariusz Wolski, ASC, faced in shooting Pirates of the Caribbean: On Stranger Tides in native stereo in faraway locations under rigorous conditions, I commented about how rapidly the stereoscopic paradigm is evolving, advancing, and shifting things on the larger industry landscape. Since that movie was shot—indeed, even since my article about the movie was written—that landscape has continued to shift. Tools, techniques, theories, and business considerations relative to major decisions like whether to capture in stereo or convert a tentpole movie later in post have all advanced dramatically.
In fact, even while I was writing the article things were shifting on the Pirates project alone. Near the end of the shoot, the project tried out Element Technica Atom stereo rigs configured with early generation lighter-weight Red Epic cameras with 5k Mysterium-X sensors after shooting the bulk of the movie with Pace rigs and Red One [4k Mysterium chip] cameras. That was done, in part, to familiarize Wolski’s crew with those rigs and the Epics, since Wolski was preparing to transition to Ridley Scott’s Promethius using them—a project Wolski and his guys were working on at press time.
Also, Rob Engle served as the movie’s 3D Supervisor, performing the post-production stereo processes, including convergence, plate corrections, 3D visual effects reviews, and 2D to 3D conversion for a small number of shots. I regret I didn’t realize Engle’s official credit was 3D Supervisor at the time I wrote my article (I called him a consultant), nor did I give him proper credit for the role he played early on in helping the project make crucial 3D decisions. He is a 3D veteran, of course, having supervised the stereo work on several projects, including Jerry Bruckheimer Films’ first 3D effort, G-Force (2009), which was an early benchmark that quality conversion of major feature films was possible.
On Pirates, also produced by Bruckheimer Films, Engle was brought in early and played a crucial role in certain key decisions about how to capture the film. Chief among them were the decisions to shoot with native stereo camera rigs, rather than using a conversion process, and recommending the use of the 3ality Digital SIP system to monitor rig alignment (see my June article for more on all this). He also suggested shooting with a large image pad, allowing parallel photography and convergence in post—a suggestion supported by the visual effects team. Due to commitments to other projects, Engle was unable to serve as on-set stereographer, and so he recommended industry colleague Dave Drzewiecki to handle that role. (Drzewiecki left late in production to transition to another show, and Wolski’s second assistant, James Goldman, finished the on-set part of the stereography job.)
Since then, several huge feature films (Promethius, The Amazing Spider-Man, The Hobbit and Jack the Giant Killer to name just a few) are now doing the on-location stereo capture thing, and a large swath of other major feature films, originally captured in 2D, are now either being converted to produce a 3D version, or at least, putting that option on the table for serious consideration. The recently released Thor exemplifies that approach, as does Green Lantern, a film whose production story you can read from me in American Cinematographer’s July issue, and there are many others.
Since the tools, people, timelines, financial arrangements, and creative use of 3D vary wildly from project to project, I was curious to figure out what part of the equation is, in fact, consistent, and who is indispensible. The answer to both those questions is, loosely, “a 3D expert or experts” to one degree or another, although further clarification is needed to explain whether we are referring to the on-set stereographer, a post stereographer, a combination of both, or particular combinations of “3D consultants,” or “3D supervisors” in various permutations. But no matter what you call them, no matter how many of them there are, and no matter how they divide the labor, what is clear is that the good ones all have both extensive experience on stereoscopic projects and a great deal of education about the science and art of composing stereo imagery. Thus, every 3D production for mass distribution needs their help at some level, and no one more than cinematographers.
To clarify these and other points, I went back to chat with both Drzewiecki and Engle, who normally works as an effects and 3D supervisor at Sony Pictures Imageworks, but who worked independently on Pirates on behalf of Bruckheimer Films. Both men have worked on several major 3D films in various capacities in recent years, are involved with education programs and events designed to teach industry professionals about the nuances of making movies in stereo, and both have strong views about where the 3D trend is going right now, and where it needs to go next.
Information is Out There
Both emphasize that the current 3D feature film trend has some things in common with its synched projector predecessors in the 1950’s, and briefly, the single-strip processes of the 1970’s and ‘80s—namely that capturing in native stereo remains bulky, risky, and complicated, although perhaps less so than when “they used to have to put two full-size Mitchell [film] cameras together in a blimp configuration as monster rigs,” as Drzewiecki recalls. Rather, what has changed is the maturation of digital cinema, which allows theaters to exhibit 3D movies en masse, removing many of the barriers that made it difficult to seamlessly, consistently, and accurately shoot, manipulate, and project 3D in a film format in earlier eras. Drzewiecki, who is something of a history buff regarding the ebbs and flows of stereoscopic feature filmmaking, adds that is why 3D is finally viable enough to do commercially on a continual basis.
But it remains less than easy to do well, whether the stereo images are captured in a controlled environment, on location, or built later in post. The biggest problem, both men agree, is that modern filmmakers, including many cinematographers, are stuck in 2D conventions when it comes to composition, framing, depth of field, and other things. Therefore, they suggest, the industry needs to get educated, and in the meantime, needs to open normally close-knit creative teams to input from 3D experts.
“It took Avatar to make [3D] believable for the industry in a business sense,” says Drzewiecki. “Before that, 3D had ups-and-downs but was never consistent, and as a result, there were not a lot of 3D filmmakers or cinematographers with 3D experience around to pass their knowledge down the line. For the information to be carried out to the next generation, the next person, the next production, you need a consistency of execution—an educational process has to happen, and that will take time.
“But, that said, the information is out there. What is new is the equipment—the cameras and the rigs. But that is just hardware. The principles of stereo, how you use it effectively to block a shot or frame a shot—that information has been around for years. They had specialists all along, going back to the days of House of Wax (1953)—even earlier. They weren’t always understood, but they were there, and they knew the mathematics, the geometry of the relationships between the left eye and right eye images. I’ve read articles from the 1950’s, even the ‘30s, which fully understood the geometrical relationships of the camera environment versus the projection or viewing environment. So, in that sense, with that information available, and better technology, there is now no real excuse for not doing 3D well—except brute force ignorance. It’s been documented and studied, but only a small group of people has that information. Will they be listened to or not?”
Central to this discussion is a simple technical challenge—how best to align imagery to get the easiest to absorb, clearly visible image for audiences to enjoy without taxing their eyes and brains. Rob Engle suggests that it’s essentially impossible to photograph a 100% perfect 3D image to put onto a big screen. But, he adds, with the right knowledge, pipeline, tools, time, and money, stereo can now entertain the masses as a legitimate cinematic device. But that means forgetting the notion that, even with the most sophisticated technology, one can just point-and-shoot one’s way to stereoscopic success.
“My definition of a perfect 3D image is where you switch between left and right eye images, and the only thing that changes is the horizontal position of the objects in the picture,” Engle explains. “The main idea when you are shooting 3D in the field is to try and capture in-camera as much of the actual depth as you find in the real world. The challenge there is that, even with these amazing rigs, you have two different cameras with two different optical paths. It is nearly impossible to align them perfectly, but you want the alignment to be as good as possible. You can then do more work on the images in post, because after all, you are showing people a two-hour-plus movie, and any misalignment would cause eye fatigue and take them away from enjoying your film.”
Therefore, Engle emphasizes, a significant part of the equation involves giving the movie a stereo massage in post, even for movies shot in stereo. That is a big part of what he does on shows like Pirates, which he was just finishing up when we spoke at press time. Working with a modern 3D-capable DI system, he routinely pores over edited versions of such films.
“I’m looking with filmmakers at the edit for creative things mostly,” Engle says. “Where to play things in depth, correcting technical errors like color, focus, and alignment, and working with visual effects’ vendors to give them feedback on what works or doesn’t in terms of their work. With the extra image pad we had on Pirates, it wasn’t uncommon to make slight framing adjustments in 3D to give a better stereoscopic composition, as well. Generally, I’m overseeing all aspects of the 3D to make sure the film is satisfying creatively in terms of what the filmmakers want, and technically, to make sure it does not cause people fatigue to watch it. So I feel, regardless of size of budget, that it is important to have someone in post who knows something about 3D to oversee those details.”
The Current Embryonic Stage
All of which means that people like Drzewiecki and Engle are rapidly rising on the ladder of importance in terms of creative collaborators who intimately partner with directors and cinematographers on 3D features. They, after all, are the ones with, as Engle calls it, “a 3D mindset.” Engle suggests the stereographer’s job on set, for instance, is to monitor the production and “feed that 3D information back to the DP.”
“The stereographer is part of the camera team,” Engle says. “The DP can’t possibly do it all at the same time—he can’t shoot the film creatively and think about every aspect of 3D. So he needs someone to come back and say if you change the shot this way, or move the camera that way, it will work more effectively in 3D. At this stage in the evolution of 3D, most DP’s are not used to thinking in (stereoscopic depth), and appreciate advice from someone who has that experience. I’m working on The Amazing Spider-Man now, and have been having a great time working with [cinematographer] John Schwartzman [ASC]. We started our relationship on Spider-Man, but he and I also got to trade feedback on Green Hornet, which he shot and for which I supervised some of the conversion work. That collaboration helped build our relationship, and is now paying dividends on Spider-Man. ”
Drzewiecki emphasizes that regardless of the technical challenges or limitations, the real roadblock the industry faces in terms of stereo production lies in educating filmmakers how best to use the medium creatively. He warns that, “often 3D is still a gimmick forced upon [films] by studios, and the filmmakers do nothing creative with the format. Sometimes, 3D is not even a creative consideration during shooting. The truth is, few [filmmakers] really know today how to use 3D creatively and effectively. The subject often gets too caught up in conversations about equipment, not technique.”
Therefore, when experts like Drzewiecki and Engle show up, the way they can most efficiently interact and collaborate with cinematographers, directors, and others remains an evolving process currently. Those relationships are new, and the technology is new—sometimes brand new. Indeed, in the short term, it’s worth remembering that even through all the movie industry’s attempts to make 3D into a lasting, substantive business, the latest attempts to do so, and the technology and techniques that go along with those attempts, are all embryonic at this point—essentially in their earliest stages. The Polar Express’ arrival as the first IMAX full-length animated 3D feature (its 3D was also supervised by Engle) was only seven years ago, and Avatar’s creative, technical, and financial breakthroughs only took place a couple of years ago. 3D tentpole features therefore were, until just recently, rare and risky events.
That’s changing, but not seamlessly or easily or cost-effectively in many cases. Thus, for the time being, it appears that with all the “wow” power of 3D, it will, in the short term, make filmmaking harder, not easier, and will reduce, rather than expand, options available to filmmakers in the field. Both Engle and Drzewiecki feel this will remain the case for some time to come. Therefore, despite their own vested interest in seeing 3D grow and thrive, they are cautioning it is not always appropriate, when or if resources, time, or the visions of particular filmmakers do not coincide with the requirements of high-end stereo production.
As far as stereo acquisition goes, Drzewiecki compares the situation to the introduction of sound to motion pictures. In the long run, that development fundamentally changed the industry forever. In the short run, when it first happened, he suggests, filmmakers got bogged down with all sorts of technical limitations while trying to figure out how to make the darn thing work.
“Having to record sound threw the filmmaking process overall back many years in terms of creativity when it first happened,” Drzewiecki suggests. “There were all sorts of limits placed on sets and actors and equipment to permit them to record sound adequately when the whole thing began. At this point, 3D rigs are still bulky, unwieldy, and in fact, plagued with problems from time to time. So more time will have to pass before you get a larger group of technicians that are comfortable and confident, and equipment providers that have the bugs worked out of their systems. That will all take time.”
But the stereo conversion process has its own limitations, as well. One need look no further than (2010’s) Clash of the Titans for an illustration of those limitations. The 3D problems on that picture, however, were largely a result of business getting ahead of the process—“the production simply did not allow enough time for an adequate conversion to be done on that show,” Engle suggests.
“Going into [Pirates], we knew we had a very short post-production schedule which would not lend itself well to a conversion,” Engle adds. “Part of the decision to shoot native on ‘Pirates’ was due to that inflexible post-production window. What the industry learned in 2010 is that it is not possible to convert a movie to 3D with an eight-week conversion schedule and do a decent job. That is what got conversion a lot of negative feedback. So, now, there is an evolution across the industry that 3D can be used as a creative tool, but only if you start early enough, plan it well, and take enough time. Some studios have really understood that—[Dreamworks Animation] has built 3D into its pipeline and finishes all [CG] movies in 3D as part of their official process, for instance. And now, across the industry, despite time and cost pressures, people are building more time into their post schedules for 3D conversions. At the same time, there are now several companies that can do a great job with that if you give them enough time to do it properly.”
These and other issues return us to the topic of education. Since early 2010, the Sony 3D Technology Center has been offering training programs and educational opportunities for industry types interested in learning stereo, and Drzewiecki has been teaching classes there and elsewhere in partnership with Local 600 on the subject. Engle has spoken on these topics at numerous industry educational events for ASC, AMPAS, Siggraph, DGA, VES, I3DS, and others. And the ASC Technology Committee, of course, is closely studying emerging technologies and standards for stereoscopic filmmaking.
But even with those important programs and projects, and with more on the way, Drzewiecki points out that the industry still lacks enough qualified instructors who have actual in-the-field experience making stereoscopic films, rather than mere theoretical knowledge. And, besides, he adds, “the 3D experience is different for everyone anyway. We all have different levels of depth perception—it’s a lot like color in that regard. Some people feel that ‘less is more’ when it comes to 3D—just adding a minimal amount of depth can make the images fascinating. But others feel just the opposite. So it’s definitely a matter of taste.”
But as the industry travels further down this path, as technology continues to improve things, as education proliferates and professionals get more stereoscopic experience, there is every reason to presume the cost and complexities involved with making stereo movies will eventually come down. Therefore, it is reasonable to assume that the business of 3D movie-making is, this time, here to stay.
But on the other hand, as Engle emphasizes, that depends on that whole series of assumptions coming to fruition. At the end of the day, he says, it will all come down to whether or not risks associated with this kind of filmmaking go up, or come down, as far as filmmakers, studios, and financiers are concerned.
“It will become easier to shoot native stereo faster and more cost effectively, and it will also become cheaper and easier to convert movies,” Engle says. “If that continues, we’ll see a lot more stereo, and it will go far past being a fad as it was in the past. With that in mind, there is likely to be a seesaw effect as to whether or not to shoot in 3D or convert as each technique becomes better and most cost-effective. Additionally, each production will have unique requirements, which may push the decision one way or another. Currently, it seems that conversion is popular with many films if for no other reason than it allows the studio to defer the risk of deciding to make the film in 3D until later in the process.”
For another, detailed take on where the 3D trend may be taking us, from the point of view of a veteran cinematographer, check out this fascinating blog posting John Bailey, ASC, put up on this website last year.