THE DIGITAL DILEMMA 2

WWI battlefield scene from the 1927 motion picture “Wings.”

ONE

On January 18, 2012, the silent film Wings, directed by William Wellman, was screened in the Academy’s Goldwyn Theater in a new restoration, with color tinting duplicating the original Handschiegl process, and featuring live organ accompaniment by Clark Wilson. But this was in no way a recreation of the original release. In 1927, Wings was heralded for its lustrous 35mm. imagery and for thrilling aerial stunts done in a pre-rear screen era, every frame captured as real events on nitrate negative film. This Academy screening, however, was made from a DCP, a digital file, and shown on the Academy’s 2 K Barco digital projector. Like many of the treasures of the silent and early sound eras, the original negative is long lost; the restoration was made from the Cinemateque Française fifth generation print using state of the art digital technology to restore (as far as possible) the shimmering glow of the black and white nitrate film original.

Several weeks after this screening, the Sci-Tech Council of the Academy released The Digital Dilemma 2 report, the long awaited follow-up to its 2007 paper that discusses preservation and archiving in the digital age. That original paper explained challenges confronting the major studios; the TDD 2 paper focuses more narrowly on indie and independent movies as well as on audiovisual archives.

Motion picture preservation is no longer the concern only of studio librarians, museums, and professional archivists. As we rush headlong into the era of digital cinema, archiving and preservation issues become more urgent, and need to be addressed even before a film concludes its brief theatrical or television run. In the world of low-budget indie and documentary filmmaking that is the focus of the TDD 2 report, these responsibilities now fall mainly into the arms of the filmmakers themselves, and unfortunately, there is often no money available for them to enlist professional, outsourced archiving. Discussing this past summer’s Academy screening series of documentaries, one staffer confessed to me that some documentary filmmakers keep their digital masters in their own homes and, in one case, under his bed.

The first century of movies was photographed on film: first on fragile, flammable nitrate base stock and by 1951, on acetate safety base. The mantra for storing original elements such as black and white and color negatives, 3 strip Technicolor matrices, and reversal masters has been “store and ignore.” This may be a too reductive aphorism but it does define the lack of concern, even the confidence, with which motion picture film elements have been stored in sometimes problematic film vaults, vaults often built more for fire protection than for the critical control factors of temperature and humidity.

In June of last year, David Pierce wrote a report commissioned by the National Film Preservation Board titled, Survey of the Survival of American Silent Feature Films: 1912-1930. Despite its unwieldy title, the report is a fascinating look at the history of the American Silent Film with its attendant erratic distribution patterns. It tracks the many differing gauges and formats both in original production and in downstream distribution and exhibition. The discussion of the many different versions of a film, its print formats and preservation masters reads like a jigsaw that bears more than a token resemblance to that of ever increasing iterations of digital video media, formats, and codecs, a maelstrom that is swamping us. Any cinematographer bold enough to buy a high-end video camera today must expect to profit on its rental within a several year life span—before the camera’s inevitable obsolescence.

A look at the number of silent American films that have survived until today may provide a cautionary tale for filmmakers in our digital age. There are 1,538 American silent films that exist in complete 35 mm. domestic versions. That may seem like a lot until you realize it is only 14% of those that were produced; another 1,090 (10%) are complete but exist in only lesser formats or foreign release versions. (It was common practice for multiple cameras to provide several negatives for international distribution, taken simultaneously at different camera angles and with slightly different shot sizes.) Less than one fourth of all American silent motion pictures exist today in any complete form—and they were all photographed on the medium that is still universally considered to be the only archival one—film. To be sure, there was little thought in the pre-TV, pre-home video age that there would be anything like a “long tail” life for these movies. Films were largely disposable. Several studios emptied their silent film vaults in the thirties, reclaiming the valuable, dense deposits of silver that were embedded in the nitrate negatives and prints.

These survival statistics, though, come from the “store and ignore” mindset. If those early movies had been stored on digital media, it is certain that nothing would have survived. When many of us were film students, older movies, especially documentaries and foreign films, were mostly available for study on badly duped and scratched 16mm. prints. It is one of those wonderful ironies that it is by the magic of state of the art digital technology that today the masterpieces of early cinema are being given anew to us in pristine digital editions. “Wings” is such an example. Several months ago, TCM broadcast this movie as well as glorious digital restorations of the 1921 Four Horsemen of the Apocalypse and the 1925 Ben-Hur.

Alice Terry and Rudolph Valentino in “The Four Horsemen of the Apocalypse.”

Ramon Navarro in the 1925 “Ben-Hur.”

Three strip Technicolor films such as The Wizard of Oz and The Red Shoes, with their unevenly shrunk matrices, have most benefitted from the tools of digital restoration and have allowed us to experience anew the glories of Jack Cardiff’s cinematography.

Moira Shearer in “The Red Shoes.”

Like its earlier model, The Digital Dilemma 2 report is well structured and very readable, devoid of cant and techno-babble. It is clearly intended for working film professionals in many disciplines, not just for archive moles and equipment nerds. In separate chapters, the report deals with independent filmmakers, documentarians, film festivals (marketing and distribution), and in-depth surveys by several of the eighteen audiovisual archives that participated in the study. There is an introduction and brief history for each section. An eight page appendix features an interview with Peter Mavromates, a director of post-production, who has worked with many studios and filmmakers, notably with David Fincher on several of his “born digital” feature films, as well as on the recent Iron Man 2. Mavromates’ perspective paints a confident picture of an eventual all digital future, but for now at least, a cautious one. When asked, “What do you do over the long term with the digital files (for a feature film),” he answers, “If you are asking is the film-out an interim solution—no. It is a delivery requirement and although the film-out is certainly an archival element, we consider the DI the mother lode. For both now and later, the solution is data migration.” This may well be true, but the problem for many digital filmmakers who do not have the unlimited resources available to Mr. Mavromates’ studio clients or the deep pockets of well-endowed audiovisual archives, data migration on a regular basis is a pipe dream.

Just how precarious the preservation/migration issue is for indie filmmakers and documentarians is reflected in a Jan. 18 article in Variety written by David S. Cohen. He interviewed Milt Shefter, Sci-Tech Council member, who co-authored the report with Andy Maltz, Academy Director of the Council.

Shefter told Variety, ”The bottom line is we’re running out of time. The time for studies is past. We have to find solutions or we’re going to lose a lot of material.” For documentarians this ticking clock is critical because it is not at all unusual for their films to achieve distribution only after the “shelf life” of the non-migrated master has passed. Even worse, according to page 4 of the report’s executive summary, “Most of the filmmakers surveyed and interviewed… were not aware of the perishable nature of digital content, or how short its unmanaged lifespan is.” Many documentarians depend on access to historical footage; they seemed equally unaware that news and documentary footage they may draw upon is just as vulnerable to being lost.

It is in our nature to believe that somehow one  technology will always ride to the rescue of another technology; that may not be the case here. The report’s executive summary paints the unvarnished reality as clearly as possible:

Well-established, time-tested analog preservation practices do not apply to digital holdings; digital materials are fundamentally different from motion picture film and other analog materials. Suitable long-term preservation and access mechanisms for digital motion picture materials have not yet been developed.  (my boldface)

And until they are developed every motion picture image that exists solely in the digital realm is at risk—just as are our personal iPhoto files on our laptops and iPhones… Have you lost any files or photos? “Future-proofing” in the digital world is just a slogan, not a reality.

One of the most intriguing sections of the report deals with nonprofit audiovisual archives. They are the real repositories of our A-V history and their custodians are equally alarmed. According to the National Film Preservation Board, there are 550 moving-image archives in the United States and its territories and another 310 worldwide. Preservation of film materials in the United States began early. According to TDD 2, an early version of “migration” of movies began in 1942 when “the Library of Congress discovered paper print copies of several thousand motion pictures produced between 1894 and 1912— deteriorating in one of their vaults. The Library restored and converted approximately 3,000 of these paper prints to motion picture film through collaborations with the Academy of Motion Picture Arts and Sciences and Renovare” (a company formed for this specific project). Paper prints were originally made frame-by-frame from the OCN or fine grain master positives because there was not yet copyright protection law for film— but there was for material created on paper. Some of the early Griffith films exist today only because of this measure.

One section of the report tries to explain the concept of “digital preservation.” It includes this caveat from the United Kingdom’s Joint Information Systems Committee:

Digital objects will cease to be accessible without active management and intervention. The biggest risk to the accessibility of digital objects is the continual development of computing hardware and software. Many digital files or formats are dependent on a particular computing environment for accurate presentation of their content. Any change to the rendering environment could result in change to the rendered representation of a resource (or result in not being able to render the resource at all).

Once you dig into the dry jargon embedded in the statement, the message is clear. New systems and upgrades can exacerbate the problems they are trying to solve and even destroy content. Any of us who has done a home computer operating system upgrade, only to find that one or more applications no longer work, can understand the dilemma. It’s not unreasonable to say that the term “digital preservation” is an oxymoron. “Digital data management” is more accurate.

TWO

To put a human face on all this cascading information, I asked Milt Shefter if he would answer a few questions that came up for me as I read The Digital Dilemma 2 report. Milt is a good friend and an associate member of the American Society of Cinematographers. He speaks here as a film preservationist, not as an official Academy spokesman. Milt is also well known to many of us as the resonant voice of the annual ASC Awards show. Here are my (admittedly, sometimes leading) questions, with Milt’s responses:

JOHN BAILEY, ASC: Almost five years have passed between the first and second Digital Dilemma reports. Do you feel that filmmakers are more aware today of the challenges posed by archiving movies, television, and documentaries only on digital platforms?

MILT SHEFTER: No. And the indie filmmakers don’t seem to realize that because of the time lag to distribution, their digital master may be technologically obsolete before that.

Why does there seem to be so little awareness and action by indie and documentary filmmakers to do long term preservation of their work? Is it simply lack of funding? Just a few years ago, YCM separations or IPs were routinely included as a line item in budgets.

That’s two questions. One, we can’t peer inside the minds of filmmakers. Many seem enamored with the perceived benefits of digital image capture and workflow in post-production. But, the lack of long term guaranteed access in digital storage technology is not considered. They’re used to “store and ignore” analog (film) and may not understand that digital records require continuous and active management. Two, YCMs and IPs coming from film masters are economical. With a digital source master, you have to go through the Digital Intermediate master system, which is very expensive. Even just scanning back to a single strand (IP) film from a digital master is much more than most indie filmmakers can afford.

Are you surprised by the large number of filmmakers, esp. in indies and documentaries, who are still unaware of the vulnerability of work stored only as digital tape masters or on drives?

Yes, especially the documentarians who use archival footage for many of their projects. What will they use in 25 years when they want to use material from today, and most of it won’t be in existence?

The major studios seem to be acutely aware of the archiving and preservation issues at hand. Have they reached any consensus about what to do? Do they have in sight any method or system other than media migration that is comparable to film for long-term preservation?

The majors take one version of their digital source master and scan back, usually to three-color [film] separations. That’s their preservation master, good for the proverbial and proven 100-year life.

It has been over ten years since cinematographer Roger Deakins did one of the first DIs on a major theatrical film, O, Brother Where Art Thou? This exceeds the recommended 5-year “shelf life” for media before migration is necessary. We can trust or hope that this master DI has been migrated. But what about the perhaps hundreds of movies that have employed the DI process—and may not have such keen studio oversight, movies that failed at the box office or whose distribution arm has gone out of business? What fate awaits them?

Those that may have survived technological obsolescence are still faced with the issue that the DI facilities all used their own “secret sauce” software, so if those post houses are gone, even the surviving record needs “forensic” (read that as expensive) action to determine if the software (movie) can be salvaged.

We talk about “migration” as if it were a foolproof system, like a text clone; but in fact, the ones and zeros that constitute a digital record of a two-hour feature length film at 2K resolution, is a monstrously hungry data system. How accurate is such complex media migration? Has enough time passed, and enough migration generations, for us to know the safety and accuracy of the migration process?

Just the opposite. We already see issues with migrated materials. And, while it may work for a very small number of titles with active management, migration doesn’t scale for larger collections: this from the Storage Industry Network Association study. The Associate Librarian of Congress, a major migrator of digital data, admits that each time they migrate, they face the danger of losing information.

Is it safe to assume that most migration is done on robot systems without a human eye as intermediary to check for accuracy? If migration errors happen, and they are almost inevitable, how does this affect subsequent migrations? The question of migration errors is not addressed in the TDD2 report. Perhaps, it’s too early in migration history to even know the answer.

Our role was to ask about possible solutions for the long term. Migration, as stated by others (see above, Mavrotes) isn’t a long-term solution. It’s part of active management and has to be done before the five-year “guaranteed” life of the hard drive is over.

Given the importance of the original Digital Dilemma report, the Academy must hope for a greater awareness of the issues it addresses again in TDD 2. Yet, in general conversation, even among my cinematographer peers, I find much more interest in new cameras, LUTs, workflow, and DI systems than in any discussion about preservation and archiving. This dismays me, as we all seem, on the surface at least, to believe that the best of our work will outlive us. Do you have any idea why this lack of concern is so broad among those of us who are otherwise so techno-saavy?

As you know, TDD,2 was written from the perspective of the independent filmmakers and nonprofit audiovisual archives. The majors and other big organizations and businesses have resources and built-in ecosystems. The indies and archives are for the most part under-resourced.

As for the cinematographers, their concentration seems to be on the image capture systems and perhaps on some part of the post work flow (like timing.) It’s sad to think that much of their creative efforts won’t survive for future generations of audience or filmmakers.

How does the Academy hope to get the word out, especially to young and emerging filmmakers who are mainly digerati, that their digitally captured movies are a “slow fire,” what has been called “digital nitrate”?

The Academy has committed to “get the word out” through film schools and film festivals to educate those filmmakers about the lack of long term guaranteed access in current digital systems. More importantly, it encourages the effort to try and organize collaborative activities, to get standardization and perhaps to push the envelope in creating an archival system that either meets or exceeds the parameters of the current analog (film) system of preservation.

THREE

“Getting the word out” becomes more critical by the month. The transition to digital production rushes at us now at a breakneck pace, despite the very real concerns discussed in this report. Near professional quality image capture is simply so easy with prosumer cameras. New and ever improving camera systems such as DSLRs are seductive, and with Arri’s breakthrough Alexa camera and the arrival of Sony’s F65, film negative quality in image capture is a near reality. It is so tempting to focus only on the technology we see. The speed of its ascendancy has surprised everyone. Momentum counts in movie production, just as it does in a sports event. A heady techno-fervor sweeps aside sober reality. That’s just the way it is. Motion picture film diehards, even those of us who have photographed many digital movies, are hard put to defend a “mature” analog system that reeks to the hipster’s nose of a steampunk technology (film)—all those chemicals and cans, ugh. And the great, now beleaguered, company Kodak seems to be in a tailspin from the top, as if digital camera manufacturers had planted a mole to dismantle the company from within. The irony is that Kodak’s Motion Picture Imaging division is one of its few bright spots; the long term (mis)management of  an iconic American company only unleashes more grief for our colleagues at Kodak Hollywood.

I doubt any of us knows just how all this will play out in the next several years.The dilemma of The Digital Dilemma 2 is just that— a catalog of challenges seeking solutions. The new digital image generating systems that we embrace with such passion are also the sources of their own content demise: in short, unless we find solutions beyond the limits of our current technology—many filmmakers are facing a digital fadeout. Or a total blackout.

The Digital Dilemma 2 report is easily accessible at the Academy’s website as a downloadable PDF. Here’s the link:

Oscars.org: Science & Technology Council projects: Digital Dilemma 2

Get your copy today. It’s a necessary but disturbing read.

Milt Shefter

Next: The Three Graces, The Snapshot, and Roland Barthes. .  . Please subscribe to the blog in the box at the bottom of the comments column.

9 Responses to “THE DIGITAL DILEMMA 2”

Leave a Comment


  • John ,
    Always great to read your thougthfull posts .
    Thank you for pointing out once more the importance of film preservation . Even more so at the digital age .
    You’ve placed the right questions that unfortunately don’t have a precise answer .
    The Digital Dillema 2 is a very important contribuition on this mater for all independent filmakers around the world .
    yours
    Lauro

  • John,

    Thanks for a wonderfully clear and concise consideration of this issue. As the founder of the Film Noir Foundation, a non-profit lobbying and fundraising group dedicated to rescuing at-risk films in the genre, I can’t stress enough how critical it is to explain that having a film in some digital form is NOT preservation. It’s especially crucial for us, because we are trying to ensure that archival 35mm prints exist for everything in the American noir canon — and it’s getting difficult because people mistakenly believe that if there’s a digital copy, the title has been “preserved.” There won’t be any digital copies for convenient distribution via disc or download unless the film is first preserved as FILM. I’m going to share your article, thank you.

    And P.S. you may recall coming to my reading at Skylight Books when my first book on film noir, DARK CITY: The Lost World of Film Noir came out. It was wonderful chatting with you that night, many years ago now.

    John’s Reply: Yes, Eddie, I remember quite well the evening at Skylight Books, which continues to be a wonderful venue for readings, one of those rare indies that actually seems to be thriving. I am a fan of your continuing exploration of film noir. Your books are a major reason why film noir has achieved a level of attention way beyond what any of the filmmakers of the time could have imagined. Thanks.

  • Dear Mr. Bailey:

    Your post should be required reading for everyone who cares about preserving our motion picture legacy. And it is also an alarming call to action. As a “film” (outdated term already) professor at Miami Dade College, I am proposing an initiative that would look into finding the best available “migration strategies” that would preserve our students’ films. Think globally, act locally.

    John’s reply: Yes, it is so crucial we get the information to film students. It is their work that is most vulnerable and it is they who have most drunk the corporate kool-aid about “digital preservation.” I firmly believe it is the film school teachers who can best make young filmmakers aware of what we all face. Thank you for your comment.

  • What a timely and rich set of comments! By chance I wrote a blog entry on the same subject today, though mine discusses archival presentation in general. It’s here:

    http://www.davidbordwell.net/blog/2012/02/13/pandoras-digital-box-pix-and-pixels/

    This is always a wonderful site. And I appreciate John’s last comment about film education. We need to show students the importance of preserving work old and new.

  • (Sadly) It’s not only films that are being lost to either neglect or technological advances but also valuable scientific data from the early days of Space Program, both manned and unmanned; a serious loss for humankind. NASA flat out lost a lot the data and original video footage of the Apollo 11 mission, due to sloppy handling. http://www.hq.nasa.gov/alsj/a11/Apollo_11_TV_Tapes_Report.pdf
    There was actually high definition video of the first steps on the Moon that were seen in mission control but had to be converted to NTSC so it would be viewable on home television sets of that time, hence the reason for the murky low quality video. The original tapes seem to be forever lost.
    Technology advances have also rendered a lot of scientific information gathered by many of the early space probes unreadable. Most of the data sent by the early satellites and planetary probes was recorded on two inch magnetic tape. So much so that the tapes were stored for later analysis, but the years pass and things change and now we don’t know what’s on those tapes because two inch tape has been obsolete for years and the people who built those machines and knew how they worked have long since retired or died. There is no Digital Rosetta Stone to translate the old scientific data language into the language of the moment. Important knowledge has been lost; ET might have called and left a message that he / she would like to come visit our neighborhood sometime soon but we didn’t return the call.
    Tragic !!!

    John’s Note:
    Richard, your points are so important. We in film are, as always, hyper-focused on our own world. But it is our real life history (the story of man) that is being, and is going to be, lost, unless we find a solution. Human knowledge that has been recorded and passed down for centuries, such as the recently discovered paper texts found in Nepalese caves, is far more durable than that of our high-tech digital information systems.

  • A great summary of the urgency of this issue.

  • “It’s not unreasonable to say that the term “digital preservation” is an oxymoron.”

    Yes it is. There is such a thing as digital preservation, and it is not that hard, though it does take regular management. I work for a state digital archive and forward-migration is a key part of what we do. Digital preservation takes digital archivists.

    John’s reply: God bless you, Larry. You have the true faith.”Forward migration” is indeed the ONLY way. But I hope you are spared migration errors and obsolete formats along the way; if so, you will be the first.

  • A great discussion by some very knowledgable gentlemen, but
    I’m afraid the bottom line is always going to be money.I can
    only speak for myself,in that I have always used Cine film, have
    collected film “on film”, both 8mm gauges and 16mm and have
    never owned a video camera or a digital one.Film is tactile,
    organic and universal.There is nothing like the feeling one gets
    in tracking down a print of a much loved film and screening it,
    seeing the movie as it was meant to be seen, and as the
    gentlemen have already said,it will be there to be viewed again
    and again.Unfortunately that is not what commerce wants, and
    will again and again tell us in ads, that what we have is outdated
    and here’s something “better” as they did with video and how
    many people caught that cold.It’s a bit like watching a child
    playing with hot coals, no matter how many warnings are given
    no notice is taken until it’s too late.

    John’s reply: The nostalgia factor is one thing–it will likely disappear as the film generation retires from the field but will resurface as retro hip— like the presence of old vinyl discs in the coolest apts. But there are encouraging signs from many young filmmakers who have been educated in digital filmmaking and who are more than eager to see what “film” is all about.

  • Adrian Winchester

    I’m pleased to see the efforts being made to increase awareness in relation to this immensely important issue. But I can’t help thinking that many inpendent film-makers will remain unprepared to factor-in the cost of a film print and will keep their fingers crossed that their material remains usable in future. Suffering a major disaster may be the one thing that brings about a change in attitude!

    On a different aspect of the digital v film issue, I’d also like to see the Sci-Tech Council of the Academy comment on cinema audiences now having to tolerate far more projection problems than applied to 35mm, due to the frequency of picture freezing, breakdowns, etc. Is this progress?

    John’s reply: Adrian, the “major disaster” of loss of digital content is not a what if, but a when. It’s inevitable. And saying so does not make one a “Neinsager.” As for bad digital projection, that is a tough one for most of us cinematographers. We are spared the inevitable badly timed and badly duped analog film prints especially in second tier markets. But until the digital bugs are killed off, we are going to see more rather than fewer of these problems as maintenance of digital projectors becomes more erratic in the hinterlands that are now being compelled to install this temperamental equipment.

Leave a Reply


× 6 = eighteen