How Video Storage is Changing for the Better
We’re an award-winning video production company with a hundred-thousand hours of film in storage. Each of those hours represents irreplaceable moments in a story. They are testaments of client trust and the product of our team’s workmanship. That’s why in our twelve years of operation we’ve never deleted a single recording file. It’s not the industry standard, but we’re going to tell you why it should be.
Data Can Be Lost at Any Time
Our clients’ trust is often unspoken but unmistakable in the filmmaking process. We treat footage as though every clip were precious because those shots are irreplaceable. Clients expect that we will protect their valuable footage but most give little thought to how we actually secure the vault.
There is no worse feeling in our industry than seeing a ‘data corrupted’ message. The sunken moment of, “oh no, I’ve lost something,” is dreadful. Considering all of the things that have to come together for a project: the scheduling, travel, and single-moment opportunities. There is just so much capital that gets spent in production and you might have wasted it all. It’s not just a matter of production insurance because in documentary work you may only get one chance to capture something. We make every effort to avoid this possibility.
A lot of video producers assume that losing footage just means they’re out some money and a client relationship. To the client, the consequences could be catastrophic. They may be losing expensive ad buys or no longer have a video ready in time for their fundraising gala. Taking steps to avoid this situation is a crucial way to be good stewards of our clients’ trust.
In 2013, our Creative Director Chris Ryan was in Haiti interviewing then-President Michel Martelly. Time was scarce and he had to stay light-footed throughout production. After their interview Chris rushed to the airport with our client. As he sat in the motorcade, Chris took the brief opportunity to dump the data from his camera’s memory card onto a backup drive. Chris returned the card to his camera and stashed the dump drive in a separate carry-on. When the group returned to St. Louis, they found that much of their luggage had not made the trip home with them- including Chris’ camera case.
The client was reasonably concerned that all of the footage they had just captured had been lost. Thankfully, Chris had made that backup in the car ride and kept it with his personal belongings. The client was overjoyed and their irreplaceable footage went on to complete a valuable project. This anecdote is one example of the many times that data backups have proven critical to our worry-free production and excellent client relationships.
Film is Changing
In the earliest days of cinema, producers wrote over their films after they had been shown. Then, as today, storage of media was expensive and production companies thought the film’s life ended after showing. That mindset continues more than a century later in commercial video production. Erasing data to increase capacity happens at all levels of our industry. Even some large production companies still delete footage after they deliver their clients’ projects. We work with clients who have experienced this first hand. One such client was unable to update a half-million dollar film with their previous agency because the raw footage had been deleted after the original project was finished.
Modern video production is changing. Projects are rarely completed and never used again. Instead we regularly work with clients to develop a visual library of work that they can pull from again in future projects. Some of our clients have a thousand clips with us. With the complexity, depth, and travel to some of our projects we could never recreate all of that footage. So we operate on the assumption that every client will come back to update their project in the future. Now in our twelfth year of business, we still have the raw footage from our first projects securely stored and easily accessible.
Commitment to risk-averse data storage is a great indicator of a quality video production company. Filmmakers who fly by the seat of their pants are showing little regard for their clients’ interests. One copy on a hard drive isn’t enough anymore. Deleting the backups to save money on buying new hard drives is inviting future disaster. It’s not an exciting part of filmmaking, but it affects everyone when there’s a problem.
Once’s Best Practices
One of our favorite sayings in teaching new filmmakers is that, “if you have one, you have none.” Our team at Once Films use a ‘3-2-1’ model as our data management standard. That means having three copies of all data, on at least two different types of media, and at least one of those storage methods must be kept off-site. Here’s our workflow for data backups:
We shoot raw footage onto a memory card. We can also shoot redundantly on a hard drive if the situation calls for it.
We create an on-location backup using an independent hard drive. Sometimes we also save the footage to a laptop before transporting.
When we get back to the studio we copy the data from the memory cards onto our in-house server. That data gets duplicated on the server automatically. Our independent hard drive from earlier remains air gapped from everything and stored separately.
As a project is completed we back all of its data onto a tape drive and store that off-site. Finished client files are often shared by cloud methods.
As you can see we aim to strike a balance between financial costs, swiftness, and data security. We don’t recycle memory cards until we have three copies elsewhere. When we perform our data dumps we don’t try to organize it or change file names. Oftentimes with data, the worst thing you can do is introduce the opportunity for human error.
We organize through metadata and our editing software. Each project has working files saved by versions. These are entirely new ‘save-as’ files. We start new project files at every reliable milestone. Sometimes that’s every couple of days. Sometimes that’s a new pass at interview strings or B-roll. Anytime data goes off-site and travels we will start a new version. All of this is so that if something gets corrupted we have a recent backup. We also make even more frequent duplicate sequences in Premier. This helps us in the development process when we want to try different music or compare some different cuts by reverting back and forth.
We will work from a new duplicate if we need to make fundamental changes to an earlier project. Most fundamental changes are the result of a new project anyway so it receives the same ‘3-2-1’ treatment as the original. Long gone are the days of melting down film to recycle it into new projects- and that’s a good thing.
Storage Solutions
There is a cost to this process and the solution isn’t simply throwing money at storage. Video producers need to balance financial investment with data security. We are clever with our expenses to reduce the costs to our clients. This all starts with being scalable.
Scalability
As a video production company grows it needs to add more storage. Small filmmakers are often tempted to prioritize fancy camera gear over backup storage. Using less-expensive storage options wisely would achieve some of the same results without a huge investment. The goal is to transfer from expensive data storage to cheaper options whenever reasonable. Small production companies can easily afford backup storage if they dump to hard drives more frequently and recycle their memory cards.
Other Backup Methods
As we grew we added larger hard drives, servers, tape backup and most importantly- dedicated processes to use these methods correctly. Today, we operate a large RAID server and tape backup system. The internal server is modular and currently provides a petabyte of storage. All of our editing suites are network attached to this server and work from the same raw files. We can pull up that data instantly- even faster than a connected SSD. Our tape backup is uncommon because of its initial system cost, but it actually saves us money in the longterm because the storage medium is less expensive than common alternatives. The data saved on these tapes can sit for thirty to fifty years before it degrades. These are substantial investments but they pay dividends for an established production company.
Cloud Storage
One solution that hasn’t caught up to our production needs is cloud storage. We don’t use remote access for our raw data because the files are prohibitively large. We tried to cloud backup our server one time- the system calculated that it would take 47,000 years for it to upload our content at that time. That’s on pause for the moment but we still use cloud services for lighter stuff like finished files and project files. We share a lot of our exports through cloud services.
Conclusion
We wish more clients asked about our data storage. It’s one of the things that we consider a gold standard in our field. We hope to see the whole industry follow suit as data storage becomes more affordable. So many master recordings have been lost in history. We take these deliberate steps to ensure that our clients’ projects are not among them.