They say hindsight is 20/20, and often, when looking back at a cyber disaster, it is easy to see how a worst-case scenario could have been avoided with a few simple steps. Namely, using data protection best practices. Data loss isn’t exclusive to any one industry or corporation, it can affect anyone at any time. While celebrating World Backup Day, we wanted to bring your attention to three scary stories from the past and give you some tips to avoid similar situations .
Schrodinger’s backup: the condition of any backup is unknown until a restore is attempted
Let’s talk about that famous incident at Pixar Studios. Back in the 90’s, Pixar was nearly two years into creating Toy Story 2 when disaster struck. One of the film’s animators mistakenly entered a “rm -rf * ” (remove command) at the root level of this movie project and caused almost the entirety of the movie’s files to be deleted from Pixar’s servers. The team was quick to react by bringing their backups in place. However, their first attempt to recover the data failed. Back in 1998, they used tape as the backup target. Unfortunately for them, the movie project had grown up over 10GB in size (way above tape capacity of 4GB) and the error log was also saved at the end of the tape, having zero bytes in size, rendering all backups effectively useless, but no one realized that until they actually attempted to restore the data.
Luckily, the movie’s supervising technical director was able to save the day. She had been working from home following the recent birth of her child and was able to carry her computer into the office where the team successfully recovered a two-week old backup with almost all the original data — allowing them to resume working and deliver the finished film on schedule. This near disaster at Pixar underscores the importance of backups, especially for critical data, and most importantly backup testing. Without that, the loss of data can be catastrophic. However, as this story shows, even a single backup can make all the difference in the world.
Tip #1: Use the backup solution for any meaningful data and test backups regularly!
The highest price of a targeted cyber attack
Not all data disasters are caused by accidental human error though — cyberattacks are one of the leading causes of data loss worldwide and back in 2014, Code Spaces, a company that provided services to developers, learned this unfortunate lesson first-hand. Code Spaces experienced a distributed denial-of-service (DDoS) attack that was followed by a ransom demand. Code Spaces initially tried to mitigate the attack by changing their login credentials and contacting their hosting provider, but the attackers had already gained access to their systems. The attackers then proceeded to delete all of the company’s data, including their onsite backups, and changed the passwords to all of their systems, effectively locking Code Spaces out of their own infrastructure. As a result of this attack, Code Spaces was forced to shut down their entire business, leaving their customers without access to their code and collaboration tools. The lesson from this story is clear: offsite and air-gapped backups are absolutely essential for any company that relies on digital infrastructure. Without backups, a company is vulnerable to losing all of its data in the event of a cyberattack or other disaster. Code Spaces learned this lesson the hard way, and their story serves as a cautionary tale for any company that is tempted to skimp on backup solutions.
Tip #2: Follow 3-2-1 rule by creating multiple copies of data in different media types.
The never-ending ransomware tail
More recently, in May of 2021, Colonial Pipeline was attacked by ransomware and ended up having to shut down their entire pipeline system, which transported gasoline, diesel and jet fuel from Texas to New York. As a result, widespread panic-buying and fuel shortages occurred in many states along the East Coast, leading to higher gas prices and long lines at gas stations. The company decided to pay the ransom to the hackers, as they were unsure how long the recovery would take and hoped to speed the process along. Regardless, full recovery took several days to complete, which led to huge financial loss and an FBI investigation. All this gives us a good idea about the next tip. You should be prepared for the worst and plan recovery accordingly. Having a solid disaster recovery plan (DR plan), where all the systems are sorted, starting from the most important one, is critical. Another important aspect is to have a solution that would allow your DR testing and data recoverability verification in an automated way.
Tip #3: Imagine the unthinkable and create your recovery action plan just in case.
So, on this World Backup Day, it’s important to remember that successfully backing up your data (and keeping it backed up) can save you and/or your industry from expensive and unspeakable data loss. What’s your favorite story about backup data importance? Join our community and tell us there!