A drive day is 1 drive in operation for one day, so one drive in operation for ten days is 10 drive days. To address this issue, CTO Brian (current CTO Brian that is) worked with professors from UC Santa Cruz on the problem and the idea of Drive Days was born. In other words, the number of drives is probably not consistent across a given time period. There are new drives, failed drives, migrated drives, and so on. On a daily basis, drives enter and leave the system. That works for static systems, but data centers are quite different. So, if over the last year, you had 10 drives and 1 failed, you could say the 10% failure rate for the year. To compute a failure rate of a given group of drives over a given time period, you might start with 2 pieces of data: the number of drives, and the number of drive failures over that period of time. The answers, we hoped, would help us make better purchasing and drive deployment decisions. Questions that were top of mind included: Which drive models lasted longer? Which SMART attributes really foretold drive health? What is the failure rate of different models? And so on. Our initial goal in collecting the Drive Stats data was to determine if it would help us better understand the failure rates of the HDDs we were using to store data. Still the basic process-try something, evaluate it, tweak it, and try again-still applies, and over the years, such experiments have led to the development of our Storage Pods and our Drive Farming efforts. Conducting such experiments has been part of our DNA since we started and we continue today, albeit with a little more planning and documentation. The effort was more than a skunkworks project, but certainly not a full-fledged engineering project. We would collect and save basic drive information, including the SMART statistics for each drive, each day. In April 2013, our Drive Stats data collection experiment began.
The split could be 90/10 or even 80/20, but that’s how I think it went down. Edison – founder Brian being the 1% that is inspiration, and engineer Brian being the 99% that is perspiration. Someone had to write the Python scripts to capture and process the data, and then deploy these scripts across our fleet of shiny red Storage Pods and other storage servers, and finally someone also had to find a place to store all this newly captured data.
The two Brians – founder Brian Wilson, our CTO before he retired and engineer Brian Beach, our current CTO – take turns eating humble pie and giving each other credit for this grand experiment.īut, beyond the idea, one Brian or the other also had to make it happen. I have never been able to confirm whose idea it was to start saving the Drive Stats data. I’ve been at Backblaze longer than Drive Stats and probably know the drive stats data and history better than most, so let’s spend the next few minutes getting beyond the quarterly and lifetime tables and charts and I’ll tell you some stories from behind the scenes of Drive Stats over the past 10 years. Little did we know at the time that we’d be collecting the data for the next 10 years or writing various Drive Stats reports that are read by millions, but here we are. That was the beginning of the Backblaze Drive Stats reports that we know today. It had decided to start saving these daily snapshots to improve its understanding of the burgeoning collection of HDDs the firm was using to store customer data. On April 10, 2013, Backblaze saved its first daily HDD snapshot file. This blog, published on April 10, 2023, was written by Andy Klein, principal cloud storage storyteller, Backblaze Inc.ġ0 Stories From 10 Years of Drive Stats Data
0 kommentar(er)
