DEFCON 27 BRIEFING
The DDV kicked it in gear this year! With a lot of hard work earlier in the year (thank you, Will), we increased our total number of duplication bays from 2018 with over 50 more drive slots and even received a donation (thank you, Alex) to bring us up to a total of 145 duplicator bays we could run 24x4!
General Intake
DC27 DDV Intake
Many thanks go out to our phenomenal team of dedicated volunteers, we setup extremely quickly and started accepting drives over 2 hours earlier than planned! Our first duplication run started at 1:45pm on Thursday. We used every single system 24x4 swapping out drives from 3-6am for non-stop duplication. By 2:00pm Saturday, we received 521 drives, completed three duplication cycles, and started our final duplication runs.
6 of the drives entrusted to us were SAS (oops) and 5 drives were unusable (hardware problems) even after several duplication attempts. We were, however, able to duplicate every single functional drive given to us! We processed all 521 drives and (again) cleared out all of the available 6TB drives in Vegas this year (we really should get commissions from Fry's and Best Buy). Here’s the breakdown of our 2019 requests:
Infocon.org archive: 231
Hash Tables 1 of 2: 154
Hash Tables 2 of 2: 136
DDV Drive content requests
In addition, we were able to return all but 2 drives. Some people didn’t pick up their drives before the 11:00am deadline on Sunday and donated their drives to the DDV (thank you).
DRIVE DETAILS
This year we continued to catalog the brand and model of each drive. We were able to capture information for all 521 drives we processed - including the 11 we couldn’t image. This gave us a small sample of manufacturer information for 521 drives. We've dropped that detail for you below:
Intake by Manufacturer
BRAND MODEL QTY FAILS % FAIL
-------- ------------------ ---- ----- ------
Seagate Barracuda (dsktop) 50 1 2.0%
Barracuda Pro 3 0 0.0%
Ironwolf 81 2 2.5%
Ironwolf Pro 14 0 0.0%
Skyhawk 3 0 0.0%
Exos 13 1 7.7%
Toshiba X300 127 5 3.9%
N300 39 3 7.7%
S300 1 0 0.0%
WD Black 51 1 2.0%
Gold 5 0 0.0%
Purple 6 0 0.0%
Yellow (AE) 1 0 0.0%
Red 28 0 0.0%
Red Pro 26 1 3.9%
Blue 32 1 3.1%
Green 1 0 0.0%
White & EMAZ 14 0 0.0%
HGST Deskstar 24 1 4.2%
Other 2 0 0.0%
TOTAL 521 16 3.1%
DRIVE FAILURES
Last year, there were no fails with the Toshiba drives but nothing’s perfect forever. This year, we only received drive types from four different manufacturers but we did see a few failures with all of them. All of the failures, however, were in a really good (acceptable) range and there were no manufacturers that showed any unusually high failure rates! We set any problematic drives aside after the first attempt and took them over to 1:1 duplicators to work out any issue. At the end of the con there were only 5 drives we couldn’t duplicate to because of drive functionality issues.
Number of Drive failures
Statistics Blurb: Keep in mind that any failures with a small number of drives (i.e. 50 or less) can easily show a disproportionately higher failure rate than should be expected in real world conditions. You can dive as deep as you want to in the statistical calculations but realize that a drive failure percentage for HSRT, for example, is inflated with such a small sample.
Making it happen
Drive stickers that are applied upon completion
With the 66 hour window that Defcon allows for us to accept, tag, copy, and return all the drives (1600 Thursday to 1100 on Sunday), we know we really have to hustle. Last year, we were missing some duplicators and it really hurt us. We took 5 systems in for hardware review and repair (one didn’t survive) and we purchased 4 shiny new 1:5 Startech duplicators in preparation for this year’s demand. In addition, we had one generous donation of an additional 1:5 duplicator which almost doubled the number of duplication bays we had this year!
Each 6TB drive copy cycle takes 11-20 hours to complete so we targeted completing four cycles for each duplicator - giving us a theoretical maximum of a 540 drive throughput. — We were really close!
We kept an extremely close eye on all of the duplicators for anything slowing the process down and only left the duplicators with a few minutes of rest between cycles when swapping drives out. We were even struck by one brief power problem (a power strip failed and cost us about 6 hours of processing time). We recovered quickly by keeping things closely monitored 24x4. It must have worked because we were able to duplicate 521 drives this year with some room to spare!
The final duplication broke down to the following:
Infocon.org archive: 231
Hash Tables 1 of 2: 150
Hash Tables 2 of 2: 129
Grand Total Completed: 510
BONUS CONTENT
We had an addition to the DDV this year - a live “Pick and Pull” system run by one of our volunteers! John setup a system of 5 Raspberry Pis (4 file servers and 1 control system) attached to 4 StarTech 1:1 duplicators set in USB attached mode. The Pis created an isolated network where anyone could connect an pull down any of the Infocon.org data! We transferred 112 GB from this system and it also allowed us to run 4 additional duplications each night when the village was closed! Not bad for a beta test! Look to see expansions of this in 2020!
Wrap up
In the end we were able to push a little over 3 Petabytes out the door to our community! We rocked the village this year! We’re already gearing up for next year to able to take all comers in 2020 in the new location.
Remember, if you're one of those who wasn't able to get a drive in to us, all the content is also available through torrents or via Infocon.org. If you have the bandwidth, please seed them too!
Big thanks to DT, Will, Zant, the QM Stores, Goons, and all our volunteers for helping make DDV such a success this year! See you in 2020!