Jeff Atwood of Coding Horror posted an interesting article after news of the disappearance of Turing award-winning researcher Jim Gray. In this article he analyzes the economics of bandwidth when sending large amounts of data.
Jeff cites an ACM interview in which Jim Gray describes "TeraScale SneakerNet" boxes he created to transmit 40 TB of data across the country. He found that the cost and time to send such a large amount of data was actually lower using a scaled-up version of SneakerNet – the old-school version of networking that involved physically running floppy disks from computer to computer. They created these high-capacity, ruggedized external drives that they would fill with data then ship via FedEx.
JG We built more than 20 of these boxes we call TeraScale SneakerNet boxes. Three of them are in circulation. We have a dozen doing TeraServer work; we have about eight in our lab for video archives, backups, and so on. It's real convenient to have 40 TB of storage to work with if you are a database guy. Remember the old days and the original eight-inch floppy disks? These are just much bigger.
DP "Sneaker net" was when you used your sneakers to transport data?
JG In the old days, sneaker net was the notion that you would pull out floppy disks, run across the room in your sneakers, and plug the floppy into another machine. This is just TeraScale SneakerNet. You write your terabytes onto this thing and ship it out to your pals. Some of our pals are extremely well connected—they are part of Internet 2, Virtual Business Networks (VBNs), and the Next Generation Internet (NGI). Even so, it takes them a long time to copy a gigabyte. Copy a terabyte? It takes them a very, very long time across the networks they have.
DP When they get a whole computer, don't they still have to copy?
JG Yes, but it runs around their fast LAN at gigabit speeds as opposed to the slower Internet. The Internet plans to be running at gigabit speeds, but if you experiment with your desktop now, I think you'll find that it runs at a megabyte a second or less.
DP Megabyte a second? We get almost 10 megabytes sustained here.
JG That translates to 40 gigabytes per hour and a terabyte per day. I tend to write a terabyte in about 8 to 10 hours locally. I can send it via UPS anywhere in the U.S. That turns out to be about seven megabytes per second.
DP How do you get to the 7-megabytes-per-second figure?
JG UPS takes 24 hours, and 9 hours at each end to do the copy.
DP Wouldn't it be a lot less hassle to use the Internet?
JG It's cheaper to send the machine. The phone bill, at the rate Microsoft pays, is about $1 per gigabyte sent and about $1 per gigabyte received—about $2,000 per terabyte. It's the same hassle for me whether I send it via the Internet or an overnight package with a computer. I have to copy the files to a server in any case. The extra step is putting the SneakerNet in a cardboard box and slapping a UPS label on it. I have gotten fairly good at that. Tape media is about $3,000 a terabyte. This media, in packaged SneakerNet form, is about $1,500 a terabyte.
This prompted Jeff to perform an impressive analysis of the exact economy of data bandwidth, and the results were quite surprising. The article is definitely worth the read, but the costs he cites appear to be without loop charges. Based on my experience, here is a more likely cost structure:
| |
Cost/mo. |
Transfer Rate |
| T1 |
$550 |
192 KB/sec |
| DS3 |
$4,000 |
5.4 MB/sec |
| OC-3 |
$14,000 |
19 MB/sec |
Adjusting his cost assumptions to include loops as above, here are the results he found to transmit a 20GB file:
| |
Download 20 GB |
Upload 20 GB |
| 56K Modem |
49 days |
$24.27 |
61 days |
$30.34 |
| DSL |
1.25 days |
$1.26 |
10 days |
$10.11 |
| Premium DSL |
15 hours |
$1.05 |
5 days |
$8.43 |
| Cable |
19 hours |
$1.35 |
8 days |
$13.48 |
| Premium Cable |
10 hours |
$1.08 |
4 days |
$10.79 |
| T1 |
1.25 days |
$22.92 |
1.25 days |
$22.92 |
| DS3 |
1 hour |
$5.56 |
1 hour |
$5.56 |
| OC-3 |
17 minutes |
$5.51 |
17 minutes |
$5.51 |
I found it interesting that a DS3 is so significantly less expensive in this scenario than a T1, in terms of operating expense.
Also interesting is the comparatively low cost of DSL and cable, which often attracts most small businesses using their connection primarily for surfing the Internet. What these costs don't demonstrate, however, is the uptime statistics and the result of oversubscription, a standard engineering strategy for most broadband connections.
Even still, if a company is utilizing their connection for meaningful data transmissions, either for file transfers, remote applications, or hosting; DS3 seems to provide the most bang for the buck.