They say we don’t need fat pipes. They say that if we were offered it, we wouldn’t buy it, and that the telcos don’t want to sell it to us either.
But there are a few use cases, as Amazon pointed out last week. In a blog post, Amazon CTO Werner Vogels noted that the company is advising any customer with a 1 TB or larger data set to ship it to Amazon on a portable storage device.
The company calls the service AWS Import/Export and it is currently available in the U.S., with availability in the EU planned.
“For each portable storage device to be loaded, a manifest explains how and where to load the data, and how to map file to Amazon S3 object keys. After loading the data into Amazon S3, AWS Import/Export stores the resulting keys and MD5 Checksums in log files such that you can check whether the transfer was successful,” Vogels wrote.
As speeds in the core of the Internet rise to 100 Gbps and major institutions such as the New York Stock Exchange embrace the service, most businesses operate with smaller pipes.
Vogels calculated that it would take a 100 Mbps optical pipe running full throttle between one and two days to deliver a terabyte of data to Amazon, and for those operating on copper it would be impossible. A T-1 line would require 82 days and DSL (assuming an upstream of 768 Kbps) would take 164 days.
So we return to the past, to “sneaker net.” “Before networks were everywhere, the easiest way to transport information from one computer in your machine room was to write the data to a floppy disk, run to the computer and load the data there from that floppy. This form of data transport was jokingly called ‘sneaker net,'” Vogels wrote.
He added that business and research institutions are using ever larger data sets, and that those data sets are increasing in size at a rate that’s faster than the rate of increase of network capacity. Don’t expect to see the end of “sneaker net” soon.
As Andrew Tenenbaum once said (h/t Ken DiPietro), “never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.”