AWS adds speed

Today, customers are moving an increasing number of applications and large volumes of data to the AWS Cloud – everything from app log files to digital media, genomes, and petabytes of sensor data from connected devices. While AWS Direct Connect provides customers with a dedicated, fast connection to the AWS network, AWS Snowball and Amazon Kinesis Firehose are ideal for customers that need to transfer data in large batches, have data located in distributed locations, or require continuous loading of streaming data. With AWS Snowball and Amazon Kinesis Firehose, AWS customers now have two new ways to more easily and cost-effectively get large data sets and streaming data into the AWS Cloud.

“It has never been easier or more cost-effective for companies to collect, store, analyze, and share data than it is today with the AWS Cloud,” said Bill Vass, Vice President, AWS Storage Services. “As customers have realized that their data contains key insights that can lead to competitive advantage, they’re looking to get as much data into AWS as quickly as possible. AWS Snowball and Amazon Kinesis Firehose give customers two more important tools to get their data into AWS.”

AWS Snowball: the fastest way to transfer large amounts of data to AWS securely

Customers who need to transfer large amounts of data to AWS face a challenge – the time it takes to upload data. For instance, if a company committed 100 megabits per second of their total bandwidth capacity to transferring data to AWS, transferring 100 TB of data via that connection would take about 100 days. Companies could choose to spend more money expanding their bandwidth capability or upgrading their network, but most don’t want to do so simply to support sending more data to the cloud. Now, AWS provides a solution to this problem with AWS Snowball – a durable and tamper-resistant, encrypted, and portable storage appliance that customers can use to move that same 100 TB of data to AWS in less than a week, and at as little as one-fifth of the cost of using high-speed Internet. Customers create a job using the AWS Management Console, AWS ships the appliance directly to the customer, and the customer, upon receiving the appliance, simply plugs it into their local network. AWS Snowball provides a simple data transfer client which customers use to encrypt and transfer 50TB of data to each appliance. Customers can use multiple AWS Snowball appliances in parallel to transfer larger data sets within the same time frame. Once a customer’s data is completely loaded onto an AWS Snowball, its E Ink shipping label is automatically updated with the AWS shipping address, and customers can track the status of the transfer job using Amazon Simple Notification Service (SNS), text messages, or the AWS Management Console.

AWS Snowball appliances use multiple layers of security to protect customer data. In addition to tamper-resistant enclosures, AWS Snowball also employs end-to-end, 256-bit encryption, along with an industry-standard Trusted Platform Module (TPM) designed to ensure both security and full chain-of-custody for customer data. Once a customer's data has been transferred from the AWS Snowball to AWS's data stores (initially Amazon S3), AWS erases all data from the AWS Snowball appliance, following the standards defined by the National Institute of Standards and Technology (NIST) guidelines for media sanitization. Customers can get a report and confirm that all of their data has successfully been loaded into AWS before deleting the local copy of their data.

BuzzFeed is a global media company that produces and distributes original news, entertainment, and video to over 200 million unique monthly visitors and over 1.5 billion viewers. “As one of the world’s largest media companies, Buzzfeed generates a massive amount of content that we need to preserve and maintain,” said Eugene Ventimiglia, Director of Technical Operations, Buzzfeed. “We’ve been wanting to implement an archive solution based on Amazon Glacier, but our 250TB of archive data was too much data to migrate on individual disk drives, and using high-speed Internet would simply take too long. The release of AWS Snowball completely changes our thinking here as we now have a simple, secure, and cost-effective way to migrate the entire 250TB library to AWS in less than a week.”

Sony's Ci Media cloud platform provides contribution, collaboration, and transformation to professional media workflows. “Many of our customers have petabytes of data and limited bandwidth which has been a barrier for them to moving some of their heaviest media workflows into Sony Media Cloud (Ci),” said David Rosen, Vice President, Solutions Business Development, Sony Professional Solutions Americas. “AWS Snowball has the potential to solve this problem for us in a big way. From our early look at AWS Snowball, we are very excited at the prospect of a secure and cost-effective alternative to Internet transfers of the massive amount of media assets we have into AWS.”

Amazon Kinesis Firehose: Easily load streaming data to AWS

Mobile devices, software applications and services, wearables, industrial sensors, and IT infrastructure generate staggering amounts of data – sometimes TBs per hour. In 2013, AWS introduced Amazon Kinesis Streams to allow customers to build applications that collect, process, and analyze this streaming data with very high throughput. Many customers use Amazon Kinesis Streams to capture streaming data and load it into Amazon S3 or Amazon Redshift. Until now, this required customers to manage the Amazon Kinesis data streams and write custom code to load the data. Now, Amazon Kinesis Firehose makes this as easy as an API call. Amazon Kinesis Firehose captures data from hundreds of thousands of different sources and loads it directly into AWS, in real-time. Customers simply create an Amazon Kinesis Firehose Delivery Stream in the AWS Management Console and specify the target Amazon S3 bucket or Amazon Redshift table, and the time frequency at which they want fresh data delivered to the destination. Customers can also configure Amazon Kinesis Firehose to batch, compress, and encrypt streaming data before delivery at specified time intervals.

Hearst Corporation is one of the largest media and information companies in the world, with over 250 sites worldwide. “We monitor trending content on our digital properties, and generate terabytes of streaming data every day,” said Rick McFarland, Vice President, Data Services at Hearst Corporation. “This is the data currency of our company and feeds our business intelligence and product development. We are excited that we can just point our fleet of web servers at Amazon Kinesis Firehose, and it takes care of all aggregation, compression, and delivery of our streaming data to Amazon S3 without any intervention from us. This will reduce operational complexity from our data pipeline and allow us to focus on analytics so we can provide the best content to our customers.”

AWS Snowball and Amazon Kinesis Firehose are both available today. Customers can request an AWS Snowball appliance by visiting aws.amazon.com/importexport/request-access. Customers can access Amazon Kinesis Firehose using the AWS Management Console, AWS Command Line Interface (CLI), or AWS SDKs. Amazon Kinesis Firehose is initially available in the US East (N. Virginia), US West (Oregon), and EU (Ireland) Regions, and will expand to additional Regions in the coming months.