Moving big data files fast

One of the commonly heard rumblings about the relationship between cloud services and big data is the issue of moving the big data around through the cloud. In many ways, offering big data analytics services in the cloud is the ideal way to develop a large market for such capabilities. The downside is that moving Terabyte-plus files around in the cloud cannot be done easily, and most assuredly not quickly.

Moves are now afoot, however, the overcome this problem, with one of the contenders, US-based Signiant, announcing a new service aimed squarely at this issue just recently.

This is Signiant SkyDrop, which the company claims is the only commercially available hybrid SaaS solution that accelerates the movement of big data and professional media files into and out of cloud storage. Unlike other acceleration offerings, Signiant states that SkyDrop is a genuine cloud-formed solution that leverages managed cloud infrastructure, in contrast to software that customers must deploy and manage.

Signiant claims that SkyDrop offers the high transfer speed, low operational overhead and scalability that enterprises need to move even the largest data sets into and out of cloud storage. Workflows that require fast exchange of files to and from cloud storage, including active archiving, media processing and big data analysis, can now be implemented much more easily and cost-effectively.

Utilising the same hybrid SaaS platform as the company’s Media Shuttle product, SkyDrop customers benefit from a cloud-connected architecture that minimises the software management burden of deployment, ongoing monitoring and upgrades.

The cloud compute resources and software required for a highly available, high-speed storage connection are operated by Signiant as a fully managed service, requiring customers to simply install lightweight client software and connect to their cloud storage tenancy.

The service is integrated with Amazon S3 and Microsoft Azure storageand will be available initially for Amazon S3. The complete SkyDrop solution will, however, end up being offered for Azure, Google Cloud Platform and other cloud platforms as market demand grows.

“The cloud is transforming how companies in media & entertainment and a wide variety of other industries are doing business – and now they need an easier, faster and more reliable way to move big data into and out of cloud environments,” said Rick Clarkson, VP of Product Management at Signiant. “They need immediate access to data and often don’t have the resources or expertise to configure and manage a cloud infrastructure. Signiant SkyDrop helps solve these problems, ensuring enterprises have the scalability, flexibility, security and control they need.”

Signiant’s patented technology provides transfer speeds that are up to 200 times faster than standard HTTP-based access, enabling time-critical workflows and reducing on-premise storage requirements while the content waits for upload to the cloud. And as a hybrid SaaS solution with a managed cloud infrastructure, SkyDrop also minimises both technical support resource requirements and direct cloud compute costs.

It autoscales during high-volume transfer cycles to maintain performance, and is fully redundant and load-balanced for reliability, and provides developers with a choice of tools to make integration easy.

Signiant’s solutions are already used broadcasters, studios, media service providers, sports networks and games developers to facilitate the secure movement of large media files between users, applications and systems – from pre-production content and mobile news to finished-format movies, trailers and TV episodes.