The Imagine Communications Research and Development data team challenge was twofold. First, they had to reduce prohibitive overhead and operational costs associated with developing a new product on Azure using Microsoft SQL Server. Second, they needed to deliver a new analytics platform built to leverage their existing analytic tools, provide instant elasticity, integrate multiple data sources, and reduce time to deliver new products and services.
This analytics system transforms TV ratings data provided by Nielsen and ComScore spanning more than 200 geographic markets in a single serverless Data Lake.
After a thorough consultation and evaluation, the Beyondsoft Big Data team proposed a solution built on AWS cloud-native services, comprised of Athena, Glue, Redshift, and Redshift Spectrum. The project also included the easy deployment of Redshift clusters and event-driven pipelines to allow Imagine Communications to move large amounts of daily generated data to a Redshift cluster.
Beyondsoft also implemented tuning and revision of the former workflow. Changes included:
By implementing the AWS managed services stack and architectural advancements, the Beyondsoft Big Data team successfully delivered a scalable, truly elastic, and highly performant data solution that also met Imagine Communications’ stringent success criteria. By moving to the AWS cloud platform, the overall cost to operate the new solution dropped from an average of $200,000 per month to an average of $5,000 per month.
During the engagement with Imagine Communications, the Beyondsoft team also provided education and training, so that the Imagine Communications Data Analytics Team could take ownership of the newly developed environment. As a practice, the Beyondsoft Big Data Team developed and delivered a runbook, so that Imagine Communications could not only manage the newly provided data platform, but also add to the solution in the future.
Migrating from Microsoft SQL Server on Azure to Redshift on Amazon Web Services provided Imagine Communications with increased performance, high scalability, and faster data processing by leveraging a sophisticated data lake solution.