A Future State Big Data Solution For a Big Data Giant

Clarity Insights Big Data Consultants Help Improve Performance

Challenge

A System Slowing Everyone Down 

This client had more than 1 Billion customers a day interacting with 2M advertisers on this site. This generated a lot of data; 22 terabytes to be precise. 

As a result, the client has hundreds of petabytes of raw data stored in Hadoop with multiple terabytes of new data ingested every hour. Fast-paced growth and increasing data ingestion were causing data management challenges. In order to scale and meet growing business needs, the client needed a better solution.

 

Solution

Rearchitecting the system

In order to improve the system's performance, Clarity:

  • Architected scalable, end-to-end processes to consume large volumes of complex data from Hive, Scribe and3P APIs
  • Integrated datasets into Hive, Vertica and 3P APIs
  • Built data integration pipelines for advertising campaigns
  • Debugged Big Data ETL pipelines
  • Developed custom UDFs for data transformation

Outcome

Faster, cheaper, better 

  • Significant cost savings compared with traditional, legacy environment.
  • Development of a next-generation platform for user insight and ad-hoc analytics.
  • Support for EDW-class, structured analytics for multiple petabytes on a multi-node EDW cluster.
  • Sample analytics, including a monetization roadmap to increase demand in the social media platform and insight to increase revenue from existing customers through media-mix optimization.
Contact us

Technologies

vertica_logo_1.png
hadoop.jpeg

Latest News

Understanding the Potential of Augmented Data Analytics

Gartner recently published its report, “Top 10 Data and Analytics Technology Trends That Will Change Your Business.” As the authors of the report note: “These data and analytics technology trends

5 Reasons You Need Customer Data Analytics

Professor Leonard Schlesinger of the Harvard Business School, considered one of America’s foremost experts on customer service, had one of his greatest moments of insight while waiting in a grocery

Benefits of a Cloud Data Warehouse

The data warehouse (DW) has proven to be the main ingredient for business intelligence (BI) and advanced analytics. It’s very success, though, has led to a situation where companies are constantly

why-clarity.jpg

Find out how Clarity Insights can help you 

Contact us